The Great Data Engineering Reset: From Pipelines to Agents
Description
For years, data engineering was a story of predictable pipelines: move data from point A to point B. But AI just hit the reset button on our entire field. Now, we're all staring into the void, wondering what's next. While the fundamentals haven't changed, data remains challenging in the traditional areas of data governance, data management, and data modeling, which still present challenges. Everything else is up for grabs.
Abstract
This talk will cut through the noise and explore the future of data engineering in an AI-driven world. We'll examine how team structures will evolve, why agentic workflows and real-time systems are becoming non-negotiable, and how our focus must shift from building dashboards and analytics to architecting for automated action. The reset button has been pushed. It's time for us to invent the future of our industry.
Key Takeaways
- Understand the fundamental shift in data engineering from pipelines to agents
- Explore the evolution of team structures in the AI era
- Learn why agentic workflows and real-time systems are essential
- Discover how to shift focus from dashboards to automated action
- Gain insights into shaping the future of data engineering
Speaker
Joe Reis
Co-author of Fundamentals of Data Engineering
Joe Reis, a seasoned data professional with over two decades of experience, is a globe-trotting author and speaker. Known as a "recovering data scientist," his expertise encompasses data engineering, architecture, and machine learning. He is highly sought-after as a keynote speaker at major data conferences worldwide, advising and investing in innovative data product companies. Additionally, he contributes regularly to his personal blog and Practical Data Modeling, and hosts popular data podcasts like "The Monday Morning Data Chat" and "The Joe Reis Show." In his spare time, Joe is passionate about authoring new books and articles, continuously striving to push the boundaries of the data industry.