{ "S0001": { "title": "Beyond Isolated Data: Agentic Orchestration of Multi-Domain Knowledge for Predictive Analytics with MCP", "speakerIds": [ "RS0001" ], "type": "talk", "description": "Traditional data pipelines suffer from a fundamental philosophical limitation: they process data in isolation, without understanding context or causal relationships. This epistemological (knowledge) isolation—where systems only \"know\" their immediate datasets—leads to brittle predictions and missed insights. When we build ETL pipelines that only consider historical load data without weather patterns, economic indicators, or seasonal behaviours, we're essentially creating blind predictive systems.", "abstract": "This talk bridges the gap between philosophical AI concepts and practical data engineering implementation, providing both strategic vision and hands-on technical guidance for building the next generation of intelligent analytics systems. We'll explore how agentic AI systems and Multi-Modal Control Protocols (MCP) represent a paradigm shift from isolated data processing to contextual reasoning, using real-world electricity load forecasting as a practical example.", "key_takeaways": [ "Traditional data isolation limits predictive accuracy and insight generation", "Agentic AI systems can autonomously orchestrate multi-domain knowledge", "MCP enables practical implementation of philosophical AI concepts in data engineering", "Loosely coupled, agent-driven architectures represent the future of enterprise analytics", "The time to experiment with agentic data engineering is now" ], "implementation_levels": [ { "title": "Standalone MCP Application", "description": "A focused electricity prediction service where Claude directly accesses power grid data, weather APIs, and economic indicators to generate enhanced forecasts" }, { "title": "Enterprise Pipeline in Mage", "description": "Full integration into data engineering workflows, where agentic reasoning becomes embedded in ETL processes, enabling dynamic data source discovery and intelligent orchestration across the entire analytics stack" } ], "cities": [ "Sydney" ], "duration": "30", "level": "Intermediate", "tags": [ "AI", "Data Engineering", "Machine Learning", "Architecture", "Philosophy" ] }, "S0002": { "title": "Mission Analytics: Building Autonomous Data Intelligence", "speakerIds": [ "TS0007" ], "type": "talk", "description": "Transform your data teams from SQL writers to AI orchestrators. This session reveals how agentic workflows are revolutionizing data analysis by creating autonomous agents that can reason through complex business questions, coordinate multi-step analyses, and deliver insights without human intervention—all while democratizing data access for non-technical users.", "abstract": "Join us for an in-depth exploration of how autonomous data intelligence is reshaping the future of analytics. Learn how to leverage agentic workflows and platform-native AI operators to create sophisticated, self-reasoning data systems that can handle complex business questions and deliver insights autonomously.", "key_takeaways": [ "Agent Architecture Patterns: Design frameworks for autonomous data analysis workflows that scale from simple queries to complex multi-step investigations", "Intelligent Task Coordination: Implement agents that reason through analytical problems, validate their own work, and orchestrate sophisticated data workflows", "Platform-Native Integration: Leverage cutting-edge native AI operators that bring machine learning directly into SQL (filtering, classification, aggregation), seamlessly integrated with agentic frameworks for multimodal data processing", "Democratized Data Access: Enable business users to interact naturally with data through conversational AI that understands context and business logic", "Production Implementation: Real-world deployment strategies, performance optimization techniques, and lessons learned from enterprise implementations" ], "cities": [ "Sydney", "Melbourne" ], "duration": "40", "level": "Advanced", "tags": [ "AI", "Data Engineering", "Machine Learning", "Snowflake", "Analytics", "Architecture" ] }, "S0003": { "title": "Escape Vendor Lock-in: Streaming Analytics with Apache Polaris", "speakerIds": [ "TS0007" ], "type": "talk", "description": "Break free from vendor lock-in with open-source streaming analytics. Through live demos, we'll build real-time pipelines using Apache Polaris and Open Catalog, showing how one dataset can power analytics across multiple engines - Spark, Trino, Snowflake, and more. See true multi-engine freedom in action.", "abstract": "Join us for a hands-on session where we'll demonstrate how to build vendor-neutral streaming analytics pipelines using Apache Polaris. Learn how to create portable analytics solutions that work across multiple engines and infrastructures, breaking free from traditional vendor lock-in constraints.", "key_takeaways": [ "Build streaming analytics pipelines using Apache Polaris and Open Catalog", "Demonstrate true multi-engine data access across Spark, Trino, and Snowflake", "Implement vendor-neutral streaming architectures", "Understand Apache Polaris catalog management and security models", "Deploy portable analytics that work across any infrastructure" ], "prerequisites": [ "Basic understanding of streaming concepts and SQL" ], "cities": [ "Sydney", "Melbourne" ], "duration": "30", "level": "Intermediate", "tags": [ "Streaming", "Data Engineering", "Apache Polaris", "Open Source", "Architecture", "Snowflake" ] }, "S0004": { "title": "Bringing stories to life with AI, data streaming and generative agents", "speakerIds": [ "TS0005" ], "type": "talk", "description": "Storytelling has always been a way to connect and imagine new worlds. Now, with Generative Agents - AI-powered characters that can think, act, and adapt - we can take storytelling to a whole new level. But what if these agents could change and grow in real time, driven by live data streams?", "abstract": "Inspired by Stanford's paper \"Generative Agents: Interactive Simulacra of Human Behavior\", this session explores how to build dynamic, AI-driven worlds using Apache Kafka and Apache Flink. We'll use a Large Language Model for conversation and agent decision-making, integrate Retrieval-Augmented Generation (RAG) for memory storage and retrieval, and use JavaScript to tie it all together. Along the way, we'll examine different approaches for data processing, storage, and analysis.", "key_takeaways": [ "Build dynamic, AI-driven worlds using Apache Kafka and Apache Flink", "Implement Generative Agents with real-time adaptation capabilities", "Integrate Large Language Models for conversation and decision-making", "Utilize Retrieval-Augmented Generation (RAG) for memory management", "Create evolving virtual communities through data streaming and AI" ], "cities": [ "Sydney", "Melbourne" ], "duration": "40", "level": "Intermediate", "tags": [ "AI", "Data Streaming", "Apache Kafka", "Apache Flink", "LLM", "RAG", "JavaScript" ] }, "S0005": { "title": "New Era of Intelligence: Why Data and AI Engineering is More Critical Than Ever", "speakerIds": [ "TS0006" ], "type": "keynote", "description": "A new era of intelligence is here. But who will actually build, manage, and ensure these advanced AI systems work correctly and reliably? The traditional lines of engineering are blurring, demanding skillsets that are both deeper technically and broader in scope than ever before.", "abstract": "This keynote will break down the significant changes ahead for our field. We'll explore why Data and AI Engineers are set to become some of the most vital and in-demand roles in the industry. We'll dive into the future of data engineering that is now critically intertwined with AI, demanding new levels of expertise in architecting and managing the high-throughput, real-time systems that intelligent applications depend on. Join us to understand the evolving challenges and opportunities, and how you can become a key player in building the future of intelligent systems.", "key_takeaways": [ "Understand the evolving role of Data and AI Engineers in the new era of intelligence", "Explore the critical intersection of data engineering and AI system management", "Learn about the emerging challenges in validating and debugging AI systems", "Discover the new skillsets required for orchestrating complex AI ecosystems", "Gain insights into becoming a key player in shaping the future of intelligent systems" ], "cities": [ "Sydney", "Melbourne" ], "duration": "40", "level": "All Levels", "tags": [ "AI", "Data Engineering", "Career Development", "Future of Tech", "Architecture", "Leadership" ] }, "S0006": { "title": "From Pipelines to Purpose: What Data Engineers Should Know About Data Governance", "speakerIds": [ "TS0003" ], "type": "talk", "description": "Data governance doesn't have to be heavy, top-down, or bureaucratic. In this talk, we'll explore a pragmatic approach that data engineers can apply: Minimum Valuable Governance.", "abstract": "Learn how to move just a few floors up the \"data elevator\" from the engine room of technical pipelines toward the strategic layer where business questions live. Through a structured \"pay-as-you-go\" methodology, you'll learn how to capture, implement, and validate the knowledge that matters most. The result? Smarter systems, clearer definitions, and reusable work that drives real value.", "key_takeaways": [ "Understand the concept of Minimum Valuable Governance for data engineering", "Learn how to bridge technical pipelines with business strategy", "Master the pay-as-you-go methodology for data governance", "Discover how to capture and validate critical business knowledge", "Build smarter, more reusable data systems with clear definitions" ], "cities": [ "Sydney", "Melbourne" ], "duration": "40", "level": "Intermediate", "tags": [ "Data Governance", "Data Engineering", "Architecture", "Best Practices", "Strategy" ] }, "S0007": { "title": "The Missing Layer: Why Semantics and Knowledge Graphs Are Essential for AI-Ready Data Systems", "speakerIds": [ "TS0003" ], "type": "talk", "description": "In the current wave of Generative AI, data engineers are flooded with technology, tools, blogs, and quick fixes. But to build systems that are truly AI-ready, we must return to the fundamentals: what does the data actually mean.", "abstract": "This talk unpacks what \"semantics\", \"semantic layers\" and \"knowledge graphs\" really mean, why they matter now more than ever, and how data engineers can start applying them today. We'll walk through concrete, tactical ways to apply them in your daily work. Whether you're wrangling tables or building LLM pipelines, this talk will help you move from experimentation to intentionality.", "key_takeaways": [ "Understand the fundamental importance of semantics in AI-ready systems", "Learn practical applications of semantic layers and knowledge graphs", "Discover how to implement semantic understanding in data engineering", "Bridge the gap between traditional data engineering and AI systems", "Move from experimental to intentional data system design" ], "cities": [ "Sydney", "Melbourne" ], "duration": "40", "level": "Intermediate", "tags": [ "AI", "Data Engineering", "Knowledge Graphs", "Semantics", "Architecture", "LLM" ] }, "S0008": { "title": "The Great Data Engineering Reset: From Pipelines to Agents", "speakerIds": [ "TS0002" ], "type": "talk", "description": "For years, data engineering was a story of predictable pipelines: move data from point A to point B. But AI just hit the reset button on our entire field. Now, we're all staring into the void, wondering what's next. While the fundamentals haven't changed, data remains challenging in the traditional areas of data governance, data management, and data modeling, which still present challenges. Everything else is up for grabs.", "abstract": "This talk will cut through the noise and explore the future of data engineering in an AI-driven world. We'll examine how team structures will evolve, why agentic workflows and real-time systems are becoming non-negotiable, and how our focus must shift from building dashboards and analytics to architecting for automated action. The reset button has been pushed. It's time for us to invent the future of our industry.", "key_takeaways": [ "Understand the fundamental shift in data engineering from pipelines to agents", "Explore the evolution of team structures in the AI era", "Learn why agentic workflows and real-time systems are essential", "Discover how to shift focus from dashboards to automated action", "Gain insights into shaping the future of data engineering" ], "cities": [ "Sydney", "Melbourne" ], "duration": "40", "level": "All Levels", "tags": [ "AI", "Data Engineering", "Future of Tech", "Architecture", "Leadership", "Strategy" ] }, "S0009": { "title": "Autonomous Data Products for the Autonomous Era: Rethinking Data Architecture for GenAI", "speakerIds": [ "TS0001" ], "type": "keynote", "description": "Zhamak Dehghani, Founder & CEO, Nextdata. As enterprises scale their deployment of Generative AI (Gen AI), a central constraint has come into focus: the primary limitation is no longer model capability, but data infrastructure. Existing platforms, optimized for human interpretation and batch-oriented analytics, are misaligned with the operational realities of autonomous agents that consume, reason over, and act upon data continuously at machine scale.", "abstract": "In this talk, Zhamak Dehghani — originator of the Data Mesh and a leading advocate for decentralized data architectures — presents a framework for data infrastructure designed explicitly for the AI-native era. She identifies the foundational capabilities required by Gen AI applications: embedded semantics, runtime computational policy enforcement, agent-centric, context-driven discovery.\n\nThe session contrasts the architectural demands of AI with the limitations of today's fragmented, pipeline-driven systems—systems that rely heavily on human intervention and customized orchestration. Dehghani introduces autonomous data products as the next evolution: self-contained, self-governing services that continuously sense and respond to their environment. She offers an architectural deep dive and showcases their power with real-world use cases.\n\nAttendees will learn the architecture of \"data products 2.0\", and how to both use GenAI to transform to this new architecture, and how this new architecture serves GenAI agents at scale.", "key_takeaways": [ "Understand why data infrastructure, not model capability, is the new bottleneck for GenAI at scale.", "Learn the foundational capabilities required for AI-native data architectures: embedded semantics, runtime policy enforcement, agent-centric discovery.", "Contrast the needs of autonomous agents with the limitations of current, human-centric, batch-oriented data platforms.", "Discover the concept of autonomous data products: self-contained, self-governing, and continuously adaptive services.", "See real-world use cases and an architectural deep dive into 'data products 2.0' for the GenAI era." ], "cities": [ "Sydney" ], "duration": 40, "level": "Intermediate", "tags": [ "Data Mesh", "Data Products", "GenAI", "AI", "Architecture", "Autonomous Agents" ] }, "S0010": { "title": "Convergence of BI, Knowledge Management and Geospatial Data Science", "speakerIds": [ "RS0002" ], "type": "talk", "description": "This presentation outlines the important role of an organisational Data & Analytics Strategy (DAS) and alignment to Knowledge Management practices. For the Royal Automobile Club of Tasmania (RACT), a key DAS initiative was to review and deliver a consolidated view of all corporate and operational metrics definitions, and align Business Intelligence solutions to those definitions and the organisational operating model.", "abstract": "Learn how RACT extended their data strategy to capture the geospatial dimension, enabling new insights through spatial data science and Geospatial Business Intelligence (GeoBI). Discover an approach to deliver a centralised, organised, discoverable and accessible knowledge hub for Business Intelligence that acts as a gateway to internal and external content, leveraging Microsoft SharePoint to provide structured portals to different types of content.", "key_takeaways": [ "Understand how to align Business Intelligence with Knowledge Management practices", "Learn how to implement a consolidated view of corporate and operational metrics", "Discover how to extend BI solutions with geospatial capabilities", "Master the creation of a centralised knowledge hub for BI content", "Explore practical approaches to organizing content within a Knowledge Management framework" ], "cities": [ "Sydney" ], "duration": "30", "level": "Intermediate", "tags": [ "Business Intelligence", "Knowledge Management", "Geospatial", "Data Strategy", "Microsoft SharePoint", "Power BI" ] }, "S0011": { "title": "From the Engine Room: Building the Future of Analytics at Scale", "speakerIds": [ "TS0004" ], "type": "talk", "description": "What does it take to power real-time insights at scale? In this talk, we'll step behind the scenes of a modern analytics platform and see how ClickHouse is evolving to meet the growing demands of real-time data processing. From architectural decisions to support streaming and high-cardinality workloads, to major new features, this session will look at the future of fast, scalable analytics — and how it's being built today.", "abstract": "Join us for an in-depth look at the future of analytics at scale. We'll explore how ClickHouse is evolving to meet the growing demands of real-time data processing, from architectural decisions to support streaming and high-cardinality workloads, to major new features that are shaping the future of fast, scalable analytics.", "key_takeaways": [ "Understand the architectural decisions behind modern analytics platforms", "Learn how ClickHouse supports streaming and high-cardinality workloads", "Discover the latest features for real-time data processing", "Explore the future of fast, scalable analytics", "See how these technologies are being built and implemented today" ], "cities": [ "Sydney", "Melbourne" ], "duration": "40", "level": "Intermediate", "tags": [ "Analytics", "ClickHouse", "Real-time", "Architecture", "Data Engineering" ] }, "S0012": { "title": "Engineering the Future: Real-Time, Vector-Powered Analytics with ClickHouse", "speakerIds": [ "TS0004" ], "type": "talk", "description": "ClickHouse is powering the next wave of real-time analytics and AI-driven applications—and not just as a lightning-fast OLAP engine, but increasingly as a scalable vector database. In this talk, we'll explore how teams are using ClickHouse to handle massive, high-cardinality datasets and perform fast, cost-efficient vector search across multi-terabyte embedding stores.", "abstract": "We'll cover real-world use cases from the ClickHouse community, walk through recent feature releases announced at Open House, and share guidance on when a specialized vector database is needed—and when ClickHouse alone is more than enough.", "key_takeaways": [ "Learn how ClickHouse is evolving into a vector database", "Understand real-world use cases for vector search in ClickHouse", "Discover recent feature releases for vector-powered analytics", "Get guidance on choosing between specialized and general-purpose solutions", "Explore cost-efficient approaches to vector search at scale" ], "cities": [ "Sydney", "Melbourne" ], "duration": "40", "level": "Intermediate", "tags": [ "ClickHouse", "Vector Search", "AI", "Analytics", "Real-time" ] }, "S0013": { "title": "Catalogs, Iceberg, and Delta: Building the Foundation for Lakehouse Analytics", "speakerIds": [ "TS0004" ], "type": "talk", "description": "Data lakes promise flexibility and scale—but without real-time capabilities, they risk becoming data swamps. In this session, we'll explore how ClickHouse integrates with modern lakehouse formats like Apache Iceberg and Delta Lake to unlock real-time, high-performance analytics.", "abstract": "You'll hear how organizations are using these architectures to power faster insights, simplify data ingestion, and reduce query latency. Whether you're building a new lakehouse or optimizing an existing one, you'll leave with practical strategies to turn your data lake into a live analytics platform.", "key_takeaways": [ "Learn how ClickHouse integrates with modern lakehouse formats", "Discover practical strategies to turn your data lake into a live analytics platform", "Explore how organizations are powering faster insights with these architectures" ], "cities": [ "Sydney", "Melbourne" ], "duration": "40", "level": "Intermediate", "tags": [ "ClickHouse", "Data Lakehouse", "Iceberg", "Delta Lake", "Analytics" ] }, "S0014": { "title": "Surviving the Growth: How We Redesigned a Modern Data Platform for a Rapidly Scaling Multi-Region Business", "speakerIds": [ "RS0006" ], "type": "talk", "description": "In fast-growing companies, scaling data infrastructure isn't just about tools, it's about communication. As our business expanded across products, teams, and markets, our early data architecture started to buckle under the pressure.", "abstract": "This talk shares the real-world strategies we used to redesign and scale a modern data platform from the ground up. We'll walk through how we implemented a resilient data architecture and how we balanced flexibility with governance across business units.", "key_takeaways": [], "cities": [ "Melbourne" ], "duration": "30", "level": "Intermediate", "tags": [ "Data Platform", "Scaling", "Architecture", "Data Governance" ] }, "S0015": { "title": "Spark Structured Streaming Autoscaling", "speakerIds": [ "RS0007" ], "type": "talk", "description": "Spark Structured Streaming is a powerful tool for real-time data processing, but its default dynamic resource allocation (DRA) often falls short in handling the fluctuating workloads of modern data pipelines. This proposal introduces a custom autoscaling solution for Spark Structured Streaming, designed to optimize resource utilization, reduce costs, and improve the overall efficiency of streaming applications. This solution addresses limitations of standard DRA, offering a more granular, responsive approach to scaling Spark executors.", "abstract": "Spark Structured Streaming applications often experience varying data input rates. Standard DRA, while beneficial, may not scale in or out efficiently as its scaling logic is based on the premise of checking pending tasks, and it doesn't scale down resources during low-cycle periods, leading to inefficiency. This results in resource wastage and inefficient scaling. Our custom autoscaling solution enhances Spark Structured Streaming by providing more intelligent and responsive scaling mechanisms. Key features include metric-driven scaling, graceful executor decommissioning, configurable parameters, a client-server architecture, and seamless integration with existing infrastructure.", "key_takeaways": [ "Cost Optimization: Reduced resource wastage by dynamically scaling executors based on actual workload.", "Improved Performance: Ensures timely data processing even during peak loads.", "Increased Stability: Graceful decommissioning prevents job failures during scale-in.", "Enhanced Flexibility: Customizable parameters allow adaptation to different application requirements.", "Proactive Resource Management: Metric-driven scaling enables proactive response to changing data patterns." ], "cities": [ "Sydney" ], "duration": "30", "level": "Intermediate", "tags": [ "Spark", "Structured Streaming", "Autoscaling", "Data Engineering", "Performance", "Cost Optimization" ] }, "S0016": { "title": "Career survival for Data Engineers: Personal creativity versus corporate conformity", "speakerIds": [ "RS0008" ], "type": "talk", "description": "Data Engineers tend to be creative types that love technology but don't always fit the \"corporate mold\". Conversely, \"business\" people have expectations of IT professional that aren't always clearly communicated (or realistic).", "abstract": "What if we could adopt a \"town planning\" approach, apply it to data, and build a bridge between these two worlds? What if the business can define what it wants, and IT are valued (and rewarded) for delivering against common goals?", "key_takeaways": [], "cities": [ "Melbourne" ], "duration": "30", "level": "All Levels", "tags": [ "Career", "Data Engineering", "Culture", "Data Strategy" ] }, "S0017": { "title": "Why We Built Our Own Feature Store (and How It Scales ML at Rokt)", "speakerIds": [ "RS0009" ], "type": "talk", "description": "As machine learning systems mature, organizations face a growing need for consistent, reliable, and scalable access to features across training and inference workflows. At Rokt, we encountered challenges as our ML use cases evolved—moving from simple scoring scripts to deep learning models powering revenue-critical systems. This talk introduces our bespoke feature store infrastructure, designed to bridge the gap between experimentation and production, enabling consistent feature access, faster iteration cycles, and robust simulation capabilities.", "abstract": "Before the feature store initiative, ML workflows at Rokt suffered from fragmented, duplicated logic across Spark, SQL, and service code. This created several recurring problems including training–serving skew, slow experimentation cycles, operational fragility, and simulation bottlenecks. While open-source and managed feature store solutions exist, they didn't align well with our platform's architecture or real-time constraints, so we built a platform-native feature store from the ground up, tailored to our ML and auction use cases.", "key_takeaways": [ "Reduced Time to Production: Standardized pipelines and reusable features accelerated model rollout.", "Improved Model Integrity: Unified contracts eliminated training–serving skew and reduced production issues.", "Scalable Experimentation: Teams can now simulate auction outcomes across millions of impressions using consistent feature inputs.", "Cost Efficiency: Optimizations to Spark and batch schedules cut infrastructure spend without sacrificing freshness or coverage.", "Cross-Team Enablement: Feature discoverability and metadata visibility unlocked collaboration between engineers, data scientists, and product teams." ], "cities": [ "Sydney" ], "duration": "30", "level": "Intermediate", "tags": [ "Feature Store", "Machine Learning", "MLOps", "Data Engineering", "Scaling", "Architecture" ] }, "S0018": { "title": "Building Trino data pipelines with SQL or Python", "speakerIds": [ "TS0008" ], "type": "talk", "description": "Trino and Starburst are well known for their performance and scalability when running analytical queries, but they are equally valid platforms for executing data engineering transformation pipelines. This talk will explain how fault-tolerant execution mode adds robustness and reliability for long-running jobs that are prevalent in transformation pipelines. These pipelines can obviously be created with SQL code, but there are Python options as well. The PyStarburst and Ibis frameworks will show how data engineers can use familiar Dataframe APIs to implement their transformation logic. Finally, these options will be reviewed for their applicability to Trino and Starburst clusters.", "abstract": "Trino and Starburst are well known for their performance and scalability when running analytical queries, but they are equally valid platforms for executing data engineering transformation pipelines. This talk will explain how fault-tolerant execution mode adds robustness and reliability for long-running jobs that are prevalent in transformation pipelines. These pipelines can obviously be created with SQL code, but there are Python options as well. The PyStarburst and Ibis frameworks will show how data engineers can use familiar Dataframe APIs to implement their transformation logic. Finally, these options will be reviewed for their applicability to Trino and Starburst clusters.", "key_takeaways": [ "Understand how Trino and Starburst can be used for data engineering transformation pipelines", "Learn about fault-tolerant execution mode for robust long-running jobs", "Explore Python options with PyStarburst and Ibis frameworks", "Discover how to use familiar Dataframe APIs for transformation logic", "Evaluate the applicability of these options to Trino and Starburst clusters" ], "cities": [ "Sydney", "Melbourne" ], "duration": "30", "level": "Intermediate", "tags": [ "Trino", "Starburst", "Data Engineering", "Python", "SQL", "Data Pipelines" ] }, "S0019": { "title": "Vector data lakes and SQL-based AI workflows with Starburst Data", "speakerIds": [ "RS0010" ], "type": "talk", "description": "In this session, we explore how Starburst Data enables vector data lakes and SQL-based AI workflows, enabling seamless querying and analysis of high-dimensional data within a familiar SQL interface, while also supporting data mesh architectures for decentralized, domain-driven data ownership and governance.", "abstract": "In this session, we explore how Starburst Data enables vector data lakes and SQL-based AI workflows, enabling seamless querying and analysis of high-dimensional data within a familiar SQL interface, while also supporting data mesh architectures for decentralized, domain-driven data ownership and governance. Attendees will learn how vector data lakes enhance AI/ML pipelines, how Starburst's distributed SQL engine simplifies vector search, clustering, and LLM integrations, and real-world use cases where organizations have unified analytics and AI operations. The talk will also cover best practices for optimizing performance at scale, demonstrating how SQL-powered AI can streamline workflows without complex pipelines. Ideal for data engineers, AI practitioners, and analytics leaders, this session highlights the future of vector-enabled analytics and its impact on generative AI and real-time decision-making.", "key_takeaways": [ "Learn how vector data lakes enhance AI/ML pipelines", "Understand how Starburst's distributed SQL engine simplifies vector search and clustering", "Discover real-world use cases for unified analytics and AI operations", "Master best practices for optimizing performance at scale", "Explore the future of vector-enabled analytics and its impact on generative AI" ], "cities": [ "Sydney", "Melbourne" ], "duration": "30", "level": "Intermediate", "tags": [ "Vector Data Lakes", "AI", "SQL", "Starburst", "Data Engineering", "Machine Learning", "Analytics" ] }, "S0020": { "title": "AI Doesn't Speak SQL. Maybe You Shouldn't Either.", "speakerIds": [ "SP0011", "SP0012" ], "type": "talk", "description": "In a world where AI agents are increasingly fluent in data, why are we still writing brittle, verbose SQL by hand? This talk explores how Malloy — a modern, expressive language built for modeling and querying relational data — offers a smarter, safer, and more composable alternative to SQL in 2025. We'll walk through real-world Malloy notebooks to show how the language empowers exploration without sacrificing structure. Then we'll go under the hood with the newly released Malloy Publisher Server — the semantic layer that makes Malloy deployable — and demo the REST and MCP APIs that power everything from embedded notebooks to AI-driven agents. You'll leave with a new perspective on how Malloy: Makes data modeling feel like algebra, not acrobatics; Bridges the gap between humans and machines via the Model Context Protocol (MCP); Provides governed, scalable semantics through versioned packages; And most importantly, frees you to focus on asking better questions, not wrestling syntax. If you've ever felt like SQL wasn't designed for the data problems of today (let alone the agents of tomorrow), this talk is your invitation to explore a better way.", "abstract": "In a world where AI agents are increasingly fluent in data, why are we still writing brittle, verbose SQL by hand? This talk explores how Malloy — a modern, expressive language built for modeling and querying relational data — offers a smarter, safer, and more composable alternative to SQL in 2025. We'll walk through real-world Malloy notebooks to show how the language empowers exploration without sacrificing structure. Then we'll go under the hood with the newly released Malloy Publisher Server — the semantic layer that makes Malloy deployable — and demo the REST and MCP APIs that power everything from embedded notebooks to AI-driven agents. You'll leave with a new perspective on how Malloy: Makes data modeling feel like algebra, not acrobatics; Bridges the gap between humans and machines via the Model Context Protocol (MCP); Provides governed, scalable semantics through versioned packages; And most importantly, frees you to focus on asking better questions, not wrestling syntax. If you've ever felt like SQL wasn't designed for the data problems of today (let alone the agents of tomorrow), this talk is your invitation to explore a better way.", "level": "Intermediate", "tags": [ "GenAI", "AI", "Malloy", "SQL", "Data Modeling" ], "cities": [ "Sydney" ], "duration": "40" }, "S0021": { "title": "Designing Cost-Efficient, Metadata-driven Kafka Processing Systems at Scale", "speakerIds": [ "SP0013", "SP0014", "SP0015" ], "type": "talk", "description": "Afterpay Kafka data processing at Block powers over 9 TB of data daily using Kafka Orion Ingest (KOI)—a fully metadata-driven framework that uses dynamically rendered Airflow DAGs for orchestration and SLA alerting. To reduce the cost of cross-region data movement, KOI integrates with Project Teleport, a two-stage architecture that compresses and transfers incremental records while preserving streaming state—cutting annual egress costs by over USD 540,000. The migration of legacy pipelines to Teleport was supported by Ozone – a GitOps-based deployment system that enables versioned, peer-reviewed SQL and Python workflows, with full operational auditability.", "abstract": "Afterpay Kafka data processing at Block powers over 9 TB of data daily using Kafka Orion Ingest (KOI)—a fully metadata-driven framework that uses dynamically rendered Airflow DAGs for orchestration and SLA alerting. To reduce the cost of cross-region data movement, KOI integrates with Project Teleport, a two-stage architecture that compresses and transfers incremental records while preserving streaming state—cutting annual egress costs by over USD 540,000. The migration of legacy pipelines to Teleport was supported by Ozone – a GitOps-based deployment system that enables versioned, peer-reviewed SQL and Python workflows, with full operational auditability.", "level": "Intermediate", "tags": [ "Data Engineering", "Kafka", "Cost Optimization", "Metadata", "Airflow", "GitOps" ], "cities": [ "Melbourne" ], "duration": "40" }, "S0022": { "title": "Real-Time Financial Crime Detection: Architecting Streaming AML (anti-money laundering) Pipelines for Instant Insights", "speakerIds": [ "SP0016" ], "type": "talk", "description": "Financial crime including money laundering, fraud, and terrorism financing is a significant and rapidly evolving threat that undermines global financial institutions and economies. Traditional Anti-Money Laundering (AML) systems are reliant on batch processing and struggle to keep pace with the increasing velocity of transactions, such as real-time payments and cryptocurrency exchanges, resulting in delayed detection and increased risk exposure. This paper will explore the architecture and implementation of streaming-first Anti-Money Laundering (AML) pipelines that leverage event-driven frameworks, real-time data enrichment, graph-based anomaly detection, and AI-driven automation to detect suspicious activity instantaneously. Attendees will gain insights into how cutting-edge technologies like Apache Kafka, Apache Flink, Graph databases, and Agentic AI can be integrated to create scalable, low-latency AML solutions. I will be talking about the following: Why traditional AML tools no longer keep up with evolving financial crimes and challenges faced by regulators in catch up to the changing world. The limitations of traditional batch-based AML approaches. Modern architecture of streaming AML pipelines, transaction ingestion, transaction enrichment with KYC and risk data, real-time analytics, and automated decision-making. Real-world use cases like transaction monitoring, crypto AML and risk scoring. Challenges to watch for, like privacy concerns and integrating with old systems. This session is ideal for fintech architects, data engineers, compliance officers, and financial crime analysts interested in using the power of real-time streaming data and AI to build next-generation AML capabilities that detect financial crime as it happens.", "abstract": "Financial crime including money laundering, fraud, and terrorism financing is a significant and rapidly evolving threat that undermines global financial institutions and economies. Traditional Anti-Money Laundering (AML) systems are reliant on batch processing and struggle to keep pace with the increasing velocity of transactions, such as real-time payments and cryptocurrency exchanges, resulting in delayed detection and increased risk exposure. This paper will explore the architecture and implementation of streaming-first Anti-Money Laundering (AML) pipelines that leverage event-driven frameworks, real-time data enrichment, graph-based anomaly detection, and AI-driven automation to detect suspicious activity instantaneously. Attendees will gain insights into how cutting-edge technologies like Apache Kafka, Apache Flink, Graph databases, and Agentic AI can be integrated to create scalable, low-latency AML solutions.", "level": "Intermediate", "tags": [ "GenAI", "AML", "Streaming", "Kafka", "Flink", "Graph", "AI", "Financial Crime" ], "cities": [ "Sydney" ], "duration": "40" }, "S0023": { "title": "The Agentic Data Engineer - friend or foe ?", "speakerIds": [ "SP0017" ], "type": "talk", "description": "GenAI is transforming every part of s/w development (including data engineering) faster than any previous transformation. This talk is based on experience building an agentic solution for developing and testing data products. It explains the language model ecosystem, the agentic approach and how data engineers can get to meet their new agentic data engineer colleagues!", "abstract": "GenAI is transforming every part of s/w development (including data engineering) faster than any previous transformation. This talk is based on experience building an agentic solution for developing and testing data products. It explains the language model ecosystem, the agentic approach and how data engineers can get to meet their new agentic data engineer colleagues!", "level": "Intermediate", "tags": [ "GenAI", "Agentic AI", "Data Engineering", "Software Development" ], "cities": [ "Sydney" ], "duration": "40" }, "W0001": { "title": "Mastering GraphRAG with Neo4j", "speakerIds": [ "SP0035" ], "type": "workshop", "description": "In this workshop, you will learn about the neo4j-graphrag Python open-source package and how it can be used to build Retrieval-Augmented Generation (RAG) applications. You will also learn how to integrate Neo4j with generative AI models to enhance graph-powered applications and AI solutions.", "abstract": "Join us for an intensive hands-on workshop where you'll dive deep into building GraphRAG applications using Neo4j. This workshop will cover everything from setting up your development environment to implementing advanced RAG patterns using graph databases. You'll learn practical techniques for enhancing your AI applications with graph-powered context and reasoning capabilities.", "key_takeaways": [ "Master the neo4j-graphrag Python package for building RAG applications", "Learn how to integrate Neo4j with generative AI models", "Understand graph-based approaches to enhance RAG systems", "Implement practical solutions for graph-powered AI applications", "Gain hands-on experience with real-world GraphRAG implementations" ], "prerequisites": [ "Basic understanding of Python", "Familiarity with RAG concepts", "Basic knowledge of graph databases (helpful but not required)" ], "cities": [ "Melbourne" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "Neo4j", "RAG", "Graph Databases", "AI", "Python" ], "time_slot": "09:00-12:30", "max_participants": 20, "materials": [ "Laptop with Python 3.8+ installed", "Neo4j account (free tier available)", "GitHub account for accessing workshop materials" ], "registration_link": "https://forms.gle/YHoaFG2JssmFX1A96" }, "W0002": { "title": "Implementing the medallion architecture with Starburst", "speakerIds": [ "TS0008", "RS0010" ], "type": "workshop", "description": "The focus of this workshop is to implement tables in the bronze, silver, and gold layers of the medallion architecture using the Starburst platform. The hands-on exercises will be completed on the Trino-based Starburst Galaxy platform, so just bring your laptop and be ready to learn.", "abstract": "The focus of this workshop is to implement tables in the bronze, silver, and gold layers of the medallion architecture using the Starburst platform. The hands-on exercises will be completed on the Trino-based Starburst Galaxy platform, so just bring your laptop and be ready to learn. \n\nIn this hands-on workshop, you will complete these activities:\n\nBeginning with the bronze layer, see how Starburst ingestion tools make it easy to source data from files on the object store, as well as from a Kafka topic. \nThen you will perform transformation logic to move the raw table in the bronze layer to a business-ready structured & validated table in the silver layer.\nThe gold layer will include views that you will construct which are aligned to consumption-oriented business requirements.\nYou will learn how Starburst auto-tagging features use AI to identify PII data and apply appropriate tags that you can review.\nNext, you'll create a Starburst Data Product with your gold and silver tables and views.\nThen you will apply RBAC and ABAC security policies to ensure different roles can only access the information they should be able to see.\nFor the speedy kids, an additional exercise will be present allowing them to use the MERGE command to populate a CDC Type 2 table based on change deltas being applied to an existing table.", "key_takeaways": [ "Learn how to implement bronze, silver, and gold layers using Starburst platform", "Master data ingestion from object stores and Kafka topics", "Understand transformation logic for moving data between medallion layers", "Create business-aligned views in the gold layer", "Apply AI-powered PII auto-tagging and security policies", "Build Starburst Data Products with proper RBAC and ABAC controls" ], "prerequisites": [ "Basic understanding of SQL", "Familiarity with data lake concepts", "Laptop with web browser for accessing Starburst Galaxy" ], "cities": [ "Sydney", "Melbourne" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "Medallion Architecture", "Starburst", "Data Engineering", "Data Lake", "Security", "RBAC", "ABAC" ], "time_slot": "09:00-12:30", "max_participants": 20, "materials": [ "Laptop with web browser", "Starburst Galaxy account (will be provided)", "Basic SQL knowledge" ], "registration_links": { "Sydney": "https://forms.gle/M1CmvAXHyp9h41rr7", "Melbourne": "https://forms.gle/YHoaFG2JssmFX1A96" } }, "W0003": { "title": "High-Speed Data Engineering with ClickHouse", "speakerIds": [ "TS0004" ], "type": "workshop", "description": "As a data engineer, you know the value of fast, flexible analytics — and you've probably heard that ClickHouse is built for exactly that. But what does it really take to make it work in your stack?", "abstract": "As a data engineer, you know the value of fast, flexible analytics — and you've probably heard that ClickHouse is built for exactly that. But what does it really take to make it work in your stack?\n\nIn this instructor-led, hands-on workshop, we'll take you deep into the core mechanics of ClickHouse, from data modeling and ingestion to architecture and performance tuning. You'll walk away not just knowing how to use ClickHouse — but understanding why it works the way it does, and how to use it effectively in production.\n\nWhat you'll learn:\n- How ClickHouse handles massive scale through its architecture, indexing, and storage engine design\n- Strategies for modeling and inserting data that won't backfire as your volumes grow\n- How to work with structured files, object storage, and efficient query patterns\n- Use cases like analytics pipelines, observability systems, and cloud-native data lakes (time permitting)\n\nThis training includes:\n- Getting Started: Install ClickHouse, run your first queries, explore the formats\n- Modeling Data: MergeTree tables, primary keys, granules, and when they matter\n- Ingest at Scale: Table engines, async inserts, schema design, and best practices\n- Use Cases: From raw files to real insights — hands-on walkthroughs\n\nHands-on Labs:\n- Querying data directly from object storage\n- Modeling and inserting UK property data\n- Building tables with S3 and exploring transform patterns", "key_takeaways": [ "Understand ClickHouse architecture and performance tuning", "Learn data modeling strategies for massive scale", "Master ingestion patterns and best practices", "Get hands-on experience with real-world use cases", "Explore analytics pipelines, observability systems, and cloud-native data lakes" ], "prerequisites": [ "Basic understanding of data engineering concepts", "Familiarity with SQL", "Laptop with internet access for hands-on labs" ], "cities": [ "Melbourne" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "ClickHouse", "Data Engineering", "Analytics", "Performance Tuning", "Data Modeling", "Object Storage" ], "time_slot": "09:00-12:30", "max_participants": 20, "materials": [ "Laptop with internet access", "ClickHouse installation (instructions provided)", "Basic SQL knowledge" ], "registration_link": "https://forms.gle/YHoaFG2JssmFX1A96" }, "W0004": { "title": "Accelerate AI Success with Snowflake Masterclass", "speakerIds": [ "SF0005", "SF0006" ], "type": "workshop", "description": "Join Accelerate AI Success with Snowflake Masterclass to access the latest technologies and innovations in the AI Data Cloud. This event offers an in-depth, firsthand look at the latest AI capabilities.", "abstract": "Join Accelerate AI Success with Snowflake Masterclass to access the latest technologies and innovations in the AI Data Cloud. This event offers an in-depth, firsthand look at the latest AI capabilities. It will highlight Snowflake's AI capabilities and how enterprises can adopt robust use cases along with demos and hands-on labs. You will learn best practices and have the opportunity to get your questions answered via live Q&A sessions with the Snowflake technology team.\n\nWHAT YOU WILL EXPLORE & LEARN\nSnowflake Vision for AI/ ML\nThis session will provide an overview of Snowflake's AI capabilities and roadmap, and cover the breadth and depth of AI capabilities your team can leverage.\n\nCapabilities & Roadmap\nHosted by your Snowflake AI experts, this session will cover a range of AI capabilities your team can leverage, including:\n- Large Language Models (LLMs) available to your teams from data engineering to business stakeholders via Snowflake Cortex\n- Data Agents – leverage Snowflake Cortex Agents to simply talk to ALL of your data\n- Document Intelligence – Extract insights from all of your unstructured data using Snowflake Document AI across images, documents and emails\n- Innovations from Snowflake – embedding AI to simplify all data operations for data teams with embedded AI assistants\n\nHands-On Learning\nHands-on Workshop – Designed for both practitioners and leaders, this session offers practical experience and insights into implementing AI solutions with Snowflake. This session allows you to get hands-on with AI while learning from the Snowflake experts.", "key_takeaways": [ "Understand Snowflake's AI capabilities and roadmap", "Learn about Large Language Models via Snowflake Cortex", "Explore Data Agents and Document Intelligence", "Get hands-on experience with AI solutions" ], "prerequisites": [ "Basic understanding of data concepts", "Familiarity with cloud platforms", "Laptop with internet access" ], "cities": [ "Sydney" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "Snowflake", "AI", "Machine Learning", "Data Cloud", "Cortex", "Document AI" ], "time_slot": "09:00-12:30", "max_participants": 20, "materials": [ "Laptop with internet access", "Snowflake account (will be provided)", "Basic understanding of data concepts" ], "registration_link": "https://forms.gle/M1CmvAXHyp9h41rr7" }, "W0005": { "title": "Accelerate AI Success with Snowflake Masterclass", "speakerIds": [ "SF0001", "SF0002" ], "type": "workshop", "description": "Join Accelerate AI Success with Snowflake Masterclass to access the latest technologies and innovations in the AI Data Cloud. This event offers an in-depth, firsthand look at the latest AI capabilities.", "abstract": "Join Accelerate AI Success with Snowflake Masterclass to access the latest technologies and innovations in the AI Data Cloud. This event offers an in-depth, firsthand look at the latest AI capabilities. It will highlight Snowflake's AI capabilities and how enterprises can adopt robust use cases along with demos and hands-on labs. You will learn best practices and have the opportunity to get your questions answered via live Q&A sessions with the Snowflake technology team.\n\nWHAT YOU WILL EXPLORE & LEARN\nSnowflake Vision for AI/ ML\nThis session will provide an overview of Snowflake's AI capabilities and roadmap, and cover the breadth and depth of AI capabilities your team can leverage.\n\nCapabilities & Roadmap\nHosted by your Snowflake AI experts, this session will cover a range of AI capabilities your team can leverage, including:\n- Large Language Models (LLMs) available to your teams from data engineering to business stakeholders via Snowflake Cortex\n- Data Agents – leverage Snowflake Cortex Agents to simply talk to ALL of your data\n- Document Intelligence – Extract insights from all of your unstructured data using Snowflake Document AI across images, documents and emails\n- Innovations from Snowflake – embedding AI to simplify all data operations for data teams with embedded AI assistants\n\nHands-On Learning\nHands-on Workshop – Designed for both practitioners and leaders, this session offers practical experience and insights into implementing AI solutions with Snowflake. This session allows you to get hands-on with AI while learning from the Snowflake experts.", "key_takeaways": [ "Understand Snowflake's AI capabilities and roadmap", "Learn about Large Language Models via Snowflake Cortex", "Explore Data Agents and Document Intelligence", "Get hands-on experience with AI solutions" ], "prerequisites": [ "Basic understanding of data concepts", "Familiarity with cloud platforms", "Laptop with internet access" ], "cities": [ "Melbourne" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "Snowflake", "AI", "Machine Learning", "Data Cloud", "Cortex", "Document AI" ], "time_slot": "09:00-12:30", "max_participants": 20, "materials": [ "Laptop with internet access", "Snowflake account (will be provided)", "Basic understanding of data concepts" ], "registration_link": "https://forms.gle/YHoaFG2JssmFX1A96" }, "W0006": { "title": "Unlocking Real-Time, Reliable, and Reusable Data", "speakerIds": [ "CF0003" ], "type": "workshop", "description": "Tired of complex, high-maintenance data pipelines? This workshop tackles the challenge of moving operational data to analytics by guiding you through building a robust, end-to-end event streaming system. Unlock real-time insights and streamline your data architecture with industry-leading tools.", "abstract": "Tired of complex, high-maintenance data pipelines? This workshop tackles the challenge of moving operational data to analytics by guiding you through building a robust, end-to-end event streaming system. Unlock real-time insights and streamline your data architecture with industry-leading tools.\n\nWhat You'll Learn:\n✓ An introduction to the fundamentals of event streaming with Apache Kafka.\n✓ How Apache Flink provides powerful capabilities for real-time stream processing and transformations.\n✓ The role of Tableflow in seamlessly producing high-quality Apache Iceberg data, and how to access this data from AWS.\n✓ Practical techniques for integrating Kafka Connectors for efficient data ingestion.\n✓ How to build and deploy a complete event streaming pipeline from source to sink, leveraging these cutting-edge technologies.", "key_takeaways": [ "Master the fundamentals of event streaming with Apache Kafka", "Learn real-time stream processing and transformations with Apache Flink", "Understand how Tableflow produces high-quality Apache Iceberg data", "Integrate Kafka Connectors for efficient data ingestion", "Build and deploy complete event streaming pipelines from source to sink" ], "prerequisites": [ "Basic understanding of data engineering concepts", "Familiarity with data pipelines and ETL/ELT processes", "Interest in real-time data solutions" ], "cities": [ "Sydney" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "Confluent", "Kafka", "Tableflow", "Real-time Data", "Apache Iceberg", "Data Streaming", "Analytics", "Apache Flink" ], "time_slot": "13:00-16:30", "max_participants": 20, "materials": [ "Laptop with internet access", "Basic understanding of data engineering concepts" ], "registration_link": "https://forms.gle/M1CmvAXHyp9h41rr7" }, "W0007": { "title": "Unlocking Real-Time, Reliable, and Reusable Data", "speakerIds": [ "CF0003" ], "type": "workshop", "description": "Tired of complex, high-maintenance data pipelines? This workshop tackles the challenge of moving operational data to analytics by guiding you through building a robust, end-to-end event streaming system. Unlock real-time insights and streamline your data architecture with industry-leading tools.", "abstract": "Tired of complex, high-maintenance data pipelines? This workshop tackles the challenge of moving operational data to analytics by guiding you through building a robust, end-to-end event streaming system. Unlock real-time insights and streamline your data architecture with industry-leading tools.\n\nWhat You'll Learn:\n✓ An introduction to the fundamentals of event streaming with Apache Kafka.\n✓ How Apache Flink provides powerful capabilities for real-time stream processing and transformations.\n✓ The role of Tableflow in seamlessly producing high-quality Apache Iceberg data, and how to access this data from AWS.\n✓ Practical techniques for integrating Kafka Connectors for efficient data ingestion.\n✓ How to build and deploy a complete event streaming pipeline from source to sink, leveraging these cutting-edge technologies.", "key_takeaways": [ "Master the fundamentals of event streaming with Apache Kafka", "Learn real-time stream processing and transformations with Apache Flink", "Understand how Tableflow produces high-quality Apache Iceberg data", "Integrate Kafka Connectors for efficient data ingestion", "Build and deploy complete event streaming pipelines from source to sink" ], "prerequisites": [ "Basic understanding of data engineering concepts", "Familiarity with data pipelines and ETL/ELT processes", "Interest in real-time data solutions" ], "cities": [ "Melbourne" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "Confluent", "Kafka", "Tableflow", "Real-time Data", "Apache Iceberg", "Data Streaming", "Analytics", "Apache Flink" ], "time_slot": "13:00-16:30", "max_participants": 20, "materials": [ "Laptop with internet access", "Basic understanding of data engineering concepts" ], "registration_link": "https://forms.gle/YHoaFG2JssmFX1A96" }, "W0008": { "title": "Real-Time Data Analytics Workshop: From Batch to Stream", "speakerIds": [ "CT0001" ], "type": "workshop", "description": "Building Modern Data Pipelines with Mage, Confluent, ClickHouse, and Snowflake", "abstract": "This comprehensive 3-4 hour hands-on workshop is designed to take participants of all skill levels through the journey of building modern data pipelines. We'll start with batch processing and gradually move to real-time streaming, using real-world NSW Transport incident data.\n\nLearning Objectives\nBy the end of this workshop, you will:\n\n✅ Understand the difference between batch and streaming data processing\n✅ Build a batch pipeline using Mage AI and Apache Iceberg\n✅ Query data lakes using Starburst (replacing Athena)\n✅ Create real-time streaming pipelines with Confluent Kafka\n✅ Store and analyze streaming data in ClickHouse\n✅ Build interactive dashboards with Snowflake and Streamlit\n✅ Integrate AI/ML using Claude API and Snowflake Cortex AI\n✅ Deploy production-ready data solutions\n\n🏗️ What We'll Build Today\n\nModule 1: Batch Pipeline\n[NSW Transport API] → [Mage Batch Pipeline] → [Apache Iceberg] → [Starburst Analytics]\n\nModule 2: AI-Powered Analytics\n[Iceberg + Streaming Data] → [Snowflake] → [Streamlit App] → [Claude AI Assistant]\n\nModule 3: Real-Time Streaming\n[NSW Transport API] → [Mage Streaming] → [Confluent Kafka] → [ClickHouse] → [Real-time Dashboard]", "key_takeaways": [ "Understand batch vs streaming data processing", "Build batch pipelines with Mage AI and Apache Iceberg", "Query data lakes using Starburst", "Create real-time streaming pipelines with Confluent Kafka", "Store and analyze streaming data in ClickHouse", "Build interactive dashboards with Snowflake and Streamlit", "Integrate AI/ML using Claude API and Snowflake Cortex AI", "Deploy production-ready data solutions" ], "prerequisites": [ "Basic understanding of data engineering concepts", "Familiarity with Python", "Laptop with internet access for cloud services" ], "cities": [ "Sydney" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "Real-time Analytics", "Mage AI", "Confluent", "ClickHouse", "Snowflake", "Apache Iceberg", "Starburst", "Streamlit", "Claude AI", "Data Pipelines" ], "time_slot": "09:00-12:30", "max_participants": 20, "materials": [ "Laptop with internet access", "Python 3.8+ installed", "Cloud accounts (will be provided for workshop)", "Basic understanding of data engineering concepts" ], "registration_link": "https://forms.gle/M1CmvAXHyp9h41rr7" }, "S9998": { "title": "Placeholder Data Engineering Session", "speakerIds": [ "TBA" ], "type": "talk", "description": "This is a placeholder for a future Data Engineering session.", "tags": [ "Data Engineering", "Placeholder" ] }, "S9999": { "title": "Placeholder ML/GenAI Session", "speakerIds": [ "TBA" ], "type": "talk", "description": "This is a placeholder for a future ML/GenAI session.", "tags": [ "ML", "GenAI", "Placeholder" ] }, "S10000": { "title": "ML/AI Panel: Bringing AI to Production", "speakerIds": [ "SP0018", "SP0031", "SP0032" ], "type": "panel", "description": "A panel discussion on the real-world challenges and breakthroughs in bringing AI and ML to production, featuring Suneeta Mall (Head of AI Engineering, Harrison.ai), Somy Ayazi (Senior Software Engineer at Atlassian), and Kanishka Mohaia (AI/ML Specialist).", "abstract": "Join industry leaders as they discuss the practical challenges of deploying AI and ML systems in production environments. This panel will explore topics including MLOps best practices, scaling AI solutions, governance and compliance, and lessons learned from real-world implementations.", "key_takeaways": [ "Learn about MLOps best practices for production AI systems", "Understand challenges in scaling AI solutions across organizations", "Discover governance and compliance considerations for AI deployment", "Gain insights from real-world AI production experiences", "Explore the future of AI in enterprise environments" ], "cities": [ "Sydney" ], "duration": "40", "level": "Intermediate", "tags": [ "GenAI", "AI", "ML", "Panel", "Production", "MLOps", "Enterprise" ] }, "S10001": { "title": "Shifting Complexity Away from Data Pipelines with Tableflow", "speakerIds": [ "SP0019" ], "type": "talk", "description": "This talk highlights how building and maintaining data pipelines often involves significant operational challenges, manual work, and inefficiencies. Prerna Tiwari introduces Confluent's Tableflow as a solution that abstracts this complexity by automating integration and data movement tasks. Tableflow enables data engineering teams to focus less on tedious pipelines and more on deriving insights, delivering improved productivity, reliability, and faster deployment for modern data workflows with streamlined operations and better resource utilisation.", "tags": [ "Data Engineering", "Streaming", "Confluent", "Flink", "Tableflow", "Iceberg", "Delta Lake" ], "cities": [ "Melbourne" ], "level": "Beginner" }, "S10002": { "title": "A Brief Introduction to Hook", "speakerIds": [ "SP0020" ], "type": "talk", "description": "Traditional data warehousing methods often struggle to balance agility, clarity, and business alignment. In this session, Andrew Foad introduces Hook—a practical, business-first approach to data warehousing that simplifies integration, enhances traceability, and accelerates delivery. Developed from decades of real-world experience, Hook focuses on modelling core business concepts and their relationships in a way that is intuitive, scalable, and adaptable to change. Attendees will gain insight into the principles behind Hook and how it compares to other approaches like Data Vault and dimensional modelling.", "tags": [ "Data Engineering", "Data Warehousing", "Hook", "Business Alignment", "Agility" ], "cities": [ "Melbourne" ] }, "S10003": { "title": "Why model data?", "speakerIds": [ "SP0020", "SP0021" ], "type": "panel", "description": "A panel discussion exploring the importance, challenges, and modern approaches to data modeling in today's data-driven organizations. Featuring Andrew Foad and Patrick Cuba.", "tags": [ "Data Engineering", "Panel", "Data Modeling", "Architecture" ], "cities": [ "Sydney" ] }, "W0009": { "title": "Mastering GraphRAG with Neo4j", "speakerIds": [ "SP0035" ], "type": "workshop", "description": "In this workshop, you will learn about the neo4j-graphrag Python open-source package and how it can be used to build Retrieval-Augmented Generation (RAG) applications. You will also learn how to integrate Neo4j with generative AI models to enhance graph-powered applications and AI solutions.", "abstract": "Join us for an intensive hands-on workshop where you'll dive deep into building GraphRAG applications using Neo4j. This workshop will cover everything from setting up your development environment to implementing advanced RAG patterns using graph databases. You'll learn practical techniques for enhancing your AI applications with graph-powered context and reasoning capabilities.", "key_takeaways": [ "Master the neo4j-graphrag Python package for building RAG applications", "Learn how to integrate Neo4j with generative AI models", "Understand graph-based approaches to enhance RAG systems", "Implement practical solutions for graph-powered AI applications", "Gain hands-on experience with real-world GraphRAG implementations" ], "prerequisites": [ "Basic understanding of Python", "Familiarity with RAG concepts", "Basic knowledge of graph databases (helpful but not required)" ], "cities": [ "Sydney" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "Neo4j", "RAG", "Graph Databases", "AI", "Python" ], "time_slot": "13:00-16:30", "max_participants": 20, "materials": [ "Laptop with Python 3.8+ installed", "Neo4j account (free tier available)", "GitHub account for accessing workshop materials" ], "registration_link": "https://forms.gle/M1CmvAXHyp9h41rr7" }, "W0010": { "title": "High-Speed Data Engineering with ClickHouse", "speakerIds": [ "TS0004" ], "type": "workshop", "description": "As a data engineer, you know the value of fast, flexible analytics — and you've probably heard that ClickHouse is built for exactly that. But what does it really take to make it work in your stack?", "abstract": "As a data engineer, you know the value of fast, flexible analytics — and you've probably heard that ClickHouse is built for exactly that. But what does it really take to make it work in your stack?\n\nIn this instructor-led, hands-on workshop, we'll take you deep into the core mechanics of ClickHouse, from data modeling and ingestion to architecture and performance tuning. You'll walk away not just knowing how to use ClickHouse — but understanding why it works the way it does, and how to use it effectively in production.\n\nWhat you'll learn:\n- How ClickHouse handles massive scale through its architecture, indexing, and storage engine design\n- Strategies for modeling and inserting data that won't backfire as your volumes grow\n- How to work with structured files, object storage, and efficient query patterns\n- Use cases like analytics pipelines, observability systems, and cloud-native data lakes (time permitting)\n\nThis training includes:\n- Getting Started: Install ClickHouse, run your first queries, explore the formats\n- Modeling Data: MergeTree tables, primary keys, granules, and when they matter\n- Ingest at Scale: Table engines, async inserts, schema design, and best practices\n- Use Cases: From raw files to real insights — hands-on walkthroughs\n\nHands-on Labs:\n- Querying data directly from object storage\n- Modeling and inserting UK property data\n- Building tables with S3 and exploring transform patterns", "key_takeaways": [ "Understand ClickHouse architecture and performance tuning", "Learn data modeling strategies for massive scale", "Master ingestion patterns and best practices", "Get hands-on experience with real-world use cases", "Explore analytics pipelines, observability systems, and cloud-native data lakes" ], "prerequisites": [ "Basic understanding of data engineering concepts", "Familiarity with SQL", "Laptop with internet access for hands-on labs" ], "cities": [ "Sydney" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "ClickHouse", "Data Engineering", "Analytics", "Performance Tuning", "Data Modeling", "Object Storage" ], "time_slot": "13:00-16:30", "max_participants": 20, "materials": [ "Laptop with internet access", "ClickHouse installation (instructions provided)", "Basic SQL knowledge" ], "registration_link": "https://forms.gle/M1CmvAXHyp9h41rr7" }, "W0011": { "title": "Real-Time Data Analytics Workshop: From Batch to Stream", "speakerIds": [ "CT0001" ], "type": "workshop", "description": "Building Modern Data Pipelines with Mage, Confluent, ClickHouse, and Snowflake", "abstract": "This comprehensive 3-4 hour hands-on workshop is designed to take participants of all skill levels through the journey of building modern data pipelines. We'll start with batch processing and gradually move to real-time streaming, using real-world NSW Transport incident data.\n\nLearning Objectives\nBy the end of this workshop, you will:\n\n✅ Understand the difference between batch and streaming data processing\n✅ Build a batch pipeline using Mage AI and Apache Iceberg\n✅ Query data lakes using Starburst (replacing Athena)\n✅ Create real-time streaming pipelines with Confluent Kafka\n✅ Store and analyze streaming data in ClickHouse\n✅ Build interactive dashboards with Snowflake and Streamlit\n✅ Integrate AI/ML using Claude API and Snowflake Cortex AI\n✅ Deploy production-ready data solutions\n\n🏗️ What We'll Build Today\n\nModule 1: Batch Pipeline\n[NSW Transport API] → [Mage Batch Pipeline] → [Apache Iceberg] → [Starburst Analytics]\n\nModule 2: AI-Powered Analytics\n[Iceberg + Streaming Data] → [Snowflake] → [Streamlit App] → [Claude AI Assistant]\n\nModule 3: Real-Time Streaming\n[NSW Transport API] → [Mage Streaming] → [Confluent Kafka] → [ClickHouse] → [Real-time Dashboard]", "key_takeaways": [ "Understand batch vs streaming data processing", "Build batch pipelines with Mage AI and Apache Iceberg", "Query data lakes using Starburst", "Create real-time streaming pipelines with Confluent Kafka", "Store and analyze streaming data in ClickHouse", "Build interactive dashboards with Snowflake and Streamlit", "Integrate AI/ML using Claude API and Snowflake Cortex AI", "Deploy production-ready data solutions" ], "prerequisites": [ "Basic understanding of data engineering concepts", "Familiarity with Python", "Laptop with internet access for cloud services" ], "cities": [ "Melbourne" ], "duration": "210", "level": "Intermediate", "tags": [ "Workshop", "Real-time Analytics", "Mage AI", "Confluent", "ClickHouse", "Snowflake", "Apache Iceberg", "Starburst", "Streamlit", "Claude AI", "Data Pipelines" ], "time_slot": "13:00-16:30", "max_participants": 20, "materials": [ "Laptop with internet access", "Python 3.8+ installed", "Cloud accounts (will be provided for workshop)", "Basic understanding of data engineering concepts" ], "registration_link": "https://forms.gle/YHoaFG2JssmFX1A96" }, "LF0001": { "title": "Leaders Forum - Melbourne", "speakerIds": ["TBA"], "type": "leaders_forum", "description": "Join leaders from around the country as well as our international speakers for an incredible dining experience and discussion.", "abstract": "Join leaders from around the country as well as our international speakers for an incredible dining experience and discussion. This exclusive event provides a unique opportunity to network with industry leaders, share insights, and discuss the future of data engineering in an intimate setting.", "key_takeaways": [ "Network with industry leaders and international speakers", "Engage in meaningful discussions about data engineering trends", "Enjoy an exceptional dining experience in a premium venue", "Build lasting professional relationships" ], "cities": ["Melbourne"], "duration": "240", "level": "Leadership", "tags": ["Leaders Forum", "Networking", "Dining", "Leadership", "Exclusive"], "time_slot": "12:00-16:00", "venue": "Chancery Lane Restaurant", "sponsors": ["ClickHouse", "Snowflake", "Starburst", "Cloud Shuttle"] }, "LF0002": { "title": "Leaders Forum - Sydney", "speakerIds": ["TBA"], "type": "leaders_forum", "description": "Join leaders from around the country as well as our international speakers for an incredible dining experience and discussion.", "abstract": "Join leaders from around the country as well as our international speakers for an incredible dining experience and discussion. This exclusive event provides a unique opportunity to network with industry leaders, share insights, and discuss the future of data engineering in an intimate setting.", "key_takeaways": [ "Network with industry leaders and international speakers", "Engage in meaningful discussions about data engineering trends", "Enjoy an exceptional dining experience in a premium venue", "Build lasting professional relationships" ], "cities": ["Sydney"], "duration": "240", "level": "Leadership", "tags": ["Leaders Forum", "Networking", "Dining", "Leadership", "Exclusive"], "time_slot": "12:00-16:00", "venue": "Ilumina Level 4 Dining Room", "sponsors": ["ClickHouse", "Snowflake", "Starburst", "Cloud Shuttle"] }, "S10004": { "title": "When Systems Break Beautifully: How Atlassian Builds Resilient Platforms", "speakerIds": ["SP0022"], "type": "talk", "description": "What if breaking your systems on purpose made them stronger? At Atlassian, we don't just embrace chaos, we inject it, monitor it, and learn from it (without setting off the fire alarm).", "abstract": "What if breaking your systems on purpose made them stronger? At Atlassian, we don't just embrace chaos, we inject it, monitor it, and learn from it (without setting off the fire alarm). Join me on a behind-the-scenes journey through controlled failure, surprising scenarios, and the weird joy of watching things almost fall apart. Expect stories of resilience engineering, platform evolution, and a few war room tales, sanitized for your enjoyment.\n\nTakeaways:\nYou'll walk away with practical insights into chaos engineering, platform stability strategies, and how to make failure your co-pilot (instead of your nemesis).", "key_takeaways": [ "Learn practical insights into chaos engineering", "Understand platform stability strategies", "Discover how to make failure your co-pilot instead of your nemesis", "Explore resilience engineering and platform evolution", "Gain insights from real-world war room experiences" ], "cities": ["Sydney"], "duration": "30", "level": "Intermediate", "tags": ["Chaos Engineering", "Platform Engineering", "Resilience", "Atlassian", "System Design", "Engineering Management"] }, "S10005": { "title": "Everything we've been doing with data has been a prototype. Where to from here?", "speakerIds": ["SP0037"], "type": "talk", "description": "Modern data engineering has been around for about 25 years and it's created a monster. Siloed and disparate operational data with point solutions on top of point solutions masquerading as a modern data stack.", "abstract": "Modern data engineering has been around for about 25 years and it's created a monster. Siloed and disparate operational data with point solutions on top of point solutions masquerading as a modern data stack. But it has created a valuable end-to-end solution and solves real world problems. Now that the prototype is complete, how do we simplify and move towards \"Data, the Sequel\" (please enjoy the pun). And where does AI fit into the creation and management of data.", "key_takeaways": [ "Understand the evolution of data engineering over 25 years", "Learn about the challenges of siloed and disparate operational data", "Explore strategies for simplifying the modern data stack", "Discover how AI fits into data creation and management", "Gain insights into the future of data engineering" ], "cities": ["Sydney"], "duration": "30", "level": "Intermediate", "tags": ["Data Engineering", "Data Strategy", "AI", "Modern Data Stack", "System Architecture", "Future of Data"] }, "S10006": { "title": "PromptSQL - Using LLMs for Query Optimisation", "speakerIds": ["RS0004"], "type": "talk", "description": "PromptSQL harnesses LLM intelligence the right way. Knowledge graphs analyze entire query pipelines and provide complete context including your schema, indexes, and database-specific best practices. Every optimization is then validated for both speed and cost improvements. Come and learn more.", "abstract": "PromptSQL harnesses LLM intelligence the right way. Knowledge graphs analyze entire query pipelines and provide complete context including your schema, indexes, and database-specific best practices. Every optimization is then validated for both speed and cost improvements. Come and learn more.", "cities": ["Melbourne"], "duration": "30", "level": "Intermediate", "tags": ["GenAI", "LLM", "SQL", "Query Optimization"] }, "S10007": { "title": "Autonomy in the Agentic Era", "speakerIds": ["RS0005"], "type": "talk", "description": "2025 has been branded 'the year of AI Agents', but what are they really? In this talk, I'll be unpacking the hype: diving into what makes agents unprecedented AI Systems, and discussing some of the governance challenges and opportunities that organisations will encounter in this new era.", "abstract": "2025 has been branded 'the year of AI Agents', but what are they really? In this talk, I'll be unpacking the hype: diving into what makes agents unprecedented AI Systems, and discussing some of the governance challenges and opportunities that organisations will encounter in this new era.", "cities": ["Sydney"], "duration": "30", "level": "Intermediate", "tags": ["GenAI", "Governance", "AI", "Agentic"] }, "S10008": { "title": "The future belongs to people who can just do things", "speakerIds": ["SP0024"], "type": "talk", "description": "Software development has enjoyed a remarkably consistent foundation for over 40 years—but that era is ending. Just as we once leapt from assembly to high-level languages, we're now entering a new phase: programming with large language models. This shift isn't incremental; it's a new layer of abstraction.", "abstract": "In this talk, Geoff Huntley explores why now is the moment to reassess your carefully crafted roadmaps, identify which assumptions no longer hold, and prepare yourself for a different kind of work. We'll talk about accelerating your \"oh-fuck\" moment—the realization that things are changing faster than expected—and how to navigate the emotional and professional turbulence that follows.", "key_takeaways": [ "Understand why the software development foundation is shifting after 40 years", "Learn how to reassess roadmaps in the age of large language models", "Identify which traditional assumptions no longer hold true", "Prepare for the transition to AI-assisted development", "Navigate the emotional and professional challenges of rapid technological change" ], "cities": ["Sydney", "Melbourne"], "duration": "40", "level": "Intermediate", "tags": ["Software Development", "AI", "Leadership", "Future of Tech", "Programming", "Technology Evolution"] }, "S10009": { "title": "Building Code-Editing Agents: The Emperor Has No Clothes", "speakerIds": ["SP0024"], "type": "talk", "description": "There's something unsettling about watching an AI agent edit the very code that defines it. Something that makes you question where the human ends and the machine begins.", "abstract": "We'll explore this strange loop by building it ourselves. 400 lines of Go. Three tools. One moment of recognition when you realize the thing you built is now building alongside you.\n\nThis is about more than productivity. It's about reimagining what programming feels like.", "key_takeaways": [ "Understand the concept of self-modifying AI agents", "Learn how to build code-editing agents in Go", "Explore the philosophical implications of AI agents editing their own code", "Experience the moment when AI becomes a true programming partner", "Reimagine the future of programming with AI collaboration" ], "cities": ["Sydney", "Melbourne"], "duration": "30", "level": "Intermediate", "tags": ["GenAI", "AI", "Programming", "Go", "Code Editing", "AI Agents", "Software Development"] }, "S10010": { "title": "Hunting PI in the Data Haystack: A Streamlit in Snowflake Detective Story", "speakerIds": ["SP0025", "SP0026"], "type": "talk", "description": "Imagine managing a cloud data warehouse serving 51 domains, with data pouring in from multiple source systems and data contracts in place for most raw data. Despite these safeguards, you suspect that sensitive personally identifiable information (PI) may still be lurking—unmasked and unnoticed—in your vast data landscape. How do you find these needles in the haystack before they become a compliance nightmare?", "abstract": "Join us as we share our journey building a self-service detective tool using Streamlit and Snowflake. We'll detail how we empowered teams to proactively cross-check potential PI in their domains, tag confirmed cases, and flag false positives for future scans—all while fostering data governance and security across a complex, multi-domain environment.\n\nThis talk is for anyone interested in data privacy, compliance, or building intuitive tools that bridge the gap between technical and non-technical teams. You'll leave with practical insights into Streamlit app development, Snowflake integrations, and strategies for scaling data privacy initiatives in the cloud.\n\nOutline:\n- The Challenge: Managing 51 domains, multiple data sources, and the risk of unmasked PI.\n- The Investigation: How we suspected PI might be slipping through despite data contracts.\n- The Solution: Scanning the data and building a self-service Streamlit app in Snowflake for domain teams to check and action potential PI.\n- The Impact: Empowering teams, improving data governance, and scaling privacy efforts.\n- Lessons Learned: Tips for building similar tools and fostering a culture of data responsibility.", "key_takeaways": [ "Learn how to build self-service tools for data privacy detection", "Understand Streamlit app development and Snowflake integrations", "Discover strategies for scaling data privacy initiatives in the cloud", "Explore techniques for managing PI across multiple domains", "Gain insights into fostering a culture of data responsibility" ], "cities": ["Melbourne"], "duration": "30", "level": "Intermediate", "tags": ["Data Governance", "Data Privacy", "Streamlit", "Snowflake", "Compliance", "PI Detection", "Data Security"] }, "S10011": { "title": "Snowflake Cortex AISQL - What, Why and How", "speakerIds": ["SP0027"], "type": "talk", "description": "Snowflake has announced the PuPr of Snowflake Cortex AISQL feature during the Snowflake Summit in early June at San Francisco, which bringing powerful AI capabilities directly into Snowflake's SQL engine. It enables users to build scalable AI pipelines across multimodal enterprise data with familiar SQL commands.", "abstract": "In this talk, I will give you high level overview of what this feature is about, why we developed such a feature, and also how to make use of the feature under what scenarios with a quick demo.", "key_takeaways": [ "Understand what Snowflake Cortex AISQL is and its capabilities", "Learn why Snowflake developed this AI feature", "Discover how to use AISQL in different scenarios", "See a live demo of the feature in action", "Explore building scalable AI pipelines with familiar SQL commands" ], "cities": ["Melbourne"], "duration": "30", "level": "Advanced", "tags": ["Snowflake", "AI", "SQL", "Cortex", "Data Engineering", "AI Pipelines", "Multimodal Data"] }, "S10012": { "title": "Data Platform Journey of Australia's leading B2B eCommerce platform: from on-prem to Modern Data Stack", "speakerIds": ["SP0028"], "type": "talk", "description": "We would like to present the transformation journey of New Aim, Australia's premier B2B e-commerce platform. As a supplier to major marketplaces such as Amazon, eBay, Catch, Myer, and Woolworths, New Aim reaches over 60% of Australian households. This talk will detail our shift from an on-prem SQL Server to a cutting-edge modern data stack, incorporating technologies such as Argo, and Kubernetes, which have significantly enhanced our scalability and operational efficiency.", "abstract": "Key Highlights:\n\n- The journey of different challenges we faced and the trade-offs we made to achieve where we are today.\n- Technical choices and in-house built tools that made us vendor-agnostic as a small team.\n- Our approach to building a low-maintenance, high-performance infrastructure through unifying DataOps and MLOps.\n\nOur story can definitely inspire other data teams who are also on the journey of transforming their data platforms.", "key_takeaways": [ "Learn about the transformation journey from on-prem SQL Server to modern data stack", "Understand the challenges and trade-offs in data platform modernization", "Discover how to build vendor-agnostic tools as a small team", "Explore approaches to unifying DataOps and MLOps", "Gain insights into building low-maintenance, high-performance infrastructure" ], "cities": ["Melbourne"], "duration": "30", "level": "Intermediate", "tags": ["Data Platform", "E-commerce", "Cloud Architecture", "Kubernetes", "Argo", "DataOps", "MLOps", "Modern Data Stack"] }, "S10013": { "title": "How Metadata-enabled Agentic AI Accelerates Enterprise Value", "speakerIds": ["SP0029"], "type": "talk", "description": "Adoption and diffusion of enterprise agentic AI systems that reliably generate business value, will be inextricably linked with the ability to provide contextually relevant governed data and metadata to those systems.", "abstract": "This session will demonstrate how the Ab Initio Data Platform's unique approach to context and governed metadata, combined with state of art AI capabilities including MCP, agentic planning and multi-agent workflows, accelerates the application of trusted, traceable, and well governed AI capabilities to high value use cases for the data driven enterprise.", "key_takeaways": [ "Understand how metadata-enabled agentic AI accelerates enterprise value", "Learn about Ab Initio Data Platform's approach to context and governed metadata", "Explore state-of-the-art AI capabilities including MCP and agentic planning", "Discover multi-agent workflows for enterprise AI applications", "Gain insights into trusted, traceable, and well-governed AI capabilities" ], "cities": ["Sydney", "Melbourne"], "duration": "30", "level": "Intermediate", "tags": ["GenAI", "AI", "Metadata", "Agentic AI", "Enterprise", "Data Platform", "MCP", "Multi-agent Workflows"] }, "S10014": { "title": "Automate Your Metadata, Deliver a Data Catalogue (by accident)", "speakerIds": ["SP0030"], "type": "talk", "description": "Have you ever caught yourself saying this?\n\"How was that calculated again?\"\n\"I'll document that later…\"\n\"I swear we built this metric... What's it called again?\"\n\nIf you're nodding.. you're not alone. Many data teams wrestle with fractured documentation, last-minute governance, and the endless game of chasing down definitions.", "abstract": "This talk is about breaking that cycle by investing upfront in automation for your configuration and extra benefits you could reap along the way.\n\nWe'll explore how building your metadata and data configs as code early on doesn't just save time; it powers your data governance portal, feeds your BI tools, and keeps definitions consistent end to end. It's not perfect, but it's honest progress and a big step up from the duct-taped reality most teams are working with.\n\nWe will talk about the practical patterns and tools we have built as a team at Pet Circle that could help your engineers and business users finally speak the same language; by using a living, breathing Data Catalogue that becomes an essential part of your data ecosystem.", "key_takeaways": [ "Learn how to automate metadata and data configuration as code", "Understand how metadata automation powers data governance portals", "Discover practical patterns for building living data catalogues", "Explore tools that help engineers and business users speak the same language", "Gain insights into breaking the cycle of fractured documentation" ], "cities": ["Melbourne"], "duration": "30", "level": "Intermediate", "tags": ["Data Engineering", "Metadata", "Data Catalogue", "Data Governance", "Automation", "Documentation", "BI Tools"] }, "S10015": { "title": "Deploying Data Models with Confidence: A CI/CD Approach in Data Engineering", "speakerIds": ["SP0033"], "type": "talk", "description": "We often rave about the latest data engineering tools - modern warehouses, transformation frameworks, and orchestration platforms but we rarely talk about how these models get deployed with confidence. CI/CD (Continuous Integration & Continuous Deployment) is often seen as a software development practice, but in reality, it plays a game-changing role in data engineering. Without CI/CD, data teams face manual deployments, broken pipelines, and inconsistent data models, leading to unreliable insights.", "abstract": "In this session, we'll explore how CI/CD can revolutionize data model deployment by bringing automation, version control, and testing into the process. Using SQLMesh, Snowflake, Apache Iceberg, and GitHub Actions, we'll walk through building a robust, automated data workflow from development to production. From ensuring transformations are validated early to managing code and data versions seamlessly, this talk provides actionable guidance for building reliable pipelines at scale.\n\nBy the end of this session, you'll learn how to:\n- Integrate CI/CD pipelines into data workflows using GitHub Actions for automated deployments.\n- Set up validation and testing for SQLMesh models to catch issues before they reach production.\n- Use Snowflake and Apache Iceberg to manage scalable, versioned datasets with confidence.\n- Improve pipeline reliability by eliminating manual steps and promoting consistency across environments.\n\nWhether you're a data engineer, analytics engineer, or platform builder, this talk will show why CI/CD is no longer optional but essential for modern, trustworthy data workflows.", "key_takeaways": [ "Integrate CI/CD pipelines into data workflows using GitHub Actions for automated deployments", "Set up validation and testing for SQLMesh models to catch issues before they reach production", "Use Snowflake and Apache Iceberg to manage scalable, versioned datasets with confidence", "Improve pipeline reliability by eliminating manual steps and promoting consistency across environments" ], "cities": ["Sydney"], "duration": "30", "level": "Intermediate", "tags": ["Data Engineering", "CI/CD", "SQLMesh", "Snowflake", "Apache Iceberg", "GitHub Actions", "DevOps", "Automation", "Testing"] }, "S10016": { "title": "Survival Tips for Data Engineers in the Age of GenAI", "speakerIds": ["TS0003", "TS0005", "TS0006"], "type": "panel", "description": "As Generative AI continues to reshape the data engineering landscape, data engineers face new challenges and opportunities. This panel brings together industry leaders Juan Sequeda (Principal Scientist & Head of AI Lab at data.world), Olena Kutsenko (Staff Developer Advocate at Confluent), and Lena Hall (CEO at Droid AI) to share practical survival strategies for data engineers navigating the GenAI revolution.", "abstract": "Join us for an engaging panel discussion where three industry experts share their insights on how data engineers can thrive in the age of Generative AI. From adapting skill sets and embracing new tools to maintaining relevance and driving innovation, this panel will provide actionable advice for data engineers at all levels. Topics will include the evolving role of data engineering, essential skills for the future, and how to leverage GenAI to enhance rather than replace human expertise.", "key_takeaways": [ "Understand how GenAI is reshaping the data engineering profession", "Learn essential skills and competencies for the GenAI era", "Discover strategies for adapting and thriving in the changing landscape", "Explore how to leverage GenAI tools to enhance productivity and innovation", "Gain insights into maintaining career relevance and competitive advantage" ], "cities": ["Sydney", "Melbourne"], "duration": "40", "level": "All Levels", "tags": ["GenAI", "AI", "Data Engineering", "Panel", "Career Development", "Future of Tech", "Leadership"] }, "S10017": { "title": "The Future of GenAI: Where We're Heading and What to Focus On", "speakerIds": ["TS0003", "TS0005", "TS0006"], "type": "panel", "description": "Join three industry leaders as they share their insights on the trajectory of Generative AI and where data engineers should focus their efforts. Juan Sequeda (Principal Scientist & Head of AI Lab at data.world), Olena Kutsenko (Staff Developer Advocate at Confluent), and Lena Hall (CEO at Droid AI) will provide a comprehensive view of emerging trends, breakthrough technologies, and strategic priorities for the next phase of AI development.", "abstract": "As Generative AI continues to evolve at breakneck speed, it's crucial to understand where the technology is heading and how to position ourselves for success. This session brings together three experts with deep insights into AI research, practical applications, and strategic implementation. They'll discuss emerging trends in AI development, breakthrough technologies on the horizon, and provide actionable guidance on where data engineers should focus their learning and development efforts. From understanding the next generation of AI models to identifying high-impact use cases and avoiding common pitfalls, this talk will help you navigate the rapidly evolving AI landscape with confidence.", "key_takeaways": [ "Understand emerging trends and breakthrough technologies in GenAI", "Identify high-impact areas where data engineers should focus their efforts", "Learn about the next generation of AI models and their implications", "Discover strategic priorities for AI development and implementation", "Gain insights into avoiding common pitfalls in AI adoption" ], "cities": ["Sydney", "Melbourne"], "duration": "40", "level": "All Levels", "tags": ["GenAI", "AI", "Future of Tech", "Data Engineering", "Strategy", "Innovation", "Technology Trends"] }, "S10018": { "title": "How AI Agents Are Reshaping the Future of Work", "speakerIds": ["SP0034"], "type": "talk", "description": "We stand at the brink of a seismic shift in how work gets done. In this talk, we explore the rise of AI agents—autonomous, intelligent collaborators that don't just respond to prompts, but plan, reason, and act. These agents are transforming industries by automating entire workflows, learning from experience, and working together in multiagent systems.", "abstract": "From personalized financial advice to real-time customer service, the future of work is being rewritten—faster, smarter, and more human-centric. Join us as we unpack how organizations can harness these agents to unlock productivity, reimagine business models, and lead boldly in the AI-powered era. We'll explore the capabilities of modern AI agents, their impact on various industries, and practical strategies for implementing agent-based solutions in your organization.", "key_takeaways": [ "Understand the capabilities and limitations of modern AI agents", "Learn how AI agents are transforming different industries", "Discover practical strategies for implementing agent-based solutions", "Explore the future of human-AI collaboration in the workplace", "Gain insights into reimagining business models with AI agents" ], "cities": ["Sydney", "Melbourne"], "duration": "30", "level": "Intermediate", "tags": ["GenAI", "AI Agents", "Future of Work", "Automation", "Business Transformation", "Productivity", "Innovation"] }, "S10019": { "title": "Agents, Assemble! Unleashing Agentic RAGs with Knowledge Graphs", "speakerIds": ["RS0003"], "type": "talk", "description": "Step into the future of Retrieval-Augmented Generation where autonomous agents don't just fetch facts — they reason with them. In this talk, we'll explore how Agentic RAGs, supercharged with Knowledge Graphs, create intelligent multi-hop workflows, unlock contextual memory, and dynamically orchestrate information across domains.", "abstract": "If you're building the next generation of LLM-powered systems, this is your blueprint for intelligent agents that think before they speak. We'll dive deep into how Knowledge Graphs enhance RAG systems by providing structured context, enabling multi-hop reasoning, and creating more intelligent retrieval mechanisms. From understanding the fundamentals of agentic RAGs to implementing practical solutions with Neo4j, this session will equip you with the knowledge to build more sophisticated AI systems.", "key_takeaways": [ "Understand how Knowledge Graphs enhance RAG systems with structured context", "Learn to implement multi-hop reasoning in agentic RAG workflows", "Discover techniques for creating intelligent retrieval mechanisms", "Explore practical implementation strategies with Neo4j", "Gain insights into building more sophisticated AI systems" ], "cities": ["Melbourne"], "duration": "30", "level": "Intermediate", "tags": ["GenAI", "RAG", "Knowledge Graphs", "Neo4j", "AI Agents", "LLM", "Multi-hop Reasoning", "Contextual Memory"] }, "S10020": { "title": "Building Enterprise Recommendation Systems with Neo4j: From Data to Personalised Insights", "speakerIds": ["SP0036"], "type": "talk", "description": "Samko Yun, Senior Solutions Engineer at Neo4j, brings extensive consulting experience across finance, retail, education, and other industries to explore how graph databases can power sophisticated recommendation engines. Through practical case studies and real-world examples, Samko will demonstrate how to leverage Neo4j's native graph capabilities to capture complex relationships between users, products, and content that traditional databases struggle to represent.", "abstract": "In this session, we'll explore how Neo4j's native graph database capabilities can revolutionize recommendation systems by capturing complex relationships that traditional databases struggle to represent. Through practical case studies from finance, retail, and education industries, we'll demonstrate how to build sophisticated recommendation engines that go beyond simple collaborative filtering to deliver truly personalized insights.", "key_takeaways": [ "Understand how graph databases capture complex relationships for recommendation systems", "Learn to leverage Neo4j's native graph capabilities for personalized insights", "Explore practical case studies from finance, retail, and education industries", "Discover techniques for building sophisticated recommendation engines", "Gain insights into moving beyond simple collaborative filtering" ], "cities": ["Sydney"], "duration": "30", "level": "Intermediate", "tags": ["Neo4j", "Recommendation Systems", "Graph Databases", "Personalization", "Enterprise", "Data Engineering", "AI"] }, "S10021": { "title": "RAG or Fine-Tune? Customizing LLMs with LoRA and MCP", "speakerIds": [ "SP0032" ], "type": "talk", "description": "When should you retrieve, and when should you fine-tune? This talk demystifies when to use RAG vs fine-tuning for GenAI applications. We'll then go hands-on with LoRA and MCP to fine-tune a model live showing how quickly you can build domain-specific LLMs on your own terms.", "abstract": "Join us for an in-depth exploration of the critical decision between RAG (Retrieval-Augmented Generation) and fine-tuning for customizing Large Language Models. This session will provide clear guidance on when to use each approach, along with practical demonstrations using LoRA (Low-Rank Adaptation) and MCP (Model Context Protocol) to fine-tune models efficiently.", "key_takeaways": [ "Understand when to use RAG vs fine-tuning for different use cases", "Learn practical implementation of LoRA for efficient model fine-tuning", "Explore MCP for enhanced model customization and control", "See live demonstrations of domain-specific LLM development", "Gain insights into building custom LLMs on your own terms" ], "cities": [ "Sydney" ], "duration": "30", "level": "Intermediate", "tags": [ "GenAI", "LLM", "RAG", "Fine-tuning", "LoRA", "MCP", "Machine Learning", "AI" ] }, "S10022": { "title": "Why is Open Source So Important in Data Engineering?", "speakerIds": [ "TS0005", "TS0006", "TS0003" ], "type": "panel", "description": "A panel discussion exploring the critical role of open source in data engineering, featuring Olena Kutsenko (Staff Developer Advocate at Confluent), Lena Hall (CEO at Droid AI), and Juan Sequeda (Principal Scientist & Head of AI Lab at data.world).", "abstract": "Join industry leaders as they discuss why open source is fundamental to the data engineering ecosystem. This panel will explore how open source drives innovation, enables collaboration, and shapes the future of data infrastructure. From community-driven development to enterprise adoption, our experts will share insights on the importance of open source in building scalable, reliable, and accessible data solutions.", "key_takeaways": [ "Understand the fundamental role of open source in data engineering innovation", "Learn how open source enables collaboration and knowledge sharing", "Discover strategies for enterprise adoption of open source solutions", "Explore the future of open source in data engineering", "Gain insights into contributing to and benefiting from open source communities" ], "cities": [ "Sydney" ], "duration": "40", "level": "All Levels", "tags": [ "Open Source", "Data Engineering", "Panel", "Community", "Innovation", "Collaboration", "Enterprise" ] }, "S10023": { "title": "Beyond Keywords: Building Intelligent Search with LLMs and Elasticsearch", "speakerIds": [ "SP0038" ], "type": "talk", "description": "Traditional search approaches struggle when users express complex, multi-faceted queries in natural language. This talk demonstrates how to bridge the gap between human intent and structured search by combining Large Language Model (LLM) functions with Elasticsearch search fundamentals.", "abstract": "In this session, you'll learn how to use LLMs to parse natural language queries and extract structured parameters, integrate external services (like geocoding) into your search pipeline through LLM function calling, leverage Elasticsearch search templates to create dynamic, parameterised queries, and build a search system that understands complex constraints without sacrificing performance.\n\nWe'll walk through a real-world hotel search implementation that showcases how this approach delivers superior precision and recall compared to traditional keyword or vector-only methods. You'll see how the LLM reasoning layer orchestrates multiple function calls - from parameter extraction to geocoding to query execution - resulting in search results that feel almost unparalleled in their accuracy.\n\nWhether you're building e-commerce search, enterprise knowledge retrieval, or location-based services, this talk will equip you with practical patterns for creating search experiences that truly understand what your users are looking for.", "key_takeaways": [ "How to combine the semantic understanding of LLMs with the structured filtering power of Elasticsearch", "Practical implementation patterns using Azure OpenAI and Elasticsearch", "Strategies for handling complex, multi-constraint queries at scale", "Code examples and notebooks you can use to implement these techniques in your own projects" ], "cities": [ "Sydney" ], "duration": "30", "level": "Intermediate", "tags": [ "GenAI", "LLM", "Elasticsearch", "Search", "Natural Language Processing", "Azure OpenAI", "Function Calling", "Search Templates" ] }, "S10024": { "title": "Beyond the Buzzwords: Building a Practical AI Playground", "speakerIds": [ "SP0023" ], "type": "talk", "description": "I've been on a bit of a rollercoaster with AI, watching the Gen AI boom from the sidelines a few years back. While everyone was buzzing, I admit, I wasn't exactly rushing to dive in and get my hands dirty. I dipped a toe in with Vertex AI at work, but the standalone nature of it, without robust external connections, felt a bit like having a fancy car with nowhere to drive.", "abstract": "Then, late last year, something truly groundbreaking happened: Model Context Protocol (MCP) burst onto the scene. It was a game-changer, quickly becoming the darling of Product Engineers worldwide. The excitement was palpable, with MCP servers popping up everywhere and promises of a new era.\n\nBut let's be honest, where there's boom, there's often a bit of hype, and I found myself wondering: \"Is this the real deal, or just another shiny object?\" That's when I decided it was time to stop spectating and start experimenting. I wanted to cut through the noise, understand the true capabilities of the current tech, and separate the genuine breakthroughs from the marketing fluff.\n\nThis curiosity sparked a personal quest. What started as a humble side project quickly transformed into something much bigger: an AI and MCP Playground. Imagine a space where our entire company can now kick the tires on AI concepts, integrate with existing systems, and build prototypes with surprising ease. It's become the go-to spot to explore what's genuinely possible with today's tools, and just as importantly, to understand their current limitations.\n\nJoin me as I pull back the curtain on this playground, share some of its cool features, and reveal how you can build your very own testing ground to supercharge your product development and run your own fascinating AI experiments. Let's explore the practical side of AI, together!", "key_takeaways": [ "Understand how to build a practical AI playground for experimentation", "Learn about Model Context Protocol (MCP) and its real-world applications", "Discover strategies for cutting through AI hype and focusing on practical implementation", "Explore how to integrate AI concepts with existing systems", "Gain insights into rapid prototyping and fearless experimentation with AI" ], "cities": [ "Melbourne" ], "duration": "30", "level": "Intermediate", "tags": [ "GenAI", "AI", "MCP", "Model Context Protocol", "Practical Implementation", "Experimentation", "Rapid Prototyping", "Product Development" ] } }