Stage takeaways:
• Enforcing data contracts to prevent silent schema breakage
• Ingesting, cleaning, and structuring massive unstructured datasets
• Modernizing workflow orchestration with agentic data engineering
• Enforcing pipeline observability while keeping operational costs under control
Constructing High-Throughput, AI-Ready Data Pipelines and Resilient Platforms
According to Informatica’s 2026 CDO benchmark, 57% of data leaders cite pipeline and data reliability as the single primary blocker preventing AI projects from advancing from pilot to production. Simply put, cutting-edge foundation models cannot fix broken, stale, or untrusted data pipelines. When the Data Innovation Summit 2027 returns to Stockholm’s Kistamässan on 18–19 May 2027, Stage M2: Data Engineering & DataOps zeroes in on building the industrial-grade plumbing required for modern enterprise intelligence. As part of the Nordics’ flagship 15-stage conference, this track brings together platform architects, data engineers, and DataOps practitioners to share real-world implementations of streaming engines, shift-left data contracts, open table formats, and automated pipeline healing and delivering the trusted pipelines needed to close the gap between data collection and bottom-line enterprise impact.
Why Production AI Starts with Resilient, Shift-Left DataOps
Without modern, clean pipelines, downstream AI initiatives stall. Stage M2: Data Engineering & DataOps addresses the engineering backbone required to sustain high-velocity analytics and foundation models. Sessions explore the intersection of streaming ingestion, unstructured data handling, and automated DataOps. Attendees will examine how data contracts prevent breaking downstream schemas, how to orchestrate hybrid pipelines, and how to maintain strict observability without runaway operational costs.
- Format: 20-minute technical sessions.
- Core Topics: AI-ready pipelines, streaming and real-time ingestion, agentic data engineering, unstructured data transformation, data contracts, workflow orchestration, pipeline automation, open table formats, pipeline observability, cost-efficient DataOps.
- Target Audience: Head of Data Engineering, Lead/Senior Data Engineer, Data Engineering Team Lead, DataOps Lead, Analytics Engineer, Streaming/Real-time Data Engineer, Head of ML Engineering.
By connecting DataOps discipline with agentic automation and real-time streaming, Stage M2 redefines the baseline for pipeline reliability. Engineers and platform teams will walk away with concrete strategies to eliminate silent data corruption, enforce strict schema contracts, and build cost-conscious pipelines that fuel high-performance AI initiatives.
Designed for Heads of Data Engineering, Senior Data Engineers, DataOps Leads, Analytics Engineers, and Streaming Specialists.