Vivekkumar Muthukrishnan is a Senior Data Engineer at Skyscanner, with extensive experience in data engineering and analytics across various industries. He has led teams and implemented innovative data solutions for both startups and established organizations, particularly in eCommerce, network security, marketing, and travel. At Shopify, he significantly improved data processing efficiency, contributing to increased merchant retention. Vivekkumar is also active in education and community engagement, writing articles for media platforms, mentoring aspiring technologists, and running a YouTube channel on programming and software development. He seeks to further contribute to the tech community through initiatives like the Hackathon Raptor Fellowship.

Introduction to Event-Driven Streaming and Apache Kafka

Overview of event-driven streaming and its significance in modern data engineering
Introduction to Apache Kafka and its architecture
Hands-on exercise: Setting up a Kafka cluster and producing/consuming messages
Discussion: Use cases, best practices, and challenges of Kafka in data engineering

Session 1

Data Processing with
Apache Flink

Overview of data processing frameworks and their applications
Introduction to Apache Flink and its real-time processing capabilities
Hands-on exercise: Implementing a simple Flink application for stream data processing
Discussion: State management, event-time processing, and advanced Flink features
Case study: Optimizing real-world Flink applications

Session 2

Advanced Data Modeling
and Cloud Optimization

Overview of advanced data modeling for big data systems
Introduction to NoSQL databases like Cassandra and HBase for cloud-based systems
Hands-on exercise: Designing an efficient data model for a cloud-based application
Discussion: Strategies for cloud cost optimization, including partitioning, indexing, and experimentation

Session 3