United Airlines - Done

Q1: Explain how Azure Event Hubs is used in your project. Share me the e2e picture from messages getting garneted, consumed by Event Hub and further passed to different downstream application. Explain this with simple example. You can think of let's say parcel delivery system. 

Q2: Scenario: There is an Azure Event Hub, suppose messages are flowing in with 30,000 messages per Second. We need to process these messages, store them. There will be some Apis who will be fetching these messages. The read requirement is like 50,000 messages per Second. How will you design such system, what are the assumptions you will make while designing this system? These 50K messages are streaming in real-time, you have to consume these many messages in real-time. 

What are different types of failures can happen in proposed architecture and how you are going to handle them?

-------------------------------------------------------------------------------------------------------------------------------------------

Answer 1:

 ┌─────────────────────┐

 │   IoT Devices                                │

 │ (Sensors, Machines)                   │

 └─────────┬───────────┘

           │  Telemetry

           │  (MQTT)

           ▼

 ┌─────────────────────┐

 │   Azure IoT Hub     │

 │  (Device Mgmt +    │

 │   Secure Ingestion)│

 └─────────┬───────────┘

           │  Event Routing

           │  (Built-in)

           ▼

 ┌──────────────────────────────────────────┐

 │      Azure Event Hub Namespace            │

 │                                          │

 │  ┌──────────────┐   ┌──────────────┐     │

 │  │  Event Hub   │   │  Event Hub   │     │

 │  │     EH1      │   │     EH2      │     │

 │  └──────┬───────┘   └──────┬───────┘     │

 │         │                   │             │

 │  ┌──────▼───────────────────▼───────┐     │

 │  │           Partitions              │     │

 │  │  P1  P2  P3  ...  P32              │     │

 │  │ (Ordered, Parallel Event Logs)     │     │

 │  └───────────────────────────────────┘     │

 │                                          │

 │  Consumer Groups (Independent Views)     │

 │  ┌───────────┐  ┌───────────┐           │

 │  │ CG-RT     │  │ CG-ANALYT │           │

 │  │ (Realtime)│  │ (Batch)   │           │

 │  └─────┬─────┘  └─────┬─────┘           │

 └────────┼──────────────┼─────────────────┘

          │              │

          ▼              ▼

 ┌─────────────────┐   ┌───────────────────┐

 │ Azure Functions │   │ Stream Analytics / │

 │ (EventHub       │   │ Web Apps / Custom  │

 │  Trigger)       │   │ Consumers          │

 └─────────────────┘   └───────────────────┘