Event Grid Blob-Created to Ingestion Pipeline

Overview

This flow delivers event-driven, low-latency file ingestion using Azure Event Grid. It subscribes to Microsoft.Storage.BlobCreated events on a source storage account, validates that each new blob is in the monitored container scope, reads the blob content from Azure Blob Storage, routes the file to the correct processing pipeline by extension, and writes a full audit row to a Dataverse ingestion-log table - every row stamped with a correlation id for end-to-end traceability.

Why it matters: polling storage is laggy and wasteful. Event Grid pushes each blob event the moment it lands, so ingestion is responsive and scales without timer churn.

Ships Off (demo).

Use Case

Files arrive in Blob Storage from external systems, partners, or upstream jobs and must be processed the instant they land - not on the next polling cycle. This flow is the entry point of an event-driven ingestion pipeline: it captures the event, fetches the file, decides which downstream pipeline (OCR, parse, image, archive) should handle it, and records an auditable log entry for every file (including ones skipped because they fall outside the monitored container).

Flow Architecture

Environment Variables

Connectors & Connections

Note — All connections are referenced as solution connection references; the flow is portable between environments as long as a connection is mapped at import time.

Customization Guide

Almost every realistic variant of this flow can be implemented by changing environment variable values. A few cases require small edits inside the flow definition — those are called out explicitly below.

Subject filters

Change the blob subject prefix to scope to a different container/prefix, or relax it to ingest all containers.

Routing

Edit the type route map to add extensions or pipeline names; add matching Switch cases for new routes.

Real processing

Replace each Switch case's Compose note with the actual downstream action (Document Intelligence/OCR, parse-and-load, image analysis, archive copy).

Dead-letter / large files

Route failures to a queue and hand very large files to Azure Functions or Data Factory.

Idempotency

For production, de-duplicate on blob path + eTag before processing to guard against repeated deliveries.

Key Expressions

The flow is intentionally light on Power Fx / WDL gymnastics — the heaviest expressions are the branch-name concatenation and the approval outcome check. They are listed below in the order they appear in the flow.

@coalesce(json(variables('varRouteMap'))?[outputs('Compose_FileExtension')], json(variables('varRouteMap'))?['default'])

@concat('/', replace(split(outputs('Compose_BlobSubject'), '/containers/')[1], '/blobs/', '/'))

startsWith(outputs('Compose_BlobSubject'), variables('varSubjectPrefix'))