Data Lake Cost and Storage Growth Report

Overview

This flow gives data and finance teams ongoing visibility into Azure Data Lake storage cost and growth. On a weekly schedule it measures the size and file count of a Data Lake zone, converts the total to GB, estimates monthly cost, compares against the previous snapshot to compute week-over-week growth, writes the result to a Dataverse trend table, refreshes a Power BI dataset, and posts either a routine scorecard or an anomaly alert to a Teams channel. Why it matters: lake storage grows quietly and cost accumulates without anyone watching. A weekly snapshot plus an automatic anomaly flag keeps spend visible and catches abnormal growth early. The flow ships Off (demo); going live requires only connection authorization and environment-variable configuration - no logic changes.

Use Case

Data platform and finance teams want to track and forecast Azure Data Lake storage cost without standing up a separate monitoring stack. This flow turns a folder listing into a trend table, a refreshed dashboard, and a Teams scorecard, with a threshold-based alert when a zone grows abnormally week-over-week.

Flow Architecture

Environment Variables

Connectors & Connections

Note — All connections are referenced as solution connection references; the flow is portable between environments as long as a connection is mapped at import time.

Customization Guide

Almost every realistic variant of this flow can be implemented by changing environment variable values. A few cases require small edits inside the flow definition — those are called out explicitly below.

Multiple zones

Wrap the measure/snapshot block in an outer loop over a comma-separated list of zones (split flowlibs_LakeScanPath), writing one snapshot row per zone.

Chargeback

Group file sizes by top-level subfolder and write a row per team/folder to attribute cost.

Forecast

Read the last N snapshots from Dataverse and project next-quarter spend using the average weekly growth rate.

Cleanup tie-in

Flag folders with the largest growth as tiering/archival candidates and route them to a Box/Blob retention flow.

Cost accuracy

For large accounts, swap the per-file enumeration for Azure Storage metrics/inventory and estimate by access tier.

Key Expressions

The flow is intentionally light on Power Fx / WDL gymnastics — the heaviest expressions are the branch-name concatenation and the approval outcome check. They are listed below in the order they appear in the flow.

@div(float(variables('varTotalBytes')), 1073741824)

@if(greater(float(outputs('Compose_PrevGB')),0), mul(div(sub(float(outputs('Compose_TotalGB')),float(outputs('Compose_PrevGB'))),float(outputs('Compose_PrevGB'))),100), 0)

@mul(float(outputs('Compose_TotalGB')), float(variables('varPricePerGb')))

@concat('flowlibs_zone eq ''', variables('varScanPath'), '''')

@equals(items('Apply_to_each_File')?['type'], 'FILE')