Deploy OneLake Infrastructure

This guide deploys the Microsoft Fabric and OneLake infrastructure required to receive Parquet files from the Haltian IoT Data API. After completing this guide, proceed to Deploy Function App to set up the automated data transfer.

Download Source Files

Download all Terraform files for this module:

Place all files in a directory (e.g., infra/onelake/) before running Terraform.

What Gets Created

The infra/onelake Terraform module provisions:

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#F6FAFA', 'primaryTextColor': '#143633', 'primaryBorderColor': '#143633', 'lineColor': '#143633', 'secondaryColor': '#C7FDE6', 'tertiaryColor': '#73F9C1', 'clusterBkg': '#ffffff', 'clusterBorder': '#143633', 'edgeLabelBackground': '#ffffff'}}}%%
flowchart TB
    subgraph Azure["Azure Subscription"]
        RG["fa:fa-folder Resource Group"]
        FabricCap["fa:fa-server Fabric Capacity<br/>F2, F4, F8, etc."]
    end

    subgraph EntraID["Microsoft Entra ID"]
        AADApp["fa:fa-id-card Azure AD Application"]
        SP["Service Principal + Secret"]
    end

    subgraph Fabric["Microsoft Fabric"]
        Workspace["fa:fa-layer-group Fabric Workspace"]
        Lakehouse["fa:fa-database Lakehouse"]
        Endpoint["OneLake DFS Endpoint"]
    end

    RG --> FabricCap
    FabricCap --> Workspace
    Workspace --> Lakehouse
    Lakehouse --> Endpoint
    AADApp --> SP
    SP -.->|"Workspace access"| Workspace

Prerequisites

• Terraform ≥ 1.5.0
• Azure CLI authenticated (az login)
• Application Administrator Azure AD role (for app registrations)
• Global Administrator (optional — required for directory roles and Graph API consent)
• Fabric Administrator (optional — required if creating new Fabric capacity)
• Microsoft Fabric license (F2 minimum for paid SKUs, or free 60-day trial)

Deployment with Terraform

Step 1: Gather Azure Information

# Get your subscription ID
az account show --query id -o tsv

# Get your tenant ID
az account show --query tenantId -o tsv

Step 2: Configure terraform.tfvars

Navigate to the infra/onelake directory and create terraform.tfvars:

# Azure Subscription Settings
subscription_id        = "YOUR_SUBSCRIPTION_ID"
fabric_tenant_id       = "YOUR_TENANT_ID"

# Resource Group
resource_group_name    = "rg-onelake-demo"
create_resource_group  = true
location               = "westeurope"

# Naming prefix (used for all resource names)
prefix                 = "s3data"

# Fabric Capacity — choose one:
# Option A: Use existing capacity (recommended)
create_fabric_capacity      = false
existing_fabric_capacity_id = "YOUR_CAPACITY_ID"

# Option B: Create new capacity
# create_fabric_capacity = true
# fabric_capacity_sku    = "F2"   # ~€240/month, can be paused

# OneLake subpath under /Files/
one_lake_subpath = "upload/"

az rest --method get \
  --url "https://api.fabric.microsoft.com/v1/capacities" \
  --resource "https://api.fabric.microsoft.com"

Step 3: Deploy

cd infra/onelake

terraform init
terraform plan
terraform apply

Deployment takes approximately 5–10 minutes.

Step 4: Capture Outputs for Function App

After deployment, generate a ready-to-use terraform.tfvars for the Function App module:

terraform output function_app_template_tfvars \
  | grep -v EOT \
  > ../../azure-function/terraform/terraform.tfvars

This file contains the OneLake credentials and workspace details needed by the Function App. You only need to add S3 credentials in the next step.

Key Outputs

# View all outputs
terraform output

# Get OneLake credentials
terraform output onelake_app_client_id
terraform output -raw onelake_app_client_secret
terraform output onelake_dfs_path

# Get Fabric IDs
terraform output fabric_workspace_id
terraform output fabric_lakehouse_id

Deployment via Azure Portal

If you prefer to create resources manually rather than using Terraform:

1. Get Fabric Capacity

Option A: Free Trial (recommended for testing)

1. Go to https://app.fabric.microsoft.com/
2. Click your profile icon → Start trial
3. Accept the trial terms (60 days free, equivalent to F64 capacity)
4. Wait 30–60 seconds for activation

Option B: Create Paid Capacity

1. In Azure Portal, search for Microsoft Fabric
2. Click + Create and configure:
   • Resource group: rg-onelake-demo
   • Capacity name: onelakefabric
   • Region: West Europe
   • Size: F2 (~€240/month) or higher
3. Click Review + create → Create

2. Create Workspace and Lakehouse

1. In Fabric Portal, click Workspaces → + New workspace
2. Name it (e.g., onelake-workspace), select your capacity under License mode
3. In the workspace, click + New → Lakehouse
4. Name it (e.g., onelakehouse)
5. Note the workspace and lakehouse names

3. Create Azure AD App Registration

1. In Azure Portal, go to Azure Active Directory → App registrations → + New registration
2. Name: onelake-access-app, single-tenant
3. Note the Application (client) ID and Directory (tenant) ID
4. Go to Certificates & secrets → + New client secret → copy the Value immediately
5. Go to API permissions → + Add a permission → APIs my organization uses
6. Search for Microsoft OneLake DFS (ID: 2eac82a2-09ca-4bb6-9712-8c0e83213eae)
7. Select Delegated permissions → check user_impersonation → Add permissions
8. Click Grant admin consent

4. Grant Workspace Access

1. In Fabric Portal, go to your workspace → Settings → Manage access
2. Add the Azure AD app or Function App identity with Contributor role

Cost Estimates

Cost optimization tips:

• Use existing_fabric_capacity_id to share capacity across workspaces
• Pause capacity when not in use (via Azure Portal)
• Start with trial capacity for development and testing

Permission Propagation

After deployment or manual setup, allow 5–15 minutes for:

• Directory role assignments to propagate
• Admin consent to take effect
• Service principal permissions to become active
• Fabric workspace access to be recognized

Next Steps

→ Deploy Function App to set up automated S3-to-OneLake data transfer