LLI Wiki — My Digital Knowledge Base

Packaging a PowerShell application for Azure Batch

Versioned ZIP, task wrapper script, environment variables, per-environment config merged at build time, retries and one-task-per-database fan-out.

An Azure Batch task is just a command line run on a node. The whole craft is giving it the right code, configuration and parameters in a reproducible way. This is how I package a PowerShell application for Batch: versions, per-environment configuration, retries and one-task-per-database fan-out.

The package: a versioned ZIP

An application package is a ZIP that Batch downloads and extracts on every node that needs it. My layout is always the same:

app-prd-2026.4.1.zip
├── applications/
│   └── myapp/            # main script, functions, SQL
├── modules/              # shared PowerShell modules
├── config/
│   └── config.json       # merged configuration, only one
├── tasks/
│   ├── run-export.ps1    # Batch entry points
│   └── run-cleanup.ps1
└── version.txt

Rules I enforce:

The version

I version with a Git tag YYYY.M.N (year, month, increment within the month). The build script reads the latest tag, validates its format with a regular expression, and falls back to 0.0.0 if invalid:

$tag = git -C $repoPath tag --sort=-version:refname | Select-Object -First 1
$version = if ($tag -match '^\d{4}\.\d{1,2}\.\d+$') { $tag } else { '0.0.0' }
$zipName = "myapp-$Environment-$version.zip"

The ZIP name carries the environment and the version; a -latest copy and a version-resolved.txt file feed the deployment pipeline. The ZIP is then uploaded as an application package with the same version, so rolling back is just changing the referenced version.

One config per environment, merged at build time

Two source files: config-common.json (shared values) and config-<env>.json (overrides). The build merges them, the environment winning, and writes one config/config.json into the ZIP:

$merged = @{}
foreach ($file in @($commonPath, $envPath)) {
    $json = Get-Content $file -Raw | ConvertFrom-Json
    foreach ($p in $json.PSObject.Properties) { $merged[$p.Name] = $p.Value }
}
$merged | ConvertTo-Json -Depth 10 | Set-Content (Join-Path $stagingRoot 'config/config.json')

The merge is shallow (per top-level key): a nested section is replaced as a whole. Pitfall: the configuration travels inside the ZIP, so changing it requires a redeploy. For anything that must change without a redeploy I prefer a pipeline parameter.

The task wrapper script

The tasks/ script is a thin adapter: it finds the package root, logs, delegates to the application script and returns its exit code. Batch reads that code to decide success.

param([Parameter(Mandatory)][string]$Environment,
      [Parameter(Mandatory)][string]$Server,
      [Parameter(Mandatory)][string]$Database)

$ErrorActionPreference = 'Stop'
$root = (Get-Item 'Env:AZ_BATCH_APP_PACKAGE_myapp' -ErrorAction SilentlyContinue)?.Value
if (-not $root) { $root = Split-Path -Parent $PSScriptRoot }   # local test

try {
    & pwsh -NoProfile -File "$root/applications/myapp/main.ps1" `
        -Environment $Environment -Server $Server -Database $Database `
        -ConfigPath "$root/config"
    exit $LASTEXITCODE
}
catch { Write-Output "FATAL: $($_.Exception.Message)"; exit 1 }

The exact name of the AZ_BATCH_APP_PACKAGE_* variable depends on the package id and version: check it on a node. The presence of this variable also tells me a real Batch run from a local test, so I can choose the right identity.

Fan-out: one task per database

One script per database, run in parallel, rather than a loop inside a single script: a failure stays isolated, retries on its own and shows up in the metrics. The pipeline queries the inventory and submits the tasks:

foreach ($db in $databases) {
    $token  = ($db.Name.ToLower() -replace '[^a-z0-9]', '-')
    $taskId = "export-$token-$(Get-Date -Format yyyyMMddHHmmss)"
    $cmd = "pwsh -NoProfile -File tasks/run-export.ps1 -Environment prd " +
           "-Server $($db.Server) -Database $($db.Name)"
    # submit task $taskId with command line $cmd
}

The task id prefix (export-, cleanup-) identifies the application in monitoring queries; task ids are limited in length and characters, so normalise the database name.

Retries

Batch can rerun a failed task (a maximum retry count in the task constraints). But a retry only helps if the script is idempotent: clean up the files of the previous attempt, leave no half results. I also handle recovery on the selection side: the query that picks databases takes again those that failed recently with no success since.

Lessons learned

Mastodon LLIMastodon SocialFosstodon