Azure Diagnostic Settings Policy: Capture Logs Before the Incident

Focus keyphrase: Azure diagnostic settings policy

An Azure diagnostic settings policy is one of those admin guardrails that feels boring right up until the outage bridge starts and nobody can find the logs. Resource logs are not collected by default for every Azure resource, so waiting until after the incident to “turn on diagnostics” is basically buying a smoke alarm after the toaster has already become a dragon.

The better pattern is simple: use Azure Policy to deploy diagnostic settings automatically, send the right resource logs to a Log Analytics workspace, and remediate existing resources before the next mystery 403, throttling event, or failed backend call eats your afternoon.

Why Azure diagnostic settings policy belongs in your baseline

Azure Monitor diagnostic settings route platform logs, metrics, and activity log data to supported destinations such as Log Analytics, Event Hubs, and Storage. The catch: diagnostic settings are configured per resource, and resource logs are not collected by default. That is fine for a lab. It is less cute in production.

For admins and platform teams, the real win is consistency. Instead of asking every project team to remember the same portal clicks, assign built-in Azure Policy initiatives where they exist, create custom policies where they do not, and use remediation tasks to bring older resources into compliance.

What to send: audit vs allLogs

Microsoft’s diagnostic settings policy guidance recommends category groups because they adjust as log categories evolve. The two groups you will see most often are audit and allLogs.

Choice	Best for	Watch out for
audit	Security, compliance, control-plane visibility, and lower-noise baselines.	May not include every operational log you want for deep troubleshooting.
allLogs	High-value production services, critical workloads, or resources with frequent support needs.	Higher ingestion volume. Review retention, table plans, and workspace cost controls.
Custom selection	Teams with mature logging standards per service type.	More policy maintenance as resource providers add categories.

Recommended rollout pattern for Azure diagnostic settings policy

Do not assign one giant “log everything everywhere forever” policy and then act surprised when Finance appears behind you like a jump scare. Roll it out deliberately.

1. Pick the destination architecture first

For most admin teams, the default operational destination is a Log Analytics workspace. It gives you KQL queries, alerts, workbooks, and a common place to troubleshoot. Event Hubs fits SIEM or downstream streaming scenarios. Storage fits long-term archive requirements. Many organizations use more than one destination, but start with the reason before enabling the pipe.

2. Separate production and sandbox logging

Production workloads need reliable retention, alerting, and access control. Sandboxes need guardrails too, but they rarely need the same volume and retention. Use management groups, subscriptions, or policy assignment parameters to avoid treating every experiment like the crown jewels.

3. Prefer built-in policies, then fill gaps

Microsoft provides built-in initiatives and policies for many resource types. Use them where available because they are easier to support over time. For resources without a built-in policy, Microsoft documents a custom policy approach and points to tooling such as Create-AzDiagPolicy for generating policy files.

4. Assign with a managed identity and remediate

Policies using the deployIfNotExists effect need a managed identity for remediation. New or updated resources can be evaluated automatically, but existing non-compliant resources need a remediation task. Translation: assigning the policy is step one; cleaning up the backlog is step two.

Azure CLI example: assign a built-in diagnostic settings initiative

The exact policy or initiative name depends on the destination and category group you choose. This example shows the shape of the rollout, not a copy/paste substitute for reviewing the policy definition in your tenant.

# 1) Find diagnostic settings policy definitions or initiatives
az policy set-definition list   --query "[?contains(displayName, 'Diagnostic settings')].[displayName, name]"   -o table

# 2) Assign the selected initiative at a management group, subscription, or resource group scope
az policy assignment create   --name deploy-diagnostic-settings-log-analytics   --display-name "Deploy diagnostic settings to Log Analytics"   --policy-set-definition <initiativeDefinitionId>   --scope <scopeId>   --params '{"logAnalytics":{"value":"<workspaceResourceId>"}}'   --assign-identity   --location eastus

After assignment, create a remediation task for existing resources. The Azure portal exposes this from the policy compliance experience, and the CLI supports remediation operations as well.

Troubleshooting when diagnostic settings policy does not deploy

If you are troubleshooting a specific Azure service, pair policy compliance with resource-specific logs. For example, App Service and Key Vault issues often become much easier to explain once platform logs are available. I covered a related service-specific troubleshooting pattern in App Service Key Vault References Not Resolving?. For broader cloud drift detection, pair this guardrail with Azure Resource Graph guardrail queries.

KQL starter checks after logs arrive

Once diagnostic settings are flowing into Log Analytics, validate the pipeline before you trust it. The exact tables vary by resource provider and destination mode, but these starter checks are useful for confirming recent ingestion and noisy resources.

// Recent AzureDiagnostics volume by resource type
AzureDiagnostics
| where TimeGenerated > ago(24h)
| summarize Events=count() by ResourceProvider, ResourceType
| order by Events desc

// Top talkers by resource
AzureDiagnostics
| where TimeGenerated > ago(24h)
| summarize Events=count() by Resource, ResourceGroup, SubscriptionId
| top 20 by Events

Use those results to tune category choices, alert rules, and retention. Logs are supposed to reduce chaos, not create a new haystack with better branding.

Governance checklist for admins

Final take

An Azure diagnostic settings policy is not glamorous, but it is exactly the sort of boring automation that saves real time. Turn it into a baseline, use built-in policies where possible, remediate the backlog, and keep an eye on cost. Future-you on the outage bridge will be grateful. Present-you may even get to finish coffee while it is still pretending to be warm.

Sources

• Microsoft Learn: Diagnostic settings in Azure Monitor
• Microsoft Learn: Enable diagnostic settings by category group using built-in policies
• Microsoft Learn: Create diagnostic settings at scale by using Azure Policy
• Microsoft Learn: Azure Policy deployIfNotExists effect
• Microsoft Learn: Resource logs in Azure Monitor
• Microsoft Learn: Log Analytics workspace overview
• Microsoft Learn: Azure Monitor Logs cost calculations and options