By Michael Branson, Founder/COO, i3solutions

Quick answer. An Azure AI pilot security checklist covers six checks before production: vault networking, Private Link, key authentication, managed identity, Azure Policy, Azure Government. Microsoft says a new vault’s firewall “is disabled”, which “doesn’t mean that any user will be able to perform operations on your key vault”.

If a business unit’s Azure OpenAI or Microsoft Foundry pilot works and now has to pass a production review, the settings that made it quick to build are the ones the security reviewer will ask about. This checklist covers the infrastructure baseline beneath the application: for each of six checks, what the pilot default is, what production needs, and how to verify it with a named setting or test. If you have not yet chosen a platform, start with Copilot Studio vs Azure AI Foundry: Which One Fits Your Compliance Posture? Every Microsoft statement below is quoted or paraphrased from Microsoft Learn as read on September 26, 2026, with its link.

The Six Checks, and How to Verify Each One

If you are preparing the production review, work through the six checks in order; each one ends with the portal setting or test to record in the change log as evidence for the reviewer.

1. Key Vault network access

When the pilot’s secrets sit in a vault created with default settings, the network path to that vault is open. Microsoft’s Configure network security for Azure Key Vault page says: “By default, when you create a new key vault, the Azure Key Vault firewall is disabled. All applications and Azure services can access the key vault and send requests to the key vault.” It continues: “This configuration doesn’t mean that any user will be able to perform operations on your key vault. The key vault still restricts access to secrets, keys, and certificates stored in key vault by requiring Microsoft Entra authentication and access policy permissions.” Authentication still applies; what the default leaves out is the network layer.

Production needs: the vault firewall on with named virtual networks or address ranges, or public access disabled with a private endpoint. Two details belong in the review. Microsoft says “Disabling public access doesn’t suppress the trusted services bypass”, so “trusted services continue to reach the vault and don’t require a private endpoint.” And “Azure Key Vault firewall rules only apply to data plane operations. Control plane operations are not subject to the restrictions specified in firewall rules.” i3solutions selects Dataverse over SharePoint as the primary relational store when a client needs scalable high-volume transactional data, and holds application secrets in Azure Key Vault.

Verify: read the vault’s networking settings and record the public access setting, the allowed networks and whether the trusted services bypass is on, and write down who decided the bypass setting and why.

2. Public network access and Private Link

If the AI resource was created with public network access enabled, it answers from the internet as well as from your network. Microsoft’s How to configure network isolation for Microsoft Foundry page says: “The PNA flag setting determines whether your project requires a private endpoint for access.” It adds: “The public network access can be set to Disabled with a private endpoint (private link) enabled, or set to Selected networks to grant specific IP addresses and virtual networks the ability to access Foundry securely.” The same page, read September 26, 2026, lists the Foundry project, storage, Key Vault, container registry and monitoring as the platform resources that Private Link isolates, so the dependencies belong to this check as well as the AI resource.

Production needs: public network access set to Disabled with a private endpoint, or set to Selected networks with the allowed ranges written down, on the AI resource and on each dependency.

Verify: for each resource, record the public network access value and the private endpoint connection with its approval state, then send one request from a network outside the allowed list and record that it is refused.

3. Key authentication off, and proven off

Before production, the pilot’s API keys stop being a way in. Microsoft’s Disable local authentication in Foundry Tools page says: “Organizations can disable local authentication methods and enforce Microsoft Entra authentication instead.” It also warns: “Disabling local authentication doesn’t take effect immediately.” Microsoft’s policy reference adds a pilot-specific catch: “Azure OpenAI Studio, typically used in development/testing, requires key access and will not function if key access is disabled.”

Production needs: local authentication disabled on the AI resource, with the callers moved to Microsoft Entra authentication first. The change is not immediate, which is why the verification below waits for it.

Verify: Microsoft’s local authentication page gives the test to record: “To confirm that the change has taken effect, send a data-plane request with the old key and verify that the service returns an HTTP 401 response”. Keep the request, the time and the response in the change log as the evidence for this check.

4. Managed identity, with secrets held in Key Vault

Once keys are off, the application needs another way to call the model. Microsoft’s How to configure Azure OpenAI in Microsoft Foundry Models with Microsoft Entra ID authentication (classic) page says: “Microsoft Entra ID lets you call your Azure OpenAI resource without storing an API key in your application. Instead of a key, your code requests a short-lived access token for an identity that you grant a role on the resource.” It adds: “There’s no secret for you to store or rotate.”

Production needs: a managed identity on the app host with a role on the AI resource and on the vault, and any secret that must remain (a third-party credential, for example) held in Key Vault rather than in application settings.

Verify: list the app host’s identity and its role assignments on the AI resource and the vault, then search the application settings and repository for a stored key and record the result. Which roles to grant, and why keys fall outside that role model, is outside this checklist (see What This Checklist Does Not Cover, below).

5. Azure Policy so the settings do not drift

After the review passes, a redeployment or a new resource can quietly bring the pilot defaults back. Microsoft’s Azure Policy built-in policy definitions for Foundry Tools page lists built-in definitions that match the checks above, including “Azure AI Services resources should have key access disabled (disable local authentication)”, “Azure AI Services resources should restrict network access”, “Azure AI Services resources should use Azure Private Link”, “Configure Cognitive Services accounts to disable public network access” and “Cognitive Services accounts should use a managed identity”.

Production needs: those definitions assigned at the subscription or management group that holds the AI resources, with a decision recorded for each on whether it audits or denies; the reference lists the effects each one allows.

Verify: read the policy assignments at that scope and the policy compliance results, and record that the AI resources appear in them.

6. The Azure Government check

If your tenant runs in Azure Government, check the model and the endpoints before the build is promoted. Microsoft’s Foundry Models sold by Azure in Azure Government page says: “Models sold by Azure include all Azure OpenAI models offered in Azure Government.” It also says: “Model availability in Azure Government varies by region.” Its region table names usgovarizona and usgovvirginia. Microsoft’s Compare Azure Government and global Azure page lists different endpoints for the two clouds, including openai.azure.us for Azure OpenAI and vault.usgovcloudapi.net for Key Vault.

Verify: record the model and version the pilot uses against the region table for the region you will deploy in, and search the code and configuration for commercial endpoints such as openai.azure.com and vault.azure.net.

When This Order Is Wrong

If the pilot was built inside a landing zone whose deployment path applies private networking and the policy assignments above to new resources, checks 1, 2 and 5 may already hold, and the work may come down to checks 3 and 6. Verify that the inherited settings reached the pilot’s resources before skipping them: a resource created outside the landing zone’s deployment path may not carry them.

What This Checklist Does Not Cover

If the open question is the role design, why API keys sit outside it, or the governance controls inside Foundry itself, that is the subject of the guide How to Govern Custom AI in Microsoft Foundry with Microsoft’s Own Controls. If it is what separates a production-grade deployment from a pilot at the application layer, see Production-Grade GenAI for Regulated Microsoft Estates. For Azure security settings beyond AI resources, see Azure Security Best Practices: The Basics Enterprises Still Get Wrong. This page covers the infrastructure baseline an Azure OpenAI or Foundry pilot needs before production, and how to verify it.

How i3solutions Reviews a Pilot Before Production

When your team brings an Azure AI pilot to i3solutions before its production review, the work starts from what the pilot actually deployed: the vault, the AI resource and its dependencies, the app host and the policy scope, checked against the six checks above. i3solutions maps governance to named control families, enforces it in the platform through Entra ID, Purview, and Azure Policy, and evidences it continuously rather than reconstructing it at audit. The wider service is described in Custom AI Consulting Services for Governed Microsoft Enterprise AI.

Key Takeaways

  • A new key vault’s firewall is disabled by default; Microsoft adds that “This configuration doesn’t mean that any user will be able to perform operations on your key vault.” Production adds the network layer.
  • Disable public network access or limit it to selected networks on the AI resource and its dependencies, with private endpoints.
  • Turn key authentication off, wait for it to take effect, and keep the HTTP 401 response to the old key in the change log as the evidence.
  • Give the app host a managed identity, keep any remaining secret in Key Vault, and assign the built-in policy definitions so the settings do not drift.
  • In Azure Government, check the model against the region table and move code to the Azure Government endpoints.

Frequently Asked Questions

Is Azure Key Vault public by default?

If the vault was created with default settings, its network path is open: Microsoft’s Key Vault network security page says “By default, when you create a new key vault, the Azure Key Vault firewall is disabled.” It also says “This configuration doesn’t mean that any user will be able to perform operations on your key vault”, because Microsoft Entra authentication and access permissions still apply. Production turns the firewall on or disables public access with a private endpoint.

How do you confirm key authentication is really off on an Azure OpenAI resource?

After you disable local authentication, allow time, because Microsoft says “Disabling local authentication doesn’t take effect immediately.” Then run the test on Microsoft’s local authentication page: “send a data-plane request with the old key and verify that the service returns an HTTP 401 response”. Keep that response in the change log as the evidence.

Does disabling public access block trusted Azure services from Key Vault?

If you disable public access on a vault, trusted services can still reach it. Microsoft says “Disabling public access doesn’t suppress the trusted services bypass”, and that “trusted services continue to reach the vault and don’t require a private endpoint.” Decide the bypass setting on purpose and record who decided it.

Is Azure OpenAI available in Azure Government?

If your tenant runs in Azure Government, Microsoft’s Azure Government models page says “Models sold by Azure include all Azure OpenAI models offered in Azure Government.” and “Model availability in Azure Government varies by region.” Check the specific model against the region you plan to use, and point the code at the Azure Government endpoints.

Bring the Pilot’s Resource List to the First Conversation

If the production review is waiting on you, bring the list of what the pilot deployed: the vault, the AI resource and its dependencies, the app host, and the cloud your tenant runs in. The conversation turns that into the order of work for the six checks. Talk to a senior architect

Sources