The Azure OpenAI retrieval gap came into focus after a Microsoft partner in Milan found that an email assistant could pass evaluations and unit tests while still returning SharePoint content the requesting user should not have been able to open. The case is notable because it did not come from a lab demo. It came from a working production system that was already resolving about 60% of inbound customer email automatically.

Egiziago Cioffi, the IT and Enterprise Architect and CEO of SynSphere Italia, built the agent himself. He wrote the indexing job, configured the Azure OpenAI retrieval pipeline, and connected it to SharePoint. The assistant performed well in the checks the team had designed for it. But when Cioffi tested the system with a low-privilege account, the outputs did not match what a high-privilege account had seen. In other words, the system was able to answer questions using content that the requesting user could not access directly in SharePoint.

That matters because retrieval-augmented systems are often judged by whether they answer correctly, not whether they answer only from data the requester is entitled to see. A system can look reliable in standard evaluations and still leak sensitive material if access control is not enforced at query time. The problem is especially important in enterprise workflows, where agents may handle email, internal documents, customer records, or other controlled information.

Azure OpenAI Retrieval Gap In Production

The Azure OpenAI retrieval gap in this case was not a failure to find documents. It was a failure to narrow results to the requester’s permissions. Cioffi’s retrieval logs showed that the assistant could surface SharePoint content beyond what the low-privilege user could have opened on their own. The evaluation scores stayed clean because the checks did not ask the permission question that mattered.

The source material points to a broader pattern in production RAG systems: some deployments answer with the indexer’s permissions instead of the requester’s. That can happen when a custom pipeline indexes content under a broadly privileged service account and no separate entitlement check is added at query time. In that setup, the assistant may appear correct while still being too broad.

Microsoft has shipped or previewed several protections that aim to reduce this risk. Azure AI Search has native document-level ACL trimming via Entra-based tokens, and SharePoint ACL sync followed in a later preview. The SharePoint ACL preview can also ingest site-group metadata through the spg: prefix in the 2026-05-01-preview API. But the material notes an important limitation: only Entra-backed principals are documented as reliably enforced at query time, and the preview path does not cover every agent deployment route.

Why This Matters For Enterprise AI

The Azure OpenAI retrieval gap matters because the practical risk is silent data exposure. In Cioffi’s case, the assistant did not obviously fail. It returned useful-looking answers. The issue was that the answers were sourced from content a specific requester should not have been able to reach. That makes the problem harder to spot than a system outage or a broken search result.

This also helps explain why standard testing can miss the issue. Unit tests and normal evaluations can verify that an assistant responds, follows prompts, or retrieves relevant material. They do not always verify that the returned material matches the user’s actual permissions. If that permission check is missing, the system may pass the test suite and still violate access boundaries in production.

The source material also notes that Azure OpenAI On Your Data supports document-level access through Azure AI Search security filters, but Microsoft’s documentation says document-level access is disabled if the permitted-groups field is not mapped. That is a fail-open default in a first-party path, which makes the configuration details especially important for teams deploying enterprise assistants.

What Teams Should Watch Next

• Whether document-level ACL trimming is enabled end to end in the retrieval path.
• Whether the pipeline uses a narrow, query-time permission check instead of relying on index-time access alone.
• Whether SharePoint and Entra-backed controls are actually enforced in the specific deployment path, not just available in preview.
• Whether evaluations include low-privilege user tests, not only functional accuracy tests.

The broader takeaway is that enterprise AI security is not just about model behavior. It is also about how content is indexed, which identity is used to retrieve it, and whether entitlement checks happen before an answer is returned. Cioffi’s case shows that an assistant can look successful from the outside while still crossing a permissions boundary inside the workflow.

Readers should watch how Microsoft’s ACL and security-filter features mature, and how teams wire them into custom RAG systems. The Azure OpenAI retrieval gap is a reminder that in enterprise search, correctness and authorization are not the same thing. Both have to work for the system to be safe.

Explore AI Craft Pad

Practical AI articles · AI tools · Step-by-step guides