Building Trust in AI: Microsoft Foundry's Role

Sharing some thoughts here about areas that I need to better understand and am actively exploring.

At Microsoft Ignite 2025, that took place this past November in San Francisco, it was unavoidable to notice the narrative mature from Build 2025 and Ignite 2024. We are now seeing more organizations moving from POCs to production workloads and wanting to ensure they are doing so in a manageable way.

The progress reminds me a lot of the early days in the Power Platform when people started to publish their first Power Apps and then more questions from IT Managers/Enterprise Architects/IT Leadership about “how do we manage all of this stuff if so many people are building these apps”? In response the Power Platform CoE Toolkit and Power Platform Admin Center emerged to better support those customers. We are now seeing a similar pattern.

I do see Generative AI solutions in a similar manner. The technology democratizes access to new capabilities. Whether that is writing code, reviewing documents or making decisions on behalf of users, the access to the technology is becoming more and more approachable, regardless of your skillset.

So, if we think about where organizations are headed, the natural question, once again is “how do we manage all of this stuff”?

With this in mind, my current learning plan includes the following capabilities found in Microsoft Foundry:

Evaluations
Guardrails
Foundry Control Plane

So let’s break these topics down to describe why they are important.

Evaluations

We often hear the terms ‘non-deterministic’ or ‘probabilistic’ when it comes to Gen AI solutions. We hear these terms for good reasons. Historically, we can go ahead and write unit tests that can be used as part of an assertion; I input X and I expect Y to be outputted. With Gen Ai solutions, it may not be as straight forward. We can use evaluations as a way to use a data-driven and more scientific approach.

Some mechanisms that we can use with evaluations include:

Performance metrics like groundedness (aka avoiding hallucinations), relevance and coherence
Regression Testing Models are obviously very powerful. Each new version of a model represents an opportunity to reduce latency, improve accuracy and hopefully reduce costs. But after you have tuned your solution for one model, how can you streamline adopting a new model.
Deployment Readiness Using Evaluations in Microsoft Foundry and including a vast dataset ensures your app is ready for production. Being able to demonstrate effectiveness through performance metrics provides a measurable benchmark that can be used to demonstrate readiness.

Guardrails

When building agentic solutions, developers can get wrapped up in all the functionality that they want to include in their solution and may have exposure in areas they haven’t even thought of. This includes:

Sensitive Content including avoiding leaking PII (personal identifiable information) or even in appropriate content being returned from a prompt response.
Corporate Standards/Alignment your agentic solution becomes an extension of your brand and policies including avoiding being tricked into unauthorized discounts.
Input/Output Filtering This helps avoid prompt injection attacks and provides a last mile of safety for agentic solutions.

Foundry Control Plane

With Evaluations and Guardrails in place, customers should feel more confident in the solutions that are being built. However, we still have a gap. How do we know where these agents live? What are the costs associated with them? Are they being used? Do they have errors? This is where the Foundry Control Plane can help us by:

Visibility into all of the agents that have been deployed whether they are based upon Microsoft stack or even 3rd party.
Consistency by ensuring all the necessary security policies and access controls are in place uniformly.
Control by enabling administrators the ability to enable/disable agents that are not performing as expected.

Want to see the Foundry Control Plane in action? Check out this video where I show an Azure Logic Apps agentic business process being managed by the Foundry Control Plane.

https://www.youtube.com/watch?v=xmzn1z2VixE&t=1s