How to Pilot the Copilot Harness in Microsoft Foundry Agent Service

what-the-copilot-harness-is

A harness is the runtime and interaction surface around an AI agent. It is the part that makes an agent usable by people: the chat experience, tool and skill routing, authentication context, approvals, conversation state, and the guardrails that sit between a natural-language request and an external action.

That distinction matters. A model is not an employee, and an agent instruction is not a security boundary. The harness is where you decide what the agent can see, which tools it can call, which identity it uses, when it must ask for approval, and how its work is logged.

The Copilot harness in Foundry Agent Service is best understood as a way to expose a Foundry agent through Microsoft's Copilot-style experience while keeping the agent's tools, knowledge, identity, and governance under an Azure deployment. It is a new delivery surface for an agent, not a reason to skip normal identity, data, and change-control work.

before-you-create-anything

Prepare these items first:

1. An Azure subscription and a resource group approved for the pilot.
2. A Foundry project in a supported region, with a model deployment your tenant is allowed to use.
3. A narrowly defined job for the agent, such as “summarize approved service-desk tickets and draft a response.” Avoid a vague goal such as “run IT operations.”
4. A test knowledge source that contains no secrets and no unapproved personal data.
5. A tool inventory. Write down every API, connector, function, or action the agent could call, the identity it uses, and whether the action changes data.
6. An owner and an expiry date for the pilot. Preview agents otherwise become permanent software with no one responsible for reviewing them.

Use the Copilot Studio agents overview to decide whether the workload belongs in Copilot Studio, Foundry Agent Service, or a combination of the two. The right choice depends on the tools, identity boundary, extensibility and deployment model—not on which product name is newest.

create-a-dedicated-foundry-project

Open Microsoft Foundry and create or select a project dedicated to the pilot. Do not start inside a large shared project unless you already have a clear resource, role, network and cost boundary.

Record the following before you continue:

• Subscription and resource group.
• Region and data-residency requirements.
• Model deployment name and quota.
• Project owners and contributors.
• Connected storage, search, or data services.
• Diagnostic logging destination and retention.

Use separate development, test, and production projects when the agent will eventually handle business data. A project boundary is not a substitute for data permissions, but it makes accidental cross-environment tool calls and cost attribution easier to detect.

define-the-agent-as-a-controlled-workflow

Create the agent with a short, testable instruction set. A useful starting structure is:

Keep policy in the system around the agent, not only in the prompt. Use Entra permissions, managed identities, API scopes, connector permissions, network controls and approval gates to enforce the boundary. The prompt should explain behavior; it should not be the only thing preventing a destructive API call.

add-knowledge-without-creating-a-data-leak

Connect only the sources needed for the defined job. Start with a small, curated document set. Test retrieval with questions that should succeed and questions that should fail.

For every source, answer:

• Who owns it?
• What classification does it carry?
• What users are allowed to retrieve it?
• How quickly are changes reflected?
• What happens when a document is deleted or access is revoked?
• Can the agent quote the source or provide a link back to it?

Do not upload a broad file share as a shortcut. An agent that can retrieve a document is not automatically entitled to disclose it to every user who can chat with the agent. Validate retrieval using least-privilege test accounts and include prompt-injection documents in the test set.

add-tools-one-at-a-time

Start with read-only tools. A sensible progression is:

1. Read-only search.
2. Read-only ticket or record lookup.
3. Draft generation.
4. Human-approved write action.
5. Automated writes only after the audit trail and rollback path are proven.

For every tool, document the input schema, output schema, authentication method, rate limit, data classification, and failure behavior. Use a dedicated managed identity or application identity with only the permissions required for the pilot. Do not reuse a highly privileged administrator identity because it is convenient.

If a tool can send mail, update a ticket, alter a group, change a policy, or trigger a deployment, make the approval step explicit. The agent should show the exact target, fields and proposed change before it acts. “Proceed?” is not enough if the user cannot see what will be changed.

configure-the-copilot-harness

In the agent's deployment or channel configuration, select the Copilot harness where it is available for your tenant and region. Treat the harness as a separate integration boundary and verify:

• Which users or groups can discover the agent.
• Whether the user's identity is passed to downstream tools.
• Whether the agent can act on behalf of the user or only as its managed identity.
• Which authentication prompts appear during a session.
• Whether conversation history is retained and where.
• How feedback, traces and tool calls are logged.
• How the agent is unpublished or rolled back.

Do not assume that a user who can open the Copilot experience can use every connected tool. Test authorization with a normal user, a denied user and an administrator. Confirm that a denied user receives a clean refusal rather than an empty answer that looks like a retrieval failure.

If the preview requires a new agent or a specific deployment path, do not modify a working production agent merely to try it. Clone the configuration into the pilot project, then compare behavior and permissions.

test-the-whole-path

Create a test matrix with at least these cases:

• A normal request with one authoritative answer.
• A request whose answer is not in the knowledge source.
• A prompt-injection attempt inside a retrieved document.
• A request from a user who should be denied.
• A read-only tool call.
• A write request that must pause for approval.
• A malformed tool response.
• A downstream timeout.
• A duplicate request after a network retry.
• A request containing sensitive data that the agent must not repeat.

For each test, capture the final answer, retrieved sources, tool calls, approval state, identity, latency, token or usage data, and logs. Test the refusal behavior as carefully as the happy path. A useful enterprise agent is predictable when it cannot comply.

add-evaluation-and-cost-controls

Define a small set of measures before inviting users:

• Answer groundedness: does the response cite the approved source?
• Task completion: did the agent perform the intended workflow?
• Refusal accuracy: did it reject unauthorized or unsupported requests?
• Tool accuracy: did it call the correct tool with valid arguments?
• Human-approval rate: how often does a write action pause?
• Latency and failure rate.
• Cost per completed task.

Set budgets, quotas and alerts for the Azure resources involved. Preview pricing and quotas can change, so use the current Azure pricing and service limits for the deployment rather than copying a number from an older guide. Keep the pilot small enough that an accidental loop, repeated retrieval, or runaway tool call is cheap to stop.

roll-out-in-rings

Use a staged rollout:

1. Owner-only testing.
2. A small group with low-risk data.
3. A monitored department.
4. Wider availability after the evaluation report is approved.

At each stage, review failed requests, unexpected tool calls, sensitive-data findings, user feedback, cost, and approval behavior. Keep a kill switch: revoke the agent's discovery group, disable the harness channel, remove the tool permission, or stop the deployment without deleting evidence needed for the incident review.

troubleshooting-checklist

<strong>The agent answers but cannot call a tool:</strong> check the tool's identity, scope, consent, network path, and input schema. A successful chat session proves only that the model can respond.

<strong>The agent retrieves documents a user should not see:</strong> stop the pilot, remove the source or connector, inspect the access path, and test with a least-privilege account. Do not fix this only by adding “do not reveal confidential data” to the prompt.

<strong>The agent loops or repeats actions:</strong> add idempotency keys, bounded retries, timeouts and a write approval gate. Review the trace to find whether the loop is in the model, tool wrapper, connector or retry policy.

<strong>Users see different results:</strong> compare identity claims, group membership, retrieval filters, tool permissions, region and conversation state. Do not assume that two users are querying the same effective data boundary.

<strong>The preview is unavailable:</strong> verify tenant eligibility, region, deployment type, feature flags and current Microsoft documentation. Do not route production traffic through an undocumented workaround.

what-to-keep-after-the-pilot

Keep the agent's instruction version, tool schemas, identity assignments, knowledge-source list, evaluation cases, approval policy, cost data and rollback procedure in source control or an approved change-management system. Record the date the preview was tested and the Microsoft documentation version used.

The Copilot harness is most useful when it makes an agent easier to govern, not merely easier to chat with. Build the boundary first, connect the model second, and expose it to users last. The fastest path to a reliable agent is a narrow job, read-only tools, visible sources, explicit approvals and a rollback that works before anyone depends on it.

sources

• 365Forge live feed
• Microsoft Azure: Foundry Agent Service
• Microsoft Learn: Copilot Studio agents overview
• Microsoft Learn: Agent harness