Design an AI Agent Control Plane: Tasks, Permission...

The system-design prompt sounds simple: “Design a platform where AI agents can complete customer tasks.” The difficulty appears when the interviewer asks who authorizes a payment, where credentials live, how duplicate workers are stopped, and what rollback means after an email has already been sent.

An agent control plane is a design framework for separating intent, policy, credentials, execution, state, approvals, recovery, and audit. It is not a prescribed product, and it must not become an excuse for unlimited autonomous background execution.

Separate the Control Plane From Execution

The control plane decides what work is allowed and under which conditions. Execution workers perform bounded steps with issued authority. Keep those responsibilities explicit.

The control plane can own:

task intake and normalization;
workflow and policy version selection;
tenant and actor authorization;
tool allowlists and action scope;
approval requirements;
credential issuance;
attempt and state coordination;
audit receipts and final outcome.

Workers can own one attempt: call the approved model, invoke allowed tools through gateways, report structured results, and stop when the lease or task ends.

This separation limits blast radius and makes failures inspectable. A worker cannot silently expand authority because the model generated a new plan.

Kubernetes documents a control plane that manages desired cluster state and worker components that run workloads. That is a useful analogy for ownership and reconciliation. Kubernetes is not an AI agent architecture standard, and its components should not be mapped one-to-one onto an agent platform.

Define Task and Policy Contracts

A task should be more than a natural-language request. Normalize it into an explicit contract that the platform can validate.

Include:

tenant, actor, and requested outcome;
allowed workflow and tool set;
input references and data classification;
maximum cost, duration, and attempts;
approval checkpoints;
side-effect limits;
success, safe-failure, and cancellation states;
policy and schema versions.

Keep user intent separate from generated plan steps. The model may propose a sequence, but the trusted platform evaluates each consequential action against policy and current state.

SDK guardrails are not a universal pre-execution barrier. Depending on execution mode, model work may begin before a tripwire stops the run. High-risk external actions therefore need policy and approval enforcement at the tool gateway before commitment.

Policy decisions should produce reason codes and versions. “Denied” is less useful than “payment amount exceeds unattended limit under policy version 14.” Avoid embedding every rule in the system prompt, where it becomes difficult to enforce and audit.

The agentic coding permission-gates guide provides a smaller example: permission is scoped to an operation and context, not granted permanently because one tool is available.

Isolate Credentials and Execution Workers

Do not place durable credentials in prompts, memory, or reusable task objects. A trusted credential broker should issue narrow, short-lived authority after tenant, policy, approval, and task checks.

Bind the credential to tool, action, resource, tenant, task, and expiry where the integration allows it. Validate authorization again at the tool gateway because task state can change between planning and execution.

Workers should be isolated by tenant and attempt. They receive only the data and tools required for their step. Network and storage access should default closed rather than inheriting the control plane's broad privileges.

Handle worker loss through leases and heartbeats. When a lease expires, the control plane determines whether the action is safe to retry. It must not assume a timeout means nothing happened externally.

The AI security engineer interview guide expands the threat model for untrusted content, tool boundaries, evidence, and containment. Control-plane design should make those security decisions enforceable.

Add Approvals and Idempotent Execution

Human approval is a state transition, not a chat message. Define what the reviewer sees, what they can change, how long approval remains valid, and whether a changed task invalidates it.

Use approval levels based on consequence. Read-only lookup may proceed automatically. Drafting a message may require user confirmation before sending. A high-value payment or destructive operation may require stronger identity and dual approval.

Idempotency prevents repeated attempts from duplicating supported side effects. Give each intended action a stable key and store the provider's result. Before retrying, query the authoritative external state when possible.

Do not call every action idempotent. Some providers lack safe keys, and some operations depend on changing context. In those cases, require manual reconciliation or design a different workflow.

Approval and idempotency solve different problems. Approval establishes authority; idempotency prevents duplicate execution of an authorized intent.

Distinguish Rollback From Compensation

Local workflow state can often be rolled back to a previous version. External reality may not cooperate.

A sent email cannot be unsent reliably. A completed payment may require a refund. A deleted record may be unrecoverable. A published message may already have been copied. Restoring a local database snapshot does not reverse these effects.

Classify actions before execution:

reversible local state;
externally reversible through a documented API;
compensatable through a separate business action;
irreversible or recovery-uncertain.

For irreversible or uncertain actions, prioritize prevention: explicit confirmation, preview, delayed commit, narrow limits, and stronger approval. When compensation exists, model it as a new authorized action with its own failure modes and audit receipt.

Never tell an interviewer that a transaction, deletion, or message can be “rolled back” merely because the local task state can be reset.

Produce Audit Receipts and Safe Failure States

Every consequential attempt should leave a structured receipt linking task intent to policy, authority, execution, state, and result.

Record:

task, tenant, actor, and attempt identifiers;
workflow, model, prompt, policy, and tool versions;
validated permission and credential scope reference;
approval identity, decision, and expiry;
idempotency key and external result;
state transitions and conflicts;
side-effect status;
final task and recovery state.

Do not store credentials or unnecessary personal data in the receipt. Use redacted references and access controls.

Safe failure states should be explicit: denied, canceled, timed out with no action, outcome unknown, partially completed, awaiting reconciliation, or compensation required. “Failed” alone is not enough.

The agent observability interview guide shows how traces and receipts locate the first observable deviation. The control plane owns the authoritative outcome contract.

Handle Multi-Tenancy, Concurrency, and Failure

Extend the design when the interviewer asks about scale.

Partition queues and state by tenant. Enforce tenant identity at every storage, policy, credential, and tool boundary. Apply quotas for model cost, tool rate, concurrent tasks, and high-risk approvals.

Use optimistic concurrency or explicit locks for shared resources. If two tasks propose conflicting updates, the control plane should detect the state-version mismatch and re-evaluate rather than letting the last writer win silently.

Design for partial infrastructure failure. A queue may redeliver, a worker may disappear after an external action, an approval may expire, or the audit store may be temporarily unavailable. Decide which conditions block execution. Consequential actions should fail closed when required receipts or authorization cannot be established.

The AI interview audit-trail guide offers a reviewable change chain. Runtime control adds tenant isolation, attempt coordination, and side-effect reconciliation.

Present the System-Design Answer in Layers

Begin with requirements and non-goals. Ask which tasks, tenants, tools, consequences, latency, and compliance needs are in scope. State that open-ended autonomous background work is excluded unless explicitly required.

Then draw five logical layers in words: task API, policy and approval control, credential broker, isolated execution workers, and state plus audit. Walk one read-only task and one consequential task through the system.

A concise answer might say: “The task service normalizes user intent and pins policy and workflow versions. The policy layer evaluates each proposed tool action and obtains approval where required. A broker issues short-lived scoped credentials to isolated workers. Attempts use leases, idempotency keys, and authoritative state checks. The platform distinguishes local rollback from external compensation and records side-effect status in an audit receipt. Unknown outcomes enter reconciliation instead of blind retry.”

Finish with failure modes, observability, privacy, and evolution. State clearly that Kubernetes is only an analogy and that irreversible side effects cannot be undone by restoring local state.

OpenAI's agent guidance describes instructions, tools, guardrails, and human intervention. NIST AI RMF 1.0 supplies a voluntary risk-management frame and is under revision at publication. These sources support design judgment; neither prescribes one mandatory control-plane product.