How to Know Which Parts of Your Business an AI Agent Should Not Run Yet

TL;DR AI agents are most useful when they handle bounded work whose errors are easy to detect and reverse. They should not receive final authority over payments, contracts, regulatory filings, production deletions, or public claims that customers may rely on unless a human approves the action and the surrounding system enforces that approval. A practical decision starts with two questions about stakes and reversibility, then adds a separate authority check covering permissions, monitoring, rollback, and ownership.

Why this decision matters

AI agents can research, draft, classify, update software, publish content, and call business systems. That range makes them valuable, but it also means that a wrong answer can become a completed action before anyone reviews it.

Founders, product managers, and marketing managers therefore need to decide which tasks an agent may execute, which tasks it may only prepare, and which decisions must remain with a named person.

The pressure to delegate is especially strong in small companies. Stripe Atlas reported that solo founders accounted for 63% of the C corporations formed through Atlas so far in the second quarter of 2026. That figure describes Stripe Atlas customers rather than all new businesses, but it shows why tools that extend one person''s capacity receive so much attention.

Where current agents still fail

The 2026 International AI Safety Report finds that leading agents remain less reliable as tasks become longer and more complex. In software evaluations, the most capable systems achieved about 50% success on tasks that took a person a little more than two hours, while an 80% success rate required tasks of roughly 25 minutes.

The report also describes uneven performance, including failures on simple steps after success on harder ones, which makes one successful demonstration a weak basis for granting broad production access.

Long workflows can amplify small error rates, although the exact result depends on how the steps interact. As a simplified illustration, a chain of 100 independent steps would have only about a 50% chance of complete success even if every step were 99.3% reliable. Real workflows are not independent in this way, but the calculation explains why a modest error rate can become unacceptable when every step must work.

Research from Stanford''s SALT Lab adds a different perspective by examining where workers want automation and where they prefer to retain agency. Its WORKBank study collected preferences from 1,500 workers across 104 occupations and paired them with assessments of current technical capability. The study supports task-level decisions about automation and augmentation, while practitioner frameworks add operational questions such as reversibility, access, and approval. These are complementary sources rather than one settled scientific rule.

What the failures look like

Air Canada and customer policy

In Moffatt v Air Canada, a customer relied on incorrect information from the airline''s website chatbot about claiming a bereavement fare after travel. Air Canada refused the requested refund, but the British Columbia Civil Resolution Tribunal held the company liable for negligent misrepresentation because the chatbot was part of its website.

The operational lesson is specific, a customer-facing agent should not create or interpret a policy when its answer can establish a price, refund, eligibility decision, or other commitment.

PocketOS and production access

In April 2026, a coding agent used by PocketOS found a cloud token in an unrelated file and deleted the company''s production database and its backups. The deletion took seconds, while recovery took more than two days and relied on an older offsite backup plus records from other systems. Written instructions had prohibited destructive commands, but the agent still possessed credentials that allowed the action.

The useful control was therefore not a stronger prompt; production credentials, deletion privileges, and backup administration needed separate technical boundaries.

These cases involve different functions, yet both show the same management error. The organization allowed the agent''s output or access to create an external consequence without an effective checkpoint between the proposed action and the result.

Use stakes and reversibility

You should Classify an exact task rather than an entire department. For example, assess issuing a refund under a defined limit instead of assessing customer service as a whole, because the same function contains research, drafting, record changes, financial commitments, and legal obligations with very different risk profiles.

Stakes	Reversibility	Default agent role	Examples and required controls
Low	Easy	Execute within limits	Internal classification, formatting, test data, and draft generation. Keep logs and sample completed work.
High	Easy	Recommend or act under a cap	Pricing analysis, budget scenarios, and ranked options. A person owns the decision and reviews exceptions.
Low	Difficult	Require approval before action	Routine external messages, bulk updates, and scheduled posts. Use previews, rate limits, and a clear undo plan.
High	Difficult	Prepare only	Payments, contracts, regulatory filings, production deletion, access changes, and public claims. A person must approve through a control outside the agent.

The category can change with context. A scheduled social post may be low stakes during a routine campaign and high stakes during a product recall, while a data migration is only reversible when the backup has been tested and the team knows how long restoration will take. The classification should describe the real operating conditions rather than the task''s label.

Add an authority gate

Stakes and reversibility describe the risk, but they do not determine whether the system is ready for autonomy. Before an agent receives permission to act, confirm that its access is limited to the task, significant errors can be detected quickly, rollback has been tested, every action is logged, and a named person owns the outcome.

If any of these conditions is missing, move the workflow down one level so the agent prepares or recommends while a person executes.

A prompt that tells an agent not to delete data or move money is documentation rather than enforcement. Approval gates, scoped credentials, transaction limits, separate production accounts, and permission checks must sit outside the agent so that a prohibited action cannot run even when the model ignores an instruction.

Place checkpoints before consequential actions

A useful human checkpoint pauses the workflow before an external message, payment, production write, or legal commitment takes effect.

The reviewer should receive the proposed action, the evidence used, the material uncertainties, and the exact approval requested. Monitoring after execution can support audits, but it cannot replace approval when the consequence is difficult to reverse.

Expand autonomy through measured results

A practical autonomy ladder moves a workflow from drafting to recommending, then to execution with approval, and finally to bounded autonomous operation. Advancement should depend on measured error rates, override rates, exception handling, tested rollback, and a stable period on real data.

Apollo.io, a sales intelligence and sales engagement platform, presents a staged autonomy model for revenue operations, while a separate panel of business practitioners applies stakes and reversibility to broader management decisions; both should be treated as operating guidance rather than controlled research.

Apply the framework to your role

Founders

Agents can handle research, first drafts, data formatting, code scaffolding, and bookkeeping preparation when their sources and changes remain reviewable.

A founder should retain approval for incorporation and tax filings, cap table changes, contracts, outbound payments, payroll, and any decision that creates a legal obligation. Bookkeeping automation should prepare or categorize entries without gaining unrestricted authority to post, reconcile, or transfer funds.

Product managers

Agents can summarize user feedback, draft tickets, generate test cases, compare alternatives, and prepare release notes.

They should not receive unrestricted production credentials or permission to change access, delete data, or publish roadmap commitments. The PocketOS incident shows why a product rule in a prompt cannot substitute for environment separation and technical approval on destructive operations.

Marketing managers

Agents can create variations, repurpose approved material, prepare campaign schedules, and summarize analytics.

Early deployments should require review before customer messages or public posts, especially when content makes claims about pricing, eligibility, health, finance, refunds, or partner commitments. Routine publishing can gain limited autonomy after the team measures quality, restricts source material, sets volume limits, and maintains a fast suspension process.

Keep legal duties in scope

Using an agent does not transfer the organization''s duties under laws that already apply to its activities. GDPR obligations depend on the processing of personal data, HIPAA generally applies to covered entities and business associates, and Sarbanes-Oxley requirements primarily concern public companies and their controls. A small company should identify which rules apply to the specific workflow instead of treating compliance as one generic category.

The EU AI Act also applies according to the system, the organization''s role, the use case, and the Act''s territorial scope, so serving a European customer does not answer every compliance question by itself. Article 50 transparency obligations for specified AI systems and generated content have applied since 2 August 2026, while several high risk obligations follow later dates. Teams should check the current official guidance for their use case before launching or materially changing an automated workflow.

Implementation checklist

Define one exact task, including its trigger, the systems it can access, the action it may take, and the condition that marks completion.
Rate the task''s stakes and reversibility under normal conditions and under the most credible failure scenario.
List every credential, record, account, and external channel the agent can reach, then remove access that the task does not require.
Choose the initial role as draft, recommend, execute with approval, or bounded autonomous execution, and record who owns the final result.
Place approval before payments, destructive writes, access changes, external commitments, and regulated or rights affecting decisions.
Test rollback with realistic data and measure how long recovery takes, because an untested backup should not count as reversibility.
Log inputs, sources, outputs, tool calls, approvals, and final actions so that incidents can be understood without relying on the agent''s explanation.
Pilot with limits on volume, money, recipients, and permissions, while tracking errors, overrides, exceptions, and time saved.
Increase autonomy only after the workflow meets a defined reliability target and the controls continue to work during real operation.
Set a review date because models, integrations, business rules, and legal requirements can change after the initial approval.

A workable operating rule

The safest way to use agents is to separate preparation from authority. Let them gather information, draft work, and execute bounded actions where errors are visible and recoverable, while a person retains control over decisions that move money, change rights, alter production systems, or make commitments to customers and regulators. The boundary can move as evidence improves, but accountability should remain assigned to a person throughout that process.

If you are mapping these boundaries across your own workflows, our AI readiness audit and AI agents and MCP development work exist for exactly this stage.

Why this decision matters

Where current agents still fail