Agents that only chat don’t move work forward. Here’s how DevOps and security teams wire them into real internal systems so they can act, not just suggest.
See the pattern in action, tap through the tabs below:
Most “AI agents” in production today can talk about work but can’t do it. They summarize a ticket, draft a reply, suggest a rollback command, then a human has to copy, paste, and execute it anyway. The bottleneck isn’t the model’s reasoning, it’s that nobody defined a safe, callable interface between the agent and the systems that actually run your stack: the deploy pipeline, the ticketing API, the internal admin panel. Without that interface, “agentic AI” is just a faster way to write suggestions nobody automated away.
{
"name": "rollback_deploy",
"description": "Roll back a service to its last known-good release tag",
"parameters": {
"type": "object",
"properties": {
"service": { "type": "string", "description": "Service name, e.g. checkout-api" },
"environment": { "type": "string", "enum": ["staging", "prod"] },
"reason": { "type": "string", "description": "Why the rollback is needed" }
},
"required": ["service", "environment", "reason"]
},
"requires_approval": true
}
> agent: error rate on checkout-api hit 6.2% after 14:02 deploy
> agent: calling rollback_deploy(service="checkout-api", environment="prod",
reason="error rate spike post-deploy")
> system: action requires_approval=true -> paging on-call (Slack #incidents)
> on-call: approved
> tool_result: rollback complete, checkout-api@release-482, 41s
> agent: error rate back to 0.3%, posting summary to #incidents
Start here, in order:
1. List the 3-5 manual actions your team repeats most (rollbacks, ticket updates, access grants).
2. Write one tool schema per action, plain inputs, one clear purpose each.
3. Put a human-approval gate on anything that touches production or spends money.
4. Log every call: who/what triggered it, inputs, result. Ship the smallest version first.
Agents that only chat aren’t automation
Every team we talk to has some version of the same setup: a chat-based assistant wired up to Slack or an internal dashboard that can read logs, summarize incidents, and draft a fix. It’s useful. It’s also not automation, because the last step, the part where something actually changes in a system, still goes through a human doing it manually. The agent did the thinking. A person still does the work.
That gap matters more than it looks. Teams that stop at “agent as chatbot” get a productivity bump but not a workflow change: on-call still gets paged at 2 a.m. to run the command the agent already recommended, the ticket still needs a human to click “resolve,” the access request still sits until someone remembers to approve it in the actual system. The fix isn’t a smarter model. It’s giving the agent a defined, scoped set of actions it’s allowed to take, and a way to call them.
Automated Custom Actions: closing the last mile
This is the workflow worth naming: Automated Custom Actions, custom tool or function definitions that expose a narrow slice of your internal systems (a deploy script, a ticketing endpoint, an access-management API) to an agent as a callable action with a strict schema. The agent doesn’t get a shell. It gets a menu of pre-approved verbs, each with typed inputs, a description the model can reason about, and, for anything risky, a human-approval gate before it executes.
Done well, this turns “the agent suggested rolling back” into “the agent rolled back, with an audit trail,” while keeping a human in the loop wherever the blast radius matters. Done poorly (broad tools, no approval gate, no logging) it turns your agent into an unsupervised admin account, which is its own incident waiting to happen.
What a “tool” actually is
Strip away the framework branding and a tool is three things: a schema describing what inputs it takes, a description the model uses to decide when to call it, and a handler function that runs server-side and returns a result. The model never executes code directly, it emits a structured request, your infrastructure validates and runs it, and the result goes back into the conversation. That boundary is what makes this safe to build incrementally.
How it’s built
The mechanics are largely interchangeable across vendors at this point. Anthropic’s tool use API and OpenAI’s function calling both work the same way: you register a JSON schema, the model decides when to invoke it, your code executes it. If you want tools shared across multiple agents or projects instead of redefined per-app, the Model Context Protocol (MCP) standardizes that as a server your agents connect to. Orchestration frameworks like LangChain and CrewAI add scaffolding on top (routing, memory, multi-agent handoffs) but the core unit, one schema plus one handler, is the same everywhere. Pick based on what your team already runs, not based on which one is trending.
Where this pays off first
- Incident response: restart a service, roll back a release, or bump a rate limit, gated behind on-call approval instead of a runbook someone has to open and follow by hand.
- Ticketing and handoffs: update status, reassign, or close out tickets automatically once the underlying condition (build passed, alert cleared) is actually true.
- Access requests: grant time-boxed, scoped access through your existing IAM API, with the approval and expiry built into the tool call, not a Slack thread someone forgets to close.
- Config and infra changes: apply a pre-validated Terraform plan or feature-flag toggle instead of pasting a diff into a PR description and waiting.
Guardrails before you wire it up
None of this is safe by default. Scope every tool to the smallest useful action, never “run arbitrary shell command.” Require explicit approval for anything that touches production, spends money, or deletes data. Log every call with who or what triggered it, the inputs, and the result, treat it like any other privileged action in your audit trail. And test each tool against a staging environment before an agent ever gets to call it against prod. The OWASP Top 10 for LLM Applications is a solid checklist for what can go wrong when tool access is scoped too broadly.
FAQ
Is this the same as MCP (Model Context Protocol)?
MCP is one way to package and share tools across multiple agents or apps instead of redefining them per project. The underlying concept, a schema plus a handler, is the same whether you use MCP, native function calling, or a framework’s tool abstraction.
Do agents need write access to production to be useful?
No. Most of the value comes from read-and-recommend plus a narrow set of gated write actions. Start with tools that require approval, expand the auto-approved surface only after you trust the logs.
How do I test a new tool safely before an agent uses it live?
Run it against staging first, feed the agent synthetic scenarios that should and shouldn’t trigger the tool, and check the schema rejects malformed inputs before any handler code runs.
Want this wired into your team’s actual stack instead of a demo? Tha-Shed’s DevOps and AI agent courses walk through building and gating custom tools step by step, including the DevOps Boot Camp track for teams standardizing this across a whole pipeline.
