How to Build Tool-Using AI Workflows with the Responses API

An AI model cannot know the current state of your orders, files, or internal systems unless your application gives it a way to check.

The model might need to search a product catalog, check an order, run a calculation, read documentation, or prepare a change for your application. Instead of pretending it knows the result, it can request a tool, receive the real output, and use that information in its final answer.

NanoGPT's OpenAI-compatible Responses API now supports two useful ways to build that workflow:

• Custom tools, where your application runs the action and sends the result back
• Public remote MCP servers, available on supported direct OpenAI Responses models, where the model can discover and use tools exposed by an external service

Custom-tool calls and their outputs can now make a complete round trip through /api/v1/responses, including streaming requests and stored response chains.

What makes the Responses API useful for tools?

Chat Completions is organized around messages. The Responses API uses a broader set of items: messages, reasoning, tool calls, tool results, and other model actions can each be represented separately.

That makes a multi-step workflow easier to follow. Your application does not have to hide a tool request inside an assistant message or invent its own transcript format. The model can return a specific tool-call item, and your application can answer it with the matching result.

OpenAI describes the Responses API as an agent-oriented interface for combining model output, tools, and state. NanoGPT keeps that familiar request and response shape while offering access through the same API key and balance used for other supported models. Compatibility is feature- and model-specific, so check NanoGPT's current reference rather than assuming every OpenAI Responses feature works on every model.

How a custom-tool round trip works

A custom tool belongs to your application. NanoGPT does not run it; your application does.

Take a support assistant with a tool named check_order. The flow looks like this:

1. Your application tells the model which tools are available and what each one does.
2. The user asks, “Has order 1234 shipped?”
3. The model returns a tool call rather than guessing.
4. Your application validates the request, looks up the order, and sends the result back.
5. The model uses that result to write the final answer.

A custom-tool call may look like:

{
  "type": "custom_tool_call",
  "call_id": "call_1234",
  "name": "check_order",
  "input": "1234"
}

After running the lookup, your application returns a matching output:

{
  "type": "custom_tool_call_output",
  "call_id": "call_1234",
  "output": "Shipped on July 20; tracking number available."
}

The call_id connects the result to the request. The output can be text and can also use supported image or file result formats for the model to inspect.

Custom tools versus function tools

Function tools are usually the better choice when the input fits a small, predictable structure. A weather lookup might accept a city and country as JSON fields. Your application can validate those fields before doing anything.

Custom tools use freeform text instead. That is useful when the natural input is a code patch, query language, command, document transformation, or another format that would be awkward to squeeze into a collection of JSON fields.

Use a function tool when a strict schema makes the action safer and clearer. Use a custom tool when the tool genuinely needs a larger freeform instruction. Do not choose freeform input merely to avoid defining validation rules; your application still needs to decide what is allowed.

What MCP changes

MCP stands for Model Context Protocol. It gives a service a standard way to publish a catalog of tools that compatible AI clients can understand.

Instead of defining and running every tool inside your own application, you can point the Responses API at a remote MCP server. The model can inspect the available tools, choose one when appropriate, and use its result while preparing the response.

A remote MCP tool entry uses this general shape:

{
  "type": "mcp",
  "server_label": "product_docs",
  "server_url": "https://example.com/mcp",
  "allowed_tools": ["search_docs"],
  "require_approval": "never"
}

allowed_tools is worth using even when you trust the server. A smaller tool list gives the model a clearer choice and reduces the chance that an unrelated action is selected.

For more background on the protocol and its risks, see OpenAI's MCP guide.

NanoGPT's current MCP limits

Remote MCP support is intentionally narrow at launch:

• It works with supported direct OpenAI Responses models.
• Requests containing MCP authorization, custom headers, or a connector are rejected. Only public, unauthenticated servers work.
• require_approval must be set to never because approval continuations are not supported yet.

That last point should shape which MCP servers you use. never means the request cannot pause and ask a person to approve a tool call. Only connect tools that are safe to run automatically in the scope you provide.

Tool arguments are sent to the third-party MCP server. Do not build a public-MCP workflow that passes personal data, credentials, proprietary code, or other sensitive material to that server.

A public documentation search or read-only reference tool can be a sensible starting point. A tool that deletes data, sends money, publishes content, or changes account permissions is not.

Streaming does not remove the tool loop

Streaming lets your application receive response events as they happen. For custom tools, this can include pieces of the tool input before the complete call is ready.

Do not run a tool from an incomplete fragment. Wait for the completed call, validate its name and input, execute it once, then return the result using the matching call_id.

Streaming is most useful when a model may spend time reasoning or preparing a large freeform tool input. It improves visibility and responsiveness, but it does not make an incomplete call safe to execute.

Continue with or without stored responses

NanoGPT does not store Responses API content by default. You can keep the workflow stateless by sending the required messages, tool calls, and outputs with each request.

If you set store: true, you can continue a chain with previous_response_id instead of replaying the entire earlier response yourself. NanoGPT's stored Responses data is encrypted and defaults to seven days of retention; the retention can be configured from 0 to 365 days. A value of 0 disables storage for that request.

Stored chains are convenient when a tool workflow spans several calls. Stateless requests offer tighter control over what is retained. Do not turn on storage only to make requests shorter; decide whether the convenience is worth retaining the chain.

The NanoGPT Responses API documentation covers storage, retention, and optional user-provided encryption keys.

Treat tool calls as requests, not permission

A model choosing a tool does not authorize the action. Your application remains responsible for deciding whether it should run.

For custom tools:

• Allow only known tool names.
• Validate every input, including freeform text. Set length limits, allowlist the permitted operation and target, and reject anything outside the format your tool expects.
• Check the user's permissions again before reading or changing data.
• Make write actions idempotent so a retry cannot apply them twice.
• Set timeouts and return clear, limited error messages.
• Do not send stack traces, credentials, or internal connection details back to the model.
• Log the tool name and status, but exclude secrets and sensitive user data.

For MCP servers:

• Prefer servers operated by the service they represent.
• Inspect the published tool list before enabling the server and test it against a non-production account or dataset first.
• Pin allowed_tools to the smallest set your workflow needs.
• Remember that data sent to the server follows that server's privacy and retention policies.
• Treat tool output as untrusted content that may contain misleading or malicious instructions.

These controls matter even for a read-only workflow. A compromised server can return false information, and a prompt-injection attack can try to steer the model toward the wrong tool or data.

When MCP is the better fit

Use a remote MCP server when a trusted public service already exposes a useful tool catalog and the available actions are safe to run without an approval step. Keep the tool in your own application when you need authentication, a human approval, private network access, or tighter control over execution.

Use no tool at all when the model can answer from the information already in the request. Every tool adds another system, permission boundary, failure mode, and possible data path.

The Responses API provides the common request format. Your application still owns tool execution, permissions, validation, state, and retries.

Create a NanoGPT API key or read the Responses API reference.