Harness engineering is the practice of building structured, safe environments for AI agents to execute code. It's not about prompt engineering or fine-tuning. It's about creating a sandbox where an agent like Codex can act autonomously without breaking your production systems.
OpenAI's recent announcement positions Codex as an agent-first tool. But running an agent in production requires more than an API key. You need a harness: a layer that controls permissions, manages state, handles errors, and enforces budgets.
What is a harness in the context of Codex?
A harness is a wrapper around an agent that provides:
- Execution environment: A sandboxed runtime (e.g., Docker container, Firecracker microVM) where the agent can run code.
- Permission model: Scoped access to files, network, and APIs. The agent can only do what you explicitly allow.
- State management: Persistence of conversations, tool outputs, and intermediate results.
- Cost control: Token limits, step limits, and budget caps per session.
- Observability: Logging, tracing, and monitoring of every action the agent takes.
Without a harness, an agent is a liability. With one, it becomes a deployable service.
How does harness engineering work with Codex?
Let's walk through a concrete example. Suppose you want an agent that can fix bugs in your codebase. The agent receives a bug report, reads relevant files, writes a fix, runs tests, and creates a PR.
Here's a simplified harness structure:
project/
├── harness/
│ ├── sandbox.py # Docker-based execution
│ ├── permissions.py # File/network access rules
│ ├── state.py # Session state management
│ ├── cost_controller.py # Token and step limits
│ └── observer.py # Logging and metrics
├── agents/
│ ├── bug_fixer.py # Agent logic (prompts + tool definitions)
│ └── codex_client.py # OpenAI API wrapper
└── main.py # Entry point
Step 1: Define tools
Codex agents use function calling. You define tools like read_file, write_file, run_test, create_pr. Each tool has a schema and a handler.
tools = [
{
"name": "read_file",
"description": "Read the contents of a file.",
"parameters": {
"type": "object",
"properties": {
"path": {"type": "string"}
},
"required": ["path"]
}
},
{
"name": "write_file",
"description": "Write content to a file.",
"parameters": {
"type": "object",
"properties": {
"path": {"type": "string"},
"content": {"type": "string"}
},
"required": ["path", "content"]
}
},
{
"name": "run_test",
"description": "Run a test command in the sandbox.",
"parameters": {
"type": "object",
"properties": {
"command": {"type": "string"}
},
"required": ["command"]
}
},
{
"name": "create_pr",
"description": "Create a pull request on GitHub.",
"parameters": {
"type": "object",
"properties": {
"title": {"type": "string"},
"body": {"type": "string"},
"branch": {"type": "string"}
},
"required": ["title", "body", "branch"]
}
}
]
Step 2: Implement the harness sandbox
Each tool handler runs inside a sandbox. For example, run_test executes in a Docker container with limited CPU and memory, no network access except to a local test database.
# sandbox.py
import docker
client = docker.from_env()
def run_in_sandbox(command: str, image: str = "python:3.11-slim") -> str:
container = client.containers.run(
image=image,
command=["sh", "-c", command],
mem_limit="512m",
network_disabled=True,
remove=True,
stdout=True,
stderr=True
)
return container.decode("utf-8")
Step 3: Wire it together
The main loop sends the user request plus tool definitions to Codex. Codex returns a function call. The harness executes the function in the sandbox, returns the result, and repeats until the task is done or limits are hit.
# main.py
from openai import OpenAI
from harness.sandbox import run_in_sandbox
from harness.cost_controller import check_limits
client = OpenAI()
def run_agent(user_request: str):
messages = [{"role": "user", "content": user_request}]
while True:
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools,
tool_choice="auto"
)
choice = response.choices[0]
if choice.finish_reason == "stop":
return choice.message.content
elif choice.finish_reason == "tool_calls":
for tool_call in choice.message.tool_calls:
func_name = tool_call.function.name
args = json.loads(tool_call.function.arguments)
# Execute in sandbox
if func_name == "run_test":
result = run_in_sandbox(args["command"])
elif func_name == "read_file":
result = read_file_sandboxed(args["path"])
# ...
messages.append({"role": "tool", "tool_call_id": tool_call.id, "content": result})
else:
break
