Run OpenAI computer use on Orgo

TL;DR

With OpenAI's computer tool on the Responses API, the model looks at a screenshot and sends back clicks, keystrokes, and scrolls. Your code carries them out and returns a new screenshot. Run that loop against an Orgo computer and the agent works on a full cloud desktop instead of your laptop (OpenAI computer use guide). Run every action in a computer_call batch in order, then send back one unresized screenshot with "detail": "original" so the model's coordinates match the computer's pixels. If you would rather not write the loop, Orgo's computer use API runs it for you.

What is OpenAI computer use on Orgo?

OpenAI's computer tool lets a model operate a desktop through the Responses API. The model reads a screenshot and returns actions. Your application performs them and sends back a screenshot of the result (OpenAI computer use docs).

Orgo provides the computer: a full cloud desktop with a browser, reachable over HTTP and VNC (Introduction). Its HTTP API has endpoints to take screenshots, click, move the pointer, drag, scroll, press keys, and type (API reference). In this setup, Orgo is not the agent. OpenAI's model decides what to do, and the Orgo computer is where the actions happen.

For the same setup with Claude, see the Claude computer use guide.

Should Orgo run the loop, or should you?

There are two ways to put GPT on an Orgo computer (OpenAI computer use guide):

The rest of this post is the second column. For the first, see Orgo computer use API.

What do you need?

• Python 3.10 or newer. The example uses match.
• An Orgo account and an Orgo API key. The script creates the computer on your plan, so a paid plan needs room for one more computer. A free account has no plan computers. With a free account, add "billing_mode": "hourly" to the create request, and Orgo launches one Medium Linux computer billed by the hour from your signup credit (Create computer).
• An OpenAI API key for a project with access to a model that supports the computer tool.

pip install openai requests
export OPENAI_API_KEY=your_openai_api_key
export ORGO_API_KEY=your_orgo_api_key

How do you run the agent loop?

The loop has five steps (OpenAI computer use guide):

1. Send the task. Call the Responses API with tools=[{"type": "computer"}].
2. Read the actions. The response holds a computer_call with a batch of actions.
3. Run them. Carry out every action in the batch, in order, on the Orgo computer.
4. Send a screenshot. Return one screenshot as a computer_call_output, chained with previous_response_id.
5. Repeat. Stop when a response has no computer_call. Its output_text is the answer.

This script creates a computer, runs the loop, and deletes the computer when it finishes. It calls the Orgo API over HTTP, so it needs no Orgo SDK. It is written for a Linux computer, which is what POST /computers creates by default.

import os
import time
 
import requests
from openai import OpenAI
 
API = "https://www.orgo.ai/api"
MODEL = "gpt-6.1-sol"
TOOLS = [{"type": "computer"}]
 
orgo = requests.Session()
orgo.headers["Authorization"] = f"Bearer {os.environ['ORGO_API_KEY']}"
 
 
def orgo_api(method, path, **kwargs):
    response = orgo.request(method, API + path, timeout=60, **kwargs)
    response.raise_for_status()
    return response.json()
 
 
# The model names keys like "ENTER" and "CTRL". Orgo presses X11 keysyms.
KEYSYMS = {
    "ENTER": "Return", "RETURN": "Return", "ESC": "Escape", "ESCAPE": "Escape",
    "BACKSPACE": "BackSpace", "TAB": "Tab", "SPACE": "space", "DELETE": "Delete",
    "ARROWUP": "Up", "ARROWDOWN": "Down", "ARROWLEFT": "Left", "ARROWRIGHT": "Right",
    "UP": "Up", "DOWN": "Down", "LEFT": "Left", "RIGHT": "Right",
    "PAGEUP": "Page_Up", "PAGEDOWN": "Page_Down", "HOME": "Home", "END": "End",
    "CTRL": "ctrl", "CONTROL": "ctrl", "SHIFT": "shift", "ALT": "alt",
    "META": "super", "CMD": "super", "SUPER": "super",
}
 
 
def keysym(key):
    # Lowercase single letters: to X11, "ctrl+L" means ctrl+shift+l.
    return KEYSYMS.get(key.upper(), key.lower() if len(key) == 1 else key)
 
 
def act(computer_id, action):
    """Run one action from a computer_call on the Orgo computer."""
    def do(endpoint, **body):
        orgo_api("POST", f"/computers/{computer_id}/{endpoint}", json=body)
 
    match action.type:
        case "click":
            button = {"wheel": "middle"}.get(action.button, action.button)
            do("click", x=action.x, y=action.y, button=button)
        case "double_click":
            do("click", x=action.x, y=action.y, double=True)
        case "move":
            do("mouse-move", x=action.x, y=action.y)
        case "drag":
            start, end = action.path[0], action.path[-1]
            do("drag", start_x=start.x, start_y=start.y, end_x=end.x, end_y=end.y)
        case "scroll":
            if action.scroll_y:
                direction = "down" if action.scroll_y > 0 else "up"
                do("scroll", x=action.x, y=action.y, direction=direction,
                   amount=max(1, abs(action.scroll_y) // 100))
        case "keypress":
            do("key", key="+".join(keysym(k) for k in action.keys))
        case "type":
            do("type", text=action.text)
        case "wait":
            time.sleep(2)
        case "screenshot":
            pass  # The loop sends a fresh screenshot after every batch.
 
 
def screenshot(computer_id):
    shot = orgo_api("GET", f"/computers/{computer_id}/screenshot",
                    params={"response_format": "base64", "format": "png"})
    return shot["image"]
 
 
def run(task, max_turns=30):
    client = OpenAI()
    workspace_id = orgo_api("GET", "/workspaces")["workspaces"][0]["id"]
    computer = orgo_api("POST", "/computers", json={
        "workspace_id": workspace_id, "name": f"gpt-{int(time.time())}", "ram": 4, "cpu": 1,
    })
    computer_id = computer["id"]
    print(f"Watch live: {computer['connection_url']}")
 
    try:
        response = client.responses.create(model=MODEL, tools=TOOLS, input=task)
        for _ in range(max_turns):
            call = next((item for item in response.output if item.type == "computer_call"), None)
            if call is None:  # No more actions: the task is done.
                print(response.output_text)
                return
 
            # OpenAI can attach safety checks to a call. A person decides whether to go on.
            checks = call.pending_safety_checks or []
            for check in checks:
                print(f"Safety check: {check.message}")
            if checks and input("Continue? [y/N] ").strip().lower() != "y":
                return
 
            for action in call.actions:
                act(computer_id, action)
            time.sleep(1)  # Let the screen settle.
 
            output = {
                "type": "computer_call_output",
                "call_id": call.call_id,
                "output": {
                    "type": "computer_screenshot",
                    "image_url": f"data:image/png;base64,{screenshot(computer_id)}",
                    "detail": "original",
                },
            }
            if checks:
                output["acknowledged_safety_checks"] = [c.model_dump(exclude_none=True) for c in checks]
            response = client.responses.create(
                model=MODEL, tools=TOOLS, previous_response_id=response.id, input=[output],
            )
        print(f"Stopped after {max_turns} turns.")
    finally:
        orgo_api("DELETE", f"/computers/{computer_id}")
 
 
if __name__ == "__main__":
    run("Open Chrome, go to news.ycombinator.com and tell me the top 3 stories")

Save it as agent.py and run python agent.py. Open the printed link to watch the agent work. To keep the computer afterwards, remove the DELETE call in finally. To reuse it later, change run() to take its ID instead of creating a new computer.

How does each action map to Orgo?

OpenAI's computer tool can return click, double_click, drag, move, scroll, keypress, type, wait, or screenshot (OpenAI computer use docs). Each one maps to one Orgo endpoint (OpenAI computer use guide):

How do coordinates and screenshots work?

The computer tool takes no display size. The model reads the size from the screenshots you send and returns x and y in those same pixels. Send each screenshot unresized with "detail": "original", and the coordinates are already the computer's pixels (OpenAI computer use guide).

Do not assume one screen size. A Linux computer created without a template boots at 1280x720, but some templates, and Mac computers, boot at 1920x1080. The base64 screenshot response includes width and height (Screenshot), and GET /computers/{id}/screens lists the size of each screen (List screens).

If you downscale a screenshot and click the raw coordinates, every click lands short of where the model aimed. Nothing errors. The agent looks like it is confidently clicking the wrong thing. If you must resize, keep the scale factor and map each coordinate back to the original resolution.

How do you move off computer_use_preview?

OpenAI shut down the computer-use-preview model on July 23, 2026 (OpenAI deprecations). An integration still built on it needs to move to the computer tool:

The table follows OpenAI's migration notes and Orgo's OpenAI computer use guide. The biggest code change is the executor: loop over actions instead of reading one action, and send one screenshot after the whole batch. Delete any hard-coded display size, which a preview integration on Orgo had to keep in step with the screen.

Why run OpenAI computer use on Orgo instead of your own machine?

Use Orgo when:

• You want the agent on its own machine. OpenAI's guidance is to run computer use in an isolated browser or VM (OpenAI computer use docs). The agent's clicks, downloads, and installs happen on that one Orgo computer instead of your laptop, and deleting the computer removes them.
• You want a full desktop the model can click and type into, not only a browser tab (Introduction).
• You want files and installed apps to stay between tasks. On a paid plan, an Orgo computer runs until you stop or delete it and keeps its disk, so you can reuse one computer ID across tasks (Orgo pricing).
• You may run other agents later. The same computers work with Claude, Gemini, Hermes Agent, or OpenClaw (Introduction).

Stay local when:

• You are debugging the loop against one trusted screen and do not need isolation.
• You only need a browser tab, not a computer. A browser tool may fit better (Cloud desktops vs. headless browsers).

How should you think about safety?

The model acts on whatever the screen shows. As of October 2026, OpenAI's guidance is to restrict the environment to the sites and actions the task needs, treat screen content as untrusted, confirm consequential actions such as purchases or sending data, and bound and verify each run (OpenAI computer use docs).

An Orgo computer covers the isolation part. It does not replace your allow lists, approvals, or step limits. The example above stops after 30 turns and asks a person before it acknowledges any pending_safety_checks OpenAI returns.

FAQ

Is Orgo an OpenAI product?

No. Orgo provides the computer. OpenAI provides the model and the computer tool. Your code, or Orgo's computer use API, connects them (Introduction).

Which OpenAI model should I use?

As of October 2026, OpenAI's computer tool examples use gpt-6.1-sol, and GPT-6 Astra also supports the tool. For GPT-6 Astra, OpenAI recommends a code execution setup, where the model writes PyAutoGUI or Playwright code, and keeps the computer tool as a supported alternative (OpenAI computer use docs). Check the model's supported tools in OpenAI's models catalog before you switch.

Why do my clicks miss?

Coordinates are in the pixel space of the image you sent. If you downscaled the screenshot, map each coordinate back to the original size before you click. The example sends screenshots at full size, so its coordinates need no mapping.

Does this work on Windows?

Not unchanged. The example is written for Linux. On a Windows computer, key names follow Windows naming, and scroll takes only direction and amount and scrolls at the pointer's current position (Press key, Scroll).

How is it billed?

OpenAI bills the model calls to your OpenAI account. Orgo bills the computer: on a paid plan it counts toward the plan, and a free account's hourly computer draws on its signup credit. Plans are in Orgo's pricing, and as plain text at /pricing.md.

Related

• What is computer use?
• Orgo computer use API
• Computer use models compared
• Gemini computer use with Orgo
• Cloud desktops vs. headless browsers
• Docs: OpenAI computer use, Claude computer use, Quickstart