Skip to main content
Version: next

Call the sandbox API from your own code

The sandbox is also reachable from your own code over gRPC (sandbox/v1):

eval "$(mocactl sbx env)"

prints, for the bound sandbox, export lines for a client: SBX_ADDR, SBX_TLS, SBX_SANDBOX_ID, SBX_WORKSPACE_KEY, SBX_TOKEN_CMD and, when the endpoint's certificate is self-issued, SBX_CA_FILE. mocactl sbx token prints a fresh token; mocactl sbx token --renew-every 600 prints a new one every 600 seconds, one line each, until stopped, for a long-lived client that reads its next token from a pipe. With --json, each line is an object with the token, its expiresAt and the sandbox's sandboxExpiresAt (epoch seconds, as the control plane returns them; a control plane from before sandboxExpiresAt leaves it out). Each token printed counts as use of the sandbox, so a --renew-every pump keeps it alive whether or not its client runs commands. The rest of this page is what a client sends.

How to run commands in a moca sandbox service sandbox from your own code (a script, an SDK, a local tool) by calling SandboxExec over gRPC. mocactl sbx is the supported command line; this guide is for everything else. For the control-plane calls behind each mocactl sbx command, and one table of errors and retry rules across REST, SSE and gRPC, see Build a client.

What it is​

proto/sandbox/v1/sandbox.proto is one self-contained file with no imports. A client uses one service, served by the sandbox service's relay:

service SandboxExec {
rpc Exec(ExecRequest) returns (stream ExecEvent); // run one command, stream its output
rpc Abort(AbortRequest) returns (AbortResponse); // cancel a running command
rpc LeaseControl(LeaseControlRequest) returns (LeaseControlResponse); // lease mode only
rpc LeaseKeepalive(LeaseKeepaliveRequest) returns (LeaseKeepaliveResponse); // lease mode only
}

The two lease calls matter only for a sandbox in a lease mode: see Lease mode. At lease+processes, an Exec can also start and manage a tracked process instead of running a command: see Tracked processes.

SandboxWorker.Attach in the same file is the workers' side; clients never call it.

A call:

ExecRequest { sandbox_id, exec: Exec { req_id, command, stdin, timeout_s, streaming, workspace_key } }
-> stream ExecEvent chunk { req_id, data, stream: STDOUT | STDERR } zero or more
lease { req_id, state, reason, at_unix_ms, ... } lease mode only
end { req_id, exit_code, truncated } then exactly one
error { req_id, message } of end / error
  • Every Exec runs bash -c <command> with /workspace as the working directory, in a fresh Firecracker VM. That is the default mode, per-exec; in a lease mode the VM is kept between Execs (Lease mode).
  • Files under /workspace persist between calls, for the sandbox's lifetime. Nothing else does: no process, and no file outside /workspace, survives the call.
  • There is no network device in the VM. A template with egress reaches only the hosts the operator allowlisted, through a proxy on the host; any other host answers 403 destination_denied.
  • There is no file RPC. Files are commands: write with cat > path and the bytes on stdin, read with cat -- path.

Getting access​

mocactl sbx login --control-plane-url https://<api host> # GitHub device flow, once
mocactl sbx templates # what a sandbox can be created from
mocactl sbx create --template <id> # prints the sandbox id and its expiry
eval "$(mocactl sbx env <sandbox id>)"

mocactl sbx env prints export lines for a direct client, never a token:

VariableIs
SBX_ADDRWhere to dial: <exec host>:443
SBX_TLS1: always TLS
SBX_CA_FILEOnly when the exec endpoint's certificate is self-issued: the CA to trust, as a PEM file
SBX_SANDBOX_IDThe worker serving your sandbox's template (the token's wrk), not your sandbox id
SBX_WORKSPACE_KEYYour sandbox's workspace (the token's wsk)
SBX_TOKEN_CMDThe command that prints a fresh token: mocactl sbx token <sandbox id>

SBX_TOKEN_CMD may also carry --control-plane-url '<url>', when the URL came from a flag or MOCACTL_SBX_URL, so that it works in a shell that has neither. Run it with a shell-style split, as the example client does.

What a client sets​

  • TLS to SBX_ADDR: against the default roots, or, when SBX_CA_FILE is set, against that CA only. In grpcio, root_certificates replaces the default roots rather than adding to them.
  • authorization: Bearer <sandbox token> as metadata on every call, Abort included.
  • ExecRequest.sandbox_id = SBX_SANDBOX_ID and Exec.workspace_key = SBX_WORKSPACE_KEY, exactly. The relay compares both for equality with the token's wrk and wsk: never a prefix, never a case-folded match.
  • req_id unique within the sandbox, and below 2^53. The proto says uint64, but the relay decodes it as a JavaScript number and refuses larger values. The relay namespaces req_id per sandbox, so your ids cannot collide with another user's, and Abort reaches only your own Execs. Use a fresh random 53-bit id for every call. Reusing an id that is still in flight in your sandbox is refused with ALREADY_EXISTS. Reusing the id of a call that has completed replays it: the worker keeps its last 256 completed results, keyed by id plus a fingerprint of command, stdin, timeout_s and streaming, and answers a match with the cached end (or timeout error) without running the command or sending its output again. The one deliberate reuse is the retry after UNAUTHENTICATED (see Tokens), and nothing ran before it.
  • timeout_s: the worker kills the command when it runs out. Always send it. The relay passes it through unchanged, and the worker treats 0 or an absent value as its own default, 30 minutes, which is longer than a 15-minute token. Send at most what the token has left minus 60 s, and never more than its lifetime (exp βˆ’ iat) minus 60 s: 840 s with the default tokens. Otherwise a command that runs past the token's exp is cancelled at exp + 30 s with DEADLINE_EXCEEDED, possibly after it has already changed the workspace. See Tokens.
  • streaming: true, so output arrives as it is produced. false buffers it until the end.
  • Keepalive every 20 s (grpc.keepalive_time_ms = 20000, and permitted without calls). The relay pings too. Without it, a command that prints nothing for minutes can lose its connection to a load balancer's idle cut.
  • stdin: raw bytes, sent once with the call. Keep it well under the 16 MiB message limit.

Tokens​

A sandbox token is an Ed25519-signed JWT (typ: moca-sandbox+jwt) bound to one sandbox: its wrk, its wsk, and the exec scope.

  • It lives 15 minutes by default (the deployment sets it), and never past the sandbox's own expiry.
  • Get one by running SBX_TOKEN_CMD. Keep it in memory; never write it to disk or a log.
  • Run the command again:
    • when the token is within 60 s of its exp;
    • once, on UNAUTHENTICATED, if no event of that call had arrived yet; then retry the call with the same req_id;
    • before an Exec that needs more time than the token has left.
  • Never re-run a command that started. An UNAUTHENTICATED (or any failure) after the first event means the command may have run; report it instead.

No Exec outlives its token. The relay checks the token when it accepts an Exec, and cancels the Exec at the token's exp (plus a 30 s leeway), ending the call with DEADLINE_EXCEEDED. So, before each Exec, a client:

  1. makes sure its token has at least timeout_s + 60 s left, and mints a fresh one if not;
  2. sends a timeout_s of at most what the token has left minus 60 s, so that the worker's own timeout fires before the relay's cancel. When it cuts a longer request, it says so.

Asking for the whole cap on every call would mint a token before every call but the first, since a held token always has a little less than its lifetime left. The example client therefore keeps a held token for a call that names no timeout while the token has at least half the cap plus 60 s left (480 s by default), and cuts that call's timeout to what the token has left minus 60 s. A call that names a timeout the held token cannot cover still gets a fresh token.

The longest single command is therefore one token lifetime. Split longer work into several calls.

A mint also moves the sandbox's deadline: to at least now plus the sandbox's idle lifetime (idleLifetimeS), never backwards. Send {"activity": false} as the body for a mint that runs no command, such as a keepalive, so that it does not count. A client that mints without running commands should always send it. Each mocactl sbx token and each --renew-every mint counts as use: a token pump keeps the sandbox alive whether or not its client runs commands, and at a 30-day lifetime the margin before the host reclaims an unreachable workspace is 2 days (spec Β§7). The token response's sandboxExpiresAt is the deadline after the mint; track it, since it moves. mocactl sbx token --json prints it beside the token and its expiresAt.

Delete and de-enrollment are not immediate. A token already issued keeps working until its exp, plus the relay's 30 s leeway: up to one token lifetime plus 30 s (15.5 minutes by default) after mocactl sbx rm, or after an operator removes your account. An Exec still running at that point is cancelled (DEADLINE_EXCEEDED). No new token is issued after either.

You can decode the token's payload (base64url, the middle segment) to read iat and exp; do not verify it, which is the relay's job.

Outcomes versus failures​

The command's outcome:

  • end ends a command that ran, whatever its exit code. A non-zero exit_code is an outcome, not an error.
    • exit_code < 0: the command was killed by a signal (your abort, an out-of-memory kill).
    • truncated: stdout went over the 8 MiB cap, and the rest was dropped.
  • An error event whose message is timeout:<n> means the command ran out of timeout_s (n seconds). That is the command timing out, not a failure of the sandbox; sbx.py exits 124 for it. The call then ends with INTERNAL, which a client ignores once it has seen the timeout.

The machinery's failure: any other error event (the VM did not boot, the worker went away), or a non-OK status. There is no exit status to report. Two have a sandbox-service meaning of their own:

  • DEADLINE_EXCEEDED, "sandbox token expired during the Exec": the command outlived its token and was cancelled. Give it a shorter timeout, or split it.
  • RESOURCE_EXHAUSTED, "sandbox busy: <n> Execs already in flight": the sandbox already has its cap of Execs in flight (2 by default: one running, one waiting). Nothing ran; retry once one finishes.

gRPC status codes:

CodeCause
UNAUTHENTICATEDMissing, expired or invalid token. Refresh once, as above
PERMISSION_DENIEDsandbox_id is not the token's worker; workspace_key is not its workspace; or Abort of an Exec that is not yours, or has finished
ALREADY_EXISTSThe req_id is already in flight in this sandbox
RESOURCE_EXHAUSTEDThe sandbox is busy
DEADLINE_EXCEEDEDThe token expired during the Exec
INVALID_ARGUMENTNo exec in the request, or req_id out of range
UNKNOWN today (UNAVAILABLE is reserved) with no live worker for...The template's worker is not attached: the template is unavailable (Exec; LeaseControl answers UNAVAILABLE)
INTERNALAn error event, as above, including a timeout

Lease mode​

A sandbox in a lease mode keeps its VM between Execs, so the page cache stays warm and /tmp and /var survive until the lease ends. Every process is still killed at the end of each Exec, and each Exec gets a fresh shell. The design is the lease-mode spec; the modes are per-exec (the default), lease and lease+processes, and the deployment decides which of them a user may pick. A sandbox's mode is set on the control plane: mode and idleTimeoutS on POST /v1/sandboxes, and PUT /v1/sandboxes/{id}/mode to change them (openapi.yaml).

The mode reaches the worker in your token. A sandbox token carries the claims mode, mode_epoch and idle_timeout_s, and the relay writes them into every Exec it routes, over whatever the client sent. Leave Exec.mode, mode_epoch and idle_timeout_s unset. After a mode change, mint a fresh token: the old one still carries the old epoch. For a sandbox created per-exec, a token from before its first mode change carries epoch 0, and it is not refused: the worker serves an epoch-0 Exec per-Exec, and that ends a live lease. A sandbox created in a lease mode starts at epoch 1 (below), so none of its tokens carries 0.

  • Exec.keepalive is the one lease field that is yours. Setting it, or calling LeaseKeepalive, turns the keepalive rule on for the rest of the lease: the lease then ends when no keepalive has arrived for 120 s (the worker's default). Use either only from a program that calls LeaseKeepalive every 30 s while it runs. Without them, the lease ends by its idle timeout, which a keepalive never resets.
  • The first set_mode. A sandbox created in a lease mode is at epoch 1, and the worker runs its Execs per-Exec until a LeaseControl set_mode makes its epoch known. After the create, fetch a set-mode token with POST /v1/sandboxes/{id}/mode-token and send it in a set_mode. Send it again whenever LeaseControl status answers epoch_known: false (after a worker lost its state file).

Lease events. A lease Exec's response can carry ExecEvent.lease events, after the output and before the end or the error. state is start, end (with an end reason such as idle, reset or mode_change) or expiring (the lease ends at its next idle).

  • They are best-effort notices. An event is delivered once, on the next lease Exec's response, and the worker counts it as delivered when it queues it to send, not when you read it. If the connection breaks in between, the event is lost; LeaseControl status still shows the lease's state. Rarely, the same event arrives twice: dedupe on at_unix_ms.
  • expiring repeats on every lease Exec until the lease ends, with the same at_unix_ms (the lease's start plus its maximum duration) and reason max_duration. Its ends_by_unix_ms is the lease's hard end, when it ends even mid-Exec: at_unix_ms plus the grace limit (30 minutes by default). It is 0 on every other state, and from a worker that predates the field.
  • mocactl sbx and its MCP server show each event as one line (the copy is in the lease-mode spec Β§5), for example [sandbox] guest restarted (idle, 14:02): ... for an idle end.
  • A per-exec Exec never carries one. A downgrade's mode_change end therefore comes back in the set_mode result's status.pending.

The lease errors, as an error event's message (and in a failed LeaseControl's details):

MessageMeans
lease_ended(<reason>)The lease ended while this Exec was queued or running (grace_limit, reset, mode_change, ...)
stale_mode: <detail>Your token predates a mode change. Mint a fresh token; nothing ran. An epoch-0 token is never refused this way (above)
lease_unsupported: <...>A microVM worker whose snapshot has no lease mode (no lease-v1), or one that does not know the mode; nothing ran

The container tier and workers older than lease mode never refuse a lease-mode Exec: they ignore the mode and run it per-Exec, without lease events. On LeaseControl, the container tier answers lease_unsupported and an older worker does not answer (DEADLINE_EXCEEDED, below).

LeaseControl {sandbox_id, op, set_mode_token} reads the lease (LEASE_OP_STATUS), ends it (LEASE_OP_RESET), applies a mode change (LEASE_OP_SET_MODE), or ends it because you are about to delete the sandbox (LEASE_OP_SANDBOX_DELETED, the end reason sandbox_deleted; send it before DELETE /v1/sandboxes/{id}, as mocactl sbx rm does, so the guest does not idle on after the record is gone). It carries your sandbox token, and set_mode also a set-mode token (typ: moca-sandbox-mode+jwt, valid 300 s) from PUT .../mode or POST .../mode-token. The answer is a LeaseStatus {mode, epoch_known, state, age_ms, idle_timeout_s, pending}, read after the op. reset, sandbox_deleted and a downgrade wait for the lease to end, which can take as long as an Exec in flight needs (up to 30 minutes).

CodeCause
UNAUTHENTICATEDMissing, expired or invalid sandbox token
PERMISSION_DENIEDA harness exec token (the harness never uses leases); sandbox_id is not the token's worker or is a workload sandbox; or the set-mode token was refused (one message, whatever the reason)
INVALID_ARGUMENTNo op
RESOURCE_EXHAUSTEDsandbox busy: the sandbox already has 4 LeaseControl calls in flight. Nothing was sent
FAILED_PRECONDITIONThe worker refused the op; the details are its message (above, or busy: too many lease calls in flight, below)
DEADLINE_EXCEEDEDThe worker did not answer within 15 s (status) or 31 minutes (reset, set_mode, sandbox_deleted): it may predate lease mode
UNAVAILABLEThe template's worker is not attached, or it went away before answering
CANCELLEDYou cancelled the call before the worker answered. The relay frees its place at once, but a reset or set_mode already sent runs on at the worker: read the outcome with status

On a PERMISSION_DENIED from set_mode, fetch a fresh set-mode token once and retry. A worker runs at most 2 reset, set_mode or sandbox_deleted calls per sandbox, and 16 in all, and answers busy past either. It runs at most 32 status calls at once, apart from those, and answers busy past that; calls blocked in a reset or set_mode never make a status busy.

LeaseKeepalive {sandbox_id} answers OK once the token checks pass, even when no worker is attached; it does nothing without a live lease. The relay does not audit it, and it passes on at most one keepalive per sandbox per second: the others are answered OK and dropped. Every 30 s is enough (mocactl sbx mcp sends one every 30 s while its token's mode is a lease mode).

Tracked processes (lease+processes)​

At level 2 a process can outlive the Exec that started it, if it was started as a tracked process. Everything else an Exec starts (&, nohup, setsid) still dies when the Exec ends. The design is the lease-mode spec Β§7; mocactl sbx proc and the MCP server's process tools are built on the calls below.

A process call is an Exec with a kind. Exec.kind (field 11) is an ExecKind, and Exec.process (field 12) its ProcessArgs:

Exec.kindProcessArgs it readsAnswer, on stdout with exit 0
EXEC_KIND_COMMAND (0)nonea command, as before: the default, and what a client that never sets the field sends
EXEC_KIND_PROCESS_STARTname (optional), cwd (in /workspace, the default)JSON {"id", "name", "pid"}. The command is Exec.command, run as bash -c <command>
EXEC_KIND_PROCESS_LISTnonea JSON array of rows, exited processes included
EXEC_KIND_PROCESS_LOGSid or name; tail (0: 200 lines)the last lines of the process's combined stdout and stderr, as text
EXEC_KIND_PROCESS_STOPid or name; signal (empty: TERM)the process's final row, as JSON
  • A row is {"id", "name", "pid", "state", "code", "started_unix_ms", "memory_bytes"}. state is running, exited (code is the exit code) or killed (code is the signal). id is a decimal string; mocactl and the MCP server show it as p<id>.
  • Selecting a process: set id or name. An id wins over an equal name.
  • A name is 1 to 64 characters of [A-Za-z0-9._-], never shaped like an id (^p?[0-9]+$), and unique among the running processes. Starting a name that only an exited process holds replaces that entry. Unnamed is allowed.
  • A process runs in its own cgroup, in its own session, with stdin from /dev/null, under a uid of its own (not the Exec uid), in the requested cwd. Its output goes to a 4 MiB ring in the guest. At most SH_LEASE_MAX_PROCESSES (4 by default) run at once; exited ones do not count, and an exited one stays listed until the lease ends. process_stop sends the signal to everything the process started, and kills what is left 10 s later.
  • Later Execs reach it through localhost ports or files in /workspace. The guest still has no network, so nothing outside the sandbox can.
  • Without a lease, list, logs and stop answer as for no processes, and start no lease.
  • Send a timeout of at least 30 s. The worker raises a lower one to 30 s: a stop can take about 20 s.

The lease reports them. TrackedProcess {id, name, pid, state, code, age_ms, memory_bytes} carries the same rows on the lease messages:

  • LeaseEvent.tracked (field 9), with their display strings, <name> (pid <n>), in LeaseEvent.processes: on an end, the processes the end stopped; on the processes_stopped event, those a downgrade from lease+processes to lease stopped while the lease goes on;
  • LeaseStatus.processes (field 7): the whole table, exited processes included;
  • LeaseStatus.stopped (field 8): in the answer to a set_mode downgrade only, the processes it stopped. It is empty when a lease Exec of yours got there first and did the stop; that Exec then carried the processes_stopped event.

The refusals. The relay's are a gRPC status; the worker's and the guest's arrive as an error event's message, as a lease error does. Nothing ran, and none of them ends the lease:

AnswerMeans
FAILED_PRECONDITION processes_not_enabled: <detail> from the relay, or the same error from the workerthe sandbox's mode is below lease+processes. Your token may predate an upgrade: mint a fresh one and retry once, as mocactl does
INVALID_ARGUMENT unknown exec kind <n>the relay does not know the kind
lease_unsupported: <detail>the worker has no level 2: a container worker, a snapshot without lease-processes-v1, or a kind it does not know
FAILED_PRECONDITION lease_unsupported: this sandbox's worker does not support tracked processes from the relaythe worker does not advertise sbx:lease-processes-v1 (below): nothing reached it
process_not_found: <id or name>no process by that id or name
process_limit: <n> running (SH_LEASE_MAX_PROCESSES)the running cap is reached
process_limit: <n> running, <m> uids held by leftover children of exited processes (process_stop them)no tracked uid is free: leftovers of exited processes hold some; process_stop those entries
process_name_taken: <name>a running process has that name
process_name_invalid: <detail>the name breaks the rule above
process_cwd: <detail>the cwd is not a directory inside /workspace
process_signal: unknown signal <s>signal is not a signal name (with or without SIG) or number

Mixed versions: a process kind is refused, never run as a command. A worker that serves the process kinds says so in its Hello, with the capability sbx:lease-processes-v1: a microVM worker whose snapshot advertises lease-processes-v1. The relay forwards a process kind only to a worker that advertises it, and refuses it otherwise with FAILED_PRECONDITION lease_unsupported: this sandbox's worker does not support tracked processes, before anything reaches the worker. That covers a worker from before level 2, which does not know Exec.kind and would run a process_start's command as a plain command. Nothing ran; retrying does not help until the operator upgrades the worker. Level 2 is off by default: only a deployment that allows lease+processes issues a token whose mode claim is lease+processes.

Releasing the workspace when you delete​

Release {sandbox_id} tells the sandbox's worker that the sandbox has ended: the worker frees its paused standby VMs and its workspace at once, rather than at its idle reclaim, and refuses any later Exec for the workspace for the workspace's idle period (released: …). Send it just before DELETE /v1/sandboxes/{id}, after a lease sandbox's LEASE_OP_SANDBOX_DELETED, as mocactl sbx rm does; best effort, since the delete stands either way and idle reclaim stays the backstop. Never send it while you still mean to use the sandbox, and if the DELETE then fails, retry the delete: the sandbox still exists but takes no more commands (mocactl sbx rm says so). It carries your sandbox token, and the relay sends the token's own workspace key, never one you name. The answer is a ReleaseOutcome: RELEASED (freed), KEPT_BUSY (an Exec or a live lease is using it: the standbys go now, the workspace when it is free) or UNKNOWN_KEY (the worker holds no workspace for it).

CodeCause
UNAUTHENTICATEDMissing, expired or invalid sandbox token
PERMISSION_DENIEDA harness exec token; sandbox_id is not the token's worker or is a workload sandbox
FAILED_PRECONDITIONrelease_unsupported: …: the worker predates release (nothing was sent); don't retry
UNIMPLEMENTEDA relay that predates Release (nothing was sent); don't retry
RESOURCE_EXHAUSTEDbusy: …: 2 releases for this sandbox already in flight, or the worker is saturated; retry later
INVALID_ARGUMENTThe worker refused the workspace key; don't retry
DEADLINE_EXCEEDEDThe worker did not answer within 15 s
UNAVAILABLEThe worker is not attached, went away before answering, or failed the release; retry, or let idle reclaim do it

Limits​

  • One command at a time per sandbox. A second concurrent call waits at the worker until the first finishes; a third is refused as busy.
  • 16 MiB per message, so send large trees as several calls.
  • 8 MiB of stdout per call.
  • The longest command is one token lifetime minus 60 s (14 minutes by default).
  • A sandbox lives 7 days past its last use by default, then it is gone with its workspace. Each mint that counts as activity moves its deadline (see Tokens), on a deployment that sets a maximum lifetime, as deploy/sbx does by default; without one the deadline stays fixed at create. Pull your work out before the deadline. mocactl sbx ls shows each sandbox's expiry.

The stability promise​

Changes to sandbox/v1 are additive only, under ADR-0024's rules: no field is removed or renumbered, and no meaning changes. Nothing else is promised: the relay's status details, the limits above and the token's lifetime may change.

The examples​

examples/sandbox_v1_client.py is a small client with every rule above: TLS with keepalive, a TokenSource that re-runs SBX_TOKEN_CMD, token freshness and the timeout cap, the retry-once-if-not-started rule, and the outcome and failure mapping (SandboxBusy, TokenExpiredDuringExec, SandboxError). Sandbox.from_env() reads what mocactl sbx env prints. SBX_TOKEN, a one-off static token that is never refreshed, is used only when SBX_TOKEN_CMD is unset.

examples/sbx.py is an example of a thin command line on top of it, for one sandbox. It is an example only: mocactl sbx run and put (increment 1c) are the supported CLI.

CommandDoes
sbx run [COMMAND…]Runs COMMAND, or with no arguments the script on stdin, in /workspace. Exits with its exit code: 124 timed out, 125 the sandbox failed, 75 refused at capacity (nothing ran)
sbx put PATHWrites stdin to PATH, creating parent directories
sbx push [DIR]Replaces /workspace with DIR's git working tree and commits it as a baseline
sbx pullPrints every change since the push as a git patch

A token that expired during the Exec also exits 125, with its own message.

A command refused at capacity is retried, then exits 75. The worker refuses an Exec before anything of it runs when its template's memory budget or run count is full (memory-budget:, max-runs:, busy: queue full), and the relay does when the sandbox already has its cap of Execs in flight (RESOURCE_EXHAUSTED sandbox busy). Since nothing ran, sbx sends the command again, each time as a new Exec with a new req_id, after about 1, 2, 4, 8 and then 8 s (with jitter), until SBX_BUSY_RETRY_S seconds of waiting are spent (default 30; 0 turns retries off, and the refusal exits 125 as before). Then it exits 75, nothing having run. A refusal after any output, and every other failure, is never retried. Both sbx.py and the Go sbx do this, and write the same SBX_LOG keys besides {t, op, ms, exit}:

  • attempt: the attempt number, from 1, on every record;
  • refused and reason: a refused attempt's class (memory_budget, max_runs, queue_full, relay_busy) and its text, cut to 300 characters;
  • busy_wait_ms: the total wait between retries, on the call's final record only.

Both need Python 3.10 or later and grpcio. From docs/api/examples:

python3 -m venv .venv && .venv/bin/pip install grpcio grpcio-tools
.venv/bin/python -m grpc_tools.protoc -I ../../../proto --python_out=. --grpc_python_out=. \
sandbox/v1/sandbox.proto # writes sandbox/v1/, which git ignores
eval "$(mocactl sbx env <sandbox id>)"
.venv/bin/python sandbox_v1_client.py # writes a file, reads it back, runs it
.venv/bin/python sbx.py run 'uname -a; ls -la'
.venv/bin/python -m unittest test_sandbox_v1_client # offline: no network, no relay

From the shell, with grpcurl:

REQ_ID="$(od -An -N6 -tu8 /dev/urandom | tr -d ' ')" # a fresh random 48-bit id for every call
SBX_BEARER="$(eval "$SBX_TOKEN_CMD")" grpcurl -import-path proto -proto sandbox/v1/sandbox.proto \
${SBX_CA_FILE:+-cacert "$SBX_CA_FILE"} \
-expand-headers -H 'authorization: Bearer ${SBX_BEARER}' \
-d "{\"sandbox_id\":\"$SBX_SANDBOX_ID\",\"exec\":{\"req_id\":\"$REQ_ID\",\"command\":\"ls -la\",
\"timeout_s\":60,\"streaming\":true,\"workspace_key\":\"$SBX_WORKSPACE_KEY\"}}" \
"$SBX_ADDR" sandbox.v1.SandboxExec/Exec

The token travels in grpcurl's environment, which -expand-headers reads, not on its command line: anything in argv is visible to every user of the machine through ps. Chunk data comes back base64-encoded in grpcurl's JSON.

Other languages: generate a client from the one proto file with buf generate or protoc. Moca's own TypeScript client is generated in packages/k8s-sandbox/src/gen/.