Upload large event archives

OrbitForge is an event routing API for teams that move operational telemetry between services. This guide explains how to allocate a multipart archive upload without losing the context required to recover from partial failures. Examples use the fictional sandbox host api.orbitforge.example and a workspace called ws_lab. Replace these values with the workspace identifier returned by your own control plane. Sandbox traffic is isolated from production billing and does not appear in production retention reports.

Contract and prerequisites

An upload becomes visible to readers only after every part has passed checksum validation. Before you begin, create a sandbox workspace, select its region, and grant a service account the narrowest permission needed for this workflow. The workspace identifier is required on every request. Region selection happens when the workspace is provisioned; sending the same identifier to another regional endpoint does not migrate its contents. Treat a missing resource response as a configuration problem until you have checked both the identifier and the endpoint.

The API accepts UTF-8 JSON and returns a request identifier in every response. Record that identifier with your application trace, but do not log bearer credentials or entire customer payloads. Dates must use an explicit offset; examples use UTC to keep comparisons unambiguous. Empty strings and omitted properties have different meanings. An omitted optional field selects the documented default, while an empty string is validated as a supplied value and may be rejected.

Request example

Use the following request as a connectivity check before adding application concurrency. The example deliberately requests a small result so you can inspect the full response and compare it with your local ledger. Supply a real sandbox token through your process environment rather than committing it to a source file.

curl --fail-with-body 'https://api.orbitforge.example/v1/uploads' \
  -H "Authorization: Bearer $ORBIT_TOKEN" \
  -H 'Orbit-Workspace: ws_lab' \
  -H 'Accept: application/json'

A successful response includes a stable resource identifier, its current state, and the server observation time. Do not infer completion from the HTTP status alone. Accepted work may still be pending in a regional queue. If the response includes an operation reference, follow that reference until it reaches a terminal state before making dependent changes. Preserve the original response when diagnosing a discrepancy; a later read may show a newer state.

Response fields

These fields form the minimum response contract. Additional fields may appear without a new API version, so parsers should ignore unknown properties while still validating the fields they actually use. A missing required field is a protocol error and should be reported with the request identifier. Do not silently replace a missing state with a success value, because that would turn a recoverable integration failure into an incorrect business decision.

Failure handling

Transport failures leave the outcome uncertain when the server may already have received the request. Read the current resource or operation status before constructing a replacement mutation. Validation errors are deterministic for the same body and should be corrected rather than retried. Permission errors require a scope or workspace change. For throttling and temporary server errors, honor the retry delay and cap the number of attempts so background recovery cannot overwhelm foreground traffic.

Operational verification

Record each part number and checksum durably, then resume only the parts missing from the server manifest. During rollout, compare a sample of server identifiers with the identifiers stored by your application. Measure both request latency and the time until the resulting state becomes usable. Those measurements answer different questions: a fast acknowledgement can still precede a long processing delay. Alert on sustained changes in terminal failure rate rather than on a single slow response, and include the region in every dashboard dimension.

Keep a compact recovery log containing the request identifier, workspace, resource identifier, attempt count, and last observed state. Retain it for at least the maximum retry horizon of your application. During an incident, pause new work before replaying that log, select a bounded batch, and compare the resulting counts with the original intent. Resume normal traffic only after a second read confirms the expected state. This procedure makes recovery auditable without copying sensitive event bodies into diagnostic storage.

Parts and completion

Allocate an upload with its expected byte length, content type, and SHA-256 digest. The server returns an upload identifier and a staging deadline two hours in the future. Parts are numbered starting at one. Every part except the last must contain at least five mebibytes, and no part may exceed sixty-four mebibytes. Upload parts in parallel if needed, but limit concurrency so the checksum workers do not compete with foreground event publishing for the entire workspace budget.

The completion request supplies an ordered manifest of part numbers and digests. The server verifies that the concatenated payload matches the declared length and digest before importing any events. A checksum mismatch returns a failed operation with the offending part number when available. Replace that part and submit a new completion attempt before staging expires. Aborting an upload deletes staged parts and cannot be undone. After successful import, staged bytes are removed automatically; the resulting event identifiers remain available in the operation report.