Export over OpenTelemetry

asz push sends what the storage root holds to an OpenTelemetry logs receiver: every landed file and every round, as OTLP logs over gRPC, or over HTTP with a protobuf body. The SkyWalking OAP accepts both, on its gRPC port and on its REST port, and so does an OpenTelemetry Collector.

# asz.yaml
export:
  otlp:
    protocol: grpc              # the default; http posts to the receiver's /v1/logs instead
    endpoint: 127.0.0.1:11800   # the OAP's gRPC port; with http, http://127.0.0.1:12800

./bin/asz push -once      # send everything not yet sent, then exit
./bin/asz push            # keep sending new files every export.otlp.interval

One log record per file

A file is sent whole: one log record whose body is the file’s bytes, unchanged. A receiver stores it as it was landed, checks its digest at once, and has nothing to put back together. Every {seq, row, block} reference in a round still means what it meant, because a row is a line of the body it names.

This is the second design. The first sent one record per line, and it was measured on one conversation of 306 files: 39,683 records, attributes 16% on top of the body even after trimming, and a receiver that had to track which lines had arrived before it could verify anything. A landed file is cut at a budget and a round is cut at the same budget, so a whole file is a small record, and the same conversation is 306 records.

Landed files and rounds are both write-once, so each is sent once. push.state in the storage root lists what was sent, with the digest each file had; a file is recorded only after the request carrying it succeeded, so a failed request leaves it for the next pass.

Transport

The request is the ExportLogsServiceRequest of the OpenTelemetry protocol, built from the protocol’s own Go definitions and sent by the gRPC client of the official module, or posted over HTTP with a protobuf body. Nothing about the encoding is the project’s own, so any OTLP logs receiver reads it.

headers travel with every request on both transports, as gRPC metadata or as HTTP headers, which is where an authorization token goes. A receiver that answers with a partial success, saying it rejected some records, is treated as having refused the request: it does not say which records, so the request is sent again whole on the next pass, and a receiver keeps the first copy of a file it already holds.

What every record carries

The resource, which names the service a record belongs to:

The scope is github.com/apache/skywalking-ai-sessionizer with the same version. Each record then says what its file is, so a receiver can route it, index it and verify it without decoding the body:

The record’s time is chosen so a receiver can bound a read by a range it already holds. A landed file is stamped with its last record time, and a file whose records carry no time, such as a child’s meta file, with the latest record time of the session as known at push, which is always inside the session’s range and cannot go stale. A round is stamped with the session’s last activity as of that round, so a receiver’s newest row per conversation is the head. Every record also carries the time it was observed.

Every scenario in the test suite is pushed to a receiver over both transports and checked against the two tables above, in both formats, so a change to the wire that this page does not describe fails the build. See Scenarios.

Checking what a receiver gets

An OpenTelemetry Collector with a file exporter writes back what it decoded, which is the easiest way to see the records before pointing at a backend:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
        max_recv_msg_size_mib: 32   # the default is 4 MiB, below the 8 MiB batches; see Size
      http:
        endpoint: 0.0.0.0:4318      # the default limit is 20 MiB, enough
exporters:
  file:
    path: /out/logs.json
service:
  pipelines:
    logs:
      receivers: [otlp]
      exporters: [file]

Point export.otlp.endpoint at 127.0.0.1:4317, or with protocol: http at http://127.0.0.1:4318, run asz push -once, and read logs.json: one JSON line per request, with the resource, the scope and the records as the Collector understood them. make e2e-collector does exactly this with a generated session and a Collector container, once over each transport, then checks every record against the root and rebuilds the root from what the Collector wrote; CI runs it on every change. Writing each record’s body to asz.file under a new root gives a root that asz verify and asz view read like the original.

Metrics

asz push sends metrics as well, when an adapter produces them, and they are Claude Code’s own metric family, name for name and attribute for attribute with the runtime’s OpenTelemetry exporter, so a receiver holds one family whichever produced it. Phase one is the one measure both can supply exactly:

With metrics: true on the claude-code-local adapter, the collector derives the points from the landed files: one count per call, never per fragment, since a main transcript repeats the usage on every fragment of a call, summed per minute the way the runtime’s SDK sums over its export interval, with query_source from the stream the call was made on. It cannot derive what a transcript does not carry: cost, latency, active time, lines of code, commits, pull requests, the session start type, or the tokens of the runtime’s auxiliary calls, which never reach a transcript. Those are the runtime’s exporter’s alone. The first derivation over a root with history is bounded by metrics_lookback, 24 hours unless set, so switching the flag on does not send a year of tokens; every later pass derives each new file whole.

The other source of the same family is the runtime’s exporter itself: the claude-code-otlp adapter receives what Claude Code sends and lands each metrics request in the same spool, bytes as received. One root sends one source: metrics may be on for the local adapter or for the receiver, and the configuration refuses both.

The points wait in the storage root’s _metrics/ spool, one write-once file per landed file with points or per request received, and go out in order under the same budget and the same once-only rule as the files. export.otlp.logs and export.otlp.metrics switch the two things a push sends, the files and rounds as logs and the spool as metrics, so a receiver that takes one and not the other is sent what it takes. On the way out the resource is normalised to asz’s identity, the service, the layer, the sender, so the OAP holds one service for the runtime. A receiver that answers with a partial success is treated as having refused the request, as for logs.

Rate

A first push sends the whole history of the storage root, which can be hundreds of megabytes, as fast as the receiver takes it. export.otlp.max_bytes_per_minute caps what goes on the wire: a pass waits before a request until a minute’s budget, refilled continuously and never holding more than a minute’s worth, covers the request’s encoded size. The budget starts full, so a pass that sends a few new files never waits, and a request larger than a minute’s budget waits for a full one and leaves it empty. Zero, the default, is no limit. asz push -once still sends everything before it exits, and the pass line’s paused= field says how long it waited.

A pass goes session by session, the session landed first going first: its files, then its rounds. A receiver rebuilds a session once it holds both, so during a long first push the sessions become complete one after another rather than all at the end.

A receiver that answers 429 over HTTP or ResourceExhausted over gRPC is asking the sender to slow down. The pass stops there, leaves the rest for the next pass, and empties the budget, so the next request waits a whole minute’s worth when a rate is set. In watch mode a Retry-After longer than the interval is honored.

Size

A request carries at most export.otlp.batch_bytes of file bytes, 8 MiB by default, which keeps a request under the 10 MiB the OAP’s HTTP server accepts. A file larger than that is sent alone, in a request of its own. A landed file is cut at max_delta_bytes, 2 MiB by default, and a round is cut at parse.max_round_bytes, also 2 MiB, so a request normally carries several files. The exception on both sides is a single unit larger than the budget: a source record is landed whole, and a round covering one landed file is published whole. The largest source record in the measured corpus is 4.5 MB.

The receiver’s limit must cover the largest single request. The OAP accepts 50 MB over gRPC and 10 MiB over HTTP by default. An OpenTelemetry Collector accepts 4 MiB over gRPC and 20 MiB over HTTP unless its receiver is configured otherwise, so its gRPC receiver needs max_recv_msg_size_mib raised, as the example above does.