№48
Kafka Rejects Records Over 1 MB. LLM Spans Do Not Care.
A record-size test exposed an accounting error: an 870 KiB text field fit, while an 880 KiB field did not.
Devlog 7 min
Real production work debriefed after the fact — what broke, what it cost, and the lessons carried forward.
3 published entries
A record-size test exposed an accounting error: an 870 KiB text field fit, while an 880 KiB field did not.
The logs table stored trace IDs, users, token counts, latency, prompt input, model output, metadata, and variables. We used the same table to show one trace, graph cost by day, filter a feature’s recent spans, and find text a user remembered typing three weeks ago.
At 6 p.m. on December 19, 2025, the product was down again.