Nagovori and Saylo: a transcription service on your own GPUs

A user uploads a recording of a call and gets back a transcript and a structured summary a few minutes later. Everything below is what sits behind those two sentences.

#The file never passes through the backend

The first decision is the cheapest and the most useful. Audio runs to hours and gigabytes; pushing it through the application means holding a connection open for the whole upload, spending memory on buffers, and running into proxy limits.

The Go backend hands out a presigned URL and removes itself: the file travels from the browser directly into the bucket. The application only learns about the recording when the frontend comes back to create a job. A pleasant side effect — the backend can be redeployed in the middle of someone's hour-long upload without breaking it.

#One card, several workers

A GPU host doesn't run a single process: ASR, TTS, whatever else. There's one card. If two workers each take a job at the same time, neither gets enough memory and both die — not immediately, but halfway through someone's hour-long recording.

So before taking a job, a worker acquires a host-slot lock in Redis and holds it until the work is done. Every worker on that host serializes through that lock, however many there are. The scheduler knows nothing about GPUs: Temporal just hands out activities, and the physical constraint lives where it physically exists — on the host.

The model itself is called through LiteLLM's OpenAI-compatible API rather than directly. That looks like a pointless layer until the model needs to change: the job carries a model id, and which checkpoint stands behind it is a gateway configuration question, not a worker deploy.

#Billing: reserve first, charge on facts

Per-minute pricing raises an awkward question: how much do you deduct when the duration is only known at the end?

Starting a job reserves an estimate, so the balance can't go negative while the workflow runs. At the finish the estimate is replaced by the real duration and the difference is returned. If the workflow failed, all of it comes back: the service pays for the service's failures.

That order is the only one that survives a crash in the middle. Charging purely at the end hands free work to anyone who starts ten jobs at once on an empty balance.

#Two brands, one codebase

Nagovori and Saylo are literally the same code. The differences are configuration keyed to the user's organisation: its own organisation in the identity provider, its own billing account, its own currency, its own allowance of free minutes, its own interface language. A request arriving from saylo.pro lands on its own billing configuration, and that is where the difference ends.

The temptation to fork for a second market is strong and expensive six months later: every change becomes two changes, and the drift is discovered by a customer. Splitting by configuration takes one careful pass to draw the boundary — after that it's free.

#Where the line is today

The Russian side is fully in production: payments, per-minute accounting, invoices for companies. On the international side card payments are still being finished — everything else, from sign-in to transcript, works.