Tolk

I understand some Swedish, but not all of it. Tolk ("interpreter" in Swedish) is for the moments where that is not enough: a receptionist, a doctor, a craftsman. The phone listens through its microphone and shows in English what the other person says, a few seconds behind, so that I can follow and answer.

It is not a two-way interpreter. It does not speak, and the other person never has to look at the screen.

Speech recognition runs on the phone, with the Parakeet model through the FluidAudio library. Every half second the model reads the open stretch of audio again (at most 15 s). Words that stay the same between two passes count as settled, and a sentence closes at its punctuation or at a pause. The closed sentence is translated on the phone by Apple's Translation framework.

A second step is optional. Each closed sentence can be sent to a larger model that is specialized in Swedish (KB-Whisper), running on a computer of my own. If that model heard something different, the Swedish and the English on screen are replaced in place and the sentence gets a check mark.

A conversation is saved as a text transcript on the phone.

Choosing the recognizer by measurement

The first build used the system's on-device dictation for Swedish. On 15 s of speech from a radio 1 to 2 m away it returned one word, while Parakeet transcribed the same recording fluently. The app was moved to Parakeet the same day. On an iPhone 16 Pro one pass takes 0.05 to 0.15 s.

Timing

A recorded 14-minute conversation, fed through the app on the phone at real-time speed, gave 292 sentences. The English appeared 2.1 s after the words were spoken and the corrected version after 3.3 s (medians). The second model changed 199 of the 292 sentences.

Context for short sentences

A sentence of two or three words can be translated wrong when it stands alone ("Tak, tak." came out as "Ceiling, ceiling." where "Thank you, thank you." was said). The translation framework has no parameter for context. Tolk puts the preceding sentences in front and marks them as not to be translated, so the model reads them and only the new sentence comes back in English. Scored blind on 40 sentences, context made 8 better and 4 worse, so the gain is real but modest.

No cuts in the middle of a sentence

The recognizer reads at most 15 s at once, and people speak longer sentences than that. Tolk holds the settled words of a long sentence and moves the window on, so the sentence still closes at its real end and is translated whole.

A replay tool

The same recognition code runs on the Mac over a recording at real-time speed and prints every pass and every closed sentence. The sentence rules are tuned there before a build goes to the phone.

Tolk records and transcribes another person, so the defaults are conservative. Recognition and translation happen on the phone. What is saved is text. Audio is written to the phone's storage only when a debug setting is switched on, and it is off by default.

The speech model is downloaded once on the first start (about 0.6 GB). With the second step switched on, the audio of each closed sentence goes to my own computer over a private network, and to nobody else. With it switched off, no audio and no text leaves the phone.

Tolk is an early build from the end of September 2026. It runs on my iPhone 16 Pro and needs iOS 26.4 or later. It is not on the App Store yet. A release under Volary AB is planned, without a date so far.

All tests so far used Swedish radio played in the room or a recorded conversation. The test with a real person across a table is still open. Also not built yet: telling my own voice apart from the other person's, so that only the other person is translated. The only language pair is Swedish to English.

The second step currently runs against a server of my own. The app also works without it.

The glyph in the icon is "Chats Teardrop" from Phosphor Icons (MIT license).

Back to the iOS apps