STT for an Unsupported Language on iPhone – Part 1

Introduction

Two years ago, Apple made it possible to run a local large language model (LLM) directly on macOS, iOS and iPadOS devices. It is called Apple Intelligence, and to run it you must meet these three requirements:

1. iOS 18 or later,
2. iPhone 15 Pro or later,
3. a supported language and region; both the device language (Language & Region) and the Siri language must be set to the same supported language.

I live in the EU, in the Czech Republic, so even though I meet the first and second requirements, I am still out of luck if I want support for my native language.

When I bought myself a watch for my birthday, I wondered how I could use Apple Intelligence and, ideally, get the most out of this combination. And since I had been keeping a journal, I decided to dictate my journal into the watch, have the iPhone transcribe it to text, and let AI act as a mentor for my moods. So what I want is one general model supporting STT, plus some general intelligence that understands the input data – that is, the text of the entry itself supplemented with metadata such as heart rate, heart rate variability, time in daylight, time of day and more – from which it assembles a structured response and picks a matching preview icon.

Transcribing the Recording: Apple SpeechTranscriber vs. Whisper as a Custom Model

The problem with Czech is that it is not a world language. It is spoken by around 10 million people, so Apple did not include Czech in Apple Intelligence. Apple Intelligence is not even used for transcribing audio recordings; since iOS 26, Apple SpeechTranscriber is used for that, and it does not support Czech yet either. Apple's older speech recognition (SFSpeechRecognizer) could transcribe Czech, but only by sending the recording to Apple's servers, which we did not want. However, Apple does allow you to run your own model on iOS, one that supports Czech. I tried many of them – Qwen3-ASR 1.7B, NVIDIA Parakeet TDT 0.6B v3, Voxtral Mini Transcribe V2 – and finally settled on WhisperKit large-v3-turbo. This model supports 99 languages, its compressed variant is only 632 MB, it genuinely understands Czech and it does not spit out gibberish or Russian, as Voxtral did, for example. I had more or less been betting on Voxtral, because I was trying to run everything within the EU, but in the end that did not work out.

As for the technical details, I will cover them in the next article; for now I will just summarize the basic problems.

Basic Problems

One of them is ensuring that an English-speaking user can use Apple Intelligence and Apple SpeechTranscriber without having to download and load into memory the small but still bulky AI model needed for an unsupported language. So the first step must be detecting whether the phone is already capable of using Apple Intelligence and, if so, whether the model is already downloaded and installed or still waiting to be downloaded. Apple Intelligence, too, must first download its LLM and then occupy the space reserved for it in storage. If Apple Intelligence is not detected, we download WhisperKit large-v3-turbo, which I uploaded to Cloudflare's R2 storage to be safe, so that downloads are very fast worldwide. Downloading the model from a server in the Czech Republic would be slow for users abroad – it is still 632 MB. Once the model in Core ML format has been downloaded, it still has to be prepared for the specific iPhone and loaded into memory. Unfortunately, this preparation takes a very long time, on the order of minutes, but once iOS converts the model into a form that your iPhone's Neural Engine (ANE) can run, it then loads into memory in under 5 seconds.

Compilation is sometimes repeated, however, because iOS occasionally discards the cache – for example after an iOS update, after an app update from TestFlight or the App Store, or when storage is running low. Then you wait a few minutes again. Compilation, however, requires the app to be in the foreground. If the app is suspended, the watch may show "Processing" longer than it should. That means the audio from the watch does reach the phone, but the phone has no model to process it, so it never sends the watch a confirmation that it has been processed.

Of course, this does not happen with a language supported by Apple, because SpeechTranscriber is part of the system (the language model is downloaded only once) and Apple has it prepared for the ANE in advance, so it starts immediately. Whisper is an "outside" model, so each iPhone has to prepare it on its own. On the other hand, Whisper runs in the background and can serve the watch even when the iPhone is locked. If someone wanted to call an external LLM via an API instead of downloading, loading and using an on-device LLM, they might not be able to get to the key, because a key stored in the Keychain with "available only when the device is unlocked" protection is not accessible while the screen is locked.

Likewise, storing an LLM key on the device is not secure; using a jailbroken iOS, someone could get hold of the key.

The result of transcribing an audio recording from the iPhone or Apple Watch is therefore clean text in Czech or another supported language. To make sure the language is recognized correctly, I try to use a toggle in the app showing the language of the transcript. And since Czech is also highly inflected, in addition to detection I will support the speaker's gender with an "I am a man / I am a woman / Prefer not to say" toggle.

Even the Whisper model is no cure-all for Czech, though: it was trained on movie subtitles, so I had to add filtering for closing sentences such as "Subtitles translated by Johny K."

AI Mentor

A journal needs a mentor who sees the user's entries in a broader context and compares them with data on heart rate and its variability, time of day, time in daylight and your calendar. All these inputs are sent from the watch and phone as text data, anonymized and non-specific. For example, whether the user's heart rate is lower than, the same as or higher than their median for that part of the day, whether they had a scheduled meeting, a scheduled phone call or a calendar event, and much more. The AI then returns structured data, a large part of which the user sees both in the entry on the phone and on their watch.

Here, however, I had to decide whether to choose one universal model for both transcription and analysis, or several models and split the system. I tried running two models on the iPhone, but it often happened that the system removed the second model at runtime just to save memory. For a language not supported by Apple Intelligence, another viable option had to be found.

In the next part: why I decided on a model running directly on the phone, what the whole solution looks like in practice and what happens on first launch.