The speech recognition wiki
A working reference for people who build with speech: how recognition works, how to measure it, how to make it understand specialist language, and how to run it on European terms.
102
articles
10
sections
364
glossary terms
10
European languages covered
Foundations
Start here. What speech AI is, and how the pieces fit.
- What is speech recognition?ASR, from sound wave to text.
- What is voice AI?Every system that listens, understands or speaks.
- Voice AI vs speech AI vs conversational AIThree overlapping terms, untangled.
- A brief history of speech recognitionFrom Audrey to transformers in seventy years.
- How speech-to-text worksFeatures, encoders, decoders, and text.
- Speech recognition vs voice recognitionWhat was said vs who said it.
- ASR model architecturesCTC, RNN-T, and attention encoder-decoders compared.
- Acoustic models and language modelsThe two halves of recognition, and how they merged.
Speech-to-text
The mechanics of turning audio into reliable transcripts.
- Real-time vs batch transcriptionStreaming and pre-recorded, and when to use each.
- Streaming speech recognitionLive transcripts over a WebSocket.
- Partial vs final resultsWhy live text rewrites itself.
- What is voice activity detection (VAD)?Telling speech apart from silence and noise.
- What is endpoint detection?Deciding when an utterance is over.
- VAD vs endpointing vs turn detectionThree detectors that get confused for each other.
- What is speaker diarization?Who spoke when.
- Diarization vs identification vs verificationRelated speaker tasks with different answers.
- Spoken language identificationDetecting which language is being spoken.
- Code-switching in speech recognitionWhen speakers change language mid-sentence.
- Word timestamps and forced alignmentPlacing every word on the timeline.
- Confidence scores in speech recognitionWhat a 0.87 does and does not tell you.
- Punctuation, casing, and inverse text normalizationFrom 'twenty euros' to €20.
- Hallucinations in speech recognitionWhen a model writes words nobody said.
- Speech-to-text latencyMeasuring the delay that users actually feel.
- Audio formats for speech recognitionSample rates, codecs, and channels.
- Transcribing noisy audioFar-field, overlap, and real-world rooms.
- Telephony transcription8 kHz audio, codecs, and call recordings.
- Long-form transcriptionHours of audio without drift or loss.
- Captions and subtitles: SRT, VTT, and timingTurning transcripts into readable captions.
- Integrating a speech-to-text APIKeys, sessions, WebSockets, and retries.
Accuracy and evaluation
How to measure whether a transcript is good enough.
- What is word error rate (WER)?The standard accuracy metric, explained.
- Speech recognition evaluation beyond WERThe errors a single number hides.
- Semantic word error rateCounting only the errors that change meaning.
- Character error rate (CER)When words are the wrong unit.
- Entity and keyword accuracyMeasuring the words that matter most.
- ASR benchmarks and leaderboardsOpen ASR Leaderboard, FLEURS, and what they test.
- Benchmark contaminationWhy training on test sets inflates scores.
- Building your own ASR test setThe evaluation that predicts production.
- How to evaluate a speech-to-text vendorA practical checklist for buyers.
Domain adaptation
Making a speech model understand your specific language.
- What is domain adaptation in ASR?Why general models fail on specialist speech.
- Fine-tuning vs context biasingRetraining the model vs steering it at runtime.
- Custom vocabulary and context biasingTelling the model which words to expect.
- Text-only domain adaptationAdapting without collecting a single hour of audio.
- Long-context biasingFrom word lists to a million tokens of context.
- Transcribing structured identifiersOrder codes, plates, IBANs, and SKUs.
- Rare words and proper nounsNames, brands, and the long tail.
- Over-biasingWhen custom vocabulary makes accuracy worse.
- Accents and regional speechWhy pronunciation variety breaks models.
- Numbers and alphanumericsDigits, dates, amounts, and spelled-out codes.
European languages
Depth in the languages Europe actually speaks.
- Speech recognition for European languagesDepth over breadth.
- Multilingual speech recognitionOne model, many languages, and the trade-offs.
- Dutch speech recognitionFlemish, compounds, and English loanwords.
- German speech recognitionCompounds, dialects, and case endings.
- French speech recognitionLiaison, homophones, and silent letters.
- Spanish speech recognitionPeninsular and Latin American variants.
- Italian speech recognitionRegional accents and fast speech.
- Polish speech recognitionRich morphology and consonant clusters.
- Portuguese speech recognitionEuropean and Brazilian Portuguese.
- Swedish speech recognitionPitch accent and a smaller data pool.
- European EnglishNon-native English at work.
- Compound words and morphologyWhy 'word' is a slippery unit.
Voice agents
Speech recognition inside AI that talks back.
- What is a voice agent?Software that holds a spoken conversation and acts on it.
- Voice agent architectureHow recognition, reasoning and speech fit together.
- Voice agent latency budgetWhere the milliseconds go.
- End-of-turn detectionKnowing when the caller has finished.
- Turn-taking and barge-inInterruptions, overlaps, and polite agents.
- Speech-to-speech models vs pipelinesOne model or three.
- Voice agent frameworksPipecat, LiveKit Agents, and orchestration.
- Voice agents on the phoneSIP, Twilio, and 8 kHz reality.
- Tool calling in voice agentsWhen the agent needs to look something up.
- Testing voice agentsEvaluating conversations before callers do.
- Multilingual voice agentsSwitching language mid-call.
- What is text-to-speech?The other half of a voice agent.
Industries
How speech recognition is used where the stakes are high.
- Medical speech recognitionClinical vocabulary, dictation, and scribes.
- Ambient clinical documentationFrom consultation audio to a clinical note.
- Legal transcriptionHearings, depositions, and citations.
- Speech recognition in financial servicesTickers, figures, and call recording rules.
- Contact centre speech analyticsEvery call, transcribed and searchable.
- Automotive voice recognitionRoad noise, commands, and in-car assistants.
- Media and broadcast transcriptionLive captions, archives, and subtitling.
- Speech recognition in the public sectorCouncil meetings, parliaments, and citizen services.
Privacy, security, and compliance
Handling voice data in regulated environments.
- GDPR and voice dataWhen audio becomes personal data.
- EU data residency for speech AIWhere audio is processed, and why it matters.
- Zero data retentionProcessing audio without keeping it.
- Data processing agreementsWhat a DPA under GDPR Article 28 covers.
- The EU AI Act and speech recognitionRisk tiers and obligations for voice systems.
- ISO 27001 for speech AI vendorsWhat the certification does and does not prove.
- On-premise and VPC deploymentRunning speech recognition inside your perimeter.
- PII redaction in transcriptsRemoving personal data from text and audio.
- Voice biometrics and consentWhen a voice identifies a person.
- Sovereign AI in EuropeControl over models, data, and infrastructure.
Audio intelligence
What happens after the transcript.
- What is audio intelligence?Extracting meaning from recorded conversations.
- Conversation summarizationFrom an hour of talk to five lines that matter.
- Sentiment analysis on speechWords, tone, and what they miss.
- Keyword spotting and wake wordsSmall models that listen for one phrase.
- Audio event detectionRecognising laughter, alarms, music and silence.
- Using transcripts with LLMsFeeding speech into language models.
Infrastructure and cost
Serving speech models fast, at scale, and within budget.
- Real-time factor (RTF and RTFx)How fast a model processes audio.
- GPU inference for speech modelsKernels, batching, and CUDA graphs.
- Concurrency and scalingThousands of streams at once.
- Model size and quantizationSmaller, faster models and what they cost in accuracy.
- On-device vs cloud speech recognitionWhere the model runs, and the trade-offs.
- Speech-to-text pricing modelsPer minute, credits, and hidden costs.