Running Whisper Locally: A Practical Quick-Start

Whisper is one of the few genuinely useful “just run it yourself” AI models — no API key, no per-minute billing, and it runs on hardware you probably already own. If you’ve been sending audio to a paid transcription API out of habit, it’s worth five minutes to check whether your own laptop can do the job.

Pick a model size first

Whisper ships in five size tiers, each with a lighter English-only variant (.en) alongside the multilingual one:

Rule of thumb: CPU-only machine → stick to tiny or base. Any consumer GPU → small or medium runs comfortably. A dedicated GPU with 10GB+ VRAM → go straight to large-v3.

The fastest path: Hugging Face Transformers

You don’t need OpenAI’s original repo to get started — the Hugging Face transformers pipeline wraps everything into a few lines:

pip install --upgrade transformers datasets[audio] accelerate

from transformers import pipeline

asr = pipeline("automatic-speech-recognition", model="openai/whisper-large-v3")
result = asr("your_audio_file.mp3")
print(result["text"])

For longer files, pass chunk_length_s=30 (the sweet spot for large-v3) so the pipeline auto-splits the audio instead of truncating it. If you have an NVIDIA GPU, add device="cuda:0" and torch_dtype=torch.float16 to the pipeline call for a meaningful speed boost.

Model pages worth bookmarking:

• openai/whisper-large-v3 — best general accuracy
• openai/whisper-large-v3-turbo — most of the accuracy, a fraction of the latency
• Transformers pipeline docs

The reference path: OpenAI’s own repo

If you’d rather use Whisper exactly as OpenAI ships it (useful for reproducing published benchmarks or CLI-only workflows):

pip install -U openai-whisper
whisper your_audio_file.mp3 --model medium

Repo: github.com/openai/whisper — the README covers ffmpeg prerequisites and every CLI flag.

What actually determines which model you should pick

Not accuracy on paper — your hardware and your patience. tiny/base on CPU is fine for quick drafts and personal notes where you’ll skim and fix typos anyway. Anything you’re publishing or feeding into another pipeline unedited deserves medium or large-v3 on a GPU. The honest answer, echoed by most people who’ve actually benchmarked this on their own machine: try two adjacent tiers on a sample of your real audio before committing — the “best” model is the smallest one that’s accurate enough for what you’re doing with the output.

Sources: openai/whisper (GitHub), Hugging Face Transformers — Whisper, Hugging Face — openai/whisper-large-v3

Opinions are my own.