VibeVoice-ASR-HF

Name: VibeVoice-ASR-HF
Author: microsoft

by microsoft

Unified ASR + diarization + timestamps in one 9B model: 60 min single-pass

295Kdl/month

9Bparams

HuggingFace Run it on your own data, free

Identifiers

Model ID

microsoft/VibeVoice-ASR-HF

Feature URI

mixpeek://transcription@v1/microsoft_vibevoice_asr_v1

Overview

VibeVoice-ASR is Microsoft's unified speech recognition model that produces structured rich transcriptions (speaker labels, word-level timestamps, and content) from up to 60 minutes of audio in a single forward pass. It replaces the traditional pipeline of separate ASR, diarization, and alignment models with one 9B parameter model.

Supporting 50+ languages with native code-switching (no language flag required), it handles meetings, interviews, podcasts, and call center recordings where knowing who said what matters as much as what was said. On Mixpeek, it powers speaker-attributed transcription for video and audio assets.

Architecture

Encoder-decoder transformer (9B parameters) with multi-task training for simultaneous ASR, speaker diarization, and timestamp alignment. Processes up to 60 minutes of audio in a single pass without sliding window chunking.

Mixpeek SDK Integration

// No extractor parameter takes a Hugging Face model id (checked against
// GET /v1/discovery/extractors, which returns 13), so VibeVoice-ASR-HF runs
// on your side and the output is upserted through POST
// /v1/namespaces/{namespace_id}/documents/upsert. On Enterprise the other
// path is to upload the weights instead: POST /v1/namespaces/{id}/models
// accepts the huggingface format and a custom plugin loads them.
const res = await fetch(
  "https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
  {
    method: "POST",
    headers: {
      Authorization: "Bearer API_KEY",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      collection_id: "col_your_collection",
      documents: [
        {
          document_id: "asset-00412",
          // The model produces text, so it lands in payload. Give the
          // collection a text vector index and embed that text to make it
          // searchable rather than only filterable.
          payload: { extracted_text: modelOutput, source_key: "archive/2026/asset-00412" },
          vectors: { "text-embedding": embeddingOfModelOutput },
        },
      ],
    }),
  },
);

// Managed alternative, if this exact model is not the requirement:
// universal_extractor@v1 runs google/gemini-embedding-2
// (3072-d) over a bucket, with no inference of your own.