NEWVectors or files. Pick a path.Start →
    Models/Text Extraction/ByteDance/Dolphin-v2
    HFOCRMIT

    Dolphin-v2

    by ByteDance

    End-to-end document parsing VLM: 21 element types, pixel-accurate layout

    Identifiers
    Model ID
    ByteDance/Dolphin-v2
    Feature URI
    mixpeek://image_extractor@v1/bytedance_dolphin_v2

    Overview

    Dolphin v2 is ByteDance's visual document parsing model that classifies and extracts 21 element categories from both digital and photographed documents: text blocks, tables, formulas, figures, code blocks, headers, footers, captions, and more. Built on a Qwen2.5-VL-3B backbone, it processes document pages end-to-end without a separate OCR pipeline.

    It scores 89.45 on OmniDocBench V1.5 overall, with standout performance on tables (TEDS: 90.48) and formulas (CDM: 86.72). The key advance over v1 is absolute pixel-coordinate spatial localization -- every extracted element comes with precise bounding box coordinates. On Mixpeek, Dolphin v2 powers structured document extraction for RAG pipelines that need to understand document layout, not just raw text.

    Architecture

    Qwen2.5-VL-3B vision-language backbone fine-tuned for document parsing. Processes pages at native resolution with adaptive tiling. Outputs structured JSON with element type, text content, and absolute pixel-coordinate bounding boxes for each of 21 element categories.

    Mixpeek SDK Integration

    // No extractor parameter takes a Hugging Face model id (checked against
    // GET /v1/discovery/extractors, which returns 13), so Dolphin-v2 runs
    // on your side and the output is upserted through POST
    // /v1/namespaces/{namespace_id}/documents/upsert. On Enterprise the other
    // path is to upload the weights instead: POST /v1/namespaces/{id}/models
    // accepts the huggingface format and a custom plugin loads them.
    const res = await fetch(
      "https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
      {
        method: "POST",
        headers: {
          Authorization: "Bearer API_KEY",
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          collection_id: "col_your_collection",
          documents: [
            {
              document_id: "asset-00412",
              // The model produces text, so it lands in payload. Give the
              // collection a text vector index and embed that text to make it
              // searchable rather than only filterable.
              payload: { extracted_text: modelOutput, source_key: "archive/2026/asset-00412" },
              vectors: { "text-embedding": embeddingOfModelOutput },
            },
          ],
        }),
      },
    );
    
    // Managed alternative, if this exact model is not the requirement:
    // universal_extractor@v1 runs google/gemini-embedding-2
    // (3072-d) over a bucket, with no inference of your own.

    Capabilities

    • 21 document element categories (text, tables, formulas, figures, code, etc.)
    • Pixel-accurate bounding box localization
    • Tables with structure preservation (TEDS: 90.48)
    • Formula recognition (CDM: 86.72)
    • MIT license, 3B parameters

    Use Cases on Mixpeek

    Structured document extraction for enterprise RAG
    Table extraction from financial reports and invoices
    Formula extraction from scientific papers and textbooks
    Layout-aware document indexing for search across mixed-content pages

    Benchmarks

    DatasetMetricScoreSource
    OmniDocBench V1.5 (overall)Score89.45ByteDance, 2026: Model Card
    OmniDocBench V1.5 (tables)TEDS90.48ByteDance, 2026: Model Card
    OmniDocBench V1.5 (formulas)CDM86.72ByteDance, 2026: Model Card

    Performance

    Input SizeVariable resolution document pages
    GPU Latency~420ms / page (A100)
    GPU Throughput~2.4 pages/sec (A100)
    GPU Memory~7 GB

    Specification

    FrameworkHF
    OrganizationByteDance
    FeatureOCR
    Outputtext + bbox
    Modalitiesvideo, image, document
    RetrieverText-in-Image
    Parameters~3B
    LicenseMIT
    Downloads/mo210K

    Research Paper

    Dolphin: A Document Parsing Model

    arxiv.org

    Build a pipeline with Dolphin-v2

    Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.

    Run it on your own data, free