nemotron-colembed-vl-8b-v2

Name: nemotron-colembed-vl-8b-v2
Author: nvidia

by nvidia

State-of-the-art late-interaction visual document retrieval

9Kdl/month

50likes

8.8Bparams

HuggingFace Run it on your own data, free

Identifiers

Model ID

nvidia/nemotron-colembed-vl-8b-v2

Feature URI

mixpeek://image_extractor@v1/nvidia_nemotron_colembed_vl_8b_v2

Overview

Nemotron ColEmbed VL is an 8B-parameter ColBERT-style multi-vector embedding model built on Qwen3-VL-8B-Instruct. It produces per-token embeddings for both queries and documents, enabling fine-grained matching between query terms and document regions. This late-interaction approach is particularly powerful for visual document retrieval, where different parts of a document page (headers, tables, figures) need to match different parts of a query.

The model ranks #1 on ViDoRe V3, the visual document retrieval benchmark, with a score of 63.54 -- surpassing ColPali and ColQwen variants.

Architecture

ColBERT-style architecture on top of Qwen3-VL-8B-Instruct. Produces multi-vector representations (one vector per token) rather than single-vector embeddings. Matching uses MaxSim: for each query token, find the maximum similarity to any document token, then sum across query tokens.

Mixpeek SDK Integration

// No extractor parameter takes a Hugging Face model id (checked against
// GET /v1/discovery/extractors, which returns 13), so nemotron-colembed-vl-8b-v2 runs
// on your side and the output is upserted through POST
// /v1/namespaces/{namespace_id}/documents/upsert. On Enterprise the other
// path is to upload the weights instead: POST /v1/namespaces/{id}/models
// accepts the huggingface format and a custom plugin loads them.
const res = await fetch(
  "https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
  {
    method: "POST",
    headers: {
      Authorization: "Bearer API_KEY",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      collection_id: "col_your_collection",
      documents: [
        {
          document_id: "asset-00412",
          // The vector name has to match a vector index on the collection.
          vectors: { "image-embedding": yourVector },
          payload: { source_key: "archive/2026/asset-00412" },
        },
      ],
    }),
  },
);

// Managed alternative, if this exact model is not the requirement:
// image_extractor@v1 runs google/siglip-base-patch16-224
// (768-d) over a bucket, with no inference of your own.