gme-Qwen2-VL-2B-Instruct
by Alibaba-NLP
Unified multimodal embedding — text, images, and visual documents in one vector space
Alibaba-NLP/gme-Qwen2-VL-2B-InstructOverview
GME (General Multimodal Embedding) turns a vision-language model — Qwen2-VL-2B-Instruct — into a single-vector embedding model. Unlike a CLIP-style dual encoder that keeps separate towers for text and images, GME feeds text, an image, or an interleaved image+text pair through the same VLM and pools the final hidden states into one dense vector. Because every modality lands in the same space, you can retrieve any-to-any: text→image, image→text, text→text, and (image+text)→(image+text).
The practical payoff is visual-document retrieval. A slide, a scanned invoice, or a chart-heavy PDF page is encoded as an image, so layout, figures, and text are embedded together without a brittle OCR-then-embed pipeline. On Mixpeek, GME powers a single multimodal_embedding index where an agent's text query reaches document pages, product photos, and screenshots side by side.
Architecture
Built on Qwen2-VL-2B-Instruct (~2.2B params). The VLM encodes the input; the last-layer hidden states are mean-pooled into one dense vector (instruction-tunable via a task prompt). Inherits Qwen2-VL's dynamic-resolution vision encoder, so images are processed at native aspect ratio without forced 224×224 cropping — important for dense document pages. Trained with contrastive learning over text, image, and fused image-text pairs plus hard negatives.
Key Capabilities
- •Single shared space for text, image, and interleaved image-text inputs
- •Any-to-any retrieval (text↔image, image↔image, fused queries)
- •Strong visual-document retrieval (ViDoRe) — embeds page layout + figures, no OCR step
- •Dynamic image resolution inherited from Qwen2-VL (no fixed crop)
- •Instruction-conditioned embeddings via a task prompt
Use Cases on Mixpeek
- •Multimodal RAG over corpora that mix text passages and images in one index
- •Visual-document search: slides, scanned PDFs, charts, and screenshots by text query
- •Product search where the query or catalog item is image + text together
- •Agent retrieval where queries and documents span modalities without per-modality pipelines
Tags
Use gme-Qwen2-VL-2B-Instruct on Mixpeek
Build multimodal processing pipelines with this model and others. Extract features, run inference, and set up retrieval in Mixpeek Studio.
Open StudioHow It Runs on Mixpeek
On Mixpeek, gme-Qwen2-VL-2B-Instruct runs as a managed extractor inside a processing pipeline. Point a bucket of sentence similarity data at it, and Mixpeek handles GPU provisioning, batching, retries, and writing the outputs into a vector store you can query.
Extractor outputs land in the Mixpeek Vector Store (MVS), where you can combine them with retrieval, reranking, and filter stages to build end-to-end search and agent-perception pipelines, no model-serving infrastructure to maintain.
Explore on Mixpeek
Compare alternatives in this category
Hand-picked tools & platforms compared
Deep-dive technical guide
See how Mixpeek runs models as extractors
Store & search embeddings at scale
Usage-based pricing for pipelines
Compare models, APIs & infrastructure
Specification
View on HuggingFace
See model card, files, and community discussion