clip-vit-large-patch14

Name: clip-vit-large-patch14
Author: openai

by openai

Contrastive Language-Image Pre-Training for zero-shot visual understanding

6.3Mdl/month

2,072likes

428Mparams

HuggingFace Run it on your own data, free

Identifiers

Model ID

openai/clip-vit-large-patch14

Feature URI

mixpeek://video_descriptor@v1/openai_clip_large_v1

Overview

CLIP (Contrastive Language-Image Pre-Training) is a neural network trained on 400M image-text pairs from the internet. It learns visual concepts from natural language supervision, enabling zero-shot transfer to downstream tasks without task-specific training data.

On Mixpeek, CLIP powers visual embedding extraction, converting video frames and images into 768-dimensional vectors that capture semantic meaning. This enables similarity search across visual content using natural language queries.

Architecture

Vision Transformer (ViT-L/14) with 24 layers, 1024-dim hidden size, 16 attention heads. Text encoder is a 12-layer transformer. Both encoders project into a shared 768-dim embedding space via contrastive learning.

Mixpeek SDK Integration

// No extractor parameter takes a Hugging Face model id (checked against
// GET /v1/discovery/extractors, which returns 13), so clip-vit-large-patch14 runs
// on your side and the output is upserted through POST
// /v1/namespaces/{namespace_id}/documents/upsert. On Enterprise the other
// path is to upload the weights instead: POST /v1/namespaces/{id}/models
// accepts the huggingface format and a custom plugin loads them.
const res = await fetch(
  "https://api.mixpeek.com/v1/namespaces/ns_your_namespace/documents/upsert",
  {
    method: "POST",
    headers: {
      Authorization: "Bearer API_KEY",
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      collection_id: "col_your_collection",
      documents: [
        {
          document_id: "asset-00412",
          // The vector name has to match a vector index on the collection.
          vectors: { "image-embedding": yourVector },
          payload: { source_key: "archive/2026/asset-00412" },
        },
      ],
    }),
  },
);

// Managed alternative, if this exact model is not the requirement:
// image_extractor@v1 runs google/siglip-base-patch16-224
// (768-d) over a bucket, with no inference of your own.

Capabilities

Zero-shot image classification without fine-tuning
Cross-modal text-to-image and image-to-text retrieval
768-dimensional dense vector embeddings
Processes 224x224 pixel image patches
Supports 40+ languages via multilingual text encoder

Use Cases on Mixpeek

Visual search across video libraries, find frames matching natural language descriptions

Content moderation, detect brand logos, inappropriate content, or specific objects

E-commerce product matching, find visually similar products across catalogs

Media asset management, auto-tag and organize image/video archives

Benchmarks

Dataset	Metric	Score	Source
ImageNet zero-shot	Top-1 Accuracy	75.3%	Radford et al., 2021: Table 11
MS-COCO (text→image)	Recall@5	56.4%	Radford et al., 2021: Table 8
Flickr30k (text→image)	Recall@1	87.1%	Radford et al., 2021: Table 8