Which embedding model should I use for text?

Start from the languages you need, the longest text you must embed in one piece, and where the model has to run. Multilingual E5-large-instruct is the default here and does well on the MTEB retrieval tasks; for self-hosting or a different size and dimension budget, see [the self-hosted embedding models compared](/curated-lists/best-self-hosted-embedding-models) and the [models directory](/models). Whatever you choose, use the same model for documents and queries, because vectors from two models do not share a space.

How many dimensions should my text embeddings have?

Common sizes run from 384 to 3072. More dimensions carry more detail and cost proportionally more storage and memory per vector, and the difference in retrieval quality between a 1024 and a 3072 dimension model is often smaller than the difference between two models of the same size. Some models are trained so that a truncated vector still works, which lets you store fewer dimensions and rescore with the full vector; [The 3072-Dimension Problem](/diagrams/the-3072-dimension-problem) shows the storage side of the trade.

Should I chunk text before embedding it?

Yes, once the text is longer than what the model reads at once or longer than the passage you want returned. E5-large-instruct reads up to 512 tokens per chunk (about 2000 characters), and longer texts are split into overlapping chunks automatically here. Chunk at natural boundaries such as headings and paragraphs, keep a table or a figure caption together with the text that refers to it, and see how to [chunk first](/guides/multimodal-chunking-strategies) for the rules by document type.

Do I embed the search query the same way as the documents?

Same model, sometimes a different prefix. Instruction-tuned models such as E5-instruct expect a short prefix that says whether the input is a query or a passage; set `input_type` to 'query' for searches and 'passage' for documents. Everything else stays identical, so the two vectors land in one space.

Where do I store text embeddings and how do I compare them?

In a vector index next to the chunk text and its metadata, so a hit can show the passage and its source. Embeddings here are L2-normalized, so cosine similarity equals the dot product and any store that supports one supports the other; read [what a similarity score means](/guides/calibrating-similarity-scores) before setting a threshold. Mixpeek stores them in collections you query through a retriever, and the [Mixpeek Vector Store](/mvs) keeps the index on the object storage you already use.

What happens when I want to change the embedding model later?

Every stored vector has to be produced again by the new model, because the old and new vectors are not comparable. Plan for it by keeping the source text and the chunk boundaries, so re-embedding is a batch job over stored chunks; the guide on how to switch models without re-embedding everything covers the migration patterns.

embedding

How do I turn text into embeddings?

How to turn text into embeddings for semantic search and RAG: which model to pick, how many dimensions you need, whether to chunk first, and where to store the vectors. Converts sentences, paragraphs and documents into dense vectors with the same model for queries and passages.

The short answer

An embedding model reads a piece of text and returns a fixed-length vector, usually 384 to 3072 numbers, positioned so that texts with similar meaning land close together. You turn text into embeddings by choosing one model, splitting long documents into chunks the model can read whole, running every chunk and every query through that same model, and storing the vectors next to the text they came from in a vector index. The model choice matters more than anything else: it fixes the vector size, the languages covered and the maximum text length, and changing it later means re-embedding the corpus.

Max file size: 10 MB

Estimated: < 1 sec per 1000 tokens

3 input formats

How It Works

Provide text content in the request body or upload a text file.

Text is optionally chunked by token count or semantic boundaries.

Each chunk is tokenized and processed through the embedding model.

Dense vectors are returned with chunk text and metadata.

Optionally, embeddings are stored directly in your namespace.

Code Examples

import os, requests

API = "https://api.mixpeek.com"
H = {"Authorization": f"Bearer {os.environ['MIXPEEK_API_KEY']}",
     "X-Namespace": os.environ["NAMESPACE_ID"]}

# 1. a bucket, with a schema that declares the field you will send
bucket = requests.post(f"{API}/v1/buckets", headers=H, json={
    "bucket_name": "text-inputs",
    "bucket_schema": {"properties": {"text": {"type": "text"}}},
}).json()

# 2. land the file as an object. the URL goes in data, on the blob
requests.post(f"{API}/v1/buckets/{bucket['bucket_id']}/objects", headers=H, json={
    "key_prefix": "run-1",
    "blobs": [{"property": "text", "type": "text",
               "data": "https://example.com/notes.txt"}],
})

# 3. a collection over that bucket, running the extractor
collection = requests.post(f"{API}/v1/collections", headers=H, json={
    "collection_name": "text-to-embeddings",
    "source": {"type": "bucket", "bucket_ids": [bucket["bucket_id"]]},
    "feature_extractor": {"feature_extractor_name": "text_extractor", "version": "v1"},
}).json()

# 4. run extraction over the bucket
requests.post(f"{API}/v1/buckets/{bucket['bucket_id']}/batches", headers=H, json={
    "collection_ids": [collection["collection_id"]],
    "auto_submit": True,
})

# 5. read the output
docs = requests.get(
    f"{API}/v1/collections/{collection['collection_id']}/documents", headers=H
).json()
print(docs)

Use Cases

Your keyword search misses documents that say the same thing in different words

You are building RAG and need the passages a question is about, not the ones that share its keywords

Support articles are not found when customers describe the problem instead of the feature

You need one vector per query that lands in the same space as your documents

Supported Input Formats

TXT

Plain text

Markdown

Quick Info

Categoryembedding

Max File Size10 MB

Est. Time< 1 sec per 1000 tokens

Extractortext-descriptor

Processing millions of files?

Run this as a managed pipeline over your whole library, no infrastructure to build or maintain. Talk to us about processing at scale.

Run it over a library

Mixpeek runs this conversion as a pipeline over a whole library in your object storage, with the output landing as queryable documents. It is not a single-file converter.

Frequently Asked Questions

Related Converters

PDF

Embeddings

PDF to Embeddings

Convert PDF documents into semantic vector embeddings for search, retrieval, and RAG applications. Pages are chunked intelligently by sections and paragraphs, then embedded using text or multimodal models.

JSON

Embeddings

JSON to Embeddings

Convert JSON objects and arrays into semantic vector embeddings. Supports nested structures, field selection, and configurable serialization strategies for optimal embedding quality.

CSV

Embeddings

CSV to Embeddings

Convert CSV files into vector embeddings by selecting and combining columns into text representations. Supports header mapping, custom delimiters, and batch processing for large datasets.

Mixed

Embeddings

Multimodal to Embeddings

Generate unified vector embeddings from mixed-modality inputs -- text, images, audio, and video combined. Enables cross-modal search where any modality can query any other modality in a single vector space.

Ready to convert text to embeddings?

Start using the Mixpeek Text to Embeddings in minutes. Sign up for a free API key and follow the documentation to get started.