LLM Enrich

LLM Enrich stage showing structured data extraction with language models

The LLM Enrich stage uses language models to extract structured data from document content. It can identify entities, classify content, extract key information, and generate structured outputs.

Stage Category: ENRICH (Enriches documents)Transformation: N documents → N documents (with extracted data added)

When to Use

Use Case	Description
Entity extraction	Extract names, dates, amounts from text
Content classification	Categorize documents by topic/type
Key information extraction	Pull specific facts from unstructured text
Structured output generation	Convert prose to structured data

When NOT to Use

Scenario	Recommended Alternative
Simple field transformation	`json_transform`
Predefined taxonomy classification	`taxonomy_enrich`
Large-scale processing	Pre-process during indexing
Real-time low-latency	Use cached extractions

Parameters

Parameter	Type	Default	Description
`provider`	string	`google`	LLM provider: `openai`, `google`, or `anthropic` (auto-inferred from `model_name` if omitted)
`model_name`	string	provider default	Specific model, e.g. `gpt-4o-mini`, `gemini-2.5-flash-lite`, `claude-3-5-haiku`
`prompt`	string	Required	Extraction instructions (template — supports `{{DOC.field}}` / `{{INPUT.field}}`)
`content_field`	string	`content`	Field to analyze
`output_field`	string	`extracted`	Field for extracted data
`output_schema`	object	`null`	JSON schema for structured output
`batch_size`	integer	`5`	Documents per LLM call

Available Models

Model	Speed	Quality	Best For
`gpt-4o-mini`	Fast	Good	Simple extractions
`gpt-4o`	Medium	Excellent	Complex analysis
`claude-3-haiku`	Fast	Good	Quick processing
`claude-3-sonnet`	Medium	Excellent	Nuanced extraction

Configuration Examples

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract all company names and person names mentioned in this document.",
      "output_field": "entities"
    }
  }
}

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o",
      "prompt": "Extract the following information from this product review.",
      "output_field": "review_analysis",
      "output_schema": {
        "type": "object",
        "properties": {
          "sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]},
          "rating_mentioned": {"type": "number", "minimum": 1, "maximum": 5},
          "pros": {"type": "array", "items": {"type": "string"}},
          "cons": {"type": "array", "items": {"type": "string"}},
          "would_recommend": {"type": "boolean"}
        }
      }
    }
  }
}

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract key facts from this news article: main event, date, location, people involved, and outcome.",
      "output_field": "key_facts",
      "output_schema": {
        "type": "object",
        "properties": {
          "main_event": {"type": "string"},
          "date": {"type": "string"},
          "location": {"type": "string"},
          "people": {"type": "array", "items": {"type": "string"}},
          "outcome": {"type": "string"}
        }
      }
    }
  }
}

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "anthropic",
      "model_name": "claude-3-haiku",
      "prompt": "Classify this document into primary and secondary topics. Be specific.",
      "output_field": "topics",
      "output_schema": {
        "type": "object",
        "properties": {
          "primary_topic": {"type": "string"},
          "secondary_topics": {"type": "array", "items": {"type": "string"}},
          "confidence": {"type": "number", "minimum": 0, "maximum": 1}
        }
      }
    }
  }
}

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract all contact information (emails, phone numbers, addresses) from this document.",
      "output_field": "contacts",
      "output_schema": {
        "type": "object",
        "properties": {
          "emails": {"type": "array", "items": {"type": "string", "format": "email"}},
          "phones": {"type": "array", "items": {"type": "string"}},
          "addresses": {"type": "array", "items": {"type": "string"}}
        }
      }
    }
  }
}

Output Schema

Define structured output using JSON Schema:

{
  "output_schema": {
    "type": "object",
    "properties": {
      "field_name": {"type": "string"},
      "numeric_field": {"type": "number"},
      "boolean_field": {"type": "boolean"},
      "array_field": {"type": "array", "items": {"type": "string"}},
      "enum_field": {"type": "string", "enum": ["option1", "option2", "option3"]}
    },
    "required": ["field_name"]
  }
}

Supported Types

Type	Description
`string`	Text values
`number`	Numeric values
`boolean`	True/false
`array`	Lists of items
`object`	Nested objects

Output Examples

Without Schema

{
  "document_id": "doc_123",
  "content": "Apple Inc. announced...",
  "entities": "Companies: Apple Inc., Microsoft\nPeople: Tim Cook, Satya Nadella"
}

With Schema

{
  "document_id": "doc_123",
  "content": "Great product, 5 stars!...",
  "review_analysis": {
    "sentiment": "positive",
    "rating_mentioned": 5,
    "pros": ["easy to use", "great value", "fast shipping"],
    "cons": ["packaging could be better"],
    "would_recommend": true
  }
}

Performance

Metric	Value
Latency	300-800ms per batch
Batch size	5 documents default
Token usage	~200 tokens per document
Parallel	Batches processed concurrently

LLM enrichment is expensive. Consider pre-computing extractions during indexing for frequently accessed data.

Common Pipeline Patterns

Search + Extract + Filter

[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 20
          }
        ],
        "final_top_k": 20
      }
    }
  },
  {
    "stage_name": "llm_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o-mini",
        "prompt": "Extract the main topic and sentiment.",
        "output_field": "analysis",
        "output_schema": {
          "type": "object",
          "properties": {
            "topic": {"type": "string"},
            "sentiment": {"type": "string", "enum": ["positive", "neutral", "negative"]}
          }
        }
      }
    }
  },
  {
    "stage_name": "attribute_filter",
    "stage_type": "filter",
    "config": {
      "stage_id": "attribute_filter",
      "parameters": {
        "field": "analysis.sentiment",
        "operator": "eq",
        "value": "positive"
      }
    }
  }
]

Entity Extraction Pipeline

[
  {
    "stage_name": "feature_search",
    "stage_type": "filter",
    "config": {
      "stage_id": "feature_search",
      "parameters": {
        "searches": [
          {
            "feature_uri": "mixpeek://multimodal_extractor@v1/vertex_multimodal_embedding",
            "query": "{{INPUT.query}}",
            "top_k": 10
          }
        ],
        "final_top_k": 10
      }
    }
  },
  {
    "stage_name": "llm_enrich",
    "stage_type": "enrich",
    "config": {
      "stage_id": "llm_enrich",
      "parameters": {
        "provider": "openai",
        "model_name": "gpt-4o",
        "prompt": "Extract all entities with their types and relationships.",
        "output_field": "entities",
        "output_schema": {
          "type": "object",
          "properties": {
            "people": {"type": "array", "items": {"type": "object", "properties": {"name": {"type": "string"}, "role": {"type": "string"}}}},
            "organizations": {"type": "array", "items": {"type": "string"}},
            "locations": {"type": "array", "items": {"type": "string"}},
            "dates": {"type": "array", "items": {"type": "string"}}
          }
        }
      }
    }
  }
]

Writing Effective Prompts

Good Prompts

✓ "Extract the product name, price, and key features from this product listing."
✓ "Identify all dates mentioned and their associated events."
✓ "Classify the sentiment as positive, neutral, or negative, and explain why."

Poor Prompts

✗ "Analyze this document" (too vague)
✗ "Get the data" (not specific)
✗ "Tell me about it" (unclear output)

Be specific about what to extract and in what format. When using output_schema, the LLM will conform to the structure.

Multimodal Query Inputs

Pass images from your query inputs to the LLM alongside document content. This enables visual comparison, brand matching, and cross-modal analysis.

Parameters

Parameter	Type	Default	Description
`multimodal_inputs`	object	`null`	Map of INPUT field names to media types (`"image"`, `"video"`)

How It Works

Declare which INPUT fields carry multimodal content via multimodal_inputs
At runtime, the stage extracts URLs from {{INPUT.field_name}}
Images are sent to the LLM alongside the prompt and document content
Works with providers that support vision (Google Gemini, OpenAI GPT-4o, Anthropic Claude)

Configuration Examples

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "google",
      "model_name": "gemini-2.5-flash-lite",
      "prompt": "Compare this brand logo to the document. Rate alignment 1-10: {{DOC.brand_name}}",
      "output_field": "brand_alignment",
      "multimodal_inputs": {
        "query_image": "image"
      }
    }
  }
}

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o",
      "prompt": "Does this product image match the description? {{DOC.description}}",
      "output_field": "visual_match",
      "output_schema": {
        "type": "object",
        "properties": {
          "matches": {"type": "boolean"},
          "confidence": {"type": "number"},
          "differences": {"type": "array", "items": {"type": "string"}}
        }
      },
      "multimodal_inputs": {
        "product_photo": "image"
      }
    }
  }
}

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "google",
      "model_name": "gemini-2.5-flash-lite",
      "prompt": "Compare the reference image against this document's content: {{DOC.title}}",
      "output_field": "comparison",
      "multimodal_inputs": {
        "reference_image": "image",
        "style_guide": "image"
      }
    }
  }
}

Calling with Multimodal Inputs

When using multimodal_inputs, pass the image URLs in the retriever’s inputs:

{
  "inputs": {
    "query": "Find products similar to this",
    "query_image": "https://storage.example.com/brand-logo.png"
  },
  "stages": [
    {
      "stage_name": "feature_search",
      "stage_type": "filter",
      "config": {
        "stage_id": "feature_search",
        "parameters": { "..." : "..." }
      }
    },
    {
      "stage_name": "llm_enrich",
      "stage_type": "enrich",
      "config": {
        "stage_id": "llm_enrich",
        "parameters": {
          "prompt": "Compare this image to: {{DOC.product_name}}",
          "output_field": "match_score",
          "multimodal_inputs": {"query_image": "image"}
        }
      }
    }
  ]
}

If a declared multimodal input field is missing from the query inputs at runtime, the stage proceeds without it (text-only mode). No error is raised.

Bring Your Own Key (BYOK)

Use your own LLM API keys instead of Mixpeek’s default keys. This gives you control over costs, rate limits, and API usage.

Why Use BYOK?

Benefit	Description
Cost Control	Use your own LLM provider account and billing
Rate Limits	Use your own rate limits instead of shared
Compliance	Keep API calls under your own account
Key Rotation	Rotate keys without changing retrievers

Setup

Store your API key as a secret

Store your LLM provider API key in the organization secrets vault:

curl -X POST "https://api.mixpeek.com/v1/organizations/secrets" \
  -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "secret_name": "openai_api_key",
    "secret_value": "sk-proj-abc123..."
  }'

Reference the secret in your stage

Use the api_key parameter with template syntax:

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Extract key entities from this document.",
      "output_field": "entities",
      "api_key": "{{secrets.openai_api_key}}"
    }
  }
}

BYOK Configuration Example

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "openai",
      "provider": "openai",
      "model_name": "gpt-4o-mini",
      "prompt": "Summarize this document in 2-3 sentences.",
      "output_field": "summary",
      "api_key": "{{secrets.openai_api_key}}"
    }
  }
}

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "anthropic",
      "provider": "anthropic",
      "model_name": "claude-3-haiku-20240307",
      "prompt": "Extract the main topics from this content.",
      "output_field": "topics",
      "api_key": "{{secrets.anthropic_api_key}}"
    }
  }
}

{
  "stage_name": "llm_enrich",
  "stage_type": "enrich",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "provider": "google",
      "provider": "google",
      "model_name": "gemini-3.1-flash-lite",
      "prompt": "Classify the sentiment of this text.",
      "output_field": "sentiment",
      "api_key": "{{secrets.google_api_key}}"
    }
  }
}

Supported Providers

Provider	Secret Name Example	Models
OpenAI	`openai_api_key`	gpt-4o, gpt-4o-mini
Anthropic	`anthropic_api_key`	claude-3-haiku, claude-3-sonnet, claude-3-opus
Google	`google_api_key`	gemini-3.1-flash-lite, gemini-2.5-pro

When api_key is not specified, the stage uses Mixpeek’s default API keys and usage is charged to your Mixpeek account.

Custom Enrichment Model (BYO Plugin)

Use your own enrichment model deployed as a custom extractor instead of a hosted LLM provider. Set feature_uri to route enrichment through your extractor’s inference endpoint.

Parameters

Parameter	Type	Default	Description
`feature_uri`	string	`null`	Feature URI of a custom enrichment plugin. Overrides `provider`/`model` when set.

Your plugin must accept {prompt: str, document: dict} and return {text: str}.

Configuration Example

{
  "stage_name": "my_enricher",
  "config": {
    "stage_id": "llm_enrich",
    "parameters": {
      "feature_uri": "mixpeek://my_summarizer@1.0.0/summarize",
      "prompt": "Summarize: {{DOC.content}}",
      "output_field": "summary"
    }
  }
}

Set inference_type: "generate" in your plugin’s manifest to declare compatibility with LLM stages.

Error Handling

Error	Behavior
LLM timeout	Retry once, then null result
Schema validation fail	Raw text in output_field
Rate limit	Automatic backoff
Empty content	Skip enrichment
Invalid API key	Error returned with auth failure

LLM Filter - Filter using LLM evaluation
Taxonomy Enrich - Predefined classification
JSON Transform - Template-based transformation

Get started

Connect your data

Extract features

Build retrievers

Enrich & organize

Integrate & operate

Resources

When to Use

When NOT to Use

Parameters

Available Models

Configuration Examples

Output Schema

Supported Types

Output Examples

Without Schema

With Schema

Performance

Common Pipeline Patterns

Search + Extract + Filter

Entity Extraction Pipeline

Writing Effective Prompts

Good Prompts

Poor Prompts

Multimodal Query Inputs

Parameters

How It Works

Configuration Examples

Calling with Multimodal Inputs

Bring Your Own Key (BYOK)

Why Use BYOK?

Setup

BYOK Configuration Example

Supported Providers

Custom Enrichment Model (BYO Plugin)

Parameters

Configuration Example

Error Handling

​When to Use

​When NOT to Use

​Parameters

​Available Models

​Configuration Examples

​Output Schema

​Supported Types

​Output Examples

​Without Schema

​With Schema

​Performance

​Common Pipeline Patterns

​Search + Extract + Filter

​Entity Extraction Pipeline

​Writing Effective Prompts

​Good Prompts

​Poor Prompts

​Multimodal Query Inputs

​Parameters

​How It Works

​Configuration Examples

​Calling with Multimodal Inputs

​Bring Your Own Key (BYOK)

​Why Use BYOK?

​Setup

​BYOK Configuration Example

​Supported Providers

​Custom Enrichment Model (BYO Plugin)

​Parameters

​Configuration Example

​Error Handling

​Related

When to Use

When NOT to Use

Parameters

Available Models

Configuration Examples

Output Schema

Supported Types

Output Examples

Without Schema

With Schema

Performance

Common Pipeline Patterns

Search + Extract + Filter

Entity Extraction Pipeline

Writing Effective Prompts

Good Prompts

Poor Prompts

Multimodal Query Inputs

Parameters

How It Works

Configuration Examples

Calling with Multimodal Inputs

Bring Your Own Key (BYOK)

Why Use BYOK?

Setup

BYOK Configuration Example

Supported Providers

Custom Enrichment Model (BYO Plugin)

Parameters

Configuration Example

Error Handling

Related