Manifests

Manifests let you define Mixpeek resources in YAML files and apply them in a single operation. This enables version-controlled, reproducible infrastructure across environments.

Quick Start

# mixpeek.yaml — a complete pipeline: ingest → index → search
version: "1.0"
metadata:
  name: "video-search-env"

namespaces:
  - name: video_search
    feature_extractors:
      - name: multimodal_extractor
        version: v1

buckets:
  - name: raw_videos
    namespace: video_search
    schema:
      properties:
        video: { type: video }

collections:
  - name: video_index
    namespace: video_search
    source:
      type: bucket
      bucket: raw_videos
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v1

retrievers:
  - name: video_search_tool
    namespace: video_search
    collections: [video_index]
    input_schema:
      query:
        type: text
        required: true
    stages:
      - stage_name: search
        stage_type: filter
        config:
          stage_id: feature_search
          parameters:
            searches:
              - collection_identifiers: [video_index]
                query:
                  input_mode: text
                  text: "{{INPUT.query}}"
                feature_uri: "mixpeek://multimodal_extractor@v1/embedding"
                top_k: 10
            final_top_k: 10

Every collection needs a source and a feature_extractor; every retriever needs namespace, collections, and its stages pipeline. validate names any missing field, warns on unknown top-level keys (a typo like retreivers: won’t silently drop your search pipeline), and checks that references (namespace, bucket, collection names) resolve within the manifest.

curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"

Applying over resources that already exist

apply takes a mode query parameter. It defaults to create_only, which fails if any resource in the manifest already exists, because the manifest is treated as a closed world: a reference to a bucket that exists but is not declared is a validation error. Use create_missing to add to a namespace that already has resources. It creates what is missing, leaves what exists alone, and resolves references against the resources already in the namespace.

# Add these collections over buckets that are already there
curl -X POST "https://api.mixpeek.com/v1/manifest/apply?mode=create_missing" \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@collections.yaml"

apply plans before it creates anything. It runs the same checks as validate, and any error refuses the whole manifest: nothing is created, and each error names the resource, the field and the fix. The plan refuses:

an extractor that no longer exists or is turned off on this deployment
a retriever stage that is not registered, such as a retired stage type
a feature URI that no collection in the manifest writes, such as one naming a renamed model
two collections in one namespace that write the same vector at different sizes, such as one multimodal_extractor v2 collection with output_dimensionality: 1536 and another at the default 3072. A namespace holds one index per vector, so give both the same size:
feature_extractor: name: multimodal_extractor version: v2 parameters: output_dimensionality: 1536 # every collection writing gemini-embedding-2 in this namespace
an alert application that does not map an input its alert’s retriever requires
a resource its create request refuses: the namespace, bucket, collection, retriever, cluster, taxonomy, trigger, webhook, alert and storage connection are each built into the same request a real create sends, so a cluster with a non-boolean hierarchical_grouping or a taxonomy with an unknown taxonomy_type is refused here and reads cluster:<name>: <field>: <reason>
a bucket sync whose sync_mode is not initial_only or continuous
a seed that names a sample key the Mixpeek sample bucket does not hold (a read-only listing; if the bucket cannot be listed, validate adds a warning and refuses nothing)

POST /v1/manifest/validate runs every check above, so a manifest it passes is one apply accepts. validate, diff and lint each take the parameters form field that apply takes. A namespace budget above your organization’s budget is set to the organization’s budget, and apply reports the change in warnings. dry_run=true returns that plan and creates nothing. If a resource fails partway through, apply rolls back the ones it already created, and if the API is stopped in the middle of an apply, it rolls that apply back when it starts again.

Core Operations

Operation	Description
Apply	Create all resources defined in the manifest. `mode=create_only` (default) fails if anything exists; `mode=create_missing` fills the gaps
Validate	Check syntax, schema, and references without changes
Export	Generate a manifest from existing resources
Diff	Compare manifest against current state

Resource Types

Manifests support these resource types, applied in dependency order:

Namespaces - Isolation boundaries with feature extractors and a budget
Extractors - The extractors collections run, declared under extractors:
Storage connections and buckets - Object storage with schemas
Bucket syncs - Scheduled pulls from a connection
Collections - Document stores with indexing
Retrievers - Search pipelines
Alerts - Retrievers or system conditions that notify you
Clusters and taxonomies - Grouping and classification
Triggers and webhooks - Automation

Extractors

An extractors: section declares the extractors the manifest’s collections run. Each entry has a name, a namespace, a version (default v1), a source and an optional compute_profile. Collections reference an entry by name in feature_extractor, and namespaces in feature_extractors. Once a manifest has an extractors: section, every extractor a collection or namespace names must be declared in it. Extractors apply before collections, so a collection never names an extractor that does not exist yet. source is one of:

marketplace: a feature extractor name from the registry, for example text_extractor. This is the default when source is omitted ({marketplace: <name>}).
model: a model your organization uploaded through POST /v1/models/uploads, as org:<model_id>. Apply validates the model if it has not been validated, then serves it as an extractor at zero replicas until the first call.
huggingface: a model repository on the Hugging Face Hub, as {repo, revision, framework, token_secret}. Apply imports the snapshot into your organization’s model registry through the same checks an upload and confirm run, then serves it exactly like model. No manual upload is needed.

extractors:
  - name: text_extractor            # marketplace (the default when source is omitted)
    namespace: video_search
  - name: brand_classifier          # a model you uploaded
    namespace: video_search
    source:
      model: org:mdl_7f3a9c
  - name: minilm                    # a model from the Hugging Face Hub
    namespace: video_search
    source:
      huggingface:
        repo: sentence-transformers/all-MiniLM-L6-v2
        revision: "<40-character commit sha, or a tag>"
        framework: sentence-transformers   # the default; transformers is also accepted
        token_secret: HF_TOKEN             # only for a gated or private repo

A huggingface source is checked against the Hub at validate (read-only: repo info, refs and the file list), and apply refuses the same things:

revision is required and must be a full 40-character commit sha or a tag. A branch such as main is refused because it moves, and the same manifest has to import the same weights on every apply.
Weights must be safetensors. The import takes .safetensors, .json, .txt and .model files and skips pickle weights (pytorch_model.bin, .pt, .ckpt), because loading a pickle file can run code. A repo with only pickle weights is refused.
The files the framework needs must be at the repo root: config.json and modules.json for sentence-transformers, config.json for transformers. An extractor can serve sentence-transformers models today, so framework: transformers is refused on an extractor.
The snapshot must fit the model upload limit (5 GB). The limit is checked from the file list and again while downloading.
A gated or private repo needs token_secret, the name of an organization secret holding a Hugging Face access token (HF_TOKEN or ${{ secrets.HF_TOKEN }}). A public repo needs none.

The import records the repo, revision, resolved commit and the model card’s license on the registry entry (GET /v1/models/{model_id} shows license). Applying the same repo and revision again in the same organization reuses that entry with no Hub call and no download, and a tag that points at an already imported commit reuses it too. While the import runs, the apply’s task shows the extractor as importing_model; after that it moves through validating_model, deploying_model and warming or ready, the same states as a model source. compute_profile carries the fields of a compute profile (resource_type, batch_size, max_concurrency, memory). Apply stores them as the namespace’s batch override for that extractor, merged over the extractor’s built-in profile when a batch is built. Two nested blocks are checked rather than stored:

batch.enabled: only true is accepted. A collection that names the extractor runs it in batch.
serve: the real-time serving block (enabled, min_replicas, max_replicas, min_reason, idle_timeout_s). Validate and apply check it; realtime still runs under the extractor’s built-in serve settings, and warnings says so. A min_replicas above 0 is refused with the monthly cost of the warm replicas named, because a template adds no always-on floor. Leave it at 0 and the first call after idle waits for a replica to start.

extractors:
  - name: text_extractor
    namespace: video_search
    version: v1
    compute_profile:
      resource_type: cpu
      batch_size: 32
  - name: brand_classifier
    namespace: video_search
    source:
      model: org:mdl_7f3a9c

collections:
  - name: transcripts
    namespace: video_search
    source: {type: bucket, bucket: raw_videos}
    feature_extractor:
      feature_extractor_name: text_extractor
      version: v1

A collection naming an extractor the section does not declare is refused at plan with the name. Applying the same manifest again is a no-op for the extractors it declares.

Parameters

A parameters: section declares the values a manifest needs from whoever applies it, so a template ships without a hand edit or a secret inside it. Reference a parameter anywhere in the document as ${{ params.<name> }}. A reference that is the whole value takes the parameter’s type; one inside a longer string is formatted into it as text. An array parameter is a list and can only be a whole value.

Field	Type	Default	What it does
`name`	string	required	The name you reference as `${{ params.<name> }}`
`type`	`string`, `integer`, `number`, `boolean`, `array`	`string`	The type a supplied value must match
`items`	`string`, `integer`, `number`, `boolean`	`string`	The type of each item when `type` is `array`
`enum`	list of values	none	The allowed values. For an `array`, every item must be one of them
`required`	boolean	`true`	Whether a value must be supplied. Ignored when `default` is set
`default`	matches `type`	none	Used when no value is supplied. A list for an `array`
`description`	string	none	Shown next to the input in Studio

parameters:
  - name: camera_url
    type: string
    description: Your camera's rtsp:// url
  - name: polling_seconds
    type: integer
    default: 300

buckets:
  - name: camera_feed
    namespace: site_cameras
    sync:
      source_url: ${{ params.camera_url }}
      interval_seconds: ${{ params.polling_seconds }}

Pass the values on validate and apply as the parameters form field, a JSON object of name to value:

curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@camera.yaml" \
  -F 'parameters={"camera_url": "rtsp://10.0.0.8/stream"}'

An array parameter with allowed values:

parameters:
  - name: search_collections
    type: array
    items: string
    enum: [products, reviews, faqs]
    default: [products, reviews]
  - name: tier
    enum: [free, pro]
    default: free

retrievers:
  - name: site_search
    namespace: shop
    description: "${{ params.tier }} plan search"   # a scalar can sit inside a string
    collections: ${{ params.search_collections }}   # an array must be the whole value
    stages: [...]

Supply an array as JSON: -F 'parameters={"search_collections": ["faqs"], "tier": "pro"}'. A value outside enum is refused with the allowed values listed, and an array written inside a longer string ("types: ${{ params.search_collections }}") is refused. A missing required parameter, a value of the wrong type, a reference to an undeclared parameter, or a name declared twice is refused before any resource is created, naming parameters.<name>.

Budget

A namespace entry takes a budget with daily_usd and monthly_usd, the same fields the namespaces API takes as spend_budget. Omit it to use your organization’s namespace default. A value above your organization’s budget is set to the organization’s budget, and apply reports the clamp in warnings.

namespaces:
  - name: video_search
    feature_extractors:
      - name: multimodal_extractor
        version: v1
    budget:
      daily_usd: 5
      monthly_usd: 100

See Budgets for what happens when work reaches the limit.

Alerts

An alerts: section creates alerts beside the retrievers they watch. An alert mirrors the create request of the alerts API, plus namespace and retriever by name.

Field	Type	Default	What it does
`name`	string	required	Unique alert name
`namespace`	string	required	Namespace by name
`source`	`retriever` or `system`	`retriever`	Run a retriever, or watch a built-in condition
`retriever`	string	none	A retriever declared in this manifest, by name (`source: retriever`)
`retriever_id`	string	none	An existing retriever’s id instead of `retriever`; set one of the two
`trigger_on`	`results` or `no_results`	`results`	When a retriever alert fires
`system_condition`	object	none	The built-in condition to watch (`source: system`)
`notification_config`	object	required	How and where to notify
`enabled`	boolean	`true`	Whether the alert runs

alerts:
  - name: no_results_for_brand_queries
    namespace: video_search
    retriever: video_search_tool
    trigger_on: no_results
    notification_config:
      channels:
        - channel_type: email
          config:
            to_addresses: [ops@example.com]

Plan refuses an alert that does not map an input its retriever requires. The alert fields are described on Alerts.

Track an apply

apply returns a task_id (absent on a dry run). GET /v1/tasks/{task_id} returns every resource of the plan with its states in order (planned, creating, created, exists, updated, or failed with the finding; a collection also passes warming and ready), each with the time it was reached. The same events arrive on the change feed and on a webhook.

curl https://api.mixpeek.com/v1/tasks/$TASK_ID \
  -H "Authorization: Bearer $API_KEY"

The apply response also carries readiness: for each collection whose extractor needs a real-time serving app, whether that app is running. An app scaled to zero is running; its first call answers 503 with Retry-After while it warms.

Secret References

Use ${{ secrets.NAME }} to reference organization secrets:

buckets:
  - name: external_data
    namespace: my_namespace
    sync:
      connection_id: ${{ secrets.S3_CONNECTION_ID }}

Secrets must exist before applying. Use the validate endpoint to check for missing secrets.

Seed Data

A seed: section gives a namespace sample objects to answer its first query. After the resources are created, apply ingests them into the named bucket with one batch, bounded by max_objects (default 50, at most 500). Applying the manifest again finds that batch and ingests nothing more. Each object names a source for each blob property under blobs. The blob type comes from the bucket schema property. A source takes one of three forms. A Mixpeek sample key, from the sample set samples/starter/<file>:

seed:
  - bucket: ad-creatives
    objects:
      - blobs: {creative_asset_url: samples/starter/videos/norelco-speed.mp4}

A public https:// URL (or http://, which object create also accepts). Mixpeek fetches it the way POST /v1/buckets/{id}/objects fetches a URL blob, with the same checks: the host must resolve to a public address, redirects are re-checked, and the response must be reachable and match the property’s type.

seed:
  - bucket: ad-creatives
    objects:
      - blobs: {creative_asset_url: "https://cdn.example.com/ads/launch.mp4"}

One object in your own storage, given as {connection, path}. connection names a storage connection declared under storage_connections in the same manifest, or one that already exists in the organization. path is the object’s key, written the way a bucket sync’s source_path is. Apply reads the object through the connection with the same provider client a bucket sync uses and copies it into the bucket’s storage. The connection’s provider must be s3, tigris, gcs, azure_blob, backblaze or supabase; Google Drive and SharePoint name files by id and Snowflake yields rows, so those connections are refused here and belong in bucket_syncs.

storage_connections:
  - name: brand_s3
    provider: s3
    config:
      bucket_name: brand-assets
      region: us-east-1
      credentials:
        type: access_key
        access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

seed:
  - bucket: ad-creatives
    objects:
      - blobs:
          creative_asset_url: {connection: brand_s3, path: ads/launch.mp4}

Validate and apply refuse the same seeds before anything is created: a connection that is not declared and does not exist, a connection whose provider cannot read one object by path (the error names the provider), a blob property the bucket schema does not declare, a URL that object create would refuse (no host, a private or internal address, unreachable, or the wrong content type), and any other s3://, gs:// or file:// string, since object storage is read only through a connection. A seed’s batch is billed to your organization when it is submitted, like any batch you create, whichever form its blobs take. When the bucket declares other properties, set them on the object beside blobs, the way the object create API takes them:

buckets:
  - name: ad-creatives
    namespace: my_namespace
    schema:
      properties:
        creative_asset_url: {type: video}
        ad_id: {type: string, required: true}
        brand_slug: {type: string, required: true}

seed:
  - bucket: ad-creatives
    max_objects: 2
    objects:
      - blobs: {creative_asset_url: samples/starter/videos/norelco-speed.mp4}
        ad_id: sample-norelco
        brand_slug: norelco

Validate and apply check every object against the bucket’s schema before anything is created. A missing required property, a property the bucket does not declare, a value of the wrong type, or a blob property given outside blobs is refused, and the error names the bucket, the object and the property.

Seed or bucket sync

Use seed for a curated handful of objects, up to 500, that the first query should find. Use bucket_syncs with sync_mode: initial_only to import everything under a prefix of a storage connection once. The sync lists the prefix, applies file_filters and schema_mapping, and ingests the files in pages, with retries and a dead-letter queue for files that fail. sync_mode: continuous keeps polling for new files.

bucket_syncs:
  - name: brand-archive
    bucket: ad-creatives
    connection: brand_s3
    source_path: ads/2025/
    sync_mode: initial_only
    file_filters:
      include_patterns: ["*.mp4"]

sync_mode takes exactly initial_only or continuous; any other value is refused at validate.

Taxonomy Labels

A flat taxonomy tags each document with its closest label. Write the labels inline on the taxonomy and name the feature to match on:

taxonomies:
  - name: creative-format
    namespace: ads
    type: flat
    match_on: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"
    target_field: creative_format
    labels:
      - {label: testimonial, description: "A customer speaks to camera about results they saw."}
      - {label: tutorial, description: "A step by step routine that teaches while it sells."}

collections:
  - name: ad-scenes
    namespace: ads
    source: {type: bucket, bucket: ads}
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v2
      input_mappings: {video: video}
      parameters: {run_multimodal_embedding: true, output_dimensionality: 1536}
    taxonomy_applications:
      - {taxonomy: creative-format, execution_mode: materialize}

Field	Type	Default	What it does
`labels`	list	none	Each entry has `label` and an optional `description`. Any other key (a string, number or boolean) is stored on the label and written to the document as `<target_field>_<key>`
`match_on`	feature URI	inferred	The feature the classified collections write. The labels are embedded into the same feature. Optional when every collection applying the taxonomy writes exactly one feature that can embed text
`target_field`	string	taxonomy name, `-` as `_`	The document field the matched label is written to

The manifest generates the resources the taxonomy runs on, and validate, the dry run and apply treat them like declared ones:

a bucket <taxonomy>-labels with content (text) and label (string) properties
a collection <taxonomy>-labels that embeds each label’s text, <label>: <description>, with the extractor and feature match_on names. It copies output_dimensionality from the collections applying the taxonomy, so labels and documents share one vector space
a retriever <taxonomy>-lookup that returns the single closest label for a document’s vector
a seed: entry for the label bucket, one object per label
the taxonomy’s config, wiring the retriever and the label collection, with label written to target_field

Each generated resource’s description starts with Generated for taxonomy '<name>', and each row of the dry run and apply response for one carries generated_for: <taxonomy>. A resource you declare under a generated name is refused; rename yours. Validate refuses:

labels on a taxonomy that is not flat
a label given twice, or an empty label list
labels together with config.source_collection or config.retriever
a match_on whose extractor cannot compute that feature from text, such as a transcription embedding, which is computed from audio
a collection applying the taxonomy that does not write the match_on feature

The older type: flat with nodes reads the same way: each node’s name becomes a label and its description the label’s description. It needs match_on unless it can be inferred as above, and is refused until it has one. The seed runs once per label bucket, like any seed. To change the labels of a namespace that is already applied, add or edit objects in the <taxonomy>-labels bucket directly. An export writes the generated resources and the taxonomy config out in full, since the label text lives in the bucket’s objects; the exported manifest applies as it is.

Acceptance

A manifest can carry an acceptance: block: the steps that show its use case works and what each must return. Apply creates nothing from it. Apply runs the steps when you ask with run_acceptance=true, and Mixpeek’s template acceptance harness runs them after every template apply.

Field	Type	What it does
`steps`	list, at least one	The steps, run in order. Step names must be unique
`steps[].name`	string	Shown when the step fails
`steps[].kind`	`query`, `documents`, `alert` or `realtime`	What the step does
`steps[].retriever`	string	`query`: a retriever declared in this manifest
`steps[].collection`	string	`documents`: a collection declared in this manifest
`steps[].alert`	string	`alert`: an alert declared in this manifest
`steps[].feature_uri`	string	`realtime`: the `mixpeek://` feature URI to call
`steps[].inputs`	object	`query` and `realtime`: the inputs sent
`steps[].expect`	object	The outcome the step must produce, below
`max_cost_usd`	number	The most one acceptance run may cost, read from the meter

Each kind reads its own expect fields:

Kind	What runs	`expect` fields
`query`	the retriever, executed with `inputs`	`min_results`; `top_field` (a dotted path into the top result, such as `metadata.brand_slug`) with `top_equals` or `top_contains`
`documents`	the collection’s document count	`min_documents`
`alert`	the alert’s executions that triggered	`min_fires`
`realtime`	`POST /v1/inference` with `feature_uri` and `inputs`	`dims`, the length of the returned vector

One step of each kind:

acceptance:
  max_cost_usd: 0.5
  steps:
    - name: seeded documents written
      kind: documents
      collection: transcripts
      expect: {min_documents: 2}
    - name: golden query
      kind: query
      retriever: video_search_tool
      inputs: {query: "electric shaver"}
      expect: {min_results: 1, top_field: metadata.brand_slug, top_equals: norelco}
    - name: shaver alert fires
      kind: alert
      alert: shaver_spotted
      expect: {min_fires: 1}
    - name: text embedding answers
      kind: realtime
      feature_uri: mixpeek://text_extractor@v1/multilingual_e5_large_instruct_v1
      inputs: {text: "electric shaver"}
      expect: {dims: 1024}

Validate refuses a step whose retriever, collection or alert this manifest does not declare, an expect field the step’s kind does not read, and a step with no outcome, each naming acceptance.steps.<name>.

Wait for the seed and run the steps on apply

Two query flags on apply make it wait until the namespace is usable and then prove it:

Flag	What it does
`wait_for_seed=true`	After the resources exist, waits until every `seed:` batch has finished. The response lists each in `seed_batches` with its `status` and `documents_written`. A batch that failed, or did not finish in time, is also named in `errors`
`run_acceptance=true`	Implies `wait_for_seed`, and also waits for the serving apps listed in `readiness`. Then runs each step in order and returns the outcome in `acceptance`. A manifest with no `acceptance:` block is refused with 400

Both wait within deadline_seconds, or 900 seconds when you omit it. One clock covers the serving apps, the seed and the polling of documents and alert steps, which re-read until their count is reached or the time runs out.

curl -X POST "https://api.mixpeek.com/v1/manifest/apply?run_acceptance=true&deadline_seconds=900" \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"

{
  "success": true,
  "seed_batches": [
    {"bucket": "creative_assets", "batch_id": "btch_123", "status": "COMPLETED",
     "terminal": true, "usable": true, "documents_written": 4}
  ],
  "acceptance": {
    "passed": false,
    "steps": [
      {"name": "seeded documents written", "kind": "documents", "passed": true,
       "expected": {"min_documents": 2}, "observed": {"documents": 4}, "error": null},
      {"name": "golden query", "kind": "query", "passed": false,
       "expected": {"min_results": 1, "top_field": "metadata.brand_slug", "top_equals": "norelco"},
       "observed": {"results": 3, "top_value": "philips"},
       "error": "top metadata.brand_slug is 'philips', expected 'norelco'"},
      {"name": "max_cost_usd", "kind": "cost", "passed": true,
       "expected": {"max_cost_usd": 0.5}, "observed": {"spent_usd": 0.012, "metered_from": "namespace creation"}, "error": null}
    ]
  },
  "errors": ["acceptance failed: golden query (top metadata.brand_slug is 'philips', expected 'norelco')"]
}

How to read the result:

success reports resource creation only, exactly as without the flags. Read acceptance.passed for the use case.
A step passes only when it ran and every expected outcome held. A step that could not run is passed: false and its error starts with not run: and says why: a seed batch failed or was still processing at the deadline, a serving app was not running, the apply rolled back, or the cost cap was reached.
A failed run is also named in errors, so a caller that only reads errors sees it.
On a dry run both flags do nothing, and warnings says so.

Billing. Acceptance steps on apply are normal executions in your organization: each query and realtime call is billed like the same call made directly. max_cost_usd is read from your usage meter (the per-namespace credits in GET /v1/organizations/billing/usage/breakdown). Before each step after the first, a run that has reached the cap stops, and the remaining steps report not run. A closing max_cost_usd row compares the spend with the cap. When this apply created the namespace, the spend counts from its creation, seed ingest included. On a namespace that already existed it counts from the first step. Metering of very small usage can lag a few minutes, so the reading is a lower bound right after the run. Re-running the steps. Re-apply the same manifest with mode=create_missing&run_acceptance=true. Every resource reads exists, nothing is created, the seed is not ingested again (its earlier batch is reused), and the steps run against the existing resources. Under the default create_only a re-apply is refused because the namespace exists, and the steps report not run.

Dependency Resolution

Resources are created in topological order. The manifest engine:

Resolves cross-resource references automatically
Detects circular dependencies
Rolls back all changes if any creation fails

Authoring manifests with an LLM

An LLM can write a complete manifest on the first or second try if it reads what the API publishes and lets validate check the result. The loop has six steps.

Read the schema. GET /v1/discovery/schema returns:
- schemas: a JSON schema for every section (parameters, namespace, extractor, storage_connection, bucket, bucket_sync, collection, retriever, alert, cluster, taxonomy, trigger, webhook, seed, acceptance) and for the whole manifest.
- dependency_graph: for each type, the types whose names it references. A bucket sync needs a bucket and a storage connection, a seed needs a bucket, an acceptance step needs a retriever, collection or alert.
- examples: a short YAML block for each type.
- example_manifest: the complete manifest shown below.
- extractor_catalog: every extractor and version this plane serves, with its input types, the feature URIs it writes and its parameters with defaults.
- stage_catalog: every retriever stage_id with its category and parameter names.
- authoring_loop: these steps in short form.
Use GET /v1/discovery/extractors and GET /v1/discovery/stages for the full parameter schemas and stage examples.
Write the manifest. Reference resources by name. Copy feature URIs from extractor_catalog. A collection writes a feature URI only when the extractor parameters that produce it are on, for example run_transcription_embedding: true for the transcript embedding.
Validate. POST /v1/manifest/validate returns valid, errors, warnings, missing_secrets, resources (a count per type) and the declared parameters. Validate refuses what apply would refuse, so fix every entry in errors and rerun. Send parameter values in the parameters form field as a JSON object.
Lint. POST /v1/manifest/lint returns suggestions that each carry a code, a location, a suggestion and a fix_example. Three rules catch wiring that validates but cannot work: UNUSED_BUCKET (no collection reads the bucket), TAXONOMY_INPUT_NOT_WRITTEN (the taxonomy reads a feature that the collection it is applied to does not write) and RETRIEVER_FEATURE_NOT_IN_COLLECTIONS (the retriever searches a feature that none of its own collections write).
Dry run. POST /v1/manifest/apply with dry_run=true returns the plan: what would be created, in dependency order, and any refusal. Nothing is created.
Apply. POST /v1/manifest/apply creates the resources and returns the result, or the task to poll for progress.

To have the API write the first draft, call POST /v1/manifest/generate with a description. The model is given the live extractor and stage catalogs and the example below. The answer goes through the same checks as validate. If they find errors, the model is shown them and asked to fix its answer, up to two times. The response keeps manifest, format, manifest_name and description, and adds validation (valid, errors, warnings) and repair_rounds. When validation.valid is false the errors are still listed, so you can fix them yourself and continue from step 3.

Complete example

This manifest syncs ad videos from S3 once, seeds a few sample objects, indexes the ads in two collections, searches them with a hybrid retriever, tags each scene with a creative format, groups scenes by look and states what a working setup returns. It validates once ads_s3_bucket is supplied and the secrets AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY exist.

version: "1.0"
metadata:
  name: ad-creative-search
  description: Search video ads by moment, tag each scene with a creative format, group scenes by look

parameters:
  - name: ads_s3_bucket
    type: string
    required: true
    description: Name of the S3 bucket that holds your ad videos
  - name: ads_prefix
    type: string
    default: ads/
    description: Folder inside that bucket to sync

namespaces:
  - name: ad-creative
    description: Ad creative search
    feature_extractors:
      - name: multimodal_extractor
        version: v2
    payload_indexes:
      - {field_name: metadata.brand, type: keyword}

storage_connections:
  - name: ads-s3
    provider: s3
    description: Read access to the bucket that holds the ad videos
    config:
      bucket: ${{ params.ads_s3_bucket }}
      region: us-east-1
      credentials:
        type: access_key
        access_key_id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        secret_access_key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

buckets:
  - name: ads
    namespace: ad-creative
    description: Video ads, plus the creative format labels (kind=label)
    schema:
      properties:
        video: {type: video}
        metadata: {type: object}
        content: {type: text}
        label: {type: string}
        kind: {type: string}

bucket_syncs:
  - name: ads-from-s3
    bucket: ads
    connection: ads-s3
    source_path: ${{ params.ads_prefix }}
    sync_mode: initial_only
    file_filters:
      include_patterns: ["*.mp4", "*.mov"]

seed:
  - bucket: ads
    max_objects: 5
    objects:
      - blobs: {video: samples/starter/videos/norelco-speed.mp4}
        kind: ad
        metadata: {brand: norelco}
      - texts: {content: "A real customer speaks to camera about the results she saw."}
        kind: label
        label: testimonial
      - texts: {content: "Skin shown before use and after use, side by side."}
        kind: label
        label: before-and-after

collections:
  - name: creative-formats
    namespace: ad-creative
    description: Creative format labels, embedded in the same space as the ad scenes
    source:
      type: bucket
      bucket: ads
      filters: {field: kind, operator: eq, value: label}
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v2
      input_mappings: {text: content}
      field_passthrough:
        - {source_path: label}
      parameters:
        run_multimodal_embedding: true
  - name: ad-scenes
    namespace: ad-creative
    description: Every ad split into scenes, with a visual embedding and a transcript embedding
    source:
      type: bucket
      bucket: ads
      filters: {field: kind, operator: ne, value: label}
    feature_extractor:
      feature_extractor_name: multimodal_extractor
      version: v2
      input_mappings: {video: video}
      field_passthrough:
        - {source_path: metadata.brand}
      parameters:
        split_method: scene
        run_multimodal_embedding: true
        run_transcription: true
        run_transcription_embedding: true
    taxonomy_applications:
      - {taxonomy: creative-format, execution_mode: materialize}

retrievers:
  - name: format-lookup
    namespace: ad-creative
    description: Finds the closest creative format label for a scene. Used by the creative-format taxonomy.
    collections: [creative-formats]
    input_schema:
      query_embedding: {type: text, required: true}
    stages:
      - stage_name: match
        stage_type: filter
        stage_id: feature_search
        parameters:
          searches:
            - feature_uri: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"
              query: {input_mode: vector, vector: "{{INPUT.query_embedding}}"}
              top_k: 1
          final_top_k: 1
  - name: ad-moment-search
    namespace: ad-creative
    description: Describe a moment and get the matching ad scenes
    collections: [ad-scenes]
    input_schema:
      query:
        type: text
        required: true
        description: Describe the moment, the claim or the format
    stages:
      - stage_name: recall
        stage_type: filter
        stage_id: feature_search
        parameters:
          searches:
            - feature_uri: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"
              query: {input_mode: text, value: "{{INPUT.query}}"}
              top_k: 50
              weight: 1.0
            - feature_uri: "mixpeek://multimodal_extractor@v2/multilingual_e5_large_instruct_v1"
              query: {input_mode: text, value: "{{INPUT.query}}"}
              top_k: 50
              weight: 0.6
          final_top_k: 20

taxonomies:
  - name: creative-format
    namespace: ad-creative
    description: Tags every scene with its creative format
    type: flat
    config:
      taxonomy_type: flat
      retriever: format-lookup
      input_mappings:
        - {input_key: query_embedding, source_type: payload, path: "mixpeek://multimodal_extractor@v2/gemini-embedding-2"}
      source_collection:
        collection: creative-formats
        enrichment_fields:
          - {field_path: label, target_field: creative_format, merge_mode: replace}

clusters:
  - name: visual-styles
    namespace: ad-creative
    description: Groups scenes by how they look
    source_collections: [ad-scenes]
    cluster_type: vector
    vector_config:
      feature_uris: ["mixpeek://multimodal_extractor@v2/gemini-embedding-2"]
      clustering_method: kmeans
      algorithm_params: {n_clusters: 6}
    llm_labeling:
      provider: google
      model_name: gemini-2.5-flash
      labeling_inputs:
        input_mappings:
          - {input_key: text, source_type: payload, path: transcription}
    enrich_source_collection: true

acceptance:
  max_cost_usd: 5
  steps:
    - name: scenes are indexed
      kind: documents
      collection: ad-scenes
      expect: {min_documents: 1}
    - name: texture query finds a scene
      kind: query
      retriever: ad-moment-search
      inputs: {query: close-up of foundation being blended on skin}
      expect: {min_results: 1}

Workflows

Environment Replication

# Export from production
curl https://api.mixpeek.com/v1/manifest/export \
  -H "Authorization: Bearer $PROD_KEY" \
  -o prod-manifest.yaml

# Apply to staging
curl -X POST https://api.mixpeek.com/v1/manifest/apply \
  -H "Authorization: Bearer $STAGING_KEY" \
  -F "manifest_file=@prod-manifest.yaml"

Describe what is in a namespace

Add detail=stats to an export to see the data as well as the configuration. The YAML gets a read-only status section with each collection’s document count, field types and most common field values, and each cluster’s groups with their labels and sizes. Apply ignores status, so the file still applies as-is. detail=stats covers one namespace, so pass namespace_id.

curl "https://api.mixpeek.com/v1/manifest/export?namespace_id=ns_abc123&detail=stats" \
  -H "Authorization: Bearer $API_KEY" \
  -o namespace-with-stats.yaml

Each stats read has a 2 second limit. A field or cluster whose read fails or times out shows unavailable with the reason, never an empty list.

Pre-deployment Validation

# Validate without applying
curl -X POST https://api.mixpeek.com/v1/manifest/validate \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml" \
  -F 'parameters={"camera_url": "rtsp://10.0.0.8/stream"}'   # only if the manifest declares parameters

# Check what would change
curl -X POST https://api.mixpeek.com/v1/manifest/diff \
  -H "Authorization: Bearer $API_KEY" \
  -F "manifest_file=@mixpeek.yaml"

CI/CD Integration

# GitHub Actions example
- name: Deploy Mixpeek Resources
  run: |
    curl -X POST https://api.mixpeek.com/v1/manifest/apply \
      -H "Authorization: Bearer ${{ secrets.MIXPEEK_API_KEY }}" \
      -F "manifest_file=@mixpeek.yaml"

Get started

Connect your data

Extract features

Build retrievers

Enrich & organize

Integrate & operate

Resources

Quick Start

Applying over resources that already exist

Core Operations

Resource Types

Extractors

Parameters

Budget

Alerts

Track an apply

Secret References

Seed Data

Seed or bucket sync

Taxonomy Labels

Acceptance

Wait for the seed and run the steps on apply

Dependency Resolution

Authoring manifests with an LLM

Complete example

Workflows

Environment Replication

Describe what is in a namespace

Pre-deployment Validation

CI/CD Integration

References

​Quick Start

​Applying over resources that already exist

​Core Operations

​Resource Types

​Extractors

​Parameters

​Budget

​Alerts

​Track an apply

​Secret References

​Seed Data

​Seed or bucket sync

​Taxonomy Labels

​Acceptance

​Wait for the seed and run the steps on apply

​Dependency Resolution

​Authoring manifests with an LLM

​Complete example

​Workflows

​Environment Replication

​Describe what is in a namespace

​Pre-deployment Validation

​CI/CD Integration

​References

Quick Start

Applying over resources that already exist

Core Operations

Resource Types

Extractors

Parameters

Budget

Alerts

Track an apply

Secret References

Seed Data

Seed or bucket sync

Taxonomy Labels

Acceptance

Wait for the seed and run the steps on apply

Dependency Resolution

Authoring manifests with an LLM

Complete example

Workflows

Environment Replication

Describe what is in a namespace

Pre-deployment Validation

CI/CD Integration

References