> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# HTTP API

> Sync items from any JSON REST endpoint into a Mixpeek bucket, one object per item.

<Note>
  Each item in the API response becomes a bucket object. The item's JSON body is stored as the object blob.
</Note>

## Overview

Mixpeek polls a JSON endpoint on a schedule. On each run it:

1. Calls the URL in `source_path` with the configured method and headers.
2. Reads the array of items at `items_path`.
3. Creates one bucket object per item, keyed on `item_id_field`.
4. Skips items whose ID already exists, unless you set `skip_duplicates` to `false`.

## Prerequisites

* An endpoint that returns JSON or newline-delimited JSON.
* An array of items, at the root or at a nested path.
* A field on each item that is unique and stable.

## Configuration

### Connection-level fields

| Field                   | Required | Description                                                   |
| ----------------------- | -------- | ------------------------------------------------------------- |
| `item_id_field`         | Yes      | Field on each item used as the deduplication ID               |
| `credentials.headers`   | No       | HTTP headers, for example `Authorization` or `X-API-Key`      |
| `http_method`           | No       | Defaults to `GET`                                             |
| `request_body`          | No       | JSON body for `POST` requests                                 |
| `items_path`            | No       | Dot-notation path to the array, for example `data.results`    |
| `item_modified_field`   | No       | Timestamp field for incremental sync (ISO 8601 or Unix epoch) |
| `response_content_type` | No       | `json` (default) or `jsonl`                                   |
| `user_agent`            | No       | Defaults to `Mixpeek HTTP API Sync/1.0`                       |
| `request_timeout`       | No       | Seconds, defaults to `30`                                     |

### Sync-level fields

| Field                      | Required | Description              |
| -------------------------- | -------- | ------------------------ |
| `source_path`              | Yes      | The full API URL to call |
| `polling_interval_seconds` | No       | Defaults to `300`        |
| `batch_size`               | No       | Defaults to `50`         |

<Tip>
  Leave `items_path` unset when the response is a bare array. Set it to `hits` for `{"hits": [...]}`.
</Tip>

## Setup

<Steps>
  <Step title="Create the storage connection">
    <CodeGroup>
      ```python Python theme={null}
      from mixpeek import Mixpeek

      client = Mixpeek(api_key="your-mixpeek-api-key")

      connection = client.organizations.connections.create(
          name="Internal Orders API",
          provider_type="http_api",
          provider_config={
              "credentials": {
                  "type": "http_header",
                  "headers": {"Authorization": "Bearer your-api-token"},
              },
              "items_path": "data.orders",
              "item_id_field": "order_id",
              "item_modified_field": "updated_at",
          },
      )
      print(f"Created connection: {connection['connection_id']}")
      ```

      ```javascript JavaScript theme={null}
      import { Mixpeek } from 'mixpeek-sdk'

      const client = new Mixpeek({ apiKey: 'your-mixpeek-api-key' })

      const connection = await client.organizations.connections.create({
        name: 'Internal Orders API',
        provider_type: 'http_api',
        provider_config: {
          credentials: {
            type: 'http_header',
            headers: { Authorization: 'Bearer your-api-token' },
          },
          items_path: 'data.orders',
          item_id_field: 'order_id',
          item_modified_field: 'updated_at',
        },
      })
      console.log('Created connection:', connection.connection_id)
      ```

      ```bash cURL theme={null}
      curl -X POST https://api.mixpeek.com/v1/organizations/connections \
        -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
        -H "Content-Type: application/json" \
        -d '{
          "name": "Internal Orders API",
          "provider_type": "http_api",
          "provider_config": {
            "credentials": {
              "type": "http_header",
              "headers": {"Authorization": "Bearer your-api-token"}
            },
            "items_path": "data.orders",
            "item_id_field": "order_id",
            "item_modified_field": "updated_at"
          }
        }'
      ```
    </CodeGroup>
  </Step>

  <Step title="Create a sync on your bucket">
    <CodeGroup>
      ```python Python theme={null}
      sync = client.buckets.syncs.create(
          bucket_id="bkt_your_bucket_id",
          connection_id=connection["connection_id"],
          source_path="https://api.example.com/v2/orders",
          polling_interval_seconds=900,
          batch_size=100,
      )
      print(f"Sync created: {sync['sync_config_id']}")
      ```

      ```bash cURL theme={null}
      curl -X POST https://api.mixpeek.com/v1/buckets/bkt_your_bucket_id/syncs \
        -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY" \
        -H "X-Namespace: ns_your_namespace_id" \
        -H "Content-Type: application/json" \
        -d '{
          "connection_id": "conn_your_connection_id",
          "source_path": "https://api.example.com/v2/orders",
          "polling_interval_seconds": 900,
          "batch_size": 100
        }'
      ```
    </CodeGroup>
  </Step>

  <Step title="Verify the connection">
    ```bash cURL theme={null}
    curl -X POST https://api.mixpeek.com/v1/organizations/connections/conn_your_connection_id/test \
      -H "Authorization: Bearer YOUR_MIXPEEK_API_KEY"
    ```
  </Step>
</Steps>

## Incremental sync

Set `item_modified_field` to a timestamp field. Mixpeek stores the highest value it has seen and requests only newer items on the next run.

Without it, every run reads the full response. Deduplication still applies, so repeated items do not create repeated objects.

## Related

<CardGroup cols={2}>
  <Card title="RSS" icon="rss" href="/docs/integrations/web-data/rss">
    Sync RSS and Atom feeds.
  </Card>

  <Card title="Sync configuration" icon="rotate" href="/docs/platform/syncs">
    Sync modes, reconciliation, and scheduling.
  </Card>
</CardGroup>