Meilisearch Plugin

Monitor Meilisearch search engine using Prometheus metrics endpoint with metrics covering HTTP performance, index statistics, database size, and task processing.

Overview

The Meilisearch plugin collects metrics from Meilisearch's Prometheus /metrics endpoint including:

• Database Metrics - Total database size, used storage, utilization percentage
• Index Statistics - Number of indices, total documents
• HTTP Performance - Request count, response times, average latency
• Indexing Status - Whether indexing is currently active
• Task Statistics - Total tasks processed

Requirements

Meilisearch Version

• Minimum: Meilisearch 1.0
• Recommended: Meilisearch 1.5 or later
• Tested with: Meilisearch 1.3, 1.4, 1.5, 1.6

IMPORTANT: Meilisearch must be started with --experimental-enable-metrics flag or MEILI_EXPERIMENTAL_ENABLE_METRICS=true environment variable to expose the Prometheus /metrics endpoint.

Python Dependencies

pip install requests>=2.28.0

Auto-installed when using PLUGINS=meilisearch during agent installation.

Meilisearch Access

The agent needs HTTP access to Meilisearch Prometheus metrics endpoint (default port 7700).

Configuration

Basic Configuration

plugins:
  meilisearch:
    enabled: true
    host: localhost
    port: 7700

With API Key

plugins:
  meilisearch:
    enabled: true
    host: localhost
    port: 7700
    api_key: your-master-key

With HTTPS

plugins:
  meilisearch:
    enabled: true
    host: search.example.com
    port: 7700
    protocol: https
    api_key: your-api-key

All Configuration Options

plugins:
  meilisearch:
    enabled: true                # Enable/disable plugin
    host: localhost              # Meilisearch host
    port: 7700                   # Meilisearch port
    protocol: http               # Protocol: http or https
    api_key: ""                  # API key (master key recommended)

Environment Variables

Configuration can be overridden with environment variables:

export MEILISEARCH_HOST="localhost"
export MEILISEARCH_PORT="7700"
export MEILISEARCH_PROTOCOL="http"  # or "https"
export MEILISEARCH_API_KEY="your-master-key"

Meilisearch Setup

Installation

Binary installation:

# Download latest release
curl -L https://install.meilisearch.com | sh

# Run Meilisearch with metrics enabled
./meilisearch --experimental-enable-metrics

# Or with master key and metrics
./meilisearch \
  --master-key="your-secure-master-key" \
  --experimental-enable-metrics

Docker:

docker run -d \
  --name meilisearch \
  -p 7700:7700 \
  -e MEILI_MASTER_KEY="your-secure-master-key" \
  -e MEILI_EXPERIMENTAL_ENABLE_METRICS="true" \
  getmeili/meilisearch:latest

As systemd service:

# Create service file
sudo nano /etc/systemd/system/meilisearch.service

[Unit]
Description=Meilisearch
After=network.target

[Service]
Type=simple
User=meilisearch
WorkingDirectory=/var/lib/meilisearch
Environment="MEILI_MASTER_KEY=your-secure-master-key"
Environment="MEILI_EXPERIMENTAL_ENABLE_METRICS=true"
Environment="MEILI_DB_PATH=/var/lib/meilisearch/data"
Environment="MEILI_HTTP_ADDR=0.0.0.0:7700"
ExecStart=/usr/local/bin/meilisearch
Restart=on-failure

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl start meilisearch
sudo systemctl enable meilisearch

API Key Configuration

Master key (recommended for metrics endpoint):

• Set via MEILI_MASTER_KEY environment variable
• Required for accessing the Prometheus /metrics endpoint
• Provides full access to all API operations

Note: The /metrics endpoint typically requires the master key. Custom API keys may not have access to this experimental endpoint.

Collected Metrics

Note: This plugin collects metrics from Meilisearch's Prometheus /metrics endpoint (requires --experimental-enable-metrics flag). The available metrics depend on what Meilisearch exposes.

Database Metrics

Index Metrics

Indexing Status

HTTP Metrics

Task Metrics

Dashboard Metrics

The StatusRadar dashboard displays:

Overview Card

• Database Size - Total and used storage (with utilization %)
• Indices - Number of indices
• Documents - Total document count across indices
• Indexing Status - Whether indexing is active

Database Size Chart

• Total database size over time
• Used database size over time
• Utilization percentage

Index Statistics Chart

• Document count over time
• Index count over time

HTTP Performance Chart

• Total HTTP requests
• Average response time (ms)
• Request rate

Task Statistics Chart

• Total tasks processed
• Task processing rate

Installation

Quick Install

PLUGINS='meilisearch' \
TOKEN='your-agent-token' \
MEILISEARCH_API_KEY='your-master-key' \
bash -c "$(curl -sL https://statusradar.dev/install-agent.sh)"

Install on Existing Agent

1. Configure agent:

   sudo nano /opt/statusradar/config/agent.yaml

   Add:

   plugins:
     meilisearch:
       enabled: true
       host: localhost
       port: 7700
       api_key: your-master-key

2. Restart agent:

   sudo systemctl restart statusradar-agent

3. Verify:

   sudo journalctl -u statusradar-agent -n 50 --no-pager | grep meilisearch

   Expected:

   INFO: Plugin meilisearch: Metrics collected successfully
   INFO: Plugin meilisearch: 5 indices, 12345 documents, 256 MB database (75% used)

Testing

Manual Plugin Test

cd /opt/statusradar
python3 plugins/meilisearch_plugin.py

Expected Output:

Plugin: meilisearch
Enabled: True
Available: True

Collecting metrics...
{
  "db_size_bytes": 268435456,
  "used_db_size_bytes": 201326592,
  "db_utilization_percent": 75.0,
  "index_count": 5,
  "index_docs_count": 12345,
  "is_indexing": 0,
  "http_requests_total": 9876,
  "http_response_time_seconds_total": 123.45,
  "http_avg_response_time_ms": 12.5,
  "nb_tasks_total": 9888
}

Test Meilisearch API

# Health check
curl http://localhost:7700/health

# Prometheus metrics endpoint (requires experimental flag)
curl http://localhost:7700/metrics

# With API key
curl http://localhost:7700/metrics \
  -H "Authorization: Bearer your-master-key"

# Check metrics are being exposed
curl http://localhost:7700/metrics | grep meilisearch

Troubleshooting

Plugin Not Collecting Metrics

Check 1: Is Meilisearch running?

sudo systemctl status meilisearch
# Or check process:
ps aux | grep meilisearch

Check 2: Is metrics endpoint enabled?

# Should return Prometheus metrics
curl http://localhost:7700/metrics

# If you get 404, Meilisearch is not started with --experimental-enable-metrics

Check 3: Can agent access Meilisearch metrics?

# Without API key
curl http://localhost:7700/metrics | head -20

# With API key
curl http://localhost:7700/metrics \
  -H "Authorization: Bearer your-master-key" | head -20

Check 4: Check agent logs

sudo journalctl -u statusradar-agent -n 100 --no-pager | grep meilisearch

Common Errors

"Connection refused"

Error:

ERROR: Plugin meilisearch: Connection refused

Causes:

1. Meilisearch not running
2. Wrong host/port
3. Firewall blocking connection

Solution:

# Check Meilisearch is running
ps aux | grep meilisearch

# Check port
sudo netstat -tlnp | grep 7700

# Test metrics endpoint
curl http://localhost:7700/metrics

"Invalid API key"

Error:

ERROR: Plugin meilisearch: 401 Unauthorized - Invalid API key

Solution:

Check API key:

# Test with API key
curl http://localhost:7700/metrics \
  -H "Authorization: Bearer your-master-key"

If forgotten, restart Meilisearch with new master key:

./meilisearch \
  --master-key="new-secure-master-key" \
  --experimental-enable-metrics

"Forbidden"

Error:

ERROR: Plugin meilisearch: 403 Forbidden

Cause: API key doesn't have required permissions for metrics endpoint

Solution: Use master key. The /metrics endpoint typically requires master key access.

Performance Impact

On Meilisearch

Minimal impact:

• Prometheus /metrics endpoint returns pre-calculated statistics
• No indexing or search operations
• Response time: < 50ms

Benchmark:

• Overhead: < 0.1% CPU
• No measurable performance degradation

On Agent

Resource usage:

• Memory: +10 MB
• CPU: +2% during collection
• Network: +1 KB per collection

Collection time: < 0.2 seconds

Use Cases

1. Index Growth Monitoring

Monitor:

• Document count per index
• Database size growth
• Index creation rate

Alert on:

• Rapid document growth
• Database size approaching limit
• Too many indices

2. Task Queue Monitoring

Monitor:

• Enqueued tasks
• Processing tasks
• Failed tasks

Alert on:

• Task queue backup (> 100 tasks)
• High failure rate
• Tasks stuck in processing

3. Search Performance

Monitor:

• Search latency (via application metrics)
• Index update frequency
• Document indexing rate

Optimize:

• Index settings
• Search relevance
• Filtering rules

Best Practices

1. Set Master Key in Production

Always use master key in production:

MEILI_MASTER_KEY="secure-random-string-at-least-32-chars"

Generate secure key:

openssl rand -base64 32

2. Monitor Database Size

Set appropriate disk space:

• Database can grow large with many documents
• Monitor disk usage
• Implement retention policies

3. Monitor Task Queue

Healthy queue:

• Enqueued: < 10 tasks
• Processing: 1-5 tasks
• Failed: 0 or minimal

If queue backs up:

• Increase resources
• Optimize document size
• Batch operations

4. Use Async Indexing

# Python example
import meilisearch

client = meilisearch.Client('http://localhost:7700', 'your-master-key')

# Async document addition
task = client.index('movies').add_documents([
    {'id': 1, 'title': 'Movie 1'},
    {'id': 2, 'title': 'Movie 2'}
])

# Check task status
client.get_task(task.task_uid)

5. Implement Health Checks

# Health endpoint
curl http://localhost:7700/health

# Should return: {"status": "available"}

Meilisearch Performance Tuning

Index Settings

{
  "searchableAttributes": ["title", "description"],
  "filterableAttributes": ["genre", "year"],
  "sortableAttributes": ["year", "rating"],
  "rankingRules": [
    "words",
    "typo",
    "proximity",
    "attribute",
    "sort",
    "exactness"
  ]
}

Memory Configuration

# Set max indexing memory (MB)
MEILI_MAX_INDEXING_MEMORY=2048 ./meilisearch

Payload Size Limit

# Set max payload size (bytes)
MEILI_HTTP_PAYLOAD_SIZE_LIMIT=104857600 ./meilisearch

Advanced Configuration

Multiple Meilisearch Instances

plugins:
  meilisearch_prod:
    enabled: true
    host: search-prod.internal
    port: 7700
    api_key: ${MEILI_PROD_KEY}

  meilisearch_staging:
    enabled: true
    host: search-staging.internal
    port: 7700
    api_key: ${MEILI_STAGING_KEY}

Docker Container

Monitor Meilisearch in Docker:

plugins:
  meilisearch:
    enabled: true
    host: meilisearch-container
    port: 7700
    api_key: ${MEILI_MASTER_KEY}

Example Configurations

Basic Production

plugins:
  meilisearch:
    enabled: true
    host: localhost
    port: 7700
    api_key: ${MEILISEARCH_MASTER_KEY}

With HTTPS

plugins:
  meilisearch:
    enabled: true
    host: search.example.com
    port: 443
    protocol: https
    api_key: ${MEILISEARCH_API_KEY}

High-Availability Setup

plugins:
  meilisearch_primary:
    enabled: true
    host: search1.internal
    port: 7700
    api_key: ${MEILI_KEY}

  meilisearch_replica:
    enabled: true
    host: search2.internal
    port: 7700
    api_key: ${MEILI_KEY}

Limitations

Current Limitations

1. No per-search metrics - Only aggregate statistics
2. No query analysis - Search query breakdown not available
3. No RAM usage - Memory metrics not exposed by API

Scalability

Tested with:

• 10M+ documents
• 100+ indices
• GB-sized databases

Performance:

• Stats API response time constant regardless of data size

Monitoring Checklist

Critical:

1. Task queue backup (> 100 tasks)
2. High task failure rate (> 5%)
3. Database approaching disk limit

Important: 4. Document count stagnant (indexing issues) 5. Tasks stuck in processing 6. Version outdated

Alert Thresholds

tasks_enqueued: > 100
tasks_failed_rate: > 0.05
database_size_percent: > 85

Next Steps

• View all plugins
• Elasticsearch Plugin - Alternative search engine
• System Requirements
• Agent Configuration Guide