MongoDB Plugin

Monitor MongoDB database performance with comprehensive metrics covering connections, operations, replication, and storage.

Overview

The MongoDB plugin collects detailed metrics from MongoDB servers including:

• Connection Statistics - Active connections, available connections, connection limits
• Operation Counters - Inserts, queries, updates, deletes, getmores, commands
• Replication Status - Replica set health, replication lag, oplog size
• Memory Usage - Resident memory, virtual memory, mapped memory
• Storage Statistics - Data size, index size, storage size
• Lock Statistics - Global lock time, database locks
• Network Metrics - Bytes in/out, requests per second

Requirements

MongoDB Version

• Minimum: MongoDB 4.0
• Recommended: MongoDB 5.0 or later
• Tested with: MongoDB 4.4, 5.0, 6.0, 7.0

Python Dependencies

pip install pymongo>=4.3.0

Auto-installed when using PLUGINS=mongodb during agent installation.

MongoDB Access

The agent needs read access to MongoDB server status and statistics.

Configuration

Basic Configuration

plugins:
  mongodb:
    enabled: true
    host: localhost
    port: 27017

With Authentication

plugins:
  mongodb:
    enabled: true
    host: localhost
    port: 27017
    username: monitor
    password: your-secure-password
    auth_database: admin

With Replica Set

plugins:
  mongodb:
    enabled: true
    host: localhost
    port: 27017
    username: monitor
    password: your-secure-password
    auth_database: admin
    replica_set: rs0

All Configuration Options

plugins:
  mongodb:
    enabled: true                    # Enable/disable plugin
    host: localhost                  # MongoDB host
    port: 27017                      # MongoDB port
    username: monitor                # Username (optional)
    password: password               # Password (optional)
    auth_database: admin             # Auth database (default: admin)
    replica_set: rs0                 # Replica set name (optional)
    ssl: false                       # Use SSL/TLS (default: false)
    ssl_ca_certs: /path/to/ca.pem   # CA certificate path
    timeout: 10                      # Connection timeout (seconds)

Environment Variables

Configuration can be overridden with environment variables:

export MONGODB_HOST="localhost"
export MONGODB_PORT="27017"
export MONGODB_USERNAME="monitor"
export MONGODB_PASSWORD="password"

MongoDB Setup

Create Monitoring User

For MongoDB 4.x and 5.x:

use admin

db.createUser({
  user: "monitor",
  pwd: "your-secure-password",
  roles: [
    { role: "clusterMonitor", db: "admin" },
    { role: "read", db: "local" }
  ]
})

For MongoDB 6.x and later:

use admin

db.createUser({
  user: "monitor",
  pwd: passwordPrompt(),  // Prompts for password securely
  roles: [
    { role: "clusterMonitor", db: "admin" },
    { role: "read", db: "local" }
  ]
})

Required Roles:

• clusterMonitor - Read cluster-wide status and statistics
• read on local database - Read oplog for replication metrics

Verify User Permissions

use admin
db.auth("monitor", "your-secure-password")

// Should succeed
db.serverStatus()
db.runCommand({ replSetGetStatus: 1 })

// Should list databases
db.adminCommand({ listDatabases: 1 })

MongoDB Configuration

No special MongoDB configuration required. The plugin uses standard monitoring commands:

• db.serverStatus() - Server statistics
• db.replSetGetStatus() - Replica set status
• db.stats() - Database statistics

These commands are available by default in all MongoDB installations.

Collected Metrics

Connection Metrics

Operation Counters

Memory Metrics

Network Metrics

Global Lock Metrics

WiredTiger Cache Metrics

Server Info Metrics

Dashboard Metrics

The StatusRadar dashboard displays:

Overview Card

• Current Connections - Active vs available
• Operations/sec - Real-time operation rate
• Replication Lag - For replica sets
• Memory Usage - Resident memory

Connections Chart

• Current connections over time
• Connection limit threshold
• Connection saturation alerts

Operations Chart

• Operations per second by type
• Insert/query/update/delete breakdown
• Command execution rate

Replication Chart

• Replication lag across members
• Oplog utilization
• Replica set health status

Memory Chart

• Resident memory usage
• Virtual memory
• Page faults

Storage Chart

• Data size growth
• Index size
• Collection and document counts

Installation

Quick Install

PLUGINS='mongodb' \
TOKEN='your-agent-token' \
MONGODB_USERNAME='monitor' \
MONGODB_PASSWORD='your-password' \
bash -c "$(curl -sL https://statusradar.dev/install-agent.sh)"

Install on Existing Agent

1. Install Python dependency:

   cd /opt/statusradar
   source venv/bin/activate  # If using venv
   pip install pymongo

2. Enable plugin in config:

   sudo nano /opt/statusradar/config/agent.yaml

   Add:

   plugins:
     mongodb:
       enabled: true
       host: localhost
       port: 27017
       username: monitor
       password: your-secure-password

3. Restart agent:

   sudo systemctl restart statusradar-agent

4. Verify:

   sudo journalctl -u statusradar-agent -n 50 --no-pager | grep mongodb

   Expected:

   INFO: Plugin mongodb: Metrics collected successfully
   INFO: Plugin mongodb: 25 metrics collected

Testing

Manual Plugin Test

cd /opt/statusradar
python3 plugins/mongodb_plugin.py

Expected Output:

Plugin: mongodb
Enabled: True
Available: True

Collecting metrics...
{
  "connections_current": 52,
  "connections_available": 767,
  "connections_total_created": 1234,
  "opcounters_insert": 123456,
  "opcounters_query": 789012,
  "opcounters_update": 345678,
  "mem_resident": 512,
  "mem_virtual": 1024,
  "data_size": 2048,
  "index_size": 512,
  "repl_lag": 0.5,
  "repl_set_health": 1
}

Test MongoDB Connection

# Using mongo shell
mongo -u monitor -p your-password --authenticationDatabase admin

# Using mongosh (MongoDB 5.0+)
mongosh -u monitor -p your-password --authenticationDatabase admin

# Test serverStatus command
use admin
db.serverStatus()

Troubleshooting

Plugin Not Collecting Metrics

Check 1: Is MongoDB running?

sudo systemctl status mongod

Check 2: Can agent connect to MongoDB?

mongo -u monitor -p password --authenticationDatabase admin --eval "db.serverStatus()"

Check 3: Is Python package installed?

python3 -c "import pymongo; print(pymongo.__version__)"

Check 4: Check agent logs

sudo journalctl -u statusradar-agent -n 100 --no-pager | grep mongodb

Common Errors

"Authentication failed"

Error:

ERROR: Plugin mongodb: Authentication failed

Causes:

1. Wrong username/password
2. Wrong auth database
3. User doesn't have required roles

Solution:

// Verify user exists
use admin
db.getUser("monitor")

// Check user roles
db.getUser("monitor").roles

// Re-create user with correct roles
db.dropUser("monitor")
db.createUser({
  user: "monitor",
  pwd: "your-secure-password",
  roles: [
    { role: "clusterMonitor", db: "admin" },
    { role: "read", db: "local" }
  ]
})

"No module named 'pymongo'"

Error:

ERROR: No module named 'pymongo'

Solution:

pip install pymongo
# Or if using venv:
cd /opt/statusradar && source venv/bin/activate && pip install pymongo

"Connection refused"

Error:

ERROR: Plugin mongodb: Connection refused

Causes:

1. MongoDB not running
2. Wrong host/port
3. Firewall blocking connection

Solution:

# Check MongoDB is running
sudo systemctl status mongod

# Check MongoDB is listening
sudo netstat -tlnp | grep 27017

# Check bind address in MongoDB config
sudo grep bindIp /etc/mongod.conf

# If MongoDB only listens on 127.0.0.1, agent must run on same host
# Or configure MongoDB to bind to 0.0.0.0

"Command replSetGetStatus is not allowed"

Error:

WARNING: Plugin mongodb: Cannot get replication status - not a replica set or insufficient permissions

Cause: MongoDB is running as standalone (not replica set)

Solution: This is a warning, not an error. Replication metrics are not collected for standalone instances. All other metrics work normally.

"SSL/TLS error"

Error:

ERROR: Plugin mongodb: SSL handshake failed

Solution:

Enable SSL in configuration:

plugins:
  mongodb:
    ssl: true
    ssl_ca_certs: /path/to/ca.pem

Performance Impact

On MongoDB

Minimal impact:

• Plugin executes serverStatus(), replSetGetStatus(), and dbStats() commands
• These are read-only monitoring commands designed for this purpose
• No write operations or locks
• Negligible CPU/memory overhead

Benchmark:

• MongoDB overhead: < 0.1% CPU per collection
• No measurable performance degradation
• Safe to run on production servers

On Agent

Resource usage:

• Memory: +20 MB
• CPU: +4% during collection
• Network: +3 KB per collection

Collection time: 0.5-2 seconds depending on database size

Use Cases

1. Connection Pool Monitoring

Monitor:

• Current connections
• Available connections
• Connection utilization

Alert on:

• Connections > 80% of max
• No available connections
• Rapid connection growth

2. Operation Performance

Monitor:

• Operations per second
• Query vs update ratio
• Command execution rate

Alert on:

• Sudden spike in operations
• High delete rate (potential issue)
• Command failures

3. Replication Health

Monitor:

• Replication lag
• Replica set member health
• Oplog size and utilization

Alert on:

• Replication lag > 10 seconds
• Replica set member down
• Oplog > 90% utilized

4. Memory Usage

Monitor:

• Resident memory growth
• Working set size
• Page faults

Alert on:

• Memory > 80% of server RAM
• High page fault rate (working set doesn't fit in RAM)

5. Storage Growth

Monitor:

• Data size growth rate
• Index size ratio
• Collection count

Alert on:

• Rapid data growth
• Index size > data size (over-indexed)
• Storage approaching disk capacity

Best Practices

1. Use Read-Only Monitoring User

Always create a dedicated monitoring user with minimal permissions:

db.createUser({
  user: "monitor",
  pwd: "secure-password",
  roles: [
    { role: "clusterMonitor", db: "admin" },
    { role: "read", db: "local" }
  ]
})

Benefits:

• Security isolation
• Audit trail
• Revocable access

2. Monitor Replica Sets

For production MongoDB:

• Always use replica sets (minimum 3 members)
• Monitor replication lag on all secondaries
• Alert on lag > 10 seconds

3. Set Connection Limits

Configure MongoDB connection limits appropriately:

# /etc/mongod.conf
net:
  maxIncomingConnections: 1000

Formula:

maxConnections = (Application pool size × App instances) + 100 (overhead)

4. Monitor Oplog Size

For replica sets:

• Monitor oplog utilization
• Ensure oplog covers at least 24 hours of operations
• Increase oplog size if utilization > 80%

Check oplog hours:

rs.printReplicationInfo()

5. Index Monitoring

• Monitor index size ratio (should be < data size)
• Check for unused indexes
• Alert on missing indexes (high query times)

6. Regular Backups

Don't rely solely on replication:

• Schedule daily backups
• Test restore procedures
• Monitor backup completion

Advanced Configuration

MongoDB Atlas

For MongoDB Atlas cloud:

plugins:
  mongodb:
    enabled: true
    host: cluster0.example.mongodb.net
    port: 27017
    username: monitor
    password: password
    auth_database: admin
    replica_set: atlas-abc123-shard-0
    ssl: true

Sharded Cluster

Monitor mongos router:

plugins:
  mongodb:
    enabled: true
    host: mongos.example.com
    port: 27017
    username: monitor
    password: password

Monitor each shard separately by deploying agent on each shard primary.

Docker Container

Monitor MongoDB in Docker:

plugins:
  mongodb:
    enabled: true
    host: mongodb-container-name  # Docker container name
    port: 27017
    username: monitor
    password: password

Or use host networking:

docker run --network host ...

Security Considerations

Password Security

Never hardcode passwords:

# ✅ Good - environment variable
export MONGODB_PASSWORD="password"

# ❌ Bad - hardcoded in config
password: "my-password"  # Visible in config file

Network Security

Firewall rules:

# Only allow agent host to connect
sudo ufw allow from <agent-ip> to any port 27017
sudo ufw deny 27017

MongoDB bind IP:

# /etc/mongod.conf
net:
  bindIp: 127.0.0.1,<private-ip>  # Don't bind to 0.0.0.0 unless necessary

SSL/TLS

For production, always use SSL:

plugins:
  mongodb:
    ssl: true
    ssl_ca_certs: /etc/ssl/mongodb/ca.pem

Example Configurations

Standalone Development

plugins:
  mongodb:
    enabled: true
    host: localhost
    port: 27017

Production Replica Set

plugins:
  mongodb:
    enabled: true
    host: mongo-primary.internal
    port: 27017
    username: monitor
    password: ${MONGODB_PASSWORD}  # From environment
    auth_database: admin
    replica_set: rs0
    ssl: true
    ssl_ca_certs: /etc/ssl/certs/ca-bundle.crt

MongoDB Atlas

plugins:
  mongodb:
    enabled: true
    host: cluster0.abc123.mongodb.net
    port: 27017
    username: monitor
    password: ${MONGODB_PASSWORD}
    auth_database: admin
    replica_set: atlas-cluster0-shard-0
    ssl: true

Docker Compose

plugins:
  mongodb:
    enabled: true
    host: mongodb  # Service name from docker-compose.yml
    port: 27017
    username: monitor
    password: monitor-password

Limitations

Current Limitations

1. No per-database metrics - Collects server-wide metrics only
2. No slow query analysis - Use MongoDB profiler for slow queries
3. No index efficiency metrics - Monitor via explain() plans
4. No sharding metrics - Limited shard cluster visibility

Scalability

Tested with:

• Standalone instances up to 1TB
• Replica sets with 7 members
• Databases with 1000+ collections

Performance:

• Collection time increases slightly with database size
• Recommended: < 10,000 collections for optimal performance

Monitoring Best Practices

Key Metrics to Watch

Critical:

1. Connections available < 10
2. Replication lag > 30 seconds
3. Memory > 90% of RAM
4. Oplog > 95% full

Important: 5. High command failure rate 6. Slow operations 7. Lock contention 8. Network errors

Alert Thresholds

# Recommended alert thresholds
connections_available: < 50
replication_lag_seconds: > 10
memory_percent: > 85
oplog_percent: > 90
operations_per_second: > baseline + 3 std_dev

Troubleshooting Performance

Slow Collection

Symptom: Plugin collection takes > 5 seconds

Solutions:

1. Check MongoDB performance:

   db.currentOp()  // Check for slow operations
   db.serverStatus().opcounters  // Check operation load

2. Reduce monitoring frequency:

   agent:
     interval: 600  # 10 minutes instead of 5

3. Check network latency:

   ping mongodb-host

High Memory Usage

Symptom: Agent using excessive memory with MongoDB plugin

Solution: This is normal - pymongo caches connection pools. Memory usage should stabilize after initial connection.

Next Steps

• View all plugins
• System Requirements
• Agent Configuration Guide
• Troubleshooting Guide