Memcached Plugin

Monitor Memcached distributed memory caching system with comprehensive metrics covering cache hit rates, memory usage, connections, and item statistics.

Overview

The Memcached plugin collects detailed metrics from Memcached servers including:

• Cache Performance - Hit rate, miss rate, cache efficiency
• Memory Statistics - Memory usage, evictions, memory limits
• Item Statistics - Current items, total items, item evictions
• Connection Metrics - Current connections, connection rate, rejected connections
• Operation Counters - Gets, sets, deletes, increments, decrements, touches
• Network Statistics - Bytes read/written, connection yields

Requirements

Memcached Version

• Minimum: Memcached 1.4
• Recommended: Memcached 1.6 or later
• Tested with: Memcached 1.4, 1.5, 1.6

Python Dependencies

pip install python-memcached>=1.59

Auto-installed when using PLUGINS=memcached during agent installation.

Memcached Access

The agent needs network access to Memcached server (default port 11211).

Configuration

Basic Configuration

plugins:
  memcached:
    enabled: true
    host: localhost
    port: 11211

With SASL Authentication

plugins:
  memcached:
    enabled: true
    host: localhost
    port: 11211
    username: memcache_user
    password: your-password

All Configuration Options

plugins:
  memcached:
    enabled: true                # Enable/disable plugin
    host: localhost              # Memcached host
    port: 11211                  # Memcached port (default: 11211)
    username: user               # SASL username (optional)
    password: password           # SASL password (optional)
    timeout: 10                  # Connection timeout (seconds)

Environment Variables

Configuration can be overridden with environment variables:

export MEMCACHED_HOST="localhost"
export MEMCACHED_PORT="11211"
export MEMCACHED_USERNAME="user"
export MEMCACHED_PASSWORD="password"

Memcached Setup

Installation

Ubuntu/Debian:

sudo apt-get update
sudo apt-get install memcached

CentOS/RHEL:

sudo yum install memcached

Start service:

sudo systemctl start memcached
sudo systemctl enable memcached

Configuration

Edit Memcached config:

Ubuntu/Debian:

sudo nano /etc/memcached.conf

CentOS/RHEL:

sudo nano /etc/sysconfig/memcached

Basic configuration:

# Memory limit (MB)
-m 64

# Listen on localhost only (security)
-l 127.0.0.1

# Port
-p 11211

# Max connections
-c 1024

# Username to run as
-u memcache

Restart after changes:

sudo systemctl restart memcached

Enable SASL Authentication (Optional)

Install SASL:

# Ubuntu/Debian
sudo apt-get install libsasl2-2 sasl2-bin

# CentOS/RHEL
sudo yum install cyrus-sasl-plain

Configure Memcached with SASL:

# Start Memcached with SASL
-S

Create SASL user:

echo "mech_list: plain" | sudo tee /etc/sasl2/memcached.conf
echo "memcache_user:your-password" | sudo tee /etc/sasl2/memcached-sasldb
sudo saslpasswd2 -a memcached -c memcache_user

Security Configuration

Bind to localhost only:

-l 127.0.0.1

Firewall rules (if listening on 0.0.0.0):

# Ubuntu/Debian
sudo ufw allow from 192.168.1.0/24 to any port 11211
sudo ufw deny 11211

# CentOS/RHEL
sudo firewall-cmd --permanent --add-rich-rule='rule family=ipv4 source address=192.168.1.0/24 port port=11211 protocol=tcp accept'
sudo firewall-cmd --reload

Collected Metrics

Cache Performance Metrics

Hit rate calculation:

hit_rate_percent = (get_hits / (get_hits + get_misses)) × 100

Memory Metrics

Memory usage calculation:

memory_used_percent = (bytes / limit_maxbytes) × 100

Item Metrics

Connection Metrics

Network Metrics

Server Metrics

Dashboard Metrics

The StatusRadar dashboard displays:

Overview Card

• Hit Rate - Cache efficiency percentage
• Memory Usage - Current vs limit
• Items - Current item count
• Connections - Active connections

Hit Rate Chart

• Cache hit rate over time
• Miss rate
• Hit/miss ratio trends

Memory Usage Chart

• Memory usage vs limit
• Memory utilization percentage
• Eviction rate

Items Chart

• Current items count
• Item growth rate
• Expired/evicted items

Connections Chart

• Active connections
• Connection rate
• Rejected connections

Operations Chart

• Gets/sets per second
• Delete/incr/decr operations
• Command distribution

Installation

Quick Install

PLUGINS='memcached' \
TOKEN='your-agent-token' \
bash -c "$(curl -sL https://statusradar.dev/install-agent.sh)"

Install on Existing Agent

1. Install Python dependency:

   cd /opt/statusradar
   source venv/bin/activate  # If using venv
   pip install python-memcached

2. Enable plugin in config:

   sudo nano /opt/statusradar/config/agent.yaml

   Add:

   plugins:
     memcached:
       enabled: true
       host: localhost
       port: 11211

3. Restart agent:

   sudo systemctl restart statusradar-agent

4. Verify:

   sudo journalctl -u statusradar-agent -n 50 --no-pager | grep memcached

   Expected:

   INFO: Plugin memcached: Metrics collected successfully
   INFO: Plugin memcached: 1234 items, 95.5% hit rate

Testing

Manual Plugin Test

cd /opt/statusradar
python3 plugins/memcached_plugin.py

Expected Output:

Plugin: memcached
Enabled: True
Available: True

Collecting metrics...
{
  "cmd_get": 123456,
  "cmd_set": 45678,
  "get_hits": 117890,
  "get_misses": 5566,
  "hit_rate": 95.5,
  "bytes": 45678912,
  "limit_maxbytes": 67108864,
  "mem_usage_percent": 68.1,
  "curr_items": 1234,
  "evictions": 56,
  "curr_connections": 12,
  "total_connections": 5678,
  "uptime": 86400
}

Test Memcached Connection

# Using telnet
telnet localhost 11211

# Commands:
stats
quit

# Using memcached-tool
memcached-tool localhost:11211 stats

# Using netcat
echo "stats" | nc localhost 11211

Troubleshooting

Plugin Not Collecting Metrics

Check 1: Is Memcached running?

sudo systemctl status memcached

Check 2: Can agent connect to Memcached?

telnet localhost 11211
# Type: stats
# Should show statistics

Check 3: Is Python package installed?

python3 -c "import memcache; print(memcache.__version__)"

Check 4: Check agent logs

sudo journalctl -u statusradar-agent -n 100 --no-pager | grep memcached

Common Errors

"No module named 'memcache'"

Error:

ERROR: No module named 'memcache'

Solution:

pip install python-memcached
# Or if using venv:
cd /opt/statusradar && source venv/bin/activate && pip install python-memcached

"Connection refused"

Error:

ERROR: Plugin memcached: Connection refused

Causes:

1. Memcached not running
2. Wrong host/port
3. Firewall blocking connection

Solution:

# Check Memcached is running
sudo systemctl status memcached

# Check port
sudo netstat -tlnp | grep 11211

# Test connection
telnet localhost 11211

"Connection timeout"

Error:

ERROR: Plugin memcached: Connection timeout

Causes:

1. Memcached overloaded
2. Network issues
3. Memcached not responding

Solution:

# Check Memcached load
echo "stats" | nc localhost 11211 | grep curr_connections

# Restart if needed
sudo systemctl restart memcached

"SASL authentication failed"

Error:

ERROR: Plugin memcached: SASL authentication failed

Cause: Wrong username/password

Solution:

# Verify SASL user
sudo sasldblistusers2 -f /etc/sasl2/memcached-sasldb

# Reset password
sudo saslpasswd2 -a memcached -c memcache_user

Performance Impact

On Memcached

Minimal impact:

• stats command returns pre-calculated statistics
• No cache invalidation or eviction
• Response time: < 1ms

Benchmark:

• Overhead: < 0.001% CPU
• No measurable performance degradation

On Agent

Resource usage:

• Memory: +5 MB
• CPU: +1% during collection
• Network: +0.5 KB per collection

Collection time: < 0.1 seconds

Use Cases

1. Cache Hit Rate Monitoring

Monitor:

• Hit rate percentage
• Miss rate trends
• Hit/miss ratio over time

Alert on:

• Hit rate < 90% (inefficient caching)
• Sudden drop in hit rate
• Miss rate spike

2. Memory Management

Monitor:

• Memory usage vs limit
• Eviction rate
• Memory utilization percentage

Alert on:

• Memory usage > 90%
• High eviction rate (memory too low)
• Evictions of unfetched items (wasted memory)

3. Connection Monitoring

Monitor:

• Current connections
• Rejected connections
• Connection limit reached

Alert on:

• Connections > 90% of max
• Rejected connections > 0
• Connection limit reached frequently

4. Item Lifecycle

Monitor:

• Item count growth
• Expired items
• Evicted items

Optimize:

• TTL settings
• Memory allocation
• Cache warming strategies

Best Practices

1. Set Appropriate Memory Limit

Calculate memory need:

Memory = (Average item size) × (Number of items) × 1.2 (overhead)

Example:
Item size: 1KB
Items: 100,000
Memory = 1KB × 100,000 × 1.2 = 120MB

Configure in Memcached:

-m 128  # Set to 128MB

2. Monitor Hit Rate

Healthy hit rate: > 95%

If hit rate < 90%:

• Increase memory (-m parameter)
• Review caching strategy
• Check TTL values
• Investigate cache invalidation patterns

3. Avoid Evictions

Evictions indicate:

• Memory limit too low
• Items not being read before expiration
• Cache churn

Solutions:

• Increase memory limit
• Optimize TTL values
• Implement cache warming

4. Set Connection Limits

-c 1024  # Max simultaneous connections

Formula:

Max connections = (Application pool size × App instances) + 50 (overhead)

5. Use Localhost Binding for Security

-l 127.0.0.1  # Bind to localhost only

Only bind to 0.0.0.0 if:

• Memcached is on separate server
• Firewall rules are configured
• SASL authentication is enabled

6. Enable Connection Limit Alerts

Monitor listen_disabled_num metric:

• 0 indicates connection limit reached

• Increase -c parameter if this happens frequently

Memcached Performance Tuning

Memory Optimization

1. Choose appropriate slab size:

# Growth factor (default: 1.25)
-f 1.25

# Smaller factor = less wasted memory, more slabs
# Larger factor = less overhead, more waste

2. Monitor slab statistics:

echo "stats slabs" | nc localhost 11211

Connection Optimization

1. Increase max connections:

-c 2048

2. Monitor connection usage:

echo "stats" | nc localhost 11211 | grep curr_connections

Thread Configuration

Multi-threaded Memcached:

-t 4  # 4 threads

Recommended:

threads = Number of CPU cores

Advanced Configuration

Memcached Cluster

Monitor multiple Memcached instances:

plugins:
  memcached_1:
    enabled: true
    host: cache1.internal
    port: 11211

  memcached_2:
    enabled: true
    host: cache2.internal
    port: 11211

Unix Socket

Memcached on Unix socket:

# Memcached config
-s /var/run/memcached/memcached.sock

Plugin config:

plugins:
  memcached:
    enabled: true
    socket: /var/run/memcached/memcached.sock

Docker Container

Monitor Memcached in Docker:

plugins:
  memcached:
    enabled: true
    host: memcached-container
    port: 11211

Docker run:

docker run -d --name memcached \
  -p 11211:11211 \
  memcached -m 64

Example Configurations

Basic Production

plugins:
  memcached:
    enabled: true
    host: localhost
    port: 11211

With SASL Authentication

plugins:
  memcached:
    enabled: true
    host: cache.internal
    port: 11211
    username: memcache_user
    password: ${MEMCACHED_PASSWORD}

High-Availability Setup

plugins:
  memcached_primary:
    enabled: true
    host: cache-primary.internal
    port: 11211

  memcached_secondary:
    enabled: true
    host: cache-secondary.internal
    port: 11211

Limitations

Current Limitations

1. No per-slab metrics - Only server-wide statistics
2. No per-key metrics - Cannot monitor individual cache keys
3. No slab rebalancing info - No slab automove statistics

Scalability

Tested with:

• Single instance: up to 10M items
• Memory: up to 64GB
• Throughput: 100,000+ ops/sec

Performance:

• Stats command overhead constant regardless of size
• No impact on Memcached performance

Memcached Best Practices

Cache Strategy

1. Set appropriate TTL:

# Short-lived data
cache.set('session_123', data, time=300)  # 5 minutes

# Long-lived data
cache.set('user_profile', data, time=3600)  # 1 hour

2. Implement cache warming:

# Pre-populate cache on startup
def warm_cache():
    for item in get_popular_items():
        cache.set(item.key, item.data, time=3600)

3. Handle cache misses gracefully:

def get_user(user_id):
    # Try cache first
    user = cache.get(f'user_{user_id}')
    if user is None:
        # Cache miss - fetch from database
        user = database.get_user(user_id)
        cache.set(f'user_{user_id}', user, time=3600)
    return user

Monitoring Checklist

Daily:

• Hit rate percentage
• Memory usage
• Eviction rate

Weekly:

• Connection patterns
• Item count trends
• Memory growth

Monthly:

• Capacity planning
• Performance trends
• Cache efficiency analysis

Next Steps

• View all plugins
• Redis Plugin - Alternative caching system
• System Requirements
• Agent Configuration Guide