304 lines
9.2 KiB
Markdown
304 lines
9.2 KiB
Markdown
# Puppeteer API Healthcheck
|
|
|
|
A Docker container that monitors multiple Puppeteer API endpoints and automatically restarts the target containers when the APIs become unresponsive.
|
|
|
|
## Features
|
|
|
|
- Monitors multiple Puppeteer API endpoints every minute
|
|
- Automatically restarts target containers after 3 consecutive failures
|
|
- Configurable via environment variables
|
|
- Comprehensive logging
|
|
- Docker socket access for container management
|
|
- Proper Docker permissions handling
|
|
- Backward compatibility with single-host configuration
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Default | Description |
|
|
| -------------------------- | ------------------------ | ---------------------------------------------- |
|
|
| `HOSTS` | `puppeteer-api` | Comma-separated list of hosts to monitor |
|
|
| `TEST_URL` | `https://www.google.com` | URL to test the API with |
|
|
| `API_KEY` | `Q7Sd#hhFkyHy*T` | API key for authentication |
|
|
| `CHECK_INTERVAL` | `60` | Health check interval in seconds |
|
|
| `MAX_CONSECUTIVE_FAILURES` | `3` | Number of failures before restarting container |
|
|
| `TIMEOUT` | `20` | Request timeout in seconds |
|
|
|
|
### Legacy Variables (for backward compatibility)
|
|
|
|
| Variable | Default | Description |
|
|
| ------------------ | ------------------------------------ | ---------------------------- |
|
|
| `BASE_URL` | `https://puppeteer.workwithkora.com` | Legacy single host URL |
|
|
| `TARGET_CONTAINER` | `puppeteer-api` | Legacy single container name |
|
|
|
|
## Usage
|
|
|
|
### Multi-Host Configuration
|
|
|
|
The healthcheck will monitor each host at `http://host:8000` and restart containers with the same name as the host.
|
|
|
|
#### Docker Run
|
|
|
|
```bash
|
|
docker run -d \
|
|
--name puppeteer-healthcheck \
|
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
|
-e HOSTS="host1,host2,host3" \
|
|
-e TEST_URL="https://www.google.com" \
|
|
-e API_KEY="your-api-key" \
|
|
-e CHECK_INTERVAL="60" \
|
|
your-registry/puppeteer-healthcheck:latest
|
|
```
|
|
|
|
#### Docker Compose
|
|
|
|
```yaml
|
|
version: "3.8"
|
|
|
|
services:
|
|
puppeteer-healthcheck:
|
|
build: .
|
|
container_name: puppeteer-healthcheck
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
environment:
|
|
- HOSTS=host1,host2,host3
|
|
- TEST_URL=https://www.google.com
|
|
- API_KEY=your-api-key
|
|
- CHECK_INTERVAL=60
|
|
restart: unless-stopped
|
|
```
|
|
|
|
### Single-Host Configuration (Legacy)
|
|
|
|
For backward compatibility, you can still use the old single-host configuration:
|
|
|
|
#### Docker Run
|
|
|
|
```bash
|
|
docker run -d \
|
|
--name puppeteer-healthcheck \
|
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
|
-e BASE_URL="http://host1:8000" \
|
|
-e TEST_URL="https://www.google.com" \
|
|
-e API_KEY="your-api-key" \
|
|
-e TARGET_CONTAINER="host1" \
|
|
-e CHECK_INTERVAL="60" \
|
|
your-registry/puppeteer-healthcheck:latest
|
|
```
|
|
|
|
#### Docker Compose
|
|
|
|
```yaml
|
|
version: "3.8"
|
|
|
|
services:
|
|
puppeteer-healthcheck:
|
|
build: .
|
|
container_name: puppeteer-healthcheck
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock
|
|
environment:
|
|
- BASE_URL=http://host1:8000
|
|
- TEST_URL=https://www.google.com
|
|
- API_KEY=your-api-key
|
|
- TARGET_CONTAINER=host1
|
|
- CHECK_INTERVAL=60
|
|
restart: unless-stopped
|
|
```
|
|
|
|
## How It Works
|
|
|
|
1. The healthcheck container reads the list of hosts from the `HOSTS` environment variable
|
|
2. For each host, it makes a GET request to `http://host:8000` every minute
|
|
3. It uses the configured test URL and API key for authentication
|
|
4. If the request fails (non-200 status or timeout), it increments a failure counter for that specific host
|
|
5. After 3 consecutive failures for a host, it attempts to restart the container with the same name as the host
|
|
6. If the restart is successful, the failure counter for that host is reset
|
|
7. The process continues indefinitely, monitoring all hosts independently
|
|
|
|
## Example Scenarios
|
|
|
|
### Monitoring Multiple Puppeteer Instances
|
|
|
|
If you have multiple puppeteer-api containers running on different hosts:
|
|
|
|
```bash
|
|
# Hosts: server1, server2, server3
|
|
# Containers: server1, server2, server3
|
|
docker run -d \
|
|
--name puppeteer-healthcheck \
|
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
|
-e HOSTS="server1,server2,server3" \
|
|
-e API_KEY="your-api-key" \
|
|
your-registry/puppeteer-healthcheck:latest
|
|
```
|
|
|
|
This will:
|
|
|
|
- Check `http://server1:8000` and restart container `server1` if needed
|
|
- Check `http://server2:8000` and restart container `server2` if needed
|
|
- Check `http://server3:8000` and restart container `server3` if needed
|
|
|
|
### Mixed Environment
|
|
|
|
You can also monitor hosts with different names than their containers:
|
|
|
|
```bash
|
|
# Hosts: api1.example.com, api2.example.com
|
|
# Containers: puppeteer-api-1, puppeteer-api-2
|
|
docker run -d \
|
|
--name puppeteer-healthcheck \
|
|
-v /var/run/docker.sock:/var/run/docker.sock \
|
|
-e HOSTS="api1.example.com,api2.example.com" \
|
|
-e API_KEY="your-api-key" \
|
|
your-registry/puppeteer-healthcheck:latest
|
|
```
|
|
|
|
**Note**: In this case, the container names must match the host names exactly. If they don't, you'll need to use separate healthcheck instances or modify the container names.
|
|
|
|
## Logging
|
|
|
|
The container logs all health check activities to both stdout and a log file (`/app/healthcheck.log`). Log levels include:
|
|
|
|
- INFO: Normal operations and successful health checks
|
|
- WARNING: Failed health checks
|
|
- ERROR: Container restart attempts and failures
|
|
|
|
Each log entry includes the host name for easy identification:
|
|
|
|
```
|
|
2024-01-15 10:30:00 - INFO - Performing health check for server1: http://server1:8000/?url=https%3A//www.google.com
|
|
2024-01-15 10:30:01 - INFO - Health check passed for server1 - API is responding correctly
|
|
2024-01-15 10:30:02 - WARNING - Health check failed for server2 - Status code: 500
|
|
```
|
|
|
|
## Security Considerations
|
|
|
|
- The container requires access to the Docker socket to restart other containers
|
|
- Ensure proper API key management
|
|
- The container runs as root for Docker socket access (required for container management)
|
|
- Consider using Docker-in-Docker (DinD) or Docker socket proxy for enhanced security in production
|
|
|
|
## Troubleshooting
|
|
|
|
### USB Device Mounting Error
|
|
|
|
If you encounter an error like:
|
|
|
|
```
|
|
error creating device nodes: mount src=/dev/bus/usb/003/003, dst=/var/lib/docker/overlay2/.../merged/dev/bus/usb/003/003: no such file or directory
|
|
```
|
|
|
|
**Solution**: Remove `privileged: true` from your Docker Compose configuration. The healthcheck container doesn't need privileged access.
|
|
|
|
**Correct configuration:**
|
|
|
|
```yaml
|
|
puppeteer-healthcheck:
|
|
image: bramkel/puppeteer-healthcheck:latest
|
|
container_name: puppeteer-healthcheck
|
|
# Remove this line: privileged: true
|
|
volumes:
|
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
|
environment:
|
|
- HOSTS=puppeteer-api-1,puppeteer-api-2
|
|
- TEST_URL=https://www.google.com
|
|
- MAX_CONSECUTIVE_FAILURES=2
|
|
- API_KEY=${PUPPETEER_API_KEY}
|
|
- CHECK_INTERVAL=60
|
|
- TIMEOUT=15
|
|
restart: unless-stopped
|
|
depends_on:
|
|
- puppeteer-api-1
|
|
- puppeteer-api-2
|
|
```
|
|
|
|
### Docker Permission Issues
|
|
|
|
If you see "Permission denied" errors when accessing the Docker socket:
|
|
|
|
1. **For Docker Run**: Add the `--group-add` flag:
|
|
|
|
```bash
|
|
--group-add $(getent group docker | cut -d: -f3)
|
|
```
|
|
|
|
2. **For Docker Compose**: Add the `group_add` section:
|
|
|
|
```yaml
|
|
group_add:
|
|
- docker
|
|
```
|
|
|
|
3. **Alternative**: Run the container as root (not recommended for production):
|
|
```bash
|
|
docker run --user root ...
|
|
```
|
|
|
|
### Common Error: "Connection aborted. PermissionError(13, 'Permission denied')"
|
|
|
|
This error occurs when the container cannot access the Docker socket. To fix:
|
|
|
|
1. **Mount the Docker socket**:
|
|
|
|
```bash
|
|
-v /var/run/docker.sock:/var/run/docker.sock:ro
|
|
```
|
|
|
|
2. **Add the container to the docker group**:
|
|
|
|
```bash
|
|
--group-add $(getent group docker | cut -d: -f3)
|
|
```
|
|
|
|
3. **Use the provided docker-compose.yml** which includes all necessary configurations.
|
|
|
|
### Container not found
|
|
|
|
- Ensure the container names match the host names exactly
|
|
- Verify the containers are running and accessible
|
|
- Check that the `HOSTS` environment variable is set correctly
|
|
|
|
### API key issues
|
|
|
|
- Verify the API key is correct and has the necessary permissions
|
|
- Check that the hosts are accessible from the container
|
|
|
|
### Multiple host configuration
|
|
|
|
- Ensure the `HOSTS` environment variable is a comma-separated list without spaces
|
|
- Each host should be accessible at `http://host:8000`
|
|
- Container names must match host names exactly
|
|
|
|
## Building
|
|
|
|
```bash
|
|
docker build -t puppeteer-healthcheck .
|
|
```
|
|
|
|
## Version
|
|
|
|
Current version: latest
|
|
|
|
### Migration from v1.x
|
|
|
|
To migrate from the single-host version to multi-host:
|
|
|
|
1. **Replace `BASE_URL` and `TARGET_CONTAINER` with `HOSTS`**:
|
|
|
|
```bash
|
|
# Old
|
|
-e BASE_URL="http://server1:8000" -e TARGET_CONTAINER="server1"
|
|
|
|
# New
|
|
-e HOSTS="server1"
|
|
```
|
|
|
|
2. **For multiple hosts, add them to the `HOSTS` variable**:
|
|
|
|
```bash
|
|
-e HOSTS="server1,server2,server3"
|
|
```
|
|
|
|
3. **Ensure container names match host names** (or rename containers accordingly)
|