118 lines
3.3 KiB
Markdown
118 lines
3.3 KiB
Markdown
# Playwright Node.js API
|
|
|
|
A Node.js API service that uses Playwright to fetch webpage HTML with SQLite or PostgreSQL-based caching.
|
|
|
|
## Features
|
|
|
|
- Fetch full HTML of any webpage using Playwright
|
|
- SQLite or PostgreSQL-based caching with configurable expiry (default: 24 hours)
|
|
- API key authentication (optional, via `x-api-key` header)
|
|
- Automatic cache cleanup and garbage collection
|
|
- Proper browser/page cleanup on errors
|
|
- Health check and cache statistics endpoints
|
|
|
|
## API Endpoints
|
|
|
|
### GET `/`
|
|
|
|
Fetch HTML content of a webpage.
|
|
|
|
**Query Parameters:**
|
|
|
|
- `url` (required): The URL to fetch
|
|
- `skipCache` (optional): Set to `true`, `1`, or `yes` to bypass cache
|
|
|
|
**Headers:**
|
|
|
|
- `x-api-key` (required if `API_KEY` env var is set): API key for authentication
|
|
|
|
**Example:**
|
|
|
|
```bash
|
|
# Without API key (if API_KEY env var is not set)
|
|
curl "http://localhost:3000/?url=https://example.com"
|
|
|
|
# With API key
|
|
curl -H "x-api-key: your-api-key" "http://localhost:3000/?url=https://example.com"
|
|
curl -H "x-api-key: your-api-key" "http://localhost:3000/?url=https://example.com&skipCache=true"
|
|
```
|
|
|
|
### GET `/health`
|
|
|
|
Health check endpoint. Returns API status and configuration.
|
|
|
|
**Note:** This endpoint is not protected by API key authentication.
|
|
|
|
### GET `/cache/stats`
|
|
|
|
Get cache statistics (total entries, valid entries).
|
|
|
|
### DELETE `/cache`
|
|
|
|
Clear cache entries.
|
|
|
|
**Query Parameters:**
|
|
|
|
- `url` (optional): Clear specific URL from cache. If omitted, clears all cache.
|
|
|
|
## Environment Variables
|
|
|
|
- `PORT`: Server port (default: `3000`)
|
|
- `CACHE_EXPIRY_HOURS`: Cache expiry time in hours (default: `24`)
|
|
- `API_KEY`: API key for authentication (optional). If set, all endpoints except `/health` require the `x-api-key` header
|
|
- `DB_PATH`: Path to SQLite database file (default: `/db/cache.db`) - only used if PostgreSQL is not configured
|
|
|
|
### PostgreSQL Configuration (optional)
|
|
|
|
If the following environment variables are set, the API will use PostgreSQL instead of SQLite:
|
|
|
|
- `POSTGRES_HOST`: PostgreSQL host (e.g., `puppeteer-postgres`)
|
|
- `POSTGRES_PORT`: PostgreSQL port (default: `5432`)
|
|
- `POSTGRES_USER`: PostgreSQL username (e.g., `postgres`)
|
|
- `POSTGRES_PASSWORD`: PostgreSQL password
|
|
- `POSTGRES_DB`: PostgreSQL database name (e.g., `puppeteer`)
|
|
|
|
If any of these PostgreSQL variables are missing, the API will fall back to SQLite.
|
|
|
|
## Docker
|
|
|
|
Build the image:
|
|
|
|
```bash
|
|
docker build -t playwright-node-api .
|
|
```
|
|
|
|
Run the container with SQLite:
|
|
|
|
```bash
|
|
docker run -d \
|
|
-p 3000:3000 \
|
|
-v /path/to/db:/db \
|
|
-e CACHE_EXPIRY_HOURS=24 \
|
|
-e API_KEY=your-secret-api-key \
|
|
playwright-node-api
|
|
```
|
|
|
|
Run the container with PostgreSQL:
|
|
|
|
```bash
|
|
docker run -d \
|
|
-p 3000:3000 \
|
|
-e POSTGRES_HOST=puppeteer-postgres \
|
|
-e POSTGRES_PORT=5432 \
|
|
-e POSTGRES_USER=postgres \
|
|
-e POSTGRES_PASSWORD=${PUPPETEER_DB_PASSWORD} \
|
|
-e POSTGRES_DB=puppeteer \
|
|
-e CACHE_EXPIRY_HOURS=24 \
|
|
-e API_KEY=your-secret-api-key \
|
|
playwright-node-api
|
|
```
|
|
|
|
## Notes
|
|
|
|
- The service automatically cleans up expired cache entries every hour
|
|
- Browser instances are reused for better performance
|
|
- Pages are always closed after use, even on errors
|
|
- Graceful shutdown is handled on SIGTERM/SIGINT
|
|
- API key authentication is optional: if `API_KEY` is not set, all endpoints are publicly accessible (except `/health` which is always public)
|