replace puppeteer with playwright
Build and Push Docker Images / build-and-push (push) Has been cancelled
Build and Push Docker Images / build-and-push (push) Has been cancelled
This commit is contained in:
@@ -1,6 +1,27 @@
|
||||
# Puppeteer API Server
|
||||
# Playwright API Server
|
||||
|
||||
A FastAPI-based server that uses Puppeteer (via pyppeteer) to scrape websites and extract various types of data.
|
||||
A FastAPI-based server that uses Playwright to scrape websites and extract various types of data.
|
||||
|
||||
## Recent Migration (v3.0)
|
||||
|
||||
### Migration from Puppeteer to Playwright
|
||||
|
||||
This project has been successfully migrated from Puppeteer (pyppeteer) to Playwright for better performance, reliability, and maintainability.
|
||||
|
||||
#### Key Benefits of Playwright
|
||||
|
||||
- **Better performance**: More efficient browser management and faster page loads
|
||||
- **Improved stability**: Better handling of modern web applications
|
||||
- **Enhanced features**: Better support for modern web standards
|
||||
- **Active development**: More frequent updates and better community support
|
||||
- **Multi-browser support**: Can easily switch between Chromium, Firefox, and WebKit
|
||||
|
||||
#### Migration Changes
|
||||
|
||||
- **Dependencies**: Updated from `pyppeteer` to `playwright`
|
||||
- **Browser management**: Improved browser pool with better resource management
|
||||
- **API compatibility**: All existing endpoints remain the same
|
||||
- **Docker image**: Updated to use Playwright browsers instead of Chrome
|
||||
|
||||
## Recent Fixes (v2.0)
|
||||
|
||||
@@ -102,19 +123,34 @@ python monitor.py your-api-key 30 http://localhost:8000 cleanup
|
||||
|
||||
```bash
|
||||
# Build the image
|
||||
docker build -t puppeteer-api .
|
||||
docker build -t playwright-api .
|
||||
|
||||
# Run with environment variables
|
||||
docker run -d \
|
||||
--name puppeteer-api \
|
||||
--name playwright-api \
|
||||
-p 8000:8000 \
|
||||
-e API_KEY=your-api-key \
|
||||
-e MAX_BROWSERS=3 \
|
||||
-e BROWSER_TTL=1800 \
|
||||
-v /path/to/cache:/db \
|
||||
puppeteer-api
|
||||
playwright-api
|
||||
```
|
||||
|
||||
## Testing the Migration
|
||||
|
||||
To verify that the Playwright migration works correctly:
|
||||
|
||||
```bash
|
||||
# Run the migration test script
|
||||
python test_playwright_migration.py
|
||||
```
|
||||
|
||||
This will test:
|
||||
|
||||
- Playwright installation and basic functionality
|
||||
- Browser utilities module
|
||||
- Browser service functionality
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### High CPU Usage
|
||||
@@ -135,6 +171,12 @@ docker run -d \
|
||||
2. Trigger emergency cleanup
|
||||
3. Restart the container if needed
|
||||
|
||||
### Playwright Installation Issues
|
||||
|
||||
1. Ensure Playwright browsers are installed: `playwright install chromium`
|
||||
2. Check Docker build logs for browser installation
|
||||
3. Verify system dependencies are installed
|
||||
|
||||
## Performance Tips
|
||||
|
||||
1. **Use caching**: The API caches results for 36 hours by default
|
||||
|
||||
Reference in New Issue
Block a user