345 lines
7.1 KiB
Markdown
345 lines
7.1 KiB
Markdown
# Migration Guide: Puppeteer to Playwright
|
|
|
|
This guide will help you migrate from Puppeteer (pyppeteer) to Playwright in your project.
|
|
|
|
## Overview
|
|
|
|
The migration from Puppeteer to Playwright provides several benefits:
|
|
|
|
- Better performance and stability
|
|
- Enhanced features for modern web applications
|
|
- Active development and community support
|
|
- Multi-browser support (Chromium, Firefox, WebKit)
|
|
|
|
## Pre-Migration Checklist
|
|
|
|
Before starting the migration, ensure you have:
|
|
|
|
- [ ] Backed up your current codebase
|
|
- [ ] Documented any custom Puppeteer configurations
|
|
- [ ] Identified all Puppeteer-specific code in your project
|
|
- [ ] Tested your current application thoroughly
|
|
|
|
## Step-by-Step Migration
|
|
|
|
### 1. Update Dependencies
|
|
|
|
Replace Puppeteer dependencies with Playwright:
|
|
|
|
**Before (requirements.txt):**
|
|
|
|
```
|
|
pyppeteer==1.0.2
|
|
```
|
|
|
|
**After (requirements.txt):**
|
|
|
|
```
|
|
playwright==1.40.0
|
|
```
|
|
|
|
### 2. Install Playwright
|
|
|
|
```bash
|
|
# Install Playwright
|
|
pip install playwright
|
|
|
|
# Install browsers (Chromium is recommended for compatibility)
|
|
playwright install chromium
|
|
|
|
# Install system dependencies (Linux)
|
|
playwright install-deps chromium
|
|
```
|
|
|
|
### 3. Update Imports
|
|
|
|
**Before:**
|
|
|
|
```python
|
|
from pyppeteer import launch
|
|
```
|
|
|
|
**After:**
|
|
|
|
```python
|
|
from playwright.async_api import async_playwright
|
|
```
|
|
|
|
### 4. Update Browser Launch
|
|
|
|
**Before (Puppeteer):**
|
|
|
|
```python
|
|
browser = await launch(
|
|
headless=True,
|
|
executablePath='/usr/bin/google-chrome',
|
|
args=['--no-sandbox', '--disable-setuid-sandbox'],
|
|
handleSIGINT=False,
|
|
handleSIGTERM=False,
|
|
handleSIGHUP=False,
|
|
ignoreHTTPSErrors=True,
|
|
autoClose=True,
|
|
)
|
|
```
|
|
|
|
**After (Playwright):**
|
|
|
|
```python
|
|
playwright = await async_playwright().start()
|
|
browser = await playwright.chromium.launch(
|
|
headless=True,
|
|
args=['--no-sandbox', '--disable-setuid-sandbox'],
|
|
ignore_default_args=['--enable-automation'],
|
|
)
|
|
```
|
|
|
|
### 5. Update Page Creation
|
|
|
|
**Before (Puppeteer):**
|
|
|
|
```python
|
|
page = await browser.newPage()
|
|
await page.setUserAgent(CUSTOM_USER_AGENT)
|
|
await page.setViewport({'width': 1280, 'height': 800})
|
|
page.setDefaultNavigationTimeout(30000)
|
|
```
|
|
|
|
**After (Playwright):**
|
|
|
|
```python
|
|
context = await browser.new_context(
|
|
user_agent=CUSTOM_USER_AGENT,
|
|
viewport={'width': 1920, 'height': 1080},
|
|
ignore_https_errors=True,
|
|
)
|
|
page = await context.new_page()
|
|
page.set_default_timeout(30000)
|
|
```
|
|
|
|
### 6. Update Navigation
|
|
|
|
**Before (Puppeteer):**
|
|
|
|
```python
|
|
response = await page.goto(url, waitUntil='networkidle2', timeout=30000)
|
|
```
|
|
|
|
**After (Playwright):**
|
|
|
|
```python
|
|
response = await page.goto(url, wait_until='networkidle', timeout=30000)
|
|
```
|
|
|
|
### 7. Update Wait Methods
|
|
|
|
**Before (Puppeteer):**
|
|
|
|
```python
|
|
await page.waitForNetworkIdle(idleTime=500, timeout=30000)
|
|
await page.waitForNavigation(waitUntil='networkidle2', timeout=10000)
|
|
```
|
|
|
|
**After (Playwright):**
|
|
|
|
```python
|
|
await page.wait_for_load_state('networkidle', timeout=30000)
|
|
await page.wait_for_load_state('networkidle', timeout=10000)
|
|
```
|
|
|
|
### 8. Update Request Interception
|
|
|
|
**Before (Puppeteer):**
|
|
|
|
```python
|
|
await page.setRequestInterception(True)
|
|
|
|
async def intercept(request):
|
|
if request.resourceType in ['image', 'media', 'font', 'stylesheet']:
|
|
await request.abort()
|
|
else:
|
|
await request.continue_()
|
|
|
|
page.on('request', lambda req: asyncio.ensure_future(intercept(req)))
|
|
```
|
|
|
|
**After (Playwright):**
|
|
|
|
```python
|
|
await page.route("**/*", lambda route: route.abort()
|
|
if route.request.resource_type in ['image', 'stylesheet', 'font', 'media']
|
|
else route.continue_())
|
|
```
|
|
|
|
### 9. Update Browser Cleanup
|
|
|
|
**Before (Puppeteer):**
|
|
|
|
```python
|
|
pages = await browser.pages()
|
|
for page in pages:
|
|
await page.close()
|
|
await browser.close()
|
|
```
|
|
|
|
**After (Playwright):**
|
|
|
|
```python
|
|
pages = browser.contexts[0].pages if browser.contexts else []
|
|
for page in pages:
|
|
await page.close()
|
|
await browser.close()
|
|
```
|
|
|
|
## Docker Migration
|
|
|
|
### Update Dockerfile
|
|
|
|
**Before:**
|
|
|
|
```dockerfile
|
|
# Install Chrome
|
|
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | gpg --dearmor -o /usr/share/keyrings/google-chrome-keyring.gpg \
|
|
&& echo "deb [arch=amd64 signed-by=/usr/share/keyrings/google-chrome-keyring.gpg] http://dl.google.com/linux/chrome/deb/ stable main" | tee /etc/apt/sources.list.d/google-chrome.list \
|
|
&& apt-get update \
|
|
&& apt-get install -y google-chrome-stable
|
|
```
|
|
|
|
**After:**
|
|
|
|
```dockerfile
|
|
# Install Playwright browsers
|
|
RUN playwright install chromium
|
|
RUN playwright install-deps chromium
|
|
```
|
|
|
|
## Testing the Migration
|
|
|
|
### 1. Run the Test Script
|
|
|
|
```bash
|
|
python test_playwright_migration.py
|
|
```
|
|
|
|
This script tests:
|
|
|
|
- Playwright installation
|
|
- Browser utilities
|
|
- Browser service functionality
|
|
|
|
### 2. Manual Testing
|
|
|
|
Test your key functionality:
|
|
|
|
- Basic page navigation
|
|
- JavaScript execution
|
|
- Screenshot capture
|
|
- PDF generation (if used)
|
|
|
|
### 3. Performance Testing
|
|
|
|
Compare performance metrics:
|
|
|
|
- Page load times
|
|
- Memory usage
|
|
- CPU usage
|
|
- Browser startup time
|
|
|
|
## Common Issues and Solutions
|
|
|
|
### Issue: Browser Not Starting
|
|
|
|
**Solution:** Ensure Playwright browsers are installed:
|
|
|
|
```bash
|
|
playwright install chromium
|
|
```
|
|
|
|
### Issue: Permission Errors
|
|
|
|
**Solution:** Install system dependencies:
|
|
|
|
```bash
|
|
playwright install-deps chromium
|
|
```
|
|
|
|
### Issue: Page Navigation Fails
|
|
|
|
**Solution:** Check timeout settings and network conditions:
|
|
|
|
```python
|
|
# Increase timeout if needed
|
|
await page.goto(url, wait_until='networkidle', timeout=60000)
|
|
```
|
|
|
|
### Issue: Memory Leaks
|
|
|
|
**Solution:** Ensure proper cleanup:
|
|
|
|
```python
|
|
# Always close context and browser
|
|
await context.close()
|
|
await browser.close()
|
|
```
|
|
|
|
## Performance Optimizations
|
|
|
|
### 1. Browser Pool Management
|
|
|
|
```python
|
|
# Use browser pooling for better performance
|
|
browser_pool = Queue(maxsize=MAX_BROWSERS)
|
|
```
|
|
|
|
### 2. Context Reuse
|
|
|
|
```python
|
|
# Reuse contexts when possible
|
|
context = await browser.new_context()
|
|
# Use context for multiple pages
|
|
```
|
|
|
|
### 3. Resource Blocking
|
|
|
|
```python
|
|
# Block unnecessary resources
|
|
await page.route("**/*", lambda route: route.abort()
|
|
if route.request.resource_type in ['image', 'stylesheet', 'font', 'media']
|
|
else route.continue_())
|
|
```
|
|
|
|
## Rollback Plan
|
|
|
|
If you need to rollback to Puppeteer:
|
|
|
|
1. Restore your backup
|
|
2. Revert requirements.txt changes
|
|
3. Revert Dockerfile changes
|
|
4. Test thoroughly
|
|
|
|
## Post-Migration Checklist
|
|
|
|
After completing the migration:
|
|
|
|
- [ ] All tests pass
|
|
- [ ] Performance is acceptable
|
|
- [ ] No memory leaks detected
|
|
- [ ] Error handling works correctly
|
|
- [ ] Documentation is updated
|
|
- [ ] Team is trained on new API
|
|
|
|
## Support
|
|
|
|
If you encounter issues during migration:
|
|
|
|
1. Check the [Playwright documentation](https://playwright.dev/python/)
|
|
2. Review the [migration guide](https://playwright.dev/python/docs/migrate-from-puppeteer)
|
|
3. Search existing issues on GitHub
|
|
4. Create a new issue with detailed information
|
|
|
|
## Additional Resources
|
|
|
|
- [Playwright Python Documentation](https://playwright.dev/python/)
|
|
- [Migration Guide from Puppeteer](https://playwright.dev/python/docs/migrate-from-puppeteer)
|
|
- [Playwright GitHub Repository](https://github.com/microsoft/playwright)
|
|
- [Community Discord](https://discord.gg/playwright)
|