replace puppeteer with playwright

2025-07-04 13:15:59 +02:00
parent 8c4a27c87e
commit 129c6a74e8
10 changed files with 1030 additions and 463 deletions
@@ -0,0 +1,344 @@
+# Migration Guide: Puppeteer to Playwright
+
+This guide will help you migrate from Puppeteer (pyppeteer) to Playwright in your project.
+
+## Overview
+
+The migration from Puppeteer to Playwright provides several benefits:
+
+- Better performance and stability
+- Enhanced features for modern web applications
+- Active development and community support
+- Multi-browser support (Chromium, Firefox, WebKit)
+
+## Pre-Migration Checklist
+
+Before starting the migration, ensure you have:
+
+- [ ] Backed up your current codebase
+- [ ] Documented any custom Puppeteer configurations
+- [ ] Identified all Puppeteer-specific code in your project
+- [ ] Tested your current application thoroughly
+
+## Step-by-Step Migration
+
+### 1. Update Dependencies
+
+Replace Puppeteer dependencies with Playwright:
+
+**Before (requirements.txt):**
+
+```
+pyppeteer==1.0.2
+```
+
+**After (requirements.txt):**
+
+```
+playwright==1.40.0
+```
+
+### 2. Install Playwright
+
+```bash
+# Install Playwright
+pip install playwright
+
+# Install browsers (Chromium is recommended for compatibility)
+playwright install chromium
+
+# Install system dependencies (Linux)
+playwright install-deps chromium
+```
+
+### 3. Update Imports
+
+**Before:**
+
+```python
+from pyppeteer import launch
+```
+
+**After:**
+
+```python
+from playwright.async_api import async_playwright
+```
+
+### 4. Update Browser Launch
+
+**Before (Puppeteer):**
+
+```python
+browser = await launch(
+    headless=True,
+    executablePath='/usr/bin/google-chrome',
+    args=['--no-sandbox', '--disable-setuid-sandbox'],
+    handleSIGINT=False,
+    handleSIGTERM=False,
+    handleSIGHUP=False,
+    ignoreHTTPSErrors=True,
+    autoClose=True,
+)
+```
+
+**After (Playwright):**
+
+```python
+playwright = await async_playwright().start()
+browser = await playwright.chromium.launch(
+    headless=True,
+    args=['--no-sandbox', '--disable-setuid-sandbox'],
+    ignore_default_args=['--enable-automation'],
+)
+```
+
+### 5. Update Page Creation
+
+**Before (Puppeteer):**
+
+```python
+page = await browser.newPage()
+await page.setUserAgent(CUSTOM_USER_AGENT)
+await page.setViewport({'width': 1280, 'height': 800})
+page.setDefaultNavigationTimeout(30000)
+```
+
+**After (Playwright):**
+
+```python
+context = await browser.new_context(
+    user_agent=CUSTOM_USER_AGENT,
+    viewport={'width': 1920, 'height': 1080},
+    ignore_https_errors=True,
+)
+page = await context.new_page()
+page.set_default_timeout(30000)
+```
+
+### 6. Update Navigation
+
+**Before (Puppeteer):**
+
+```python
+response = await page.goto(url, waitUntil='networkidle2', timeout=30000)
+```
+
+**After (Playwright):**
+
+```python
+response = await page.goto(url, wait_until='networkidle', timeout=30000)
+```
+
+### 7. Update Wait Methods
+
+**Before (Puppeteer):**
+
+```python
+await page.waitForNetworkIdle(idleTime=500, timeout=30000)
+await page.waitForNavigation(waitUntil='networkidle2', timeout=10000)
+```
+
+**After (Playwright):**
+
+```python
+await page.wait_for_load_state('networkidle', timeout=30000)
+await page.wait_for_load_state('networkidle', timeout=10000)
+```
+
+### 8. Update Request Interception
+
+**Before (Puppeteer):**
+
+```python
+await page.setRequestInterception(True)
+
+async def intercept(request):
+    if request.resourceType in ['image', 'media', 'font', 'stylesheet']:
+        await request.abort()
+    else:
+        await request.continue_()
+
+page.on('request', lambda req: asyncio.ensure_future(intercept(req)))
+```
+
+**After (Playwright):**
+
+```python
+await page.route("**/*", lambda route: route.abort()
+               if route.request.resource_type in ['image', 'stylesheet', 'font', 'media']
+               else route.continue_())
+```
+
+### 9. Update Browser Cleanup
+
+**Before (Puppeteer):**
+
+```python
+pages = await browser.pages()
+for page in pages:
+    await page.close()
+await browser.close()
+```
+
+**After (Playwright):**
+
+```python
+pages = browser.contexts[0].pages if browser.contexts else []
+for page in pages:
+    await page.close()
+await browser.close()
+```
+
+## Docker Migration
+
+### Update Dockerfile
+
+**Before:**
+
+```dockerfile
+# Install Chrome
+RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | gpg --dearmor -o /usr/share/keyrings/google-chrome-keyring.gpg \
+    && echo "deb [arch=amd64 signed-by=/usr/share/keyrings/google-chrome-keyring.gpg] http://dl.google.com/linux/chrome/deb/ stable main" | tee /etc/apt/sources.list.d/google-chrome.list \
+    && apt-get update \
+    && apt-get install -y google-chrome-stable
+```
+
+**After:**
+
+```dockerfile
+# Install Playwright browsers
+RUN playwright install chromium
+RUN playwright install-deps chromium
+```
+
+## Testing the Migration
+
+### 1. Run the Test Script
+
+```bash
+python test_playwright_migration.py
+```
+
+This script tests:
+
+- Playwright installation
+- Browser utilities
+- Browser service functionality
+
+### 2. Manual Testing
+
+Test your key functionality:
+
+- Basic page navigation
+- JavaScript execution
+- Screenshot capture
+- PDF generation (if used)
+
+### 3. Performance Testing
+
+Compare performance metrics:
+
+- Page load times
+- Memory usage
+- CPU usage
+- Browser startup time
+
+## Common Issues and Solutions
+
+### Issue: Browser Not Starting
+
+**Solution:** Ensure Playwright browsers are installed:
+
+```bash
+playwright install chromium
+```
+
+### Issue: Permission Errors
+
+**Solution:** Install system dependencies:
+
+```bash
+playwright install-deps chromium
+```
+
+### Issue: Page Navigation Fails
+
+**Solution:** Check timeout settings and network conditions:
+
+```python
+# Increase timeout if needed
+await page.goto(url, wait_until='networkidle', timeout=60000)
+```
+
+### Issue: Memory Leaks
+
+**Solution:** Ensure proper cleanup:
+
+```python
+# Always close context and browser
+await context.close()
+await browser.close()
+```
+
+## Performance Optimizations
+
+### 1. Browser Pool Management
+
+```python
+# Use browser pooling for better performance
+browser_pool = Queue(maxsize=MAX_BROWSERS)
+```
+
+### 2. Context Reuse
+
+```python
+# Reuse contexts when possible
+context = await browser.new_context()
+# Use context for multiple pages
+```
+
+### 3. Resource Blocking
+
+```python
+# Block unnecessary resources
+await page.route("**/*", lambda route: route.abort()
+               if route.request.resource_type in ['image', 'stylesheet', 'font', 'media']
+               else route.continue_())
+```
+
+## Rollback Plan
+
+If you need to rollback to Puppeteer:
+
+1. Restore your backup
+2. Revert requirements.txt changes
+3. Revert Dockerfile changes
+4. Test thoroughly
+
+## Post-Migration Checklist
+
+After completing the migration:
+
+- [ ] All tests pass
+- [ ] Performance is acceptable
+- [ ] No memory leaks detected
+- [ ] Error handling works correctly
+- [ ] Documentation is updated
+- [ ] Team is trained on new API
+
+## Support
+
+If you encounter issues during migration:
+
+1. Check the [Playwright documentation](https://playwright.dev/python/)
+2. Review the [migration guide](https://playwright.dev/python/docs/migrate-from-puppeteer)
+3. Search existing issues on GitHub
+4. Create a new issue with detailed information
+
+## Additional Resources
+
+- [Playwright Python Documentation](https://playwright.dev/python/)
+- [Migration Guide from Puppeteer](https://playwright.dev/python/docs/migrate-from-puppeteer)
+- [Playwright GitHub Repository](https://github.com/microsoft/playwright)
+- [Community Discord](https://discord.gg/playwright)