ocr api

Dockers/ocr-api/.gitignore

@@ -0,0 +1,149 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+pip-wheel-metadata/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# Test files
+test_files/
+*.pdf
+*.png
+*.jpg
+*.jpeg
+*.bmp
+*.tiff
+*.tif
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# OS
+.DS_Store
+Thumbs.db

Dockers/ocr-api/Dockerfile

@@ -0,0 +1,46 @@
+# Use Python 3.11 slim image as base
+FROM python:3.11-slim
+
+# Set environment variables
+ENV PYTHONUNBUFFERED=1
+ENV PYTHONDONTWRITEBYTECODE=1
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    tesseract-ocr \
+    tesseract-ocr-eng \
+    tesseract-ocr-fra \
+    tesseract-ocr-deu \
+    tesseract-ocr-spa \
+    tesseract-ocr-ita \
+    poppler-utils \
+    libpoppler-cpp-dev \
+    pkg-config \
+    && rm -rf /var/lib/apt/lists/*
+
+# Set working directory
+WORKDIR /app
+
+# Copy requirements first for better caching
+COPY requirements.txt .
+
+# Install Python dependencies
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Copy application code
+COPY main.py .
+
+# Create a non-root user
+RUN useradd --create-home --shell /bin/bash app \
+    && chown -R app:app /app
+USER app
+
+# Expose port
+EXPOSE 8000
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
+    CMD curl -f http://localhost:8000/health || exit 1
+
+# Run the application
+CMD ["python", "main.py"]

Dockers/ocr-api/README.md

@@ -0,0 +1,207 @@
+# OCR API
+
+A FastAPI-based OCR (Optical Character Recognition) service that can extract text from PDF files and images using Tesseract.
+
+## Features
+
+- **PDF Processing**: Convert PDF files to images and extract text using OCR
+- **Image Processing**: Direct OCR processing of image files (PNG, JPG, JPEG, BMP, TIFF)
+- **Multi-language Support**: Supports English, French, German, Spanish, and Italian
+- **Confidence Scoring**: Returns confidence scores for OCR results
+- **RESTful API**: Clean, documented API endpoints
+- **Docker Support**: Easy deployment with Docker and Docker Compose
+- **File Validation**: Built-in file type and size validation
+- **Background Processing**: Efficient file cleanup and processing
+
+## Quick Start
+
+### Using Docker (Recommended)
+
+1. **Clone and build the container:**
+
+   ```bash
+   docker-compose up --build
+   ```
+
+2. **Access the API:**
+   - API Documentation: http://localhost:8000/docs
+   - Health Check: http://localhost:8000/health
+
+### Local Development
+
+1. **Install system dependencies:**
+
+   ```bash
+   # Ubuntu/Debian
+   sudo apt-get update
+   sudo apt-get install tesseract-ocr tesseract-ocr-eng poppler-utils libpoppler-cpp-dev pkg-config
+
+   # macOS
+   brew install tesseract poppler
+
+   # Arch Linux
+   sudo pacman -S tesseract tesseract-data-eng poppler
+   ```
+
+2. **Install Python dependencies:**
+
+   ```bash
+   pip install -r requirements.txt
+   ```
+
+3. **Run the application:**
+   ```bash
+   python main.py
+   ```
+
+## API Endpoints
+
+### Health Check
+
+- **GET** `/health` - Check if the service is running
+
+### PDF OCR
+
+- **POST** `/ocr/pdf` - Extract text from PDF files
+  - **File**: PDF file (max 50MB)
+  - **Response**: Extracted text with metadata
+
+### Image OCR
+
+- **POST** `/ocr/image` - Extract text from image files
+  - **File**: Image file (PNG, JPG, JPEG, BMP, TIFF)
+  - **Response**: Extracted text with metadata
+
+## Usage Examples
+
+### Using curl
+
+**Process a PDF file:**
+
+```bash
+curl -X POST "http://localhost:8000/ocr/pdf" \
+     -H "accept: application/json" \
+     -H "Content-Type: multipart/form-data" \
+     -F "file=@document.pdf"
+```
+
+**Process an image file:**
+
+```bash
+curl -X POST "http://localhost:8000/ocr/image" \
+     -H "accept: application/json" \
+     -H "Content-Type: multipart/form-data" \
+     -F "file=@image.png"
+```
+
+### Using Python
+
+```python
+import requests
+
+# Process PDF
+with open('document.pdf', 'rb') as f:
+    files = {'file': f}
+    response = requests.post('http://localhost:8000/ocr/pdf', files=files)
+    result = response.json()
+    print(f"Extracted text: {result['text']}")
+    print(f"Processing time: {result['processing_time']}s")
+    print(f"Confidence: {result['confidence']}%")
+```
+
+### Using JavaScript/Fetch
+
+```javascript
+// Process PDF
+const formData = new FormData();
+formData.append("file", fileInput.files[0]);
+
+fetch("http://localhost:8000/ocr/pdf", {
+  method: "POST",
+  body: formData,
+})
+  .then((response) => response.json())
+  .then((data) => {
+    console.log("Extracted text:", data.text);
+    console.log("Processing time:", data.processing_time);
+    console.log("Confidence:", data.confidence);
+  });
+```
+
+## Response Format
+
+```json
+{
+  "text": "Extracted text content...",
+  "pages": 3,
+  "processing_time": 2.45,
+  "confidence": 85.2
+}
+```
+
+## Configuration
+
+### Environment Variables
+
+- `PYTHONUNBUFFERED=1` - Ensures Python output is not buffered
+- `PYTHONDONTWRITEBYTECODE=1` - Prevents Python from writing .pyc files
+
+### Tesseract Configuration
+
+The Docker image includes multiple language packs:
+
+- English (eng)
+- French (fra)
+- German (deu)
+- Spanish (spa)
+- Italian (ita)
+
+To add more languages, modify the Dockerfile and add additional `tesseract-ocr-*` packages.
+
+## Performance Tips
+
+1. **Image Quality**: Higher DPI (300) provides better OCR accuracy but slower processing
+2. **File Size**: Larger files take longer to process
+3. **Text Quality**: Clear, high-contrast text yields better results
+4. **Language**: Specify the correct language for better accuracy
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Tesseract not found**: Ensure Tesseract is installed on your system
+2. **PDF processing errors**: Check if Poppler utilities are installed
+3. **Memory issues**: Reduce DPI or process smaller files
+4. **Poor OCR quality**: Ensure images are clear and have good contrast
+
+### Logs
+
+The application logs processing information. Check Docker logs:
+
+```bash
+docker-compose logs ocr-api
+```
+
+## Development
+
+### Project Structure
+
+```
+ocr-api/
+├── main.py              # FastAPI application
+├── requirements.txt     # Python dependencies
+├── Dockerfile          # Docker configuration
+├── docker-compose.yml  # Docker Compose configuration
+└── README.md          # This file
+```
+
+### Adding Features
+
+1. **New OCR engines**: Modify the `process_pdf_with_ocr` function
+2. **Additional file formats**: Add new endpoints and processing logic
+3. **Custom preprocessing**: Add image preprocessing steps before OCR
+4. **Batch processing**: Implement queue-based processing for multiple files
+
+## License
+
+This project is open source and available under the MIT License.

Dockers/ocr-api/docker-compose.yml

@@ -0,0 +1,19 @@
+version: "3.8"
+
+services:
+  ocr-api:
+    build: .
+    ports:
+      - "8000:8000"
+    environment:
+      - PYTHONUNBUFFERED=1
+    volumes:
+      # Optional: mount a directory for testing with sample files
+      - ./test_files:/app/test_files:ro
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 40s

Dockers/ocr-api/main.py

@@ -0,0 +1,225 @@
+import os
+import tempfile
+import logging
+from typing import List, Optional
+from pathlib import Path
+
+from fastapi import FastAPI, File, UploadFile, HTTPException, BackgroundTasks
+from fastapi.responses import JSONResponse
+from pydantic import BaseModel
+import pytesseract
+from PIL import Image
+from pdf2image import convert_from_path
+import uvicorn
+
+# Configure logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)
+
+app = FastAPI(
+    title="OCR API",
+    description="API for extracting text from PDF files using OCR",
+    version="1.0.0"
+)
+
+class OCRResponse(BaseModel):
+    text: str
+    pages: int
+    processing_time: float
+    confidence: Optional[float] = None
+
+class ErrorResponse(BaseModel):
+    error: str
+    detail: str
+
+@app.get("/")
+async def root():
+    return {"message": "OCR API is running. Use /docs for API documentation."}
+
+@app.get("/health")
+async def health_check():
+    return {"status": "healthy", "service": "OCR API"}
+
+async def process_pdf_with_ocr(pdf_path: str) -> OCRResponse:
+    """
+    Process a PDF file and extract text using OCR
+    """
+    import time
+    start_time = time.time()
+
+    try:
+        # Convert PDF to images
+        logger.info(f"Converting PDF to images: {pdf_path}")
+        images = convert_from_path(pdf_path, dpi=300)
+
+        if not images:
+            raise HTTPException(status_code=400, detail="Could not extract images from PDF")
+
+        # Extract text from each page
+        all_text = []
+        total_confidence = 0
+        confidence_count = 0
+
+        for i, image in enumerate(images):
+            logger.info(f"Processing page {i+1}/{len(images)}")
+
+            # Perform OCR on the image
+            ocr_data = pytesseract.image_to_data(image, output_type=pytesseract.Output.DICT)
+
+            # Extract text and confidence scores
+            page_text = []
+            for j in range(len(ocr_data['text'])):
+                if int(ocr_data['conf'][j]) > 0:  # Filter out low confidence results
+                    page_text.append(ocr_data['text'][j])
+                    total_confidence += float(ocr_data['conf'][j])
+                    confidence_count += 1
+
+            page_text = ' '.join(page_text).strip()
+            if page_text:
+                all_text.append(f"--- Page {i+1} ---\n{page_text}")
+
+        # Calculate average confidence
+        avg_confidence = total_confidence / confidence_count if confidence_count > 0 else None
+
+        processing_time = time.time() - start_time
+
+        return OCRResponse(
+            text='\n\n'.join(all_text),
+            pages=len(images),
+            processing_time=round(processing_time, 2),
+            confidence=round(avg_confidence, 2) if avg_confidence else None
+        )
+
+    except Exception as e:
+        logger.error(f"Error processing PDF: {str(e)}")
+        raise HTTPException(status_code=500, detail=f"Error processing PDF: {str(e)}")
+
+@app.post("/ocr/pdf", response_model=OCRResponse)
+async def extract_text_from_pdf(
+    file: UploadFile = File(...),
+    background_tasks: BackgroundTasks = None
+):
+    """
+    Extract text from a PDF file using OCR
+
+    - **file**: PDF file to process
+    - Returns extracted text with metadata
+    """
+
+    # Validate file type
+    if not file.filename.lower().endswith('.pdf'):
+        raise HTTPException(
+            status_code=400,
+            detail="File must be a PDF"
+        )
+
+    # Check file size (limit to 50MB)
+    if file.size and file.size > 50 * 1024 * 1024:
+        raise HTTPException(
+            status_code=400,
+            detail="File size too large. Maximum size is 50MB"
+        )
+
+    try:
+        # Create temporary file
+        with tempfile.NamedTemporaryFile(delete=False, suffix='.pdf') as temp_file:
+            # Write uploaded file to temporary file
+            content = await file.read()
+            temp_file.write(content)
+            temp_file_path = temp_file.name
+
+        try:
+            # Process the PDF
+            result = await process_pdf_with_ocr(temp_file_path)
+
+            # Clean up temporary file
+            background_tasks.add_task(os.unlink, temp_file_path)
+
+            return result
+
+        except Exception as e:
+            # Clean up temporary file on error
+            if os.path.exists(temp_file_path):
+                os.unlink(temp_file_path)
+            raise e
+
+    except Exception as e:
+        logger.error(f"Error handling file upload: {str(e)}")
+        raise HTTPException(status_code=500, detail=f"Error processing file: {str(e)}")
+
+@app.post("/ocr/image", response_model=OCRResponse)
+async def extract_text_from_image(
+    file: UploadFile = File(...),
+    background_tasks: BackgroundTasks = None
+):
+    """
+    Extract text from an image file using OCR
+
+    - **file**: Image file to process (PNG, JPG, JPEG, etc.)
+    - Returns extracted text with metadata
+    """
+
+    # Validate file type
+    allowed_extensions = {'.png', '.jpg', '.jpeg', '.bmp', '.tiff', '.tif'}
+    file_extension = Path(file.filename).suffix.lower()
+
+    if file_extension not in allowed_extensions:
+        raise HTTPException(
+            status_code=400,
+            detail=f"Unsupported file type. Allowed types: {', '.join(allowed_extensions)}"
+        )
+
+    try:
+        # Create temporary file
+        with tempfile.NamedTemporaryFile(delete=False, suffix=file_extension) as temp_file:
+            content = await file.read()
+            temp_file.write(content)
+            temp_file_path = temp_file.name
+
+        try:
+            import time
+            start_time = time.time()
+
+            # Open and process image
+            image = Image.open(temp_file_path)
+
+            # Perform OCR
+            ocr_data = pytesseract.image_to_data(image, output_type=pytesseract.Output.DICT)
+
+            # Extract text and confidence
+            text_parts = []
+            total_confidence = 0
+            confidence_count = 0
+
+            for i in range(len(ocr_data['text'])):
+                if int(ocr_data['conf'][i]) > 0:
+                    text_parts.append(ocr_data['text'][i])
+                    total_confidence += float(ocr_data['conf'][i])
+                    confidence_count += 1
+
+            text = ' '.join(text_parts).strip()
+            avg_confidence = total_confidence / confidence_count if confidence_count > 0 else None
+            processing_time = time.time() - start_time
+
+            # Clean up temporary file
+            background_tasks.add_task(os.unlink, temp_file_path)
+
+            return OCRResponse(
+                text=text,
+                pages=1,
+                processing_time=round(processing_time, 2),
+                confidence=round(avg_confidence, 2) if avg_confidence else None
+            )
+
+        except Exception as e:
+            # Clean up temporary file on error
+            if os.path.exists(temp_file_path):
+                os.unlink(temp_file_path)
+            raise e
+
+    except Exception as e:
+        logger.error(f"Error processing image: {str(e)}")
+        raise HTTPException(status_code=500, detail=f"Error processing image: {str(e)}")
+
+if __name__ == "__main__":
+    uvicorn.run(app, host="0.0.0.0", port=8000)

Dockers/ocr-api/requirements.txt

@@ -0,0 +1,8 @@
+fastapi==0.104.1
+uvicorn[standard]==0.24.0
+python-multipart==0.0.6
+Pillow==10.1.0
+pytesseract==0.3.10
+pdf2image==1.16.3
+pydantic==2.5.0
+python-dotenv==1.0.0

Dockers/ocr-api/test_client.py

@@ -0,0 +1,109 @@
+#!/usr/bin/env python3
+"""
+Test client for the OCR API
+This script demonstrates how to use the OCR API endpoints
+"""
+
+import requests
+import json
+import sys
+from pathlib import Path
+
+# API base URL
+BASE_URL = "http://localhost:8000"
+
+def test_health():
+    """Test the health endpoint"""
+    try:
+        response = requests.get(f"{BASE_URL}/health")
+        print(f"Health check: {response.status_code} - {response.json()}")
+        return response.status_code == 200
+    except requests.exceptions.ConnectionError:
+        print("❌ Could not connect to the API. Make sure it's running on localhost:8000")
+        return False
+
+def test_pdf_ocr(file_path):
+    """Test PDF OCR endpoint"""
+    if not Path(file_path).exists():
+        print(f"❌ File not found: {file_path}")
+        return False
+
+    try:
+        with open(file_path, 'rb') as f:
+            files = {'file': f}
+            print(f"📄 Processing PDF: {file_path}")
+            response = requests.post(f"{BASE_URL}/ocr/pdf", files=files)
+
+        if response.status_code == 200:
+            result = response.json()
+            print(f"✅ PDF OCR successful!")
+            print(f"   Pages: {result['pages']}")
+            print(f"   Processing time: {result['processing_time']}s")
+            print(f"   Confidence: {result['confidence']}%")
+            print(f"   Text preview: {result['text'][:200]}...")
+            return True
+        else:
+            print(f"❌ PDF OCR failed: {response.status_code} - {response.text}")
+            return False
+
+    except Exception as e:
+        print(f"❌ Error processing PDF: {str(e)}")
+        return False
+
+def test_image_ocr(file_path):
+    """Test image OCR endpoint"""
+    if not Path(file_path).exists():
+        print(f"❌ File not found: {file_path}")
+        return False
+
+    try:
+        with open(file_path, 'rb') as f:
+            files = {'file': f}
+            print(f"🖼️  Processing image: {file_path}")
+            response = requests.post(f"{BASE_URL}/ocr/image", files=files)
+
+        if response.status_code == 200:
+            result = response.json()
+            print(f"✅ Image OCR successful!")
+            print(f"   Processing time: {result['processing_time']}s")
+            print(f"   Confidence: {result['confidence']}%")
+            print(f"   Text: {result['text']}")
+            return True
+        else:
+            print(f"❌ Image OCR failed: {response.status_code} - {response.text}")
+            return False
+
+    except Exception as e:
+        print(f"❌ Error processing image: {str(e)}")
+        return False
+
+def main():
+    """Main test function"""
+    print("🧪 OCR API Test Client")
+    print("=" * 50)
+
+    # Test health endpoint
+    if not test_health():
+        sys.exit(1)
+
+    print("\n" + "=" * 50)
+
+    # Test PDF OCR if file provided
+    if len(sys.argv) > 1:
+        file_path = sys.argv[1]
+        file_ext = Path(file_path).suffix.lower()
+
+        if file_ext == '.pdf':
+            test_pdf_ocr(file_path)
+        elif file_ext in ['.png', '.jpg', '.jpeg', '.bmp', '.tiff', '.tif']:
+            test_image_ocr(file_path)
+        else:
+            print(f"❌ Unsupported file type: {file_ext}")
+            print("Supported types: .pdf, .png, .jpg, .jpeg, .bmp, .tiff, .tif")
+    else:
+        print("📝 Usage: python test_client.py <file_path>")
+        print("   Example: python test_client.py document.pdf")
+        print("   Example: python test_client.py image.png")
+
+if __name__ == "__main__":
+    main()