intermediate commit
Build and Push Docker Images / build-and-push (push) Failing after 37s

This commit is contained in:
2025-11-17 21:56:21 +01:00
parent b516c942df
commit d72ee4a7d2
8 changed files with 667 additions and 13 deletions
+16
View File
@@ -0,0 +1,16 @@
FROM python:3.12-slim
# Set workdir
WORKDIR /app
# Copy requirements first (better caching)
COPY requirements.txt .
# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Copy source code
COPY . .
# Run script
CMD ["python", "main.py"]
+112
View File
@@ -0,0 +1,112 @@
# Docker Webhook Updater
A Docker container that monitors a Docker image for updates and triggers a webhook when a new version is detected.
## Features
- Monitors Docker images from any registry (Docker Hub, GHCR, Quay.io, custom registries)
- Detects image updates by comparing SHA256 digests
- Triggers webhook with bearer token authentication
- Persistent SHA storage across container restarts
- Configurable check interval
## Environment Variables
| Variable | Description | Required | Example |
| ----------------- | --------------------------------------- | -------- | ----------------------------------------------------- |
| `DOCKER_REPO_URL` | Docker registry URL | Yes | `docker.io`, `ghcr.io`, `quay.io`, or custom registry |
| `DOCKER_IMAGE` | Full image name with tag | Yes | `username/image:latest`, `library/nginx:1.25` |
| `WEBHOOK_URL` | URL to call when image is updated | Yes | `https://example.com/webhook` |
| `WEBHOOK_TOKEN` | Bearer token for webhook authentication | Yes | `your-secret-token` |
| `CHECK_INTERVAL` | Seconds between checks (default: 300) | No | `600` |
## Usage
### Docker Hub Example
```bash
docker run -d \
-e DOCKER_REPO_URL=docker.io \
-e DOCKER_IMAGE=library/nginx:latest \
-e WEBHOOK_URL=https://example.com/webhook \
-e WEBHOOK_TOKEN=your-secret-token \
-e CHECK_INTERVAL=300 \
your-registry/docker-webhook-updater:latest
```
### GitHub Container Registry Example
```bash
docker run -d \
-e DOCKER_REPO_URL=ghcr.io \
-e DOCKER_IMAGE=username/repo:latest \
-e WEBHOOK_URL=https://example.com/webhook \
-e WEBHOOK_TOKEN=your-secret-token \
your-registry/docker-webhook-updater:latest
```
### Docker Compose Example
```yaml
version: "3.8"
services:
webhook-updater:
image: your-registry/docker-webhook-updater:latest
environment:
- DOCKER_REPO_URL=docker.io
- DOCKER_IMAGE=library/nginx:latest
- WEBHOOK_URL=https://example.com/webhook
- WEBHOOK_TOKEN=your-secret-token
- CHECK_INTERVAL=300
restart: unless-stopped
```
## How It Works
1. The service periodically checks the Docker registry for the specified image
2. It retrieves the SHA256 digest of the image manifest
3. Compares the current SHA with the previously stored SHA
4. If different, triggers the webhook with a POST request containing:
```json
{
"event": "image_updated",
"image": "username/image:latest",
"repo_url": "docker.io"
}
```
5. Updates the stored SHA only after successful webhook trigger
## Webhook Request Format
When an image update is detected, the service sends a POST request to the webhook URL:
- **Method**: POST
- **Headers**:
- `Authorization: Bearer {WEBHOOK_TOKEN}`
- `Content-Type: application/json`
- **Body**:
```json
{
"event": "image_updated",
"image": "username/image:tag",
"repo_url": "docker.io"
}
```
## Notes
- The SHA is stored in `/app/sha.txt` inside the container
- On first run, the SHA is stored without triggering the webhook
- If the webhook fails, the SHA is not updated, so it will retry on the next check
- For private Docker images, authentication may be required (not currently supported)
- The service runs continuously and checks at the specified interval
## Building
```bash
docker build -t docker-webhook-updater:latest .
```
+257
View File
@@ -0,0 +1,257 @@
import os
import sys
import time
import logging
import requests
from typing import Optional
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s',
handlers=[
logging.StreamHandler(sys.stdout)
]
)
logger = logging.getLogger(__name__)
# Environment variables
DOCKER_REPO_URL = os.getenv('DOCKER_REPO_URL')
DOCKER_IMAGE = os.getenv('DOCKER_IMAGE')
WEBHOOK_URL = os.getenv('WEBHOOK_URL')
WEBHOOK_TOKEN = os.getenv('WEBHOOK_TOKEN')
# SHA storage file
SHA_FILE = '/app/sha.txt'
# Check interval in seconds (default: 300 = 5 minutes)
CHECK_INTERVAL = int(os.getenv('CHECK_INTERVAL', '300'))
def validate_env_vars():
"""Validate that all required environment variables are set."""
missing = []
if not DOCKER_REPO_URL:
missing.append('DOCKER_REPO_URL')
if not DOCKER_IMAGE:
missing.append('DOCKER_IMAGE')
if not WEBHOOK_URL:
missing.append('WEBHOOK_URL')
if not WEBHOOK_TOKEN:
missing.append('WEBHOOK_TOKEN')
if missing:
logger.error(f"Missing required environment variables: {', '.join(missing)}")
sys.exit(1)
def get_image_sha(repo_url: str, image: str) -> Optional[str]:
"""
Get the SHA256 digest of the Docker image from the registry.
Args:
repo_url: Docker registry URL (e.g., docker.io, ghcr.io, or custom registry)
image: Full image name with tag (e.g., 'username/image:tag' or 'username/image:latest')
Returns:
SHA256 digest string or None if failed
"""
try:
# Parse image name and tag
if ':' in image:
image_name, tag = image.rsplit(':', 1)
else:
image_name = image
tag = 'latest'
# Normalize repo URL (remove https://, http://, trailing /)
repo_url_normalized = repo_url.rstrip('/').replace('https://', '').replace('http://', '')
# Construct manifest URL
# For Docker Hub: https://registry-1.docker.io/v2/{namespace}/{image}/manifests/{tag}
# For other registries: https://{repo_url}/v2/{image}/manifests/{tag}
if repo_url_normalized in ('docker.io', 'hub.docker.com', ''):
# Docker Hub uses registry-1.docker.io
registry_url = 'https://registry-1.docker.io'
# For Docker Hub, image_name should be 'username/image' or 'library/image' for official images
# If no namespace, assume it's an official image (library namespace)
if '/' not in image_name:
image_name = f'library/{image_name}'
manifest_url = f"{registry_url}/v2/{image_name}/manifests/{tag}"
else:
# Other registries (ghcr.io, quay.io, custom registries)
registry_url = f"https://{repo_url_normalized}"
manifest_url = f"{registry_url}/v2/{image_name}/manifests/{tag}"
logger.debug(f"Fetching manifest from: {manifest_url}")
# Request manifest with Accept header for schema v2
headers = {
'Accept': 'application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json'
}
response = requests.get(manifest_url, headers=headers, timeout=30)
response.raise_for_status()
# Get SHA from Docker-Content-Digest header (preferred)
sha = response.headers.get('Docker-Content-Digest')
if not sha:
# Fallback: try to get from response if it's a manifest list
try:
manifest = response.json()
# For manifest lists, get the digest from the first platform-specific manifest
if manifest.get('mediaType') == 'application/vnd.docker.distribution.manifest.list.v2+json':
if 'manifests' in manifest and len(manifest['manifests']) > 0:
sha = manifest['manifests'][0].get('digest')
elif 'config' in manifest and 'digest' in manifest['config']:
sha = manifest['config']['digest']
elif 'digest' in manifest:
sha = manifest['digest']
except Exception as e:
logger.debug(f"Could not parse manifest JSON: {e}")
if sha:
logger.info(f"Current SHA for {image}: {sha[:16]}...")
return sha
else:
logger.warning(f"Could not extract SHA from manifest response")
return None
except requests.exceptions.HTTPError as e:
if e.response.status_code == 404:
logger.error(f"Image not found: {image} (tag: {tag})")
elif e.response.status_code == 401:
logger.error(f"Authentication required for {repo_url}. Private images may require authentication.")
else:
logger.error(f"HTTP error fetching image SHA: {e}")
return None
except requests.exceptions.RequestException as e:
logger.error(f"Failed to fetch image SHA: {e}")
return None
except Exception as e:
logger.error(f"Unexpected error fetching image SHA: {e}")
return None
def load_stored_sha() -> Optional[str]:
"""Load the previously stored SHA from file."""
try:
if os.path.exists(SHA_FILE):
with open(SHA_FILE, 'r') as f:
sha = f.read().strip()
if sha:
logger.debug(f"Loaded stored SHA: {sha}")
return sha
except Exception as e:
logger.warning(f"Failed to load stored SHA: {e}")
return None
def save_sha(sha: str):
"""Save the SHA to file."""
try:
with open(SHA_FILE, 'w') as f:
f.write(sha)
logger.debug(f"Saved SHA: {sha}")
except Exception as e:
logger.error(f"Failed to save SHA: {e}")
def trigger_webhook(url: str, token: str) -> bool:
"""
Trigger the webhook with bearer token authentication.
Args:
url: Webhook URL
token: Bearer token
Returns:
True if successful, False otherwise
"""
try:
headers = {
'Authorization': f'Bearer {token}',
'Content-Type': 'application/json'
}
# Send POST request (you can modify payload if needed)
payload = {
'event': 'image_updated',
'image': DOCKER_IMAGE,
'repo_url': DOCKER_REPO_URL
}
logger.info(f"Triggering webhook: {url}")
response = requests.post(url, json=payload, headers=headers, timeout=30)
response.raise_for_status()
logger.info(f"Webhook triggered successfully (status: {response.status_code})")
return True
except requests.exceptions.RequestException as e:
logger.error(f"Failed to trigger webhook: {e}")
return False
except Exception as e:
logger.error(f"Unexpected error triggering webhook: {e}")
return False
def main():
"""Main loop to check for image updates."""
logger.info("Starting Docker Webhook Updater")
logger.info(f"Monitoring: {DOCKER_REPO_URL}/{DOCKER_IMAGE}")
logger.info(f"Webhook URL: {WEBHOOK_URL}")
logger.info(f"Check interval: {CHECK_INTERVAL} seconds")
validate_env_vars()
# Load initial SHA
stored_sha = load_stored_sha()
if stored_sha:
logger.info(f"Initial stored SHA: {stored_sha}")
else:
logger.info("No stored SHA found. Will trigger webhook on first check if image exists.")
while True:
try:
# Get current SHA from registry
current_sha = get_image_sha(DOCKER_REPO_URL, DOCKER_IMAGE)
if current_sha:
# Compare with stored SHA
if stored_sha and current_sha != stored_sha:
logger.info(f"Image updated! Old SHA: {stored_sha[:16]}..., New SHA: {current_sha[:16]}...")
# Trigger webhook
if trigger_webhook(WEBHOOK_URL, WEBHOOK_TOKEN):
# Save new SHA only if webhook was successful
save_sha(current_sha)
stored_sha = current_sha
else:
logger.warning("Webhook failed. SHA not updated. Will retry on next check.")
elif not stored_sha:
# First run - just store the SHA without triggering webhook
logger.info(f"First run: storing initial SHA: {current_sha[:16]}...")
save_sha(current_sha)
stored_sha = current_sha
else:
logger.debug(f"No update detected. SHA: {current_sha[:16]}...")
else:
logger.warning("Could not fetch image SHA. Will retry on next check.")
# Wait before next check
logger.debug(f"Waiting {CHECK_INTERVAL} seconds before next check...")
time.sleep(CHECK_INTERVAL)
except KeyboardInterrupt:
logger.info("Received interrupt signal. Shutting down...")
break
except Exception as e:
logger.error(f"Unexpected error in main loop: {e}")
logger.info(f"Waiting {CHECK_INTERVAL} seconds before retry...")
time.sleep(CHECK_INTERVAL)
if __name__ == "__main__":
main()
+25
View File
@@ -0,0 +1,25 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1
WORKDIR /app
# System deps for curl and bash used by Teleport installer
RUN apt-get update && apt-get install -y --no-install-recommends \
curl ca-certificates bash tzdata \
&& rm -rf /var/lib/apt/lists/*
# Copy app
COPY main.py /app/main.py
# Add entrypoint
COPY entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
# Default environment knobs
ENV TELEPORT_EDITION=oss \
EXECUTE=false \
LOOP_INTERVAL_SECONDS=60
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
@@ -0,0 +1,77 @@
#!/bin/bash
set -euo pipefail
#
# Teleport setup (installed at runtime to allow choosing version/edition via env)
#
if [[ -n "${TELEPORT_VERSION:-}" ]]; then
echo "Installing Teleport ${TELEPORT_VERSION} (${TELEPORT_EDITION:-oss})..."
curl -fsSL https://goteleport.com/static/install.sh | bash -s "${TELEPORT_VERSION}" "${TELEPORT_EDITION:-oss}"
else
echo "TELEPORT_VERSION not set; skipping Teleport install. Ensure tsh/tctl are available."
fi
# Configure Teleport node if a token is provided (optional)
if [[ -n "${TELEPORT_TOKEN:-}" && -n "${TELEPORT_URL:-}" ]]; then
if [[ ! -f /etc/teleport.yaml ]]; then
echo "Configuring Teleport node with token..."
teleport configure --roles=node --token="${TELEPORT_TOKEN}" --proxy="${TELEPORT_URL}" --no-acme -o file
fi
# Ensure nodename matches container hostname
HOSTNAME=$(hostname)
sed -i "s/^ nodename:.*/ nodename: ${HOSTNAME}/" /etc/teleport.yaml || true
echo "Starting Teleport node in background..."
nohup teleport start >/var/log/teleport.log 2>&1 &
fi
#
# Build flags for tsh/tctl from environment
#
TSH_FLAGS=()
TCTL_FLAGS=()
if [[ -n "${TELEPORT_URL:-}" ]]; then
TSH_FLAGS+=("--proxy=${TELEPORT_URL}")
TCTL_FLAGS+=("--proxy=${TELEPORT_URL}")
fi
if [[ -n "${TELEPORT_IDENTITY_FILE:-}" && -f "${TELEPORT_IDENTITY_FILE}" ]]; then
echo "Using Teleport identity: ${TELEPORT_IDENTITY_FILE}"
TSH_FLAGS+=("--identity=${TELEPORT_IDENTITY_FILE}")
TCTL_FLAGS+=("--identity=${TELEPORT_IDENTITY_FILE}")
fi
# Optional: extra flags from env (space-separated)
if [[ -n "${EXTRA_TSH_FLAGS:-}" ]]; then
# shellcheck disable=SC2206
TSH_FLAGS+=( ${EXTRA_TSH_FLAGS} )
fi
if [[ -n "${EXTRA_TCTL_FLAGS:-}" ]]; then
# shellcheck disable=SC2206
TCTL_FLAGS+=( ${EXTRA_TCTL_FLAGS} )
fi
echo "Starting teleport-auto-remover main loop (interval: ${LOOP_INTERVAL_SECONDS:-60}s, execute: ${EXECUTE:-false})"
while true; do
TS=$(date -Iseconds)
echo "[${TS}] Running prune…"
set +e
# Build CLI args for main.py with repeated tsh/tctl flags (skip empties)
ARGS=(/app/main.py)
for f in "${TSH_FLAGS[@]}"; do
[[ -n "$f" ]] && ARGS+=("--tsh-flag=$f")
done
for f in "${TCTL_FLAGS[@]}"; do
[[ -n "$f" ]] && ARGS+=("--tctl-flag=$f")
done
if [[ "${EXECUTE:-false}" == "true" ]]; then ARGS+=(--execute); fi
# Debug-print the exact command for troubleshooting
printf 'Invoking: python3'
for a in "${ARGS[@]}"; do printf ' %q' "$a"; done; printf '\n'
python3 "${ARGS[@]}"
EXIT_CODE=$?
set -e
echo "Run finished with code ${EXIT_CODE}."
sleep "${LOOP_INTERVAL_SECONDS:-60}"
done
+150
View File
@@ -0,0 +1,150 @@
#!/usr/bin/env python3
"""
teleport_prune_by_hostname_v2.py
- Groups nodes by spec.hostname
- In each duplicate group, removes the node with the *oldest version*
- If versions are tied, removes the one with the earliest metadata.expires (oldest heartbeat)
- Dry-run by default; pass --execute to delete with `tctl rm node/<metadata.name> --confirm`
"""
import argparse, json, shutil, subprocess, sys
from collections import defaultdict
from datetime import datetime
from typing import Any, Dict, List, Tuple, Optional
# ---------- tiny shell helpers ----------
def ensure_binary(name: str):
if shutil.which(name) is None:
sys.exit(f"ERROR: '{name}' not found on PATH.")
def run(cmd: List[str]) -> subprocess.CompletedProcess:
try:
return subprocess.run(cmd, check=True, capture_output=True, text=True)
except subprocess.CalledProcessError as e:
sys.stderr.write(f"\nCommand failed: {' '.join(cmd)}\nSTDERR:\n{e.stderr}\n")
raise
# ---------- JSON field helpers ----------
def get_hostname(n: Dict[str, Any]) -> Optional[str]:
return (n.get("spec") or {}).get("hostname") or None
def get_uuid(n: Dict[str, Any]) -> Optional[str]:
return (n.get("metadata") or {}).get("name") or None
def get_version(n: Dict[str, Any]) -> str:
return (n.get("spec") or {}).get("version") or "0.0.0"
def get_expires(n: Dict[str, Any]) -> Optional[datetime]:
exp = (n.get("metadata") or {}).get("expires")
if not exp:
return None
# tolerate fractional seconds + Z
try:
# Example: 2025-10-26T17:45:26.132967655Z
exp = exp.rstrip("Z")
# trim to microseconds if too long
if "." in exp:
head, tail = exp.split(".", 1)
tail = ''.join(ch for ch in tail if ch.isdigit())
tail = (tail + "000000")[:6]
exp = f"{head}.{tail}"
return datetime.fromisoformat(exp)
except Exception:
return None
def parse_version_tuple(v: str) -> Tuple[int, ...]:
parts = []
for token in (v or "").split("."):
num = ""
for ch in token:
if ch.isdigit(): num += ch
else: break
parts.append(int(num) if num else 0)
return tuple(parts) if parts else (0,)
# ---------- core ----------
def list_nodes(tsh_bin: str, tsh_flags: List[str]) -> List[Dict[str, Any]]:
cp = run([tsh_bin, "ls", "--format=json", *tsh_flags])
try:
data = json.loads(cp.stdout)
except json.JSONDecodeError:
sys.exit("ERROR: Could not parse JSON from `tsh ls --format=json`.")
if not isinstance(data, list):
sys.exit("ERROR: Unexpected JSON (expected list).")
return data
def decide_deletions(nodes: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
groups: Dict[str, List[Dict[str, Any]]] = defaultdict(list)
for n in nodes:
host = get_hostname(n)
print(host)
if not host: # ignore ungroupable entries
continue
entry = {
"uuid": get_uuid(n),
"hostname": host,
"version": get_version(n),
"expires": get_expires(n),
}
entry["version_key"] = parse_version_tuple(entry["version"])
groups[host].append(entry)
to_delete: List[Dict[str, Any]] = []
for host, items in groups.items():
if len(items) <= 1:
continue
# Sort by: newest version first, then latest expiry first
items_sorted = sorted(
items,
key=lambda x: (x["version_key"], x["expires"] or datetime.min),
reverse=True
)
# choose the *oldest* by version; if tie, earliest expiry
victim = items_sorted[-1]
to_delete.append(victim)
return to_delete
def main():
ap = argparse.ArgumentParser(description="Remove the oldest Teleport node per duplicated hostname (Teleport v2 JSON).")
ap.add_argument("--tsh", default="tsh")
ap.add_argument("--tctl", default="tctl")
ap.add_argument("--tsh-flag", action="append", default=[], help="Extra flags for tsh (repeatable)")
ap.add_argument("--tctl-flag", action="append", default=[], help="Extra flags for tctl (repeatable)")
ap.add_argument("--execute", action="store_true", help="Actually delete (default is dry-run)")
args = ap.parse_args()
ensure_binary(args.tsh); ensure_binary(args.tctl)
print("Listing nodes via tsh...")
nodes = list_nodes(args.tsh, args.tsh_flag)
deletions = decide_deletions(nodes)
if not deletions:
print("No duplicate hostnames found. Nothing to do.")
return
print("\nPlanned deletions (oldest per duplicate hostname):")
for d in deletions:
exp = d["expires"].isoformat() if d["expires"] else "unknown"
print(f" host={d['hostname']:<30} uuid={d['uuid']} version={d['version']} expires={exp}")
if not args.execute:
print("\nDRY-RUN: No changes made. Re-run with --execute to delete the above nodes.")
return
print("\nDeleting via tctl…")
for d in deletions:
if not d["uuid"]:
print(f"SKIP (no UUID): host={d['hostname']} version={d['version']}")
continue
cmd = [args.tctl, *args.tctl_flag, "rm", f"node/{d['uuid']}"]
print(" ", " ".join(cmd))
try:
run(cmd); print(" -> OK")
except Exception: print(" -> FAILED")
print("\nDone.")
if __name__ == "__main__":
main()
+21 -6
View File
@@ -13,20 +13,35 @@ api:
password: "" password: ""
ota: ota:
platform: esphome
password: "" password: ""
platform: esphome
wifi: wifi:
ssid: !secret wifi_ssid ssid: "ItHurtsWhenIP"
password: !secret wifi_password password: "BKl@3660"
manual_ip:
# Set this to the IP of the ESP
static_ip: 10.0.1.50
# Set this to the IP address of the router. Often ends with .1
gateway: 10.0.0.1
# The subnet of the network. 255.255.255.0 works for most home networks.
subnet: 255.255.254.0
# Enable fallback hotspot (captive portal) in case wifi connection fails # Enable fallback hotspot (captive portal) in case wifi connection fails
ap: ap:
ssid: "Bed-Scale Fallback Hotspot" ssid: "Bed-Scale Fallback Hotspot"
password: !secret hotspot_password password: "bINLLate9rIA"
captive_portal: captive_portal:
# sensor:
# - platform: hx711
# name: "HX711 Value"
# dout_pin: D4
# clk_pin: D3
# gain: 128
# update_interval: 3s
globals: globals:
- id: constant_weight - id: constant_weight
type: float type: float
@@ -41,8 +56,8 @@ sensor:
gain: 128 gain: 128
filters: filters:
- calibrate_linear: - calibrate_linear:
- 1827200 -> 0 - 2290200 -> 0
- 3218000 -> 105 - 2856500 -> 50
on_value: on_value:
- if: - if:
condition: condition:
+9 -7
View File
@@ -3,22 +3,23 @@ substitutions:
esphome: esphome:
name: shelly1-slaapkamer name: shelly1-slaapkamer
platform: ESP8266
esp8266:
board: esp01_1m board: esp01_1m
wifi: wifi:
ssid: !secret wifi_ssid ssid: "ItHurtsWhenIP"
password: !secret wifi_password password: "BKl@3660"
manual_ip: manual_ip:
static_ip: 192.168.179.28 static_ip: 10.0.1.51
gateway: 192.168.178.1 gateway: 10.0.0.1
subnet: 255.255.254.0 subnet: 255.255.254.0
dns1: 1.1.1.1 dns1: 1.1.1.1
dns2: 8.8.8.8 dns2: 8.8.8.8
power_save_mode: none power_save_mode: none
ap: ap:
ssid: "Slaapkamer Fallback Hotspot" ssid: "Slaapkamer Fallback Hotspot"
password: !secret hotspot_password password: "bINLLate9rIA"
captive_portal: captive_portal:
logger: logger:
@@ -26,6 +27,8 @@ logger:
api: api:
ota: ota:
password: ""
platform: esphome
web_server: web_server:
port: 80 port: 80
@@ -61,7 +64,6 @@ sensor:
update_interval: 60s update_interval: 60s
id: wifi_signal_sensor id: wifi_signal_sensor
switch: switch:
- platform: gpio - platform: gpio
name: ${device_name} name: ${device_name}