updated docs and readme to showcase playbooks
This commit is contained in:
rezayaghobi-dev
140
README.md
140
README.md
@@ -5,71 +5,97 @@ A self-hosted homelab running on Docker Compose, built and maintained on an Ubun
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Overview
|
||||
|
||||
This is a real, running homelab — not a demo. It's deployed on a home Ubuntu server and managed entirely through Docker Compose, with each service isolated in its own folder. Configs here are the actual files running in production, with all secrets, API keys, and runtime state stripped out and replaced with `.env.example` placeholders.
|
||||
This is a real, running homelab — not a demo. It's deployed on a home Ubuntu server and managed entirely through Docker Compose, with each service isolated in its own folder, and updated through a shared Ansible role run via Semaphore. Configs here are the actual files running in production, with all secrets, API keys, and runtime state stripped out and replaced with `.env.example` placeholders.
|
||||
|
||||
## Services
|
||||
|
||||
| Service | Folder | Purpose |
|
||||
|---|---|---|
|
||||
| **Monitoring stack** | `monitoring-grafana-promethues-cadvisor-node-exporter/` | Prometheus for metrics collection, Grafana for dashboards, cAdvisor for per-container resource stats, node-exporter for host-level metrics |
|
||||
| **Pi-hole** | `pihole/` | Network-wide DNS-level ad and tracker blocking |
|
||||
| **Portainer** | `portainer/` | Web UI for managing and monitoring all Docker containers on the host |
|
||||
| **DockScope** | `dockscope/` | 3D visual dashboard mapping containers, networks, and dependencies, with anomaly detection and crash diagnostics — see [DockScope](docs/DockScope.md) |
|
||||
| **n8n** | `n8n/` | Workflow automation engine, backed by Postgres |
|
||||
| **Samba** | `samba/` | Network file sharing across LAN devices |
|
||||
| **Filebrowser** | `filebrowser/` | Web-based file manager for the server's storage |
|
||||
| **Media stack** | `mediastack/` | Plex (media server), Sonarr/Radarr/Prowlarr (media automation and indexing), qBittorrent (download client) |
|
||||
| **Minecraft** | `minecraft/` | Self-hosted game server |
|
||||
| **Sentinel** | `sentinel/` | Disk health monitoring integrated with smartctl, dumps output to container logs |
|
||||
| **smartctl** | `smartctl/` | Disk health monitoring (S.M.A.R.T. data) |
|
||||
| **OpenHands** | `openhands/` | Self-hosted AI coding agent (OpenHands/agent-canvas) backed by Ollama — see [AI Coding Agent](docs/AI-Agent.md) |
|
||||
| **Nginx Proxy Manager** | `nginx-proxy-manager/` | Reverse proxy giving every service a friendly `.home` hostname with locally-trusted HTTPS — see [Reverse Proxy](docs/Reverse-Proxy.md) |
|
||||
| **Semaphore** | `semaphore/` | Self-hosted Ansible UI (alternative to AWX) — see [Semaphore Guide](docs/Semaphore.md) |
|
||||
| **Monitoring stack** | `docker-compose/monitoring-grafana-promethues-cadvisor-node-exporter/` | Prometheus for metrics collection, Grafana for dashboards, cAdvisor for per-container resource stats, node-exporter for host-level metrics |
|
||||
| **Pi-hole** | `docker-compose/pihole/` | Network-wide DNS-level ad and tracker blocking |
|
||||
| **Portainer** | `docker-compose/portainer/` | Web UI for managing and monitoring all Docker containers on the host |
|
||||
| **DockScope** | `docker-compose/dockscope/` | 3D visual dashboard mapping containers, networks, and dependencies, with anomaly detection and crash diagnostics — see [DockScope](docs/DockScope.md) |
|
||||
| **n8n** | `docker-compose/n8n/` | Workflow automation engine, backed by Postgres |
|
||||
| **Samba** | `docker-compose/samba/` | Network file sharing across LAN devices |
|
||||
| **Filebrowser** | `docker-compose/filebrowser/` | Web-based file manager for the server's storage |
|
||||
| **Media stack** | `docker-compose/mediastack/` | Plex (media server), Sonarr/Radarr/Prowlarr (media automation and indexing), qBittorrent (download client) |
|
||||
| **Minecraft** | `docker-compose/minecraft/` | Self-hosted game server |
|
||||
| **Sentinel** | `docker-compose/sentinel/` | Disk health monitoring integrated with smartctl, dumps output to container logs |
|
||||
| **smartctl** | `docker-compose/smartctl/` | Disk health monitoring (S.M.A.R.T. data) |
|
||||
| **OpenHands** | `docker-compose/openhands/` | Self-hosted AI coding agent (OpenHands/agent-canvas) backed by Ollama — see [AI Coding Agent](docs/AI-Agent.md) |
|
||||
| **Nginx Proxy Manager** | `docker-compose/nginx-proxy-manager/` | Reverse proxy giving every service a friendly `.home` hostname with locally-trusted HTTPS — see [Reverse Proxy](docs/Reverse-Proxy.md) |
|
||||
| **Semaphore** | `docker-compose/semaphore/` | Self-hosted Ansible UI (alternative to AWX) — see [Semaphore Guide](docs/Semaphore.md) |
|
||||
| **Ansible playbooks** | `ansible/` | Shared update role (pull → recreate → health-check → prune) reused across every stack, run via Semaphore — see [Ansible Playbooks](docs/AnsiblePlaybooks.md) |
|
||||
|
||||
## Repository structure
|
||||
|
||||
```
|
||||
homelab/
|
||||
├── monitoring-grafana-promethues-cadvisor-node-exporter/
|
||||
│ └── docker-compose.yml
|
||||
├── pihole/
|
||||
│ ├── docker-compose.yml
|
||||
│ └── .env.example
|
||||
├── portainer/
|
||||
│ └── docker-compose.yml
|
||||
├── dockscope/
|
||||
│ └── docker-compose.yml
|
||||
├── n8n/
|
||||
│ ├── docker-compose.yaml
|
||||
│ └── .env.example
|
||||
├── samba/
|
||||
│ ├── docker-compose.yml
|
||||
│ └── .env.example
|
||||
├── filebrowser/
|
||||
│ └── docker-compose.yml
|
||||
├── mediastack/
|
||||
│ ├── plex/docker-compose.yml
|
||||
│ ├── sonarr/docker-compose.yml
|
||||
│ ├── radarr/docker-compose.yml
|
||||
│ ├── prowlarr/docker-compose.yml
|
||||
│ ├── whisparr/docker-compose.yml
|
||||
│ └── download-client-qbittorent/docker-compose.yml
|
||||
├── minecraft/
|
||||
│ └── docker-compose.yml
|
||||
├── sentinel/
|
||||
│ └── docker-compose.yml
|
||||
├── smartctl/
|
||||
│ └── docker-compose.yml
|
||||
├── openhands/
|
||||
│ └── docker-compose.yaml
|
||||
├── nginx-proxy-manager/
|
||||
│ └── docker-compose.yaml
|
||||
├── semaphore/
|
||||
│ ├── docker-compose.yml
|
||||
├── docker-compose/
|
||||
│ ├── monitoring-grafana-promethues-cadvisor-node-exporter/
|
||||
│ │ └── docker-compose.yml
|
||||
│ ├── pihole/
|
||||
│ │ ├── docker-compose.yml
|
||||
│ │ └── .env.example
|
||||
│ ├── portainer/
|
||||
│ │ └── docker-compose.yml
|
||||
│ ├── dockscope/
|
||||
│ │ └── docker-compose.yml
|
||||
│ ├── n8n/
|
||||
│ │ ├── docker-compose.yaml
|
||||
│ │ └── .env.example
|
||||
│ ├── samba/
|
||||
│ │ ├── docker-compose.yml
|
||||
│ │ └── .env.example
|
||||
│ ├── filebrowser/
|
||||
│ │ └── docker-compose.yml
|
||||
│ ├── mediastack/
|
||||
│ │ ├── plex/docker-compose.yml
|
||||
│ │ ├── sonarr/docker-compose.yml
|
||||
│ │ ├── radarr/docker-compose.yml
|
||||
│ │ ├── prowlarr/docker-compose.yml
|
||||
│ │ ├── whisparr/docker-compose.yml
|
||||
│ │ └── download-client-qbittorent/docker-compose.yml
|
||||
│ ├── minecraft/
|
||||
│ │ └── docker-compose.yml
|
||||
│ ├── sentinel/
|
||||
│ │ └── docker-compose.yml
|
||||
│ ├── smartctl/
|
||||
│ │ └── docker-compose.yml
|
||||
│ ├── openhands/
|
||||
│ │ └── docker-compose.yaml
|
||||
│ ├── nginx-proxy-manager/
|
||||
│ │ └── docker-compose.yaml
|
||||
│ └── semaphore/
|
||||
│ ├── docker-compose.yml
|
||||
│ └── README.md
|
||||
├── ansible/
|
||||
│ └── playbooks/
|
||||
│ ├── roles/
|
||||
│ │ └── docker_compose_update/
|
||||
│ │ └── tasks/main.yml
|
||||
│ ├── update-filebrowser.yml
|
||||
│ ├── update-n8n.yml
|
||||
│ ├── update-npm.yml
|
||||
│ ├── update-pihole.yml
|
||||
│ ├── update-plex.yml
|
||||
│ ├── update-portainer.yml
|
||||
│ ├── update-prowlarr.yml
|
||||
│ ├── update-radarr.yml
|
||||
│ ├── update-samba.yml
|
||||
│ ├── update-sentinel.yml
|
||||
│ ├── update-smartctl.yml
|
||||
│ ├── update-sonarr.yml
|
||||
│ ├── update-whisparr.yml
|
||||
│ ├── update-agent-canvas.yml
|
||||
│ ├── update-grafana.yml
|
||||
│ └── ping.yml
|
||||
├── docs/
|
||||
│ ├── Home.md
|
||||
│ ├── Architecture-and-Hardware.md
|
||||
@@ -83,6 +109,7 @@ homelab/
|
||||
│ ├── Lessons-Learned.md
|
||||
│ ├── Semaphore.md
|
||||
│ ├── DockScope.md
|
||||
│ ├── AnsiblePlaybooks.md
|
||||
│ └── images/
|
||||
└── .gitignore
|
||||
```
|
||||
@@ -92,17 +119,21 @@ homelab/
|
||||
Each folder is a standalone Compose stack:
|
||||
|
||||
```bash
|
||||
cd pihole
|
||||
cd docker-compose/pihole
|
||||
cp .env.example .env # fill in real values
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## Updating a service
|
||||
|
||||
Rather than pulling and recreating manually, every stack has a matching Ansible playbook that pulls the latest images, recreates containers, verifies they're healthy, and prunes unused images — run through Semaphore's web UI. See [Ansible Playbooks](docs/AnsiblePlaybooks.md) for details on how the shared role works.
|
||||
|
||||
## Security notes
|
||||
|
||||
- All `.env` files, runtime config directories, and databases are excluded via `.gitignore` — only deployment definitions (`docker-compose.yml`) and `.env.example` placeholders are tracked.
|
||||
- Passwords referenced in compose files are injected via environment variables, never hardcoded.
|
||||
- The AI coding agent (`openhands/`) currently uses a third-party API aggregator as its LLM backend — see [AI Coding Agent](docs/AI-Agent.md) for details and caveats. It has no access to the Docker socket or other containers on the host.
|
||||
- **DockScope** (`dockscope/`) mounts `/var/run/docker.sock`, the same as Portainer — this grants it effectively root-equivalent access to the Docker host, and it currently has no built-in authentication on its web UI, so it's kept LAN-only and not exposed through the reverse proxy. See [DockScope](docs/DockScope.md).
|
||||
- The AI coding agent (`docker-compose/openhands/`) currently uses a third-party API aggregator as its LLM backend — see [AI Coding Agent](docs/AI-Agent.md) for details and caveats. It has no access to the Docker socket or other containers on the host.
|
||||
- **DockScope** (`docker-compose/dockscope/`) mounts `/var/run/docker.sock`, the same as Portainer — this grants it effectively root-equivalent access to the Docker host, and it currently has no built-in authentication on its web UI, so it's kept LAN-only and not exposed through the reverse proxy. See [DockScope](docs/DockScope.md).
|
||||
|
||||
## Lessons learned
|
||||
|
||||
@@ -111,6 +142,7 @@ docker compose up -d
|
||||
- Centralized monitoring (Prometheus + Grafana + cAdvisor + node-exporter) made it possible to actually see resource usage per container, which mattered a lot on lower-spec hardware.
|
||||
- Free-tier cloud AI APIs enforce regional access restrictions automatically — worth checking a provider's terms before building around "free tier" as an assumption. See [AI Coding Agent](docs/AI-Agent.md).
|
||||
- Wildcard TLS certs (`*.home`) are rejected by browsers on single-label local domains — certificates for local reverse proxies need explicit hostnames listed instead. See [Reverse Proxy](docs/Reverse-Proxy.md).
|
||||
- Grouping all Compose stacks under one `docker-compose/` folder (rather than flat at repo root) kept the structure readable once Ansible playbooks were added as a sibling — a flat repo mixing deployment configs and automation code got confusing fast.
|
||||
|
||||
## Roadmap
|
||||
|
||||
|
||||
81
docs/AnsiblePlaybooks.md
Normal file
81
docs/AnsiblePlaybooks.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# Ansible Playbooks
|
||||
|
||||
How updates get applied across the homelab: one shared role, one thin playbook per stack, triggered through Semaphore.
|
||||
|
||||
## Why
|
||||
|
||||
Updating each Docker Compose stack by hand — SSH in, `cd` into the folder, `docker compose pull`, `docker compose up -d`, check nothing broke, prune old images — works fine for one or two services. Across a dozen-plus stacks it gets repetitive and easy to skip steps (especially the health check after recreating containers). Ansible plus Semaphore turns that into a one-click job per stack, with consistent behavior and a visible run history.
|
||||
|
||||
## Structure
|
||||
|
||||
```
|
||||
ansible/
|
||||
└── playbooks/
|
||||
├── roles/
|
||||
│ └── docker_compose_update/
|
||||
│ └── tasks/
|
||||
│ └── main.yml # the shared update logic
|
||||
├── update-filebrowser.yml
|
||||
├── update-n8n.yml
|
||||
├── update-npm.yml
|
||||
├── update-pihole.yml
|
||||
├── update-plex.yml
|
||||
├── update-portainer.yml
|
||||
├── update-prowlarr.yml
|
||||
├── update-radarr.yml
|
||||
├── update-samba.yml
|
||||
├── update-sentinel.yml
|
||||
├── update-smartctl.yml
|
||||
├── update-sonarr.yml
|
||||
├── update-whisparr.yml
|
||||
├── update-agent-canvas.yml
|
||||
├── update-grafana.yml
|
||||
└── ping.yml
|
||||
```
|
||||
|
||||
One role, reused by every playbook. Each playbook is intentionally tiny:
|
||||
|
||||
```yaml
|
||||
---
|
||||
- hosts: all
|
||||
become: true
|
||||
vars:
|
||||
compose_dir: "/home/rezayaghobi/docker/<service>"
|
||||
service_name: "<Friendly Name>"
|
||||
roles:
|
||||
- docker_compose_update
|
||||
```
|
||||
|
||||
Adding a new stack means adding a new playbook like the one above — no changes to the role itself.
|
||||
|
||||
## What the role actually does
|
||||
|
||||
`docker_compose_update` runs the same sequence for every stack:
|
||||
|
||||
1. **Records a start time**, for a duration summary at the end
|
||||
2. **Snapshots `docker system df`** before the pull, so disk usage is visible
|
||||
3. **Runs `docker compose pull`** in the stack's directory
|
||||
4. **Snapshots `docker system df`** again after the pull — comparing the two makes it obvious when an update pulled down something unexpectedly large (this is what caught a ~9GB pull for the OpenHands/Ollama stack early on — see [AI Coding Agent](AI-Agent.md))
|
||||
5. **Runs `docker compose up -d`** to recreate any containers with new images
|
||||
6. **Waits 5 seconds**, then checks `docker compose ps`
|
||||
7. **Asserts every container is actually up** — fails the run (and skips cleanup) if anything shows `Exit`, `Created`, or `Restarting` instead of running
|
||||
8. **Runs `docker image prune -f`** to reclaim space from now-unused image layers, but only if the health check passed
|
||||
9. **Prints a summary** — duration, cleanup result, pass/fail
|
||||
|
||||
If any step fails (a bad pull, a container that won't start), Ansible stops the play immediately rather than continuing on to prune images out from under a broken deployment.
|
||||
|
||||
## Running it via Semaphore
|
||||
|
||||
Semaphore ([see Semaphore Guide](Semaphore.md)) is what actually executes these — no manual `ansible-playbook` on the server. Each playbook is set up as a Task Template pointed at this repo, so updating a stack is a matter of picking its template and hitting run.
|
||||
|
||||
The Semaphore dashboard gives a quick visual of run history — successes in green, failures in red:
|
||||
|
||||
|
||||
|
||||
## Lessons learned
|
||||
|
||||
- Keeping the update logic in a single role instead of copy-pasting it into every playbook means a fix or improvement (like the disk-usage snapshots) only has to happen once — see the role's own history for an example of that in practice.
|
||||
- The `docker system df` before/after comparison earns its place: it's what made an unexpectedly large image pull for OpenHands/Ollama visible immediately, instead of just being a mystery bandwidth spike after the fact.
|
||||
- The health-check assert before pruning matters — pruning images out from under a stack that failed to come back up would make debugging (and rolling back) harder, not easier.
|
||||
|
||||
[← Back to Home](Home.md)
|
||||
@@ -15,6 +15,7 @@ Documentation for a self-hosted homelab running entirely on Docker Compose. This
|
||||
- [Reverse Proxy](Reverse-Proxy.md)
|
||||
- [Semaphore](Semaphore.md)
|
||||
- [DockScope](DockScope.md)
|
||||
- [Ansible Playbooks](AnsiblePlaybooks.md)
|
||||
|
||||
## At a glance
|
||||
|
||||
@@ -25,7 +26,7 @@ Documentation for a self-hosted homelab running entirely on Docker Compose. This
|
||||
| **Management** | Portainer, **DockScope** (3D visual dashboard) |
|
||||
| **Monitoring** | Prometheus + Grafana + cAdvisor + node-exporter |
|
||||
| **Network** | Pi-hole (DNS filtering), Samba (file sharing), nginx (proxy managing) |
|
||||
| **Automation** | n8n + **Semaphore** (Ansible Web UI) || **Media** | Plex + Sonarr/Radarr/Prowlarr + qBittorrent |
|
||||
| **Automation** | n8n + **Semaphore** (Ansible Web UI) + **Ansible** (update playbooks) || **Media** | Plex + Sonarr/Radarr/Prowlarr + qBittorrent |
|
||||
| **Disk health** | Hard Disk Sentinel |
|
||||
| **AI Agent** | OpenHands + Ollama |
|
||||
|
||||
|
||||
BIN
docs/images/semaphore-dashboard-stats.png
Normal file
BIN
docs/images/semaphore-dashboard-stats.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 76 KiB |
Reference in New Issue
Block a user