7.5 KiB
7.5 KiB
DevOps practical test
A small Python web service, containerized with Docker Compose, built and deployed by a GitLab CI/CD pipeline, monitored with Prometheus, Alertmanager and Grafana, and protected with iptables rules.
The project covers four areas: containerization, CI/CD, monitoring and alerting, and network security with restart policies. Each area has its own detailed guide in docs/.
Documentation
| Guide | What you will find |
|---|---|
| Web application and containerization | The FastAPI service, tests, Dockerfile line by line, Compose file, health check, restart policy and the crash-recovery demonstration |
| Monitoring and alerting | Prometheus, Alertmanager and Grafana: every configuration file explained, the three alert rules, how to trigger and read an alert |
| GitLab CI/CD pipeline | The pipeline stage by stage, the runner setup, how to trigger it, failure recovery |
| Network security with iptables | Why Docker needs DOCKER-USER rules, the rule set, the script, the before/after test and counter evidence |
Requirements and where they are covered
| # | Requirement | Implementation | Guide | Evidence |
|---|---|---|---|---|
| 1 | Containerize the service with Docker Compose | webapp/Dockerfile, webapp/compose.yml |
webapp.md | Service started via Compose |
| 2 | GitLab CI/CD pipeline that builds, tests and deploys | gitlab/.gitlab-ci.yml (included by the root .gitlab-ci.yml) |
gitlab.md | Passing pipeline, test report |
| 3 | Prometheus monitoring and alerting | monitoring/ (Prometheus, Alertmanager, Grafana configs) |
monitoring.md | Targets up, rules, alert firing, Grafana dashboard |
| 4 | iptables rules restricting external access | iptables/firewall.sh |
iptables.md | Test from a second machine, rule counters |
| 5 | Restart policy and automatic restart after a kill | restart: unless-stopped in webapp/compose.yml |
webapp.md, section 7 | Restart after kill |
Architecture
Developer laptop Home server
+------------------------------+ git push +-----------------------------------+
| webapp + monitoring stack | ----------------> | GitLab CE (behind Traefik, TLS) |
| - webapp :8000 | | | |
| - Prometheus :9090 | | v |
| - Alertmanager :9093 | | GitLab runner (Docker executor) |
| - Grafana :3000 | | test -> build -> deploy |
| iptables firewall | | | |
+------------------------------+ | v |
| webapp container :8000 |
+-----------------------------------+
The webapp exposes /metrics; Prometheus collects them, evaluates the alert rules and passes firing alerts to Alertmanager, which sends them to a webhook receiver. Grafana draws the data. The pipeline tests the code, builds the image and deploys it, and only reports success when the container is healthy.
Repository layout
.
├── README.md this file
├── .gitlab-ci.yml includes gitlab/.gitlab-ci.yml
├── docs/ the four guides and docs/screenshots/
├── webapp/ FastAPI service, tests, Dockerfile, compose.yml
├── monitoring/ compose.yml, prometheus/, alertmanager/, grafana/
├── gitlab/ .gitlab-ci.yml, compose.yml of the GitLab stack, runner notes
└── iptables/ firewall.sh
Setup
Requirements: Docker with the Compose plugin, Linux. For the firewall: iptables and root access.
# 1. A network shared by the two stacks (once)
docker network create monitoring-net
# 2. The web service
cd webapp
docker compose up -d --build
# 3. Monitoring
cd ../monitoring
docker compose up -d
Check that everything is running:
docker ps
curl localhost:8000/health # {"status":"ok"}
| Service | URL | Notes |
|---|---|---|
| Webapp | http://localhost:8000 | /health, /metrics, /error (returns 500 on purpose) |
| Prometheus | http://localhost:9090 | Status → Targets, Alerts |
| Alertmanager | http://localhost:9093 | |
| Grafana | http://localhost:3000 | User admin, password admin (set GRAFANA_PASSWORD to change); dashboard "Webapp Overview" |
Apply the firewall (optional, requires root; undo with down):
sudo ./iptables/firewall.sh up
sudo ./iptables/firewall.sh status
sudo ./iptables/firewall.sh down
Stop everything:
(cd monitoring && docker compose down)
(cd webapp && docker compose down)
Trigger the pipeline
- Push the repository to the GitLab project:
git push origin master. - GitLab starts a pipeline automatically. Follow it under Build → Pipelines.
- The stages are test (pytest), build (Docker image), an optional publish and deploy (default branch only). The deploy succeeds only when the container reports
healthy. - To run it again without a new commit: Build → Pipelines → Run pipeline.
Details, the runner setup and the pipeline file are in docs/gitlab.md.
Failure recovery
| Situation | What to do |
|---|---|
| The webapp container crashed | Nothing: restart: unless-stopped restarts it automatically. Check with docker ps and docker logs webapp |
The webapp was stopped by hand (docker stop) |
Start it again: docker start webapp (or docker compose up -d in webapp/) |
The container is unhealthy |
docker logs webapp, fix the cause, then docker compose up -d --build |
| An alert is firing | Read it at http://localhost:9090/alerts and docker logs alert-logger; check the Grafana dashboard and docker logs webapp |
| Prometheus rules or config changed | Validate with docker exec prometheus promtool check config /etc/prometheus/prometheus.yml, then reload with curl -X POST localhost:9090/-/reload |
| A pipeline job is stuck "pending" | The runner is offline or not enabled for the project: docker logs gitlab-runner and the GitLab runners page |
| A deploy broke the service | Revert the commit and push; the pipeline redeploys the previous version |
| The firewall blocks something unexpected | sudo ./iptables/firewall.sh down removes all rules at once |
More specific tables are at the end of each guide.
Notes
- The Compose files are named
compose.yml, the current default name that Docker Compose looks for (docker-compose.ymlalso works). - Screenshots in
docs/screenshots/were taken during the actual runs described in the guides. - This is a demonstration setup. Each guide ends with a short list of what would change in production (authentication, TLS, real alert receivers, isolated CI runners, IPv6 firewall rules, and so on).