added kubernetes (k3s) setup and docs and monitoring

docs/K3s.md

@@ -0,0 +1,116 @@
+# K3s
+
+A single-node K3s cluster running alongside the existing Docker Compose stack — the first step toward learning Kubernetes hands-on rather than purely through theory, without disrupting the services already running on this host.
+
+## Why K3s, not kind or full kubeadm
+
+Full `kubeadm` Kubernetes runs a separate etcd cluster plus independent API server, controller-manager, and scheduler processes — too much overhead for this host's RAM budget. `kind` runs each cluster node as a Docker container, which nests Kubernetes' own networking and iptables rules inside a host that already has Docker managing iptables for 25+ services — more surface area for conflicts, not less, and better suited to disposable CI/test clusters than a persistent homelab service.
+
+K3s runs as a single systemd-managed binary directly on the host, with SQLite (via `kine`) standing in for etcd on a single-node setup — the lightest option that still behaves like real Kubernetes.
+
+## Install
+
+```bash
+curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server \
+  --disable=traefik \
+  --disable=servicelb \
+  --flannel-backend=vxlan \
+  --write-kubeconfig-mode=644" sh -
+```
+
+- **`--disable=traefik` / `--disable=servicelb`** — K3s ships with its own ingress controller and LoadBalancer implementation, both of which try to bind ports 80/443. This host's reverse proxy (see [Reverse Proxy](Reverse-Proxy.md)) already owns those ports — disabling K3s's defaults avoids the conflict entirely rather than debugging it after the fact.
+- **`--flannel-backend=vxlan`** — explicit rather than relying on the default, and chosen over `host-gw` since VXLAN works across nodes that aren't on the same L2 segment, keeping the door open for a future worker node.
+- **`--write-kubeconfig-mode=644`** — makes `/etc/rancher/k3s/k3s.yaml` world-readable so a personal, non-root kubeconfig copy can be made without repeated `sudo`.
+
+## Exposure pattern: NPM in front, NodePort behind
+
+Rather than running a second ingress controller inside the cluster (which would mean two reverse proxies stacked for no benefit), each service gets a `Service` of `type: NodePort`. The existing reverse proxy adds one more proxy host pointing at `homeserver:<nodeport>` — identical to how it already fronts every Docker container. From the proxy's perspective, a K8s pod behind a NodePort is indistinguishable from a Docker container; same TLS setup, same cert renewal flow, zero new moving parts in the HTTPS chain.
+
+## Namespace and resource governance
+
+All workloads live in a dedicated `apps` namespace (not `default`), governed by a `LimitRange` and `ResourceQuota` sized for this host's RAM budget:
+
+```yaml
+apiVersion: v1
+kind: LimitRange
+metadata:
+  name: apps-limits
+  namespace: apps
+spec:
+  limits:
+  - type: Container
+    default:        { cpu: "200m", memory: "256Mi" }
+    defaultRequest:  { cpu: "50m",  memory: "64Mi" }
+    max:             { cpu: "500m", memory: "512Mi" }
+    min:             { cpu: "10m",  memory: "16Mi" }
+---
+apiVersion: v1
+kind: ResourceQuota
+metadata:
+  name: apps-quota
+  namespace: apps
+spec:
+  hard:
+    requests.cpu: "1"
+    requests.memory: "768Mi"
+    limits.cpu: "1.5"
+    limits.memory: "1.5Gi"
+    pods: "10"
+```
+
+Any container deployed without explicit resource fields automatically inherits the `LimitRange` defaults; no container can exceed the `max` bounds even if it tries; and the whole namespace is hard-capped at 1.5 CPU / 1.5Gi RAM combined — protecting the Docker services and the OS from a runaway K8s workload, regardless of how many pods end up running.
+
+## Backups
+
+The cluster's SQLite state (`/var/lib/rancher/k3s/server/db/state.db`) is backed up on a rolling schedule via a systemd timer — deliberately off the OS disk, onto a separate physical drive:
+
+```bash
+sqlite3 "$DB_PATH" ".backup '$BACKUP_FILE'"
+gzip "$BACKUP_FILE"
+# keep only the 2 most recent, prune the rest
+```
+
+Runs every 4 days via `OnUnitActiveSec=4d` with `Persistent=true`, so a missed run (host powered off) catches up on next boot rather than silently skipping. This covers cluster state — Deployments, Services, config — not data inside PersistentVolumes; there are no stateful workloads in the cluster yet, so that gap doesn't currently apply.
+
+## kubectl ergonomics
+
+- Personal kubeconfig at `~/.kube/config` (copied from `/etc/rancher/k3s/k3s.yaml`), with `KUBECONFIG` explicitly exported in `.zshrc` — avoids the k3s-bundled `kubectl` silently falling back to its own default path and disagreeing with other tools about which config is authoritative.
+- `kubectx` / `kubens` for fast context and namespace switching, plus oh-my-zsh's `kubectl` plugin for tab-completion and short aliases (`kgp`, `kgd`, `kaf`, etc.).
+- Default namespace set to `apps` via `kubens apps`, so day-to-day commands don't need `-n apps` on every invocation.
+
+## GUI: FreeLens
+
+[FreeLens](https://github.com/freelensapp/freelens) — the actively maintained, fully open-source fork of Lens, created after the original Lens IDE moved behind a paid subscription. Same core experience (cluster overview, live workload view, logs, terminal) reading the same kubeconfig, with no account and no license cost — a better fit for a self-hosted learning project than a subscription tool.
+
+![FreeLens namespaces view](images/freelens-namespaces.png)
+
+![FreeLens nodes view](images/freelens-nodes.png)
+
+## Monitoring integration
+
+Rather than deploying a second Prometheus/Grafana/Loki stack inside K3s, the existing Docker-based monitoring stack (see [Monitoring Stack](Monitoring-Stack.md)) was extended to scrape and tail the K3s cluster as an additional source — avoiding the RAM cost of duplicate infrastructure.
+
+**The gap this closes:** K3s uses containerd, not Docker, so the existing cAdvisor container (which watches the Docker socket) has no visibility into K8s pods at all. Two things bridge that gap:
+
+- **[`kube-state-metrics`](https://github.com/kubernetes/kube-state-metrics)** — deployed into the cluster, exposes object-level state (deployment replica counts, pod restarts, resource requests vs. limits) as Prometheus metrics via a NodePort.
+- **Kubelet's built-in cAdvisor** — already running as part of K3s, exposes per-pod CPU/memory at `https://<host>:10250/metrics/cadvisor`. Requires a bearer token, unlike the open Docker cAdvisor endpoint — sourced from a scoped `ServiceAccount` created specifically for external Prometheus access.
+
+Both added as standard Prometheus scrape jobs, authenticated via a `ClusterRole` limited to read-only access on nodes, pods, and services — no broader cluster privileges than the metrics actually require.
+
+**Logs** follow the same pattern: Grafana Alloy's `discovery.kubernetes` + `loki.source.kubernetes` components stream pod logs through the K8s API (each needing its own `client` block with the same kind of scoped token — the two components don't share credentials even when defined side-by-side), landing in the same Loki instance as the Docker container logs. K3s pod log streams carry a `cluster="k3s-homeserver"` label to distinguish them from Docker-origin entries at query time.
+
+## Registry access under sanctions
+
+`registry.k8s.io` (Google-hosted) returns a hard `403 Forbidden` from this network — a harder block than the intermittent Docker Hub flakiness described elsewhere in this wiki, and one that will resurface for any future addon (cert-manager, ingress-nginx, etc.) that defaults to pulling from it.
+
+**Working pattern when this happens:**
+```bash
+docker pull <community-mirror-image>       # Docker Hub / ghcr.io usually reachable
+docker save <image> -o /tmp/image.tar
+sudo k3s ctr images import /tmp/image.tar   # load directly into containerd
+```
+Sidesteps `registry.k8s.io` entirely by pulling through Docker (which has working access) and importing straight into containerd's local store, rather than waiting on network access that isn't coming.
+
+## Why this matters
+
+The instinct to keep K3s and the existing Docker stack cleanly separated — one reverse proxy, one monitoring stack, one set of conventions — paid off directly: every new K8s workload is visible in the same Grafana instance, fronted by the same TLS setup, without a second parallel toolchain to maintain. On a four-core host with limited RAM, reusing existing infrastructure instead of duplicating it isn't just tidier — it's the difference between K3s fitting on this hardware at all and it competing with everything else already running here.