52 lines
1.7 KiB
Markdown
52 lines
1.7 KiB
Markdown
# Centralized logging (Loki + Promtail + Grafana)
|
|
|
|
We ship a lightweight logging stack into the cluster so API/UI logs are searchable.
|
|
|
|
- **Loki** (single-binary) stores logs with 14d retention by default, on a PVC.
|
|
- **Promtail** DaemonSet tails container logs and ships them to Loki with `namespace`, `pod`, and `app` labels.
|
|
- **Grafana** provides the UI with a pre-wired Loki data source and TLS ingress.
|
|
|
|
## Install / upgrade
|
|
|
|
Prereqs:
|
|
- `kubectl`/`helm` access to the cluster (the script downloads Helm if missing).
|
|
- Environment: `GRAFANA_ADMIN_PASSWORD` (required), optional `LOGS_HOST` (default `logs.lomavuokraus.fi`), `GRAFANA_CLUSTER_ISSUER` (default `letsencrypt-prod`), `LOGGING_NAMESPACE` (default `logging`).
|
|
|
|
Run:
|
|
|
|
```bash
|
|
LOGS_HOST=logs.lomavuokraus.fi \
|
|
GRAFANA_ADMIN_PASSWORD='change-me' \
|
|
GRAFANA_CLUSTER_ISSUER=letsencrypt-prod \
|
|
bash deploy/install-logging.sh
|
|
```
|
|
|
|
The script:
|
|
1. Ensures Helm is available.
|
|
2. Installs/updates Loki, Promtail, and Grafana in the logging namespace.
|
|
3. Creates a Grafana ingress with TLS via the chosen ClusterIssuer.
|
|
|
|
## Access
|
|
|
|
- Grafana: `https://<LOGS_HOST>` (admin user `admin`, password from `GRAFANA_ADMIN_PASSWORD`).
|
|
- Loki endpoint (internal): `http://loki.logging.svc.cluster.local:3100`.
|
|
|
|
## Querying
|
|
|
|
Example LogQL in Grafana Explore:
|
|
|
|
```
|
|
{namespace="lomavuokraus-test", app="lomavuokraus-web"}
|
|
```
|
|
|
|
Filter by pod:
|
|
|
|
```
|
|
{namespace="lomavuokraus-test", app="lomavuokraus-web", pod=~".*"} |= "ERROR"
|
|
```
|
|
|
|
## Tuning
|
|
|
|
- Retention: `k8s/logging/loki-values.yaml` (`limits_config.retention_period`).
|
|
- PVC sizes: adjust `persistence.size` in `k8s/logging/loki-values.yaml` and `k8s/logging/grafana-values.yaml`.
|
|
- Ingress issuer/host: override via environment when running `deploy/install-logging.sh`.
|