lomavuokraus/PROGRESS.md
Tero Halla-aho ed67f4305a
Some checks failed
CI / checks (push) Waiting to run
CI / checks (pull_request) Has been cancelled
Add migration preflight and sync prod DB
2025-12-20 22:59:48 +02:00

105 lines
12 KiB
Markdown

## 2025-11-22 — Lomavuokraus infra progress
- Repo initialized with Next.js App Router scaffold:
- Health endpoint: `app/api/health/route.ts`
- Home + listings demo: `app/page.tsx`, `app/listings/[slug]/page.tsx`
- Styles: `app/globals.css`
- Config: `next.config.mjs`, `tsconfig.json`, `.eslintrc.json`, `.env.example`
- Docker: multi-stage `Dockerfile`, `.dockerignore`, `public/robots.txt`
- Deploy tooling:
- `deploy/env.sh` holds registry/namespace/host/issuer/ingress config (no secrets)
- Scripts: `deploy/build.sh`, `deploy/push.sh`, `deploy/deploy.sh`, `deploy/deploy-prod.sh`, `deploy/deploy-staging.sh`, `deploy/rollback-prod.sh`
- K8s manifests: `k8s/app.yaml` (ConfigMap/Secret/Deployment/Service/Ingress), `k8s/namespaces.yaml`, `k8s/cert-issuers.yaml`
- Cluster:
- Hetzner k3s node: `lomavuokraus-k3s` hel1 cx23 IP `157.180.66.64`
- k3s v1.33.5 installed; kubeconfig in `k3s.yaml` (git-ignored) and `~/.kube/config`
- Namespaces: `lomavuokraus-prod`, `lomavuokraus-staging`
- cert-manager v1.15.3 installed; ClusterIssuers `letsencrypt-prod`/`staging`
- App deployed to both namespaces; ingress host rules in place via Traefik
- DNS: `lomavuokraus.fi`, `staging.lomavuokraus.fi`, `api.lomavuokraus.fi` all A -> `157.180.66.64` (updated via Joker DYNDNS).
- Registry issue (open):
- Builds succeed and image `registry.halla-aho.net:443/thalla/lomavuokraus-web:1763823196` exists locally and was imported into k3s via `ctr import`.
- Pushes (docker, ctr, skopeo from k3s node) fail: connection closed while uploading blobs (`http://registry.halla-aho.net:443/... use of closed network connection`). Suspect registry reverse-proxy dropping uploads/HTTPS handling.
- Need to inspect registry host logs/config and retry push once fixed.
- Secrets:
- `AUTH_SECRET` removed from `deploy/env.sh`; export it in shell (or via `scripts/load-secrets.sh`) before deploy.
- `creds/` and `k3s.yaml` are git-ignored; contains joker DYNDNS creds and registry auth.
## 2025-11-24 — Lomavuokraus app progress
- New testing DB (`lomavuokraus_testing`) holds the previous staging/prod data; the main `lomavuokraus` DB was recreated clean with only the seeded admin user. Migration history was copied, and a schema snapshot lives at `docs/db-schema.sql`.
- Testing environment wiring added: dedicated namespace (`lomavuokraus-test`), deploy wrapper (`deploy/deploy-test.sh`), API host support, and a DNS updater for `test.lomavuokraus.fi` / `apitest.lomavuokraus.fi`.
- Access control tightened: middleware now gates admin routes, admin-only pages check session/role, API handlers return proper 401/403, and listing removal is limited to owners/admins (no more moderator overrides).
- Security: added OWASP ZAP baseline helper (`scripts/zap-baseline.sh`) and documentation (`docs/security.html`) for quick unauthenticated scans against test/staging/prod.
- Added master test suite runner (`scripts/run-test-suite.sh`) that executes npm audit, Trivy scan, and ZAP baseline and writes HTML summaries under `reports/runs/`.
- Listings: added draft saves; backend accepts draft status with nullable listing fields, and the new listing form has a “Save draft” option (publish still enforces required fields + images).
- Backend/data: Added Prisma models (User/Listing/ListingTranslation/ListingImage), seed script creates sample listing; DB on Hetzner VM `46.62.203.202`, staging secrets set in `lomavuokraus-web-secrets`.
- Auth: Register/login/verify flows; session cookie (`session_token`), NavBar shows email+role badge. Roles: USER, ADMIN, USER_MODERATOR (approve users), LISTING_MODERATOR (approve listings). Admin can change roles at `/admin/users`.
- Listing flow: create listing (session required), pending/published with admin/moderator approvals; pages for “My listings,” “New listing,” “Profile.” Quick actions tile removed; all actions in navbar.
- SMTP/DKIM: SMTP via smtp.lomavuokraus.fi (CNAME to smtp.sohva.org); DKIM key at `creds/dkim/lomavuokraus.fi/mail2025.private`, TXT needed at `mail2025._domainkey.lomavuokraus.fi`.
- Deployment/secrets: Removed Secret from k8s manifest to avoid overwrites; env now from k8s secret only. ConfigMap only for public vars. Image pull secret `registry-halla` in staging.
- Latest images built/pushed: `registry.halla-aho.net/thalla/lomavuokraus-web:1763993882` (approvals badge + FI/EN localization) and `:1763994382` (profile edit). Staging/prod rolled out.
- Security: `npm audit --audit-level=high` runs in build (warnings only). Trivy scan run; remaining CVEs mostly in tooling (cross-spawn, glob) and base OS Debian 12.10. Further reduction would require eslint-config-next 16.x and base image updates when available.
## 2025-11-24 — Recent changes
- Public browse/search page with map, address filters, and EV charging amenity; listings now store street address and geocoordinates.
- Amenities expanded: electric vehicle charging (free/paid) and air conditioning; cover image selectable per listing and used in cards.
- Home page shows a rolling feed of latest listings; navbar + CTA link to browse.
- Listing creation form captures address details, coordinates, amenities (incl. EV/AC), and cover image choice.
- Documentation moved to `docs/`; PlantUML diagrams rendered to SVG and embedded in docs pages (draw.io sources kept for architecture/infra).
- UI polish: navbar buttons gained icons, consistent button sizing, and form fields restyled for alignment.
- Auth: added forgotten password flow (email reset link + reset page).
- HTTPS redirect middleware applied to staging/prod ingress.
- FI/EN localization with navbar language toggle; UI strings translated; Approvals link shows pending count badge.
- Soft rejection/removal states for users/listings with timestamps; owner listing removal; login redirects home; listing visibility hides removed/not-published.
- Profile page now allows editing name and password (email immutable).
- Docs: Added docs in `docs/` (tracked, not shipped) with HTML + PlantUML sequences + draw.io diagrams. Ignored from deploy via runtime paths; kept in git.
- Documentation pivoted to Mermaid-only diagrams (sequence + architecture/infra/pipeline), rendered in-browser; legacy draw.io/PlantUML assets removed for simplicity.
- Updated docs to fix Mermaid syntax and labels; Mermaid renders cleanly across all pages.
- Local Docker cleanup: removed all stale images (including registry.halla-aho.net:443 tags); only current `3a5de63` and `latest` remain.
- Listing details: right rail now surfaces quick facts + amenity icons; browse map given fixed height so OpenStreetMap tiles show reliably; footer links to privacy page with version indicator.
- Listing images now stored in DB (binary) with API serving `/api/images/:id`; upload limited to 6 images (5MB each) and seed pulls from `sampleimages/` if present.
- Sample listings flagged via `isSample`, seeded demo listings marked, and UI badges added to identify them.
- Privacy page localized (FI/EN) via i18n.
- Version hash now injected via build arg (`NEXT_PUBLIC_VERSION`) and shown in footer; build scripts updated.
- In-cluster Varnish cache added in Deployment to cache `/api/images/*` and static assets.
- Added `generate_images.py` and committed sample image assets for reseeding/rebuilds.
- Price hint now stored in euros (schema field `priceHintPerNightEuros`); Prisma migration added to convert from cents, seeds and API/UI updated, and build now runs `prisma generate` automatically.
- Listing creation amenities UI improved with toggle cards and EV button group.
- Edit listing form now matches the create form styling, including amenity icon grid and price helpers.
- Centralized logging stack scaffolded (Loki + Promtail + Grafana) with Helm values and install script; Grafana ingress defaults to `logs.lomavuokraus.fi`.
- Logging: Loki+Promtail+Grafana deployed to `logging` namespace; DNS updated for `logs.lomavuokraus.fi`; Grafana admin password reset due to PVC-stored credentials overriding the secret.
- Mermaid docs fixed: all sequence diagrams declare their participants and avoid “->” inside message text; the listing creation diagram message was rewritten to prevent parse errors. Use mermaid.live or browser console to debug future syntax issues (errors flag the offending line/column).
## 2025-11-27 — Availability & filters
- Availability calendars: listings can store iCal URLs, merged into a combined availability calendar on detail pages; availability filtering added to search along with amenity filters; new migration `20251127_calendar_urls`.
- Browse amenity filters now show the same icons as listing detail; image `registry.halla-aho.net/thalla/lomavuokraus-web:e95d9e0` built/pushed and rolled out to staging.
- Home hero cleaned up (removed sample/browse CTAs), hero FI text updated, and health check link moved to About page runtime section.
- Listing creation form now supports editing all locales at once with language tabs, per-locale readiness badges, and an AI JSON helper to translate and apply copy across languages; API accepts multiple translations in one request.
- Added Swedish locale support across the app, language selector is now a flag dropdown (FI/SV/EN), and the new listing form/AI helper handle all three languages.
- Site navbar now shows the new logo above the lomavuokraus.fi brand text on every page.
- Language selector in the navbar aligned with other buttons and given higher-contrast styling.
- Listing edit page now lets owners delete individual images (with cover/order preserved), and a protected API endpoint handles image removal.
- Security hardening: npm audit now passes cleanly after upgrading Prisma patch release and pinning `glob@10.5.0` via overrides to eliminate the glob CLI injection advisory in eslint tooling.
## 2025-12-06 — Pricing & amenities
- Listings now capture separate weekday/weekend prices and new amenities (microwave, free parking) across schema, API, UI, and seeds.
- Deployed pricing/amenity update image `registry.halla-aho.net/thalla/lomavuokraus-web:bee691e` to staging and production.
## 2025-12-17 — Accessibility & admin UX
- New amenities added: kitchen, dishwasher, washing machine, barbecue; API/UI/i18n updated and seeds randomized to populate missing prices/amenities. Prisma migration `20250210_more_amenities` applied to shared DB; registry pull secret added to k8s Deployment to avoid image pull errors in prod.
- Added About and Pricing pages (FI/EN), moved highlights/runtime config to About, and linked footer navigation.
- Added site favicon generated from the updated logo (`public/favicon.ico`).
- New admin monitoring dashboard at `/admin/monitor` surfaces Hetzner node status, Kubernetes nodes/pods health, and PostgreSQL connection/size checks with auto-refresh.
- Netdata installed on k3s node (`node1.lomavuokraus.fi:8443`) and DB host (`db1.lomavuokraus.fi:8443`) behind self-signed TLS + basic auth; DB Netdata includes Postgres metrics via dedicated `netdata` role.
- Footer now includes a minimal cookie usage statement (essential cookies only; site requires acceptance).
- Forgejo deployment scaffolding added: Docker Compose + runner config guidance and Apache vhost for git.halla-aho.net, plus CI workflow placeholder under `.forgejo/workflows/`.
- Amenities: added separate EV charging flags (on-site vs nearby) plus wheelchair accessibility, including browse filters and admin approvals view badges.
- Navbar: combined admin actions (approvals/users/monitoring) under a single “Admin” dropdown menu.
- Pricing copy: treat listing prices as indicative “starting from” values and show starting-from line on browse cards + home latest carousel.
- Site settings page added with a toggle to require login for listing contact details; contact info is now hidden from logged-out visitors.
## 2025-12-20 — Migration history repair
- Restored missing migration `20251212_agent_billing` (agent billing columns + listing billing settings table) so Prisma history matches the DB.
- Reconciled test DB migration history: aligned checksum for `20251212_agent_billing` and marked `20260310_site_settings` and `20260311_billing_preferences` as applied to stop Prisma errors and surface listings again.
- Applied the same agent billing schema to staging/prod DB (`lomavuokraus`) and marked the migration as applied; Prisma status now clean there too.
- Deploy script now runs a Prisma migration status preflight using DATABASE_URL from env or in-cluster secret and fails fast on drift before applying manifests.