Getting Started¶
Local dev procedures. Each section is self-contained — skim the headings and jump.
Prerequisites¶
| Tool | Version | Purpose |
|---|---|---|
| Go | 1.25+ | Compiler and runtime |
| Task | latest | Task runner (brew install go-task/tap/go-task) |
| Docker | latest | Postgres + Redis + MinIO + Photon (local infra) |
| Air | latest | Hot reload (installed by task setup) |
| swag | latest | Swagger doc generator (installed by task setup) |
| wire | latest | DI codegen (installed by task setup) |
| sqlc / goose | latest | Query codegen + migrations (installed by task setup) |
| sqlfluff | latest | SQL lint/format (installed by task setup) |
One-time machine setup¶
task setup # deps, dev toolchain, git hooks, docs toolchain
task setup needs a working Docker daemon and a Python that can create virtual
environments. On macOS, Docker Desktop covers the first and the system Python
covers the second. On Linux both need a one-time step, and they need the
machine owner because they touch system services and group membership:
# 1. Start the Docker daemon and have it come back after a reboot.
sudo systemctl enable --now docker.socket
# 2. Let your user talk to the daemon without sudo.
sudo usermod -aG docker "$USER"
# 3. Pick up the new group. Log out and back in, or start a fresh login shell.
newgrp docker
# 4. Confirm, then run setup.
docker ps
task setup
Debian and Ubuntu also need sudo apt install python3-venv, without which
task setup:python-tools cannot build the toolchain venv. Fedora and Arch
ship it with the base Python package.
The Python dev tools (sqlfluff for SQL lint, mkdocs for the docs site)
install into a repo-local .venv-tools/, which is gitignored. Distro Pythons
mark the system environment externally managed (PEP 668), so pip install
--user fails on most current Linux distributions; a venv at a fixed path works
everywhere and lets the tasks call the tools by absolute path without anyone
having to activate anything. Rebuild it any time with task setup:python-tools.
Start local dev¶
# Terminal 1 — backend + infra
task dev
# Terminal 2 — frontend (Metro + web; press i/a to attach an installed native dev binary)
task dev:frontend
# One-time per Simulator/emulator (and after native dep changes)
task dev:ios # builds + installs the iOS dev binary on the Simulator
task dev:android # builds + installs the Android dev binary on the emulator
task dev brings up Postgres, Redis, MinIO, and Photon via Docker Compose, runs migrations, regenerates Swagger docs, pulls GCP secrets (if available), and starts the backend on :8080 with Air for hot reload. The frontend runs on :8081 via the Expo dev server. Native JS edits hot-reload via Fast Refresh; only native dep or app.json plugin changes need another task dev:ios / task dev:android rebuild. See Docker Compose for the underlying stack.
Swagger / OpenAPI¶
Once task dev is running, the interactive API explorer is at:
http://localhost:8080/swagger/index.html
After changing handler annotations, regenerate the spec with task gen:docs. See API → Swagger.
Seed test data¶
task db:test:setup populates the database with realistic data:
task db:test:setup
What you get:
- 50 culturally diverse users with avatars uploaded to MinIO
- 15 accepted friends for
testuser1@tomoda.life(password:password) - A 600–700 message DM thread between
testuser1andtestuser2for infinite-scroll testing - 200 events across 55 categories, scattered across 15 global cities
- Multilingual chat history (English / 日本語 / 한국어 / 中文 / emoji)
To seed events around your own location, export SEED_LAT and SEED_LNG first:
SEED_LAT=35.6895 SEED_LNG=139.6917 task db:test:setup
Run simulation workers¶
task test:simulation wipes + migrates the DB, re-seeds test data, loads the location catalog from GCS (needs gcloud login), then starts sim workers that mimic live user activity. Use it to test discovery, chat, and presence under load:
task test:simulation
Per simulated user, the workers:
- Walk a random path within ~30 km of
testuser1's live position, every 2–6 minutes - Heartbeat presence every 60 seconds; go offline 5–20 min at 8% chance
- Send DMs to
testuser1every 8–30 minutes - Maintain 3–6 upcoming events near
testuser1's location
Infrastructure must already be running — task test:simulation does not start Docker.
Database migrations¶
In development (ENV != production), the server calls db.Migrate() on every boot, applying any pending goose migrations. To run migrations manually:
task db:migrate # apply all + runtime post-steps
task db:migrate:down # roll back the last migration
task db:migrate:create -- add_new_column # scaffold a new migration
In production, Migrate is skipped. Schema changes must ship through explicit SQL migrations.
Reset local DB¶
When migrations are in a weird state or you want to start over:
task db:reset # docker:clean -> docker:up -> migrate
task db:test:setup # re-seed ~20+ users, events, chats
db:reset runs task docker:clean (which is docker-compose down -v --remove-orphans) so it wipes the named volumes too — your local Postgres data is gone. It then waits on task docker:wait before migrating, because a cold-booted Postgres accepts TCP connections while initdb is still running and a migration started in that window dies with connection reset by peer.
Point devices at this machine¶
task dev and every task dev:frontend* / dev:ios* / dev:android task
resolve the LAN IPv4 of this machine and pin it into
backend/config.local.yaml (the S3 presign host) and frontend/.env.local
(EXPO_PUBLIC_API_URL / EXPO_PUBLIC_WS_URL). Without that, a phone follows a
presigned URL to localhost and hangs against its own loopback.
Detection lives in scripts/dev-host-ip.sh, which skips loopback, VPN
(wg, tun, tailscale), and container/bridge (docker, br-, veth)
interfaces — those route somewhere a phone cannot follow. Override it when the
guess is wrong or when the device tunnels back through the host:
TOMODA_DEV_HOST=localhost task dev:frontend # adb reverse workflow
TOMODA_DEV_HOST=192.168.1.50 task dev # pin explicitly
Quality gates that need Python tools¶
task db:sql:lint and task docs:build fail with an install hint when
sqlfluff / mkdocs are missing, rather than passing quietly. A check that
skips itself is indistinguishable from one that succeeded, which is how SQL
lint violations reach main unnoticed. Run task setup:python-tools to
install them. If a machine genuinely cannot, opt out explicitly:
SKIP_SQL_LINT=1 git commit ... # pre-commit runs task db:sql:lint
SKIP_DOCS_BUILD=1 task docs:build
Pull GCP secrets¶
If your task dev is starting without real secrets (you see "GCP credentials not available — running with local defaults only" in the output), auth with gcloud:
gcloud auth login
gcloud config set project development-485000
task dev # re-run — secrets will now be exported into the shell
The full mechanism is documented in Secrets.
Tail backend logs¶
Locally¶
The backend logs to stdout via Zap. In structured-logging mode this is JSON; in ENV=local it's the colored console encoder. Just look at the terminal where task dev is running.
To pipe through jq for readability:
task dev 2>&1 | jq -R 'fromjson? // .'
Production¶
Backend logs go to Cloud Logging (each Pod's stdout is collected by the GKE logging agent). Use the Logs Explorer in the GCP Console with a query like:
resource.type="k8s_container"
resource.labels.namespace_name="<dev-or-prod>"
resource.labels.container_name="backend"
For a faster CLI workflow:
gcloud logging read \
'resource.type="k8s_container" AND resource.labels.container_name="backend"' \
--limit 100 --format json --project development-485000
Deploy a backend hotfix¶
For a small, urgent fix on main:
git checkout main && git pull
# make the fix, commit, push
.github/scripts/release.sh # bump VERSION, tag, push tag, optional GH release
The tag push triggers Cloud Build → image lands in Artifact Registry → ArgoCD picks it up → rolling update on GKE. See Deployment for the full chain.
If you need to ship a fix without bumping a tag (rare and discouraged), bump only the image reference in the devops/ repo and let ArgoCD reconcile.
Rollback¶
Two paths:
- ArgoCD UI — open the
tomodaApplication, click History and Rollback, select the previous sync, and confirm. ArgoCD re-applies the previous manifest set, including the previous image SHA. Fastest option. - Git revert in the devops repo —
git revertthe commit that bumped the image (indevops/k8s/apps/tomoda/) and push. ArgoCD reconciles on its next sync.
Avoid kubectl rollout undo unless ArgoCD itself is broken — it'll drift from the Git source of truth.
Native app rollback¶
The native (iOS + Android) app is a separate beast because rollback means different things depending on the release vehicle.
| Type of release | Rollback options |
|---|---|
| OTA update (self-hosted Xavia, JS-only) | Activate a prior release for the prod server in the Xavia dashboard (ota.tomoda.life), which re-points runtimeVersion at the previous bundle. |
| Native release (new binary submitted to App Store / Play Store) | You can't unship a binary that's been approved. Submit a hotfix build with an incremented version. For Android you can halt the staged rollout in Play Console; for iOS you can pull from sale (App Store Connect → Pricing) but that's a sledgehammer. |
See Native → Release for the full release flow.
Add a new env var¶
A new backend env var is a multi-step change:
- Add the field to the config struct (
backend/internal/config/config.goor equivalent — see../backend/infrastructure/config.md). - Add the default / per-env value in
backend/config.local.yamlandbackend/config.{dev,prod}.yaml. - Update
backend/env.example.localso other developers see it. - Add to
scripts/pull-secrets.shif the value is sensitive (sotask devpicks it up). - Create the secret in GCP Secret Manager (
gcloud secrets create ...) for non-local environments. - Add a key to the K8s
backend-secretsSecret in thedevops/repo so it gets injected into the Pod.
Skipping step 6 will leave the Pod running without the value — usually a silent failure in the affected code path.
Add a new scheduled task¶
Scheduled jobs are not K8s CronJobs — they're handled by the in-process scheduler. See ../backend/infrastructure/async.md for how to register a new periodic task.
Investigate a slow query¶
task db:console # opens psql against the local Postgres
Then:
EXPLAIN (ANALYZE, BUFFERS) SELECT ...;
For production, use Cloud SQL Insights in the GCP Console — it tracks the top queries by total time and exposes execution plans without needing a direct DB shell.
Common culprits in this codebase:
- Missing index on a column added via a recent migration. Check
backend/migrations/and compare againstpg_indexes. - A spatial query missing the GiST geography index (
idx_*_coordinates_geog). Discovery and presence queries lean on these. - Sequential scan on a JSONB column. Add a GIN index or pull the field out into a column.
Investigate WebSocket flapping¶
Symptoms: users report repeated reconnects, real-time updates stop arriving, presence shows users as offline that are actually online.
The Hub is in-memory per pod; cross-pod fanout flows through Redis pub/sub on
chat:event:*. Pod restarts drop the connections owned by that pod (clients
reconnect, but it's user-visible); a Redis outage breaks cross-pod fanout
specifically — same-pod chat still works.
Triage steps:
- Check Pod restart count.
kubectl -n <ns> get pods -l app.kubernetes.io/name=tomoda-api— ifRESTARTSis climbing, the Pod is OOMing or panicking. Look at logs (kubectl logs ... --previous). - Check liveness probe. A slow
/health(>30s withperiodSeconds: 30) will get the Pod killed. The/healthroute is registered inbackend/internal/wiring/router.go. - Check ping/pong intervals. The Hub sends pings; the client must pong within the deadline. Mismatched timeouts between the proxy in front (Cloudflare / Ingress) and the Hub will look like flapping. See
../backend/infrastructure/websocket.md. - Check Ingress idle timeout. GKE Ingress defaults can drop idle WS connections — verify in
devops/k8s/apps/tomoda/base/ingress.yaml. - Check cross-pod fanout. If users on one pod see messages but users on a sibling pod don't, the issue is the Hub subscriber. Logs include
Hub subscriber loop error; reconnectingon Redis pub/sub failures;MONITORRedis briefly to confirmchat:event:*traffic if needed.
If the Pod is healthy and connections are still flapping, the issue is client-side or network-side, not on the server.