Skip to content

Backend Deployment

  1. Managing Secrets
  2. Google Cloud Setup
  3. Pre-Deploy Checks
  4. Check Current System State
  5. Running a Deploy
  6. Monitoring & Rollback

Managing Secrets

You must have proper IAM roles: Artifact Registry Admin, Cloud SQL Admin, Kubernetes Admin, Storage Admin, etc.

Ensure you have all required secrets configured. See Secrets Management for details on adding new secrets and keys to the secrets flake.

  1. Ensure you're on the latest main branch.
  2. Confirm that RB_SECRET_CI_KUBECONFIG ran successfully.

Google Cloud Setup

  1. Run just setup → will launch browser auth; use the correct Google account.
  2. Run just gke get pods to check GKE.
  3. Run just k3s get pods for K3s.
  4. Run just rbtg plan to see Terraform diff (Terragrunt wrapper). Can feed diff to LLM for clarity.

Terraform Dependencies

If Terraform dependencies have changed, you may need to run terraform init from the terraform directory:

cd terraform
terraform init

Pre-Deploy Checks

Think through schema/db state and compatibility before deploying.

  1. Use just prod <command> for production context (e.g. just prod alembic current).
  2. Get current DB revision (alembic current) and compare with alembic history.
  3. Review image history in:
  4. Google Cloud > Artifact Registry
  5. Gitea commit actions on master
  6. railbird-full image includes all dependencies.
  7. Run just current-api-image to see current deployed hash.
  8. Re-auth with just reauth as needed.

Database Migration

If DB is not at HEAD:

Run just upgrade-prod-db.

Check Current System State

  • Pod status: just gke get pods
  • Pod details: just gke describe pods, just describe <pod-name>
  • Node info: just gke get nodes
  • just scripts are in .just files (e.g., build.just)

Running a Deploy

Ignore changes in terraform lock.

  1. Run: just rbtg-apply-bump-only-api-om
  2. Deploys all services using main railbird image (not web/frontend).

Before Deploying

Check image status:

  1. just ar-latest-tag
  2. just ar-image-exists <tag> <image> (e.g., railbird-full)
  3. Follow the prompt and enter yes to proceed.

Monitoring & Rollback

Monitoring

Monitor with: watch -n 5 'just gke get pods'

Rollback

To roll back:

  1. Use just upgrade-all-images <tag> where tag = Git hash from just current-api-image
  2. Always track the currently deployed tag for safety.