356 lines
14 KiB
Markdown
356 lines
14 KiB
Markdown
|
# Cloud Build Environment Configuration
|
||
|
|
|
||
|
|
This guide explains how to set up separate staging and production triggers with environment-specific configurations.
|
||
|
|
|
||
|
|
## Overview
|
||
|
|
|
||
|
|
The `cloudbuild.yaml` uses substitution variables to make it environment-agnostic. You'll create **two separate triggers** (one for staging, one for production) with different substitution variable values.
|
||
|
|
|
||
|
|
**Note**: The Dockerfile uses a single final stage. Environment differentiation (staging vs production) is handled entirely through Cloud Run environment variables, not at build time. This simplifies the build process - the same Docker image can be used for both environments.
|
||
|
|
|
||
|
|
## Architecture
|
||
|
|
|
||
|
|
```
|
||
|
|
┌─────────────────┐ ┌──────────────────────┐
|
||
|
|
│ Git Branch │ │ Cloud Build Trigger │
|
||
|
|
│ - main │─────▶│ (Production) │
|
||
|
|
│ │ │ Variables: prod │
|
||
|
|
└─────────────────┘ └──────────────────────┘
|
||
|
|
|
||
|
|
┌─────────────────┐ ┌──────────────────────┐
|
||
|
|
│ Git Branch │ │ Cloud Build Trigger │
|
||
|
|
│ - develop │─────▶│ (Staging) │
|
||
|
|
│ │ │ Variables: staging │
|
||
|
|
└─────────────────┘ └──────────────────────┘
|
||
|
|
```
|
||
|
|
|
||
|
|
## Required Substitution Variables
|
||
|
|
|
||
|
|
### Common Variables (Different per Environment)
|
||
|
|
|
||
|
|
| Variable | Production Value | Staging Value |
|
||
|
|
| ----------------------- | ------------------------ | ---------------------------- |
|
||
|
|
| `$_NODE_ENV` | `production` | `staging` |
|
||
|
|
| `$_SERVICE_NAME` | `xtablo-api` | `xtablo-api-staging` |
|
||
|
|
| `$_SUPABASE_URL` | Production Supabase URL | Staging Supabase URL |
|
||
|
|
| `$_STREAM_CHAT_API_KEY` | Production key | Staging key |
|
||
|
|
| `$_EMAIL_USER` | `noreply@xtablo.com` | `staging@xtablo.com` |
|
||
|
|
| `$_EMAIL_CLIENT_ID` | Production OAuth client | Staging OAuth client |
|
||
|
|
| `$_R2_ACCOUNT_ID` | Production R2 account | Staging R2 account |
|
||
|
|
| `$_CORS_ORIGIN` | `https://app.xtablo.com` | `https://staging.xtablo.com` |
|
||
|
|
| `$_XTABLO_URL` | `https://app.xtablo.com` | `https://staging.xtablo.com` |
|
||
|
|
| `$_LOG_LEVEL` | `info` | `debug` |
|
||
|
|
|
||
|
|
### Build Configuration (Same for Both)
|
||
|
|
|
||
|
|
| Variable | Value |
|
||
|
|
| ----------------- | ---------------------------------- |
|
||
|
|
| `$_AR_PROJECT_ID` | Your GCP project ID |
|
||
|
|
| `$_AR_REPOSITORY` | `xtablo` (or your repository name) |
|
||
|
|
|
||
|
|
### Secret Manager References (Different per Environment)
|
||
|
|
|
||
|
|
Each environment should have its own secrets in Secret Manager:
|
||
|
|
|
||
|
|
| Variable | Production Secret Name | Staging Secret Name |
|
||
|
|
| ------------------------------------- | --------------------------------- | ------------------------------------ |
|
||
|
|
| `$_SUPABASE_SERVICE_ROLE_KEY_SECRET` | `supabase-service-role-key-prod` | `supabase-service-role-key-staging` |
|
||
|
|
| `$_SUPABASE_CONNECTION_STRING_SECRET` | `supabase-connection-string-prod` | `supabase-connection-string-staging` |
|
||
|
|
| `$_SUPABASE_CA_CERT_SECRET` | `supabase-ca-cert-prod` | `supabase-ca-cert-staging` |
|
||
|
|
| `$_STREAM_CHAT_API_SECRET_SECRET` | `stream-chat-api-secret-prod` | `stream-chat-api-secret-staging` |
|
||
|
|
| `$_STRIPE_SECRET_KEY_SECRET` | `stripe-secret-key-prod` | `stripe-secret-key-staging` |
|
||
|
|
| `$_STRIPE_WEBHOOK_SECRET_SECRET` | `stripe-webhook-secret-prod` | `stripe-webhook-secret-staging` |
|
||
|
|
| `$_EMAIL_CLIENT_SECRET_SECRET` | `email-client-secret-prod` | `email-client-secret-staging` |
|
||
|
|
| `$_EMAIL_REFRESH_TOKEN_SECRET` | `email-refresh-token-prod` | `email-refresh-token-staging` |
|
||
|
|
| `$_R2_ACCESS_KEY_ID_SECRET` | `r2-access-key-id-prod` | `r2-access-key-id-staging` |
|
||
|
|
| `$_R2_SECRET_ACCESS_KEY_SECRET` | `r2-secret-access-key-prod` | `r2-secret-access-key-staging` |
|
||
|
|
|
||
|
|
## Setup Instructions
|
||
|
|
|
||
|
|
### Step 1: Create Secrets in Secret Manager
|
||
|
|
|
||
|
|
#### Production Secrets
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Supabase (Production)
|
||
|
|
echo -n "prod-service-role-key" | gcloud secrets create supabase-service-role-key-prod --data-file=-
|
||
|
|
echo -n "prod-connection-string" | gcloud secrets create supabase-connection-string-prod --data-file=-
|
||
|
|
echo -n "prod-ca-cert" | gcloud secrets create supabase-ca-cert-prod --data-file=-
|
||
|
|
|
||
|
|
# Stream Chat (Production)
|
||
|
|
echo -n "prod-stream-secret" | gcloud secrets create stream-chat-api-secret-prod --data-file=-
|
||
|
|
|
||
|
|
# Stripe (Production)
|
||
|
|
echo -n "prod-stripe-key" | gcloud secrets create stripe-secret-key-prod --data-file=-
|
||
|
|
echo -n "prod-webhook-secret" | gcloud secrets create stripe-webhook-secret-prod --data-file=-
|
||
|
|
|
||
|
|
# Email (Production)
|
||
|
|
echo -n "prod-client-secret" | gcloud secrets create email-client-secret-prod --data-file=-
|
||
|
|
echo -n "prod-refresh-token" | gcloud secrets create email-refresh-token-prod --data-file=-
|
||
|
|
|
||
|
|
# R2 (Production)
|
||
|
|
echo -n "prod-access-key-id" | gcloud secrets create r2-access-key-id-prod --data-file=-
|
||
|
|
echo -n "prod-secret-access-key" | gcloud secrets create r2-secret-access-key-prod --data-file=-
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Staging Secrets
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Supabase (Staging)
|
||
|
|
echo -n "staging-service-role-key" | gcloud secrets create supabase-service-role-key-staging --data-file=-
|
||
|
|
echo -n "staging-connection-string" | gcloud secrets create supabase-connection-string-staging --data-file=-
|
||
|
|
echo -n "staging-ca-cert" | gcloud secrets create supabase-ca-cert-staging --data-file=-
|
||
|
|
|
||
|
|
# Stream Chat (Staging)
|
||
|
|
echo -n "staging-stream-secret" | gcloud secrets create stream-chat-api-secret-staging --data-file=-
|
||
|
|
|
||
|
|
# Stripe (Staging - use test keys)
|
||
|
|
echo -n "staging-stripe-key" | gcloud secrets create stripe-secret-key-staging --data-file=-
|
||
|
|
echo -n "staging-webhook-secret" | gcloud secrets create stripe-webhook-secret-staging --data-file=-
|
||
|
|
|
||
|
|
# Email (Staging)
|
||
|
|
echo -n "staging-client-secret" | gcloud secrets create email-client-secret-staging --data-file=-
|
||
|
|
echo -n "staging-refresh-token" | gcloud secrets create email-refresh-token-staging --data-file=-
|
||
|
|
|
||
|
|
# R2 (Staging)
|
||
|
|
echo -n "staging-access-key-id" | gcloud secrets create r2-access-key-id-staging --data-file=-
|
||
|
|
echo -n "staging-secret-access-key" | gcloud secrets create r2-secret-access-key-staging --data-file=-
|
||
|
|
```
|
||
|
|
|
||
|
|
### Step 2: Grant Access to Secrets
|
||
|
|
|
||
|
|
```bash
|
||
|
|
PROJECT_ID="your-project-id"
|
||
|
|
SERVICE_ACCOUNT="${PROJECT_ID}@appspot.gserviceaccount.com"
|
||
|
|
|
||
|
|
# Grant access to all production secrets
|
||
|
|
for secret in \
|
||
|
|
supabase-service-role-key-prod \
|
||
|
|
supabase-connection-string-prod \
|
||
|
|
supabase-ca-cert-prod \
|
||
|
|
stream-chat-api-secret-prod \
|
||
|
|
stripe-secret-key-prod \
|
||
|
|
stripe-webhook-secret-prod \
|
||
|
|
email-client-secret-prod \
|
||
|
|
email-refresh-token-prod \
|
||
|
|
r2-access-key-id-prod \
|
||
|
|
r2-secret-access-key-prod
|
||
|
|
do
|
||
|
|
gcloud secrets add-iam-policy-binding $secret \
|
||
|
|
--member="serviceAccount:${SERVICE_ACCOUNT}" \
|
||
|
|
--role="roles/secretmanager.secretAccessor"
|
||
|
|
done
|
||
|
|
|
||
|
|
# Grant access to all staging secrets
|
||
|
|
for secret in \
|
||
|
|
supabase-service-role-key-staging \
|
||
|
|
supabase-connection-string-staging \
|
||
|
|
supabase-ca-cert-staging \
|
||
|
|
stream-chat-api-secret-staging \
|
||
|
|
stripe-secret-key-staging \
|
||
|
|
stripe-webhook-secret-staging \
|
||
|
|
email-client-secret-staging \
|
||
|
|
email-refresh-token-staging \
|
||
|
|
r2-access-key-id-staging \
|
||
|
|
r2-secret-access-key-staging
|
||
|
|
do
|
||
|
|
gcloud secrets add-iam-policy-binding $secret \
|
||
|
|
--member="serviceAccount:${SERVICE_ACCOUNT}" \
|
||
|
|
--role="roles/secretmanager.secretAccessor"
|
||
|
|
done
|
||
|
|
```
|
||
|
|
|
||
|
|
### Step 3: Create Cloud Build Triggers
|
||
|
|
|
||
|
|
#### Production Trigger
|
||
|
|
|
||
|
|
Via gcloud CLI:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
gcloud builds triggers create github \
|
||
|
|
--name="xtablo-api-production" \
|
||
|
|
--repo-name="xtablo-source" \
|
||
|
|
--repo-owner="your-github-org" \
|
||
|
|
--branch-pattern="^main$" \
|
||
|
|
--build-config="apps/api/cloudbuild.yaml" \
|
||
|
|
--substitutions='
|
||
|
|
_NODE_ENV=production,
|
||
|
|
_SERVICE_NAME=xtablo-api,
|
||
|
|
_AR_PROJECT_ID=your-project-id,
|
||
|
|
_AR_REPOSITORY=xtablo,
|
||
|
|
_SUPABASE_URL=https://your-prod-project.supabase.co,
|
||
|
|
_STREAM_CHAT_API_KEY=your-prod-stream-key,
|
||
|
|
_EMAIL_USER=noreply@xtablo.com,
|
||
|
|
_EMAIL_CLIENT_ID=your-prod-oauth-client-id,
|
||
|
|
_R2_ACCOUNT_ID=your-prod-r2-account,
|
||
|
|
_CORS_ORIGIN=https://app.xtablo.com,
|
||
|
|
_XTABLO_URL=https://app.xtablo.com,
|
||
|
|
_LOG_LEVEL=info,
|
||
|
|
_SUPABASE_SERVICE_ROLE_KEY_SECRET=supabase-service-role-key-prod,
|
||
|
|
_SUPABASE_CONNECTION_STRING_SECRET=supabase-connection-string-prod,
|
||
|
|
_SUPABASE_CA_CERT_SECRET=supabase-ca-cert-prod,
|
||
|
|
_STREAM_CHAT_API_SECRET_SECRET=stream-chat-api-secret-prod,
|
||
|
|
_STRIPE_SECRET_KEY_SECRET=stripe-secret-key-prod,
|
||
|
|
_STRIPE_WEBHOOK_SECRET_SECRET=stripe-webhook-secret-prod,
|
||
|
|
_EMAIL_CLIENT_SECRET_SECRET=email-client-secret-prod,
|
||
|
|
_EMAIL_REFRESH_TOKEN_SECRET=email-refresh-token-prod,
|
||
|
|
_R2_ACCESS_KEY_ID_SECRET=r2-access-key-id-prod,
|
||
|
|
_R2_SECRET_ACCESS_KEY_SECRET=r2-secret-access-key-prod
|
||
|
|
'
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Staging Trigger
|
||
|
|
|
||
|
|
Via gcloud CLI:
|
||
|
|
|
||
|
|
```bash
|
||
|
|
gcloud builds triggers create github \
|
||
|
|
--name="xtablo-api-staging" \
|
||
|
|
--repo-name="xtablo-source" \
|
||
|
|
--repo-owner="your-github-org" \
|
||
|
|
--branch-pattern="^develop$" \
|
||
|
|
--build-config="apps/api/cloudbuild.yaml" \
|
||
|
|
--substitutions='
|
||
|
|
_NODE_ENV=staging,
|
||
|
|
_SERVICE_NAME=xtablo-api-staging,
|
||
|
|
_AR_PROJECT_ID=your-project-id,
|
||
|
|
_AR_REPOSITORY=xtablo,
|
||
|
|
_SUPABASE_URL=https://your-staging-project.supabase.co,
|
||
|
|
_STREAM_CHAT_API_KEY=your-staging-stream-key,
|
||
|
|
_EMAIL_USER=staging@xtablo.com,
|
||
|
|
_EMAIL_CLIENT_ID=your-staging-oauth-client-id,
|
||
|
|
_R2_ACCOUNT_ID=your-staging-r2-account,
|
||
|
|
_CORS_ORIGIN=https://staging.xtablo.com,
|
||
|
|
_XTABLO_URL=https://staging.xtablo.com,
|
||
|
|
_LOG_LEVEL=debug,
|
||
|
|
_SUPABASE_SERVICE_ROLE_KEY_SECRET=supabase-service-role-key-staging,
|
||
|
|
_SUPABASE_CONNECTION_STRING_SECRET=supabase-connection-string-staging,
|
||
|
|
_SUPABASE_CA_CERT_SECRET=supabase-ca-cert-staging,
|
||
|
|
_STREAM_CHAT_API_SECRET_SECRET=stream-chat-api-secret-staging,
|
||
|
|
_STRIPE_SECRET_KEY_SECRET=stripe-secret-key-staging,
|
||
|
|
_STRIPE_WEBHOOK_SECRET_SECRET=stripe-webhook-secret-staging,
|
||
|
|
_EMAIL_CLIENT_SECRET_SECRET=email-client-secret-staging,
|
||
|
|
_EMAIL_REFRESH_TOKEN_SECRET=email-refresh-token-staging,
|
||
|
|
_R2_ACCESS_KEY_ID_SECRET=r2-access-key-id-staging,
|
||
|
|
_R2_SECRET_ACCESS_KEY_SECRET=r2-secret-access-key-staging
|
||
|
|
'
|
||
|
|
```
|
||
|
|
|
||
|
|
### Step 4: Configure via Console (Alternative)
|
||
|
|
|
||
|
|
If you prefer using the Google Cloud Console:
|
||
|
|
|
||
|
|
1. Go to **Cloud Build > Triggers**
|
||
|
|
2. Click **Create Trigger**
|
||
|
|
3. Configure:
|
||
|
|
|
||
|
|
- **Name**: `xtablo-api-production` (or `xtablo-api-staging`)
|
||
|
|
- **Event**: Push to a branch
|
||
|
|
- **Repository**: Your GitHub repository
|
||
|
|
- **Branch**: `^main$` (or `^develop$` for staging)
|
||
|
|
- **Build configuration**: Cloud Build configuration file
|
||
|
|
- **Location**: `apps/api/cloudbuild.yaml`
|
||
|
|
|
||
|
|
4. Scroll to **Substitution variables** and add all variables from the table above
|
||
|
|
|
||
|
|
5. Save the trigger
|
||
|
|
|
||
|
|
## Verification
|
||
|
|
|
||
|
|
### Test Production Deployment
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Trigger manually
|
||
|
|
gcloud builds triggers run xtablo-api-production --branch=main
|
||
|
|
|
||
|
|
# Check logs
|
||
|
|
gcloud builds list --limit=5
|
||
|
|
gcloud builds log BUILD_ID
|
||
|
|
|
||
|
|
# Verify deployment
|
||
|
|
gcloud run services describe xtablo-api --region=europe-west1
|
||
|
|
```
|
||
|
|
|
||
|
|
### Test Staging Deployment
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Trigger manually
|
||
|
|
gcloud builds triggers run xtablo-api-staging --branch=develop
|
||
|
|
|
||
|
|
# Check logs
|
||
|
|
gcloud builds list --limit=5
|
||
|
|
|
||
|
|
# Verify deployment
|
||
|
|
gcloud run services describe xtablo-api-staging --region=europe-west1
|
||
|
|
```
|
||
|
|
|
||
|
|
### Verify Environment Variables
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Production
|
||
|
|
gcloud run services describe xtablo-api --region=europe-west1 \
|
||
|
|
--format="value(spec.template.spec.containers[0].env)"
|
||
|
|
|
||
|
|
# Staging
|
||
|
|
gcloud run services describe xtablo-api-staging --region=europe-west1 \
|
||
|
|
--format="value(spec.template.spec.containers[0].env)"
|
||
|
|
```
|
||
|
|
|
||
|
|
## Best Practices
|
||
|
|
|
||
|
|
1. **Secret Naming**: Use `-prod` and `-staging` suffixes for secrets
|
||
|
|
2. **Service Names**: Use different names for each environment (`xtablo-api` vs `xtablo-api-staging`)
|
||
|
|
3. **Branches**: Deploy production from `main`, staging from `develop`
|
||
|
|
4. **Stripe Keys**: Use Stripe test keys in staging
|
||
|
|
5. **Supabase**: Use separate Supabase projects for staging and production
|
||
|
|
6. **Monitoring**: Set up different alert thresholds for staging vs production
|
||
|
|
7. **Resource Limits**: You may want lower limits for staging to save costs
|
||
|
|
|
||
|
|
## Quick Reference
|
||
|
|
|
||
|
|
### Environment Comparison
|
||
|
|
|
||
|
|
| Aspect | Production | Staging |
|
||
|
|
| ------------------ | ---------------- | -------------------- |
|
||
|
|
| **Branch** | `main` | `develop` |
|
||
|
|
| **Service Name** | `xtablo-api` | `xtablo-api-staging` |
|
||
|
|
| **Domain** | `app.xtablo.com` | `staging.xtablo.com` |
|
||
|
|
| **Log Level** | `info` | `debug` |
|
||
|
|
| **Stripe** | Live keys | Test keys |
|
||
|
|
| **Min Instances** | 1 (optional) | 0 |
|
||
|
|
| **Secrets Suffix** | `-prod` | `-staging` |
|
||
|
|
|
||
|
|
## Troubleshooting
|
||
|
|
|
||
|
|
### "Secret not found" error
|
||
|
|
|
||
|
|
- Ensure secret names in substitution variables match actual secret names
|
||
|
|
- Verify secrets exist: `gcloud secrets list | grep staging` or `grep prod`
|
||
|
|
|
||
|
|
### Wrong environment variables in deployment
|
||
|
|
|
||
|
|
- Check trigger substitution variables in Cloud Build console
|
||
|
|
- Verify the correct trigger is being used for the branch
|
||
|
|
|
||
|
|
### Services pointing to wrong databases
|
||
|
|
|
||
|
|
- Ensure `SUPABASE_URL` is different for staging/production
|
||
|
|
- Check that secret names include correct environment suffix
|
||
|
|
|
||
|
|
## Example: Adding a New Environment Variable
|
||
|
|
|
||
|
|
To add a new environment variable (e.g., `FEATURE_FLAG`):
|
||
|
|
|
||
|
|
1. Add to `cloudbuild.yaml` in the `--set-env-vars` line:
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
"NODE_ENV=$_NODE_ENV,...,FEATURE_FLAG=$_FEATURE_FLAG"
|
||
|
|
```
|
||
|
|
|
||
|
|
2. Add to **both triggers** with different values:
|
||
|
|
|
||
|
|
- Production: `_FEATURE_FLAG=false`
|
||
|
|
- Staging: `_FEATURE_FLAG=true`
|
||
|
|
|
||
|
|
3. Update app code to read `process.env.FEATURE_FLAG`
|