xtablo-source/docs/GOOGLE_SECRET_MANAGER_SETUP.md

Google Secret Manager Setup

This guide explains how to set up and use Google Secret Manager for storing application secrets in production and staging environments.

Overview

The application uses Google Secret Manager to securely store sensitive configuration values (API keys, database credentials, etc.) in production and staging environments, while using local .env files for development.

Environment-based Configuration

• Development/Test: Uses .env.development or .env.test files (via dotenv)
• Production/Staging: Uses Google Secret Manager

Prerequisites

1. Google Cloud Project: You need a GCP project with Secret Manager API enabled
2. Service Account: A service account with Secret Manager access
3. Authentication: Proper credentials configured for your deployment environment

Setup Steps

1. Enable Secret Manager API

# Enable the Secret Manager API in your GCP project
gcloud services enable secretmanager.googleapis.com --project=YOUR_PROJECT_ID

2. Create Secrets in Google Cloud

For each required secret, create it in Google Secret Manager:

# Example: Create SUPABASE_URL secret
echo -n "https://your-project.supabase.co" | \
  gcloud secrets create SUPABASE_URL \
  --data-file=- \
  --project=YOUR_PROJECT_ID

# Example: Create STRIPE_SECRET_KEY secret
echo -n "sk_live_xxxxx" | \
  gcloud secrets create STRIPE_SECRET_KEY \
  --data-file=- \
  --project=YOUR_PROJECT_ID

Required Secrets

Create the following secrets in Google Secret Manager:

Secret Name	Description	Example
SUPABASE_URL	Supabase project URL	https://xxx.supabase.co
SUPABASE_SERVICE_ROLE_KEY	Supabase service role key	eyJxxx...
SUPABASE_CONNECTION_STRING	PostgreSQL connection string	postgresql://...
SUPABASE_CA_CERT	Supabase CA certificate (optional)	-----BEGIN CERTIFICATE-----...
STREAM_CHAT_API_KEY	Stream Chat API key	xxx
STREAM_CHAT_API_SECRET	Stream Chat API secret	xxx
STRIPE_SECRET_KEY	Stripe secret key	sk_live_xxx
STRIPE_WEBHOOK_SECRET	Stripe webhook signing secret	whsec_xxx
EMAIL_USER	Gmail/email account	yourapp@gmail.com
EMAIL_CLIENT_ID	OAuth client ID for email	xxx.apps.googleusercontent.com
EMAIL_CLIENT_SECRET	OAuth client secret	xxx
EMAIL_REFRESH_TOKEN	OAuth refresh token	xxx
R2_ACCOUNT_ID	Cloudflare R2 account ID	xxx
R2_ACCESS_KEY_ID	Cloudflare R2 access key	xxx
R2_SECRET_ACCESS_KEY	Cloudflare R2 secret key	xxx
TASKS_SECRET	Secret for task authentication	xxx

3. Grant Service Account Access

Your service account needs permission to access secrets:

# Grant Secret Manager Secret Accessor role
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
  --member="serviceAccount:YOUR_SERVICE_ACCOUNT@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"

4. Set Environment Variables

The application needs these environment variables to work with Google Secret Manager:

# Required in production/staging
export GCP_PROJECT=your-project-id        # or GOOGLE_CLOUD_PROJECT
export NODE_ENV=production                # or staging

# Optional: Use non-default port
export PORT=8080

# Optional: Override CORS origins
export CORS_ORIGIN=https://app.xtablo.com

# Optional: Override XTablo URL
export XTABLO_URL=https://app.xtablo.com

Authentication

Local Development (Testing Secret Manager)

For local testing of Secret Manager integration:

# Authenticate with gcloud
gcloud auth application-default login

# Set project
export GCP_PROJECT=your-project-id
export NODE_ENV=staging

# Run your app
npm run dev

Google Cloud Run / Cloud Functions

Authentication is automatic when running on Google Cloud services. Just ensure:

1. The service account has the roles/secretmanager.secretAccessor role
2. The GCP_PROJECT environment variable is set (or use the project's default)

# Example Cloud Run configuration (cloudbuild.yaml)
steps:
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/api:$COMMIT_SHA', '.']
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push', 'gcr.io/$PROJECT_ID/api:$COMMIT_SHA']
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: gcloud
    args:
      - 'run'
      - 'deploy'
      - 'xtablo-api'
      - '--image=gcr.io/$PROJECT_ID/api:$COMMIT_SHA'
      - '--region=us-central1'
      - '--platform=managed'
      - '--set-env-vars=NODE_ENV=production,GCP_PROJECT=$PROJECT_ID'
      - '--service-account=YOUR_SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com'

Usage in Code

Accessing Configuration

The configuration is loaded asynchronously at startup:

// In index.ts (server startup)
import { getConfig } from './config.js';

async function startServer() {
  const config = await getConfig();
  // Use config.STRIPE_SECRET_KEY, config.SUPABASE_URL, etc.
}

Backwards Compatibility

For synchronous access (after initialization):

import { config } from './config.js';

// This works ONLY after getConfig() has been called
// (automatically done in index.ts startup)
const stripeKey = config.STRIPE_SECRET_KEY;

Manual Secret Loading

If you need to load additional secrets not in the main config:

import { loadSecret } from './secretManager.js';

// Load a single secret
const apiKey = await loadSecret('MY_CUSTOM_SECRET');

// Load with fallback to environment variable
import { loadSecretWithFallback } from './secretManager.js';
const token = await loadSecretWithFallback('MY_TOKEN', 'MY_TOKEN_ENV_VAR');

Secret Versioning

Google Secret Manager supports versioning:

# Add a new version of a secret
echo -n "new-secret-value" | \
  gcloud secrets versions add SUPABASE_URL \
  --data-file=- \
  --project=YOUR_PROJECT_ID

# The app automatically uses the "latest" version
# You can also specify a specific version in code if needed

Security Best Practices

1. Never commit secrets to version control
2. Use different secrets for each environment (dev, staging, production)
3. Rotate secrets regularly using Secret Manager versioning
4. Limit access to Secret Manager using IAM roles
5. Audit access using Cloud Audit Logs
6. Use service accounts with minimal permissions

Troubleshooting

"Missing required environment variable" error

Problem: App fails to start with validation errors

Solution: Ensure all required secrets are created in Secret Manager

# List all secrets in your project
gcloud secrets list --project=YOUR_PROJECT_ID

# View a specific secret's value
gcloud secrets versions access latest --secret=SUPABASE_URL --project=YOUR_PROJECT_ID

"Permission denied" errors

Problem: App can't access secrets

Solution: Check IAM permissions

# Check service account permissions
gcloud projects get-iam-policy YOUR_PROJECT_ID \
  --flatten="bindings[].members" \
  --filter="bindings.members:serviceAccount:YOUR_SERVICE_ACCOUNT@YOUR_PROJECT_ID.iam.gserviceaccount.com"

"GCP_PROJECT environment variable must be set"

Problem: App doesn't know which GCP project to use

Solution: Set the environment variable

export GCP_PROJECT=your-project-id
# or
export GOOGLE_CLOUD_PROJECT=your-project-id

Development mode still uses Secret Manager

Problem: Want to use dotenv in development but it's using Secret Manager

Solution: Ensure NODE_ENV is set to development

export NODE_ENV=development
npm run dev

Cost Considerations

Google Secret Manager pricing (as of 2024):

• Secret versions: $0.06 per active secret version per month
• Access operations: $0.03 per 10,000 accesses

For typical usage:

• ~15 secrets = ~$1/month
• Caching reduces access operations (app caches secrets after first load)

Migration from Environment Variables

If migrating from environment variables to Secret Manager:

1. Create all secrets in Google Secret Manager
2. Update deployment configs to set NODE_ENV=production and GCP_PROJECT
3. Remove sensitive env vars from deployment configs
4. Keep non-sensitive configs as env vars (CORS_ORIGIN, PORT, etc.)
5. Test thoroughly in staging before production

References

• Google Secret Manager Documentation
• IAM Roles for Secret Manager
• @google-cloud/secret-manager NPM

Support

For issues with Secret Manager integration:

1. Check application logs for detailed error messages
2. Verify IAM permissions in Google Cloud Console
3. Ensure all required secrets exist in Secret Manager
4. Check that service account authentication is working