xtablo-source/docs/GOOGLE_SECRET_MANAGER_SETUP.md

# Google Secret Manager Setup

This guide explains how to set up and use Google Secret Manager for storing application secrets in production and staging environments.

## Overview

The application uses **Google Secret Manager** to securely store sensitive configuration values (API keys, database credentials, etc.) in production and staging environments, while using local `.env` files for development.

### Environment-based Configuration

- **Development/Test**: Uses `.env.development` or `.env.test` files (via dotenv)
- **Production/Staging**: Uses Google Secret Manager

## Prerequisites

1. **Google Cloud Project**: You need a GCP project with Secret Manager API enabled
2. **Service Account**: A service account with Secret Manager access
3. **Authentication**: Proper credentials configured for your deployment environment

## Setup Steps

### 1. Enable Secret Manager API

```bash
# Enable the Secret Manager API in your GCP project
gcloud services enable secretmanager.googleapis.com --project=YOUR_PROJECT_ID
```

### 2. Create Secrets in Google Cloud

For each required secret, create it in Google Secret Manager:

```bash
# Example: Create SUPABASE_URL secret
echo -n "https://your-project.supabase.co" | \
  gcloud secrets create SUPABASE_URL \
  --data-file=- \
  --project=YOUR_PROJECT_ID

# Example: Create STRIPE_SECRET_KEY secret
echo -n "sk_live_xxxxx" | \
  gcloud secrets create STRIPE_SECRET_KEY \
  --data-file=- \
  --project=YOUR_PROJECT_ID
```

### Required Secrets

Create the following secrets in Google Secret Manager:

| Secret Name | Description | Example |
|-------------|-------------|---------|
| `SUPABASE_URL` | Supabase project URL | `https://xxx.supabase.co` |
| `SUPABASE_SERVICE_ROLE_KEY` | Supabase service role key | `eyJxxx...` |
| `SUPABASE_CONNECTION_STRING` | PostgreSQL connection string | `postgresql://...` |
| `SUPABASE_CA_CERT` | Supabase CA certificate (optional) | `-----BEGIN CERTIFICATE-----...` |
| `STREAM_CHAT_API_KEY` | Stream Chat API key | `xxx` |
| `STREAM_CHAT_API_SECRET` | Stream Chat API secret | `xxx` |
| `STRIPE_SECRET_KEY` | Stripe secret key | `sk_live_xxx` |
| `STRIPE_WEBHOOK_SECRET` | Stripe webhook signing secret | `whsec_xxx` |
| `EMAIL_USER` | Gmail/email account | `yourapp@gmail.com` |
| `EMAIL_CLIENT_ID` | OAuth client ID for email | `xxx.apps.googleusercontent.com` |
| `EMAIL_CLIENT_SECRET` | OAuth client secret | `xxx` |
| `EMAIL_REFRESH_TOKEN` | OAuth refresh token | `xxx` |
| `R2_ACCOUNT_ID` | Cloudflare R2 account ID | `xxx` |
| `R2_ACCESS_KEY_ID` | Cloudflare R2 access key | `xxx` |
| `R2_SECRET_ACCESS_KEY` | Cloudflare R2 secret key | `xxx` |
| `TASKS_SECRET` | Secret for task authentication | `xxx` |

### 3. Grant Service Account Access

Your service account needs permission to access secrets:

```bash
# Grant Secret Manager Secret Accessor role
gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \
  --member="serviceAccount:YOUR_SERVICE_ACCOUNT@YOUR_PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/secretmanager.secretAccessor"
```

### 4. Set Environment Variables

The application needs these environment variables to work with Google Secret Manager:

```bash
# Required in production/staging
export GCP_PROJECT=your-project-id        # or GOOGLE_CLOUD_PROJECT
export NODE_ENV=production                # or staging

# Optional: Use non-default port
export PORT=8080

# Optional: Override CORS origins
export CORS_ORIGIN=https://app.xtablo.com

# Optional: Override XTablo URL
export XTABLO_URL=https://app.xtablo.com
```

## Authentication

### Local Development (Testing Secret Manager)

For local testing of Secret Manager integration:

```bash
# Authenticate with gcloud
gcloud auth application-default login

# Set project
export GCP_PROJECT=your-project-id
export NODE_ENV=staging

# Run your app
npm run dev
```

### Google Cloud Run / Cloud Functions

Authentication is automatic when running on Google Cloud services. Just ensure:

1. The service account has the `roles/secretmanager.secretAccessor` role
2. The `GCP_PROJECT` environment variable is set (or use the project's default)

```yaml
# Example Cloud Run configuration (cloudbuild.yaml)
steps:
  - name: 'gcr.io/cloud-builders/docker'
    args: ['build', '-t', 'gcr.io/$PROJECT_ID/api:$COMMIT_SHA', '.']
  - name: 'gcr.io/cloud-builders/docker'
    args: ['push', 'gcr.io/$PROJECT_ID/api:$COMMIT_SHA']
  - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
    entrypoint: gcloud
    args:
      - 'run'
      - 'deploy'
      - 'xtablo-api'
      - '--image=gcr.io/$PROJECT_ID/api:$COMMIT_SHA'
      - '--region=us-central1'
      - '--platform=managed'
      - '--set-env-vars=NODE_ENV=production,GCP_PROJECT=$PROJECT_ID'
      - '--service-account=YOUR_SERVICE_ACCOUNT@$PROJECT_ID.iam.gserviceaccount.com'
```

## Usage in Code

### Accessing Configuration

The configuration is loaded asynchronously at startup:

```typescript
// In index.ts (server startup)
import { getConfig } from './config.js';

async function startServer() {
  const config = await getConfig();
  // Use config.STRIPE_SECRET_KEY, config.SUPABASE_URL, etc.
}
```

### Backwards Compatibility

For synchronous access (after initialization):

```typescript
import { config } from './config.js';

// This works ONLY after getConfig() has been called
// (automatically done in index.ts startup)
const stripeKey = config.STRIPE_SECRET_KEY;
```

### Manual Secret Loading

If you need to load additional secrets not in the main config:

```typescript
import { loadSecret } from './secretManager.js';

// Load a single secret
const apiKey = await loadSecret('MY_CUSTOM_SECRET');

// Load with fallback to environment variable
import { loadSecretWithFallback } from './secretManager.js';
const token = await loadSecretWithFallback('MY_TOKEN', 'MY_TOKEN_ENV_VAR');
```

## Secret Versioning

Google Secret Manager supports versioning:

```bash
# Add a new version of a secret
echo -n "new-secret-value" | \
  gcloud secrets versions add SUPABASE_URL \
  --data-file=- \
  --project=YOUR_PROJECT_ID

# The app automatically uses the "latest" version
# You can also specify a specific version in code if needed
```

## Security Best Practices

1. **Never commit secrets** to version control
2. **Use different secrets** for each environment (dev, staging, production)
3. **Rotate secrets regularly** using Secret Manager versioning
4. **Limit access** to Secret Manager using IAM roles
5. **Audit access** using Cloud Audit Logs
6. **Use service accounts** with minimal permissions

## Troubleshooting

### "Missing required environment variable" error

**Problem**: App fails to start with validation errors

**Solution**: Ensure all required secrets are created in Secret Manager

```bash
# List all secrets in your project
gcloud secrets list --project=YOUR_PROJECT_ID

# View a specific secret's value
gcloud secrets versions access latest --secret=SUPABASE_URL --project=YOUR_PROJECT_ID
```

### "Permission denied" errors

**Problem**: App can't access secrets

**Solution**: Check IAM permissions

```bash
# Check service account permissions
gcloud projects get-iam-policy YOUR_PROJECT_ID \
  --flatten="bindings[].members" \
  --filter="bindings.members:serviceAccount:YOUR_SERVICE_ACCOUNT@YOUR_PROJECT_ID.iam.gserviceaccount.com"
```

### "GCP_PROJECT environment variable must be set"

**Problem**: App doesn't know which GCP project to use

**Solution**: Set the environment variable

```bash
export GCP_PROJECT=your-project-id
# or
export GOOGLE_CLOUD_PROJECT=your-project-id
```

### Development mode still uses Secret Manager

**Problem**: Want to use dotenv in development but it's using Secret Manager

**Solution**: Ensure `NODE_ENV` is set to `development`

```bash
export NODE_ENV=development
npm run dev
```

## Cost Considerations

Google Secret Manager pricing (as of 2024):

- **Secret versions**: $0.06 per active secret version per month
- **Access operations**: $0.03 per 10,000 accesses

For typical usage:
- ~15 secrets = ~$1/month
- Caching reduces access operations (app caches secrets after first load)

## Migration from Environment Variables

If migrating from environment variables to Secret Manager:

1. **Create all secrets** in Google Secret Manager
2. **Update deployment configs** to set `NODE_ENV=production` and `GCP_PROJECT`
3. **Remove sensitive env vars** from deployment configs
4. **Keep non-sensitive configs** as env vars (CORS_ORIGIN, PORT, etc.)
5. **Test thoroughly** in staging before production

## References

- [Google Secret Manager Documentation](https://cloud.google.com/secret-manager/docs)
- [IAM Roles for Secret Manager](https://cloud.google.com/secret-manager/docs/access-control)
- [@google-cloud/secret-manager NPM](https://www.npmjs.com/package/@google-cloud/secret-manager)

## Support

For issues with Secret Manager integration:

1. Check application logs for detailed error messages
2. Verify IAM permissions in Google Cloud Console
3. Ensure all required secrets exist in Secret Manager
4. Check that service account authentication is working