diff --git a/justfile b/justfile index 4862610..c7b77b5 100644 --- a/justfile +++ b/justfile @@ -19,6 +19,7 @@ mod keycloak mod jupyterhub mod k8s mod kserve +mod langfuse mod lakekeeper mod longhorn mod metabase diff --git a/langfuse/.gitignore b/langfuse/.gitignore new file mode 100644 index 0000000..5072523 --- /dev/null +++ b/langfuse/.gitignore @@ -0,0 +1 @@ +langfuse-values.yaml diff --git a/langfuse/README.md b/langfuse/README.md new file mode 100644 index 0000000..eecf1b7 --- /dev/null +++ b/langfuse/README.md @@ -0,0 +1,342 @@ +# Langfuse + +Open source LLM observability and analytics platform with Keycloak OIDC authentication. + +## Overview + +This module deploys Langfuse using the official Helm chart with: + +- **Keycloak OIDC authentication** for user login +- **PostgreSQL backend** for application data +- **ClickHouse database** for analytics and traces +- **Redis (Valkey)** for caching and queues +- **MinIO/S3 storage** for event uploads and batch exports +- **Traefik ingress** for HTTPS access +- **External Secrets Operator integration** for secure credential management + +## Prerequisites + +- Kubernetes cluster (k3s) +- Keycloak installed and configured +- PostgreSQL cluster (CloudNativePG) +- ClickHouse cluster +- MinIO object storage +- External Secrets Operator (optional, for Vault integration) + +## Installation + +### Basic Installation + +```bash +just langfuse::install +``` + +You will be prompted for: + +- **Langfuse host (FQDN)**: e.g., `langfuse.example.com` + +### What Gets Installed + +- Langfuse web application (1 replica) +- Langfuse worker (background job processor) +- Redis (Valkey) for caching and queues +- PostgreSQL database `langfuse` with dedicated user +- ClickHouse database `langfuse` with dedicated user +- MinIO bucket `langfuse` for storage +- Keycloak OAuth client (confidential client) +- Keycloak user `langfuse` for system access +- Vault secrets (if External Secrets Operator is available) + +## Configuration + +Environment variables (set in `.env.local` or override): + +```bash +LANGFUSE_NAMESPACE=langfuse # Kubernetes namespace +LANGFUSE_CHART_VERSION= # Helm chart version +LANGFUSE_HOST=langfuse.example.com # External hostname +LANGFUSE_OIDC_CLIENT_ID=langfuse # Keycloak client ID +``` + +### Architecture Notes + +**Langfuse**: + +- Next.js application with FastAPI backend +- Redis/Valkey for session management and job queues +- ClickHouse for analytics queries +- PostgreSQL for application metadata +- S3-compatible storage for file uploads + +**Authentication Flow**: + +- OIDC via Keycloak with Authorization Code flow +- Username/password authentication disabled (`AUTH_DISABLE_USERNAME_PASSWORD=true`) +- Account linking enabled (`AUTH_KEYCLOAK_ALLOW_ACCOUNT_LINKING=true`) +- New users automatically provisioned on first SSO login +- Sign-up disabled for anonymous users + +**Database Structure**: + +- `langfuse` PostgreSQL database: Application data, experiments, projects +- `langfuse` ClickHouse database: Traces, observations, scores for analytics +- Redis: Session storage, job queues, caching + +## Usage + +### Access Langfuse + +1. Navigate to `https://your-langfuse-host/` +2. Click "Keycloak" button to authenticate via SSO +3. On first login, your account will be automatically created +4. Access the dashboard and start tracking LLM applications + +### Create API Keys + +1. Log in to Langfuse UI +2. Navigate to **Settings** → **API Keys** +3. Click **Create new API key** +4. Copy the public and secret keys +5. Use these keys in your LLM applications + +## Architecture + +```plain +External Users + ↓ +Cloudflare Tunnel (HTTPS) + ↓ +Traefik Ingress (HTTPS) + ↓ +Langfuse Web (HTTP inside cluster) + ├─ Next.js + ├─ OAuth → Keycloak (authentication) + ├─ PostgreSQL (metadata) + ├─ ClickHouse (analytics) + ├─ Redis/Valkey (cache & queues) + └─ MinIO (file storage) + ↓ +Langfuse Worker (background jobs) + ├─ Job queues (Redis) + ├─ Data processing + └─ Analytics aggregation +``` + +**Key Components**: + +- **Web UI**: Next.js application for dashboard and API +- **Worker**: Background job processor for async tasks +- **Redis**: Session management, job queues, caching +- **PostgreSQL**: Application data (projects, users, API keys) +- **ClickHouse**: Analytics data (traces, observations, scores) +- **MinIO**: S3-compatible storage for event uploads and batch exports + +## Authentication + +### User Login (OIDC) + +- Users authenticate via Keycloak +- Standard OIDC flow with Authorization Code grant +- Users automatically created on first login +- Username/password authentication is disabled +- Account linking enabled for users with same email + +### API Authentication + +- Public/Secret key pairs for programmatic access +- API keys are created per user in the Langfuse UI +- Keys are stored securely and can be rotated +- Each key is associated with a specific project + +### Access Control + +- Project-based access control +- Users can be invited to specific projects +- Role-based permissions (Owner, Admin, Member, Viewer) +- API keys are scoped to specific projects + +## Management + +### Upgrade Langfuse + +To upgrade Langfuse to a new version: + +```bash +just langfuse::upgrade +``` + +### Uninstall + +```bash +just langfuse::uninstall +``` + +This removes: + +- Helm release and all Kubernetes resources +- Namespace +- Keycloak client and Vault secrets + +**Note**: The following resources are NOT deleted and must be removed manually if needed: + +- PostgreSQL user and database +- ClickHouse user and database +- MinIO user and bucket +- Keycloak user + +### Clean Up Specific Resources + +```bash +# Delete PostgreSQL user and database +just langfuse::delete-postgres-user-and-db + +# Delete ClickHouse user and database +just langfuse::delete-clickhouse-user + +# Delete MinIO user and bucket +just langfuse::delete-minio-user + +# Delete Keycloak user +just langfuse::delete-keycloak-user +``` + +## Troubleshooting + +### Check Pod Status + +```bash +kubectl get pods -n langfuse +``` + +Expected pods: + +- `langfuse-web-*` - Web application (1 replica) +- `langfuse-worker-*` - Background worker (1 replica) +- `langfuse-redis-primary-0` - Redis/Valkey instance + +### OAuth Login Fails + +**Error**: `OAuthCallback: Invalid client or Invalid client credentials` + +**Cause**: Client secret mismatch between Keycloak and Langfuse + +**Solution**: Verify client secret is synchronized: + +```bash +# Get secret from Keycloak +just keycloak::get-client-secret langfuse + +# Compare with Vault +just vault::get keycloak/client/langfuse client_secret + +# If mismatched, update Vault and restart pods +just vault::put keycloak/client/langfuse client_id=langfuse client_secret= +kubectl rollout restart deployment/langfuse-web -n langfuse +``` + +**Error**: `Sign up is disabled` + +**Cause**: New SSO users cannot be created due to configuration + +**Solution**: This should not occur with the current configuration (`signUpDisabled: false`). If it does, verify Helm values: + +```bash +helm get values langfuse -n langfuse | grep signUpDisabled +# Should show: signUpDisabled: false +``` + +### Redis Connection Errors (Startup Only) + +**Symptoms**: Logs show `Redis error connect ECONNREFUSED` during pod startup + +**Cause**: Timing issue where web/worker pods start before Redis is ready + +**Impact**: None - these are transient errors during startup. Once Redis is ready, connections succeed and the application functions normally. + +**Solution**: No action needed. If you want to eliminate these startup errors, Redis pod can be deployed with a headstart, or init containers can be added to wait for Redis readiness. + +### Database Connection Issues + +Check PostgreSQL connectivity: + +```bash +kubectl exec -n langfuse deployment/langfuse-web -- \ + psql -h postgres-cluster-rw.postgres -U langfuse -d langfuse -c "SELECT 1" +``` + +Check ClickHouse connectivity: + +```bash +kubectl exec -n clickhouse clickhouse-clickhouse-0 -- \ + clickhouse-client --user=langfuse --password=$(just vault::get clickhouse/user/langfuse password) \ + --query "SELECT 1" +``` + +### Storage Issues + +Check MinIO credentials: + +```bash +kubectl get secret minio-auth -n langfuse -o yaml +``` + +Verify bucket exists: + +```bash +just minio::bucket-exists langfuse +``` + +### Check Logs + +```bash +# Web application logs +kubectl logs -n langfuse deployment/langfuse-web --tail=100 + +# Worker logs +kubectl logs -n langfuse deployment/langfuse-worker --tail=100 + +# Redis logs +kubectl logs -n langfuse langfuse-redis-primary-0 --tail=100 + +# Real-time logs +kubectl logs -n langfuse deployment/langfuse-web -f +``` + +### Common Issues + +**Blank page after login**: Check browser console for errors. Ensure `NEXTAUTH_URL` matches the actual hostname. + +**API requests fail**: Verify API keys are correct and associated with the correct project. + +**Slow dashboard**: Check ClickHouse query performance. Large trace volumes may require index optimization. + +**Missing traces**: Ensure SDK is configured with correct host and API keys. Check network connectivity from application to Langfuse. + +## Configuration Files + +Key configuration files: + +- `langfuse-values.gomplate.yaml` - Helm values template +- `keycloak-auth-external-secret.yaml` - Keycloak credentials +- `postgres-auth-external-secret.gomplate.yaml` - PostgreSQL credentials +- `clickhouse-auth-external-secret.gomplate.yaml` - ClickHouse credentials +- `redis-auth-external-secret.yaml` - Redis password +- `minio-auth-external-secret.yaml` - MinIO credentials + +## Security Considerations + +- **Secrets Management**: All credentials stored in Vault and synced via External Secrets Operator +- **OIDC Authentication**: No local password storage, authentication delegated to Keycloak +- **API Key Security**: Keys are hashed and stored securely in PostgreSQL +- **TLS/HTTPS**: All external traffic encrypted via Traefik Ingress +- **Network Isolation**: Internal services communicate via cluster network +- **Database Credentials**: Unique user per application with minimal privileges + +## References + +- [Langfuse Documentation](https://langfuse.com/docs) +- [Langfuse GitHub](https://github.com/langfuse/langfuse) +- [Langfuse Helm Chart](https://github.com/langfuse/langfuse-k8s) +- [Langfuse Python SDK](https://langfuse.com/docs/sdk/python) +- [Langfuse OpenAI Integration](https://langfuse.com/docs/integrations/openai) +- [Keycloak OIDC](https://www.keycloak.org/docs/latest/securing_apps/#_oidc) diff --git a/langfuse/justfile b/langfuse/justfile new file mode 100644 index 0000000..2c73964 --- /dev/null +++ b/langfuse/justfile @@ -0,0 +1,509 @@ +set fallback := true + +export LANGFUSE_NAMESPACE := env("LANGFUSE_NAMESPACE", "langfuse") +export LANGFUSE_CHART_VERSION := env("LANGFUSE_CHART_VERSION", "1.5.10") +export LANGFUSE_HOST := env("LANGFUSE_HOST", "") +export LANGFUSE_OIDC_CLIENT_ID := env("LANGFUSE_OIDC_CLIENT_ID", "langfuse") +export EXTERNAL_SECRETS_NAMESPACE := env("EXTERNAL_SECRETS_NAMESPACE", "external-secrets") +export K8S_VAULT_NAMESPACE := env("K8S_VAULT_NAMESPACE", "vault") +export KEYCLOAK_REALM := env("KEYCLOAK_REALM", "buunstack") +export KEYCLOAK_HOST := env("KEYCLOAK_HOST", "") +export MINIO_HOST := env("MINIO_HOST", "") +export MINIO_USER := "langfuse" + +[private] +default: + @just --list --unsorted --list-submodules + +# Add Helm repository +add-helm-repo: + helm repo add langfuse https://langfuse.github.io/langfuse-k8s + helm repo update + +# Remove Helm repository +remove-helm-repo: + helm repo remove langfuse + +# Create Langfuse namespace +create-namespace: + kubectl get namespace ${LANGFUSE_NAMESPACE} &>/dev/null || \ + kubectl create namespace ${LANGFUSE_NAMESPACE} + +# Delete Langfuse namespace +delete-namespace: + kubectl delete namespace ${LANGFUSE_NAMESPACE} --ignore-not-found + +# Install Langfuse +install: + #!/bin/bash + set -euo pipefail + + just create-namespace + just create-keycloak-user + + # Create PostgreSQL user and database with auto-generated password + if ! just postgres::user-exists langfuse &>/dev/null; then + PG_PASSWORD=$(just utils::random-password) + just postgres::create-user-and-db langfuse langfuse "${PG_PASSWORD}" + # Store password in Vault for later retrieval + just vault::put postgres/user/langfuse username=langfuse password="${PG_PASSWORD}" + else + echo "PostgreSQL user langfuse already exists, skipping creation" + if ! just postgres::db-exists langfuse &>/dev/null; then + just postgres::create-db langfuse + fi + fi + + # Check if ClickHouse is installed (required) + if ! helm list -n clickhouse 2>/dev/null | grep -q clickhouse; then + echo "Error: ClickHouse is not installed. Please install ClickHouse first:" + echo " just clickhouse::install" + exit 1 + fi + just create-clickhouse-user + just create-clickhouse-secret + + # Check if MinIO is installed (required) + if ! helm list -n minio 2>/dev/null | grep -q minio; then + echo "Error: MinIO is not installed. Please install MinIO first:" + echo " just minio::install" + exit 1 + fi + if ! just minio::user-exists langfuse &>/dev/null; then + just minio::create-user langfuse langfuse + else + echo "MinIO user langfuse already exists, skipping creation" + fi + just create-salt + just create-nextauth-secret + just create-redis-password + just create-keycloak-client + just create-secrets + + just add-helm-repo + export MINIO_HOST=$(kubectl get ingress -n minio minio -o jsonpath='{.spec.rules[0].host}' 2>/dev/null || echo "") + LANGFUSE_SALT=$(just salt) \ + NEXTAUTH_SECRET=$(just nextauth-secret) \ + gomplate -f langfuse-values.gomplate.yaml -o langfuse-values.yaml + helm upgrade --install langfuse langfuse/langfuse \ + --version ${LANGFUSE_CHART_VERSION} -n ${LANGFUSE_NAMESPACE} --wait \ + -f langfuse-values.yaml + +# Uninstall Langfuse +uninstall: + #!/bin/bash + set -euo pipefail + helm uninstall langfuse -n ${LANGFUSE_NAMESPACE} --wait --ignore-not-found + kubectl delete namespace ${LANGFUSE_NAMESPACE} --ignore-not-found + + # Clean up Keycloak client and Vault secrets to avoid stale credentials + just delete-keycloak-client || true + + echo "Langfuse uninstalled successfully" + echo "" + echo "Note: The following resources were NOT deleted:" + echo " - PostgreSQL user and database (langfuse)" + echo " - ClickHouse user and database (langfuse)" + echo " - MinIO user and bucket (langfuse)" + echo " - Keycloak user (langfuse)" + echo "" + echo "To delete these resources, run:" + echo " just langfuse::delete-postgres-user-and-db" + echo " just langfuse::delete-clickhouse-user" + echo " just langfuse::delete-minio-user" + echo " just langfuse::delete-keycloak-user" + +# Create all secrets (PostgreSQL, Keycloak, MinIO, Redis) +create-secrets: + #!/bin/bash + set -euo pipefail + + # Get PostgreSQL credentials + pg_host="postgres-cluster-rw.postgres" + pg_port="5432" + pg_user="langfuse" + pg_password=$(just vault::get postgres/user/langfuse password) + pg_database="langfuse" + database_url="postgresql://${pg_user}:${pg_password}@${pg_host}:${pg_port}/${pg_database}" + + # Get OAuth client secret + # Prioritize temporary secret (freshly created) over Vault (potentially stale) + if kubectl get secret langfuse-oauth-temp -n ${LANGFUSE_NAMESPACE} &>/dev/null; then + oauth_client_id=$(kubectl get secret langfuse-oauth-temp -n ${LANGFUSE_NAMESPACE} \ + -o jsonpath='{.data.client_id}' | base64 -d) + oauth_client_secret=$(kubectl get secret langfuse-oauth-temp -n ${LANGFUSE_NAMESPACE} \ + -o jsonpath='{.data.client_secret}' | base64 -d) + elif helm status vault -n ${K8S_VAULT_NAMESPACE} &>/dev/null && \ + just vault::get keycloak/client/langfuse client_secret &>/dev/null; then + oauth_client_id=$(just vault::get keycloak/client/langfuse client_id) + oauth_client_secret=$(just vault::get keycloak/client/langfuse client_secret) + else + echo "Error: Cannot retrieve OAuth client secret. Please run 'just langfuse::create-keycloak-client' first." + exit 1 + fi + + # Get Redis password + redis_password=$(just vault::get langfuse/redis secret) + + if helm status external-secrets -n ${EXTERNAL_SECRETS_NAMESPACE} &>/dev/null; then + echo "External Secrets Operator detected. Storing secrets in Vault..." + + # Store PostgreSQL credentials in Vault + just vault::put langfuse/postgres \ + username="${pg_user}" \ + password="${pg_password}" \ + url="${database_url}" + + # Store OAuth credentials in Vault + just vault::put keycloak/client/langfuse \ + client_id="${oauth_client_id}" \ + client_secret="${oauth_client_secret}" + + # Redis password is already in Vault (created by create-redis-password) + + # MinIO credentials are already in Vault (created by create-minio-service-account) + + # Delete existing secrets and ExternalSecrets + kubectl delete secret postgres-auth -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl delete externalsecret postgres-auth-external-secret -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl delete secret keycloak-auth -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl delete externalsecret keycloak-auth-external-secret -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl delete secret redis-auth -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl delete externalsecret redis-auth-external-secret -n ${LANGFUSE_NAMESPACE} --ignore-not-found + + # Create ExternalSecrets + gomplate -f postgres-auth-external-secret.gomplate.yaml | kubectl apply -f - + kubectl apply -n ${LANGFUSE_NAMESPACE} -f keycloak-auth-external-secret.yaml + kubectl apply -n ${LANGFUSE_NAMESPACE} -f redis-auth-external-secret.yaml + kubectl delete secret minio-auth -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl delete externalsecret minio-auth-external-secret -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl apply -n ${LANGFUSE_NAMESPACE} -f minio-auth-external-secret.yaml + + echo "Waiting for ExternalSecrets to sync..." + kubectl wait --for=condition=Ready externalsecret/postgres-auth-external-secret \ + -n ${LANGFUSE_NAMESPACE} --timeout=60s + kubectl wait --for=condition=Ready externalsecret/keycloak-auth-external-secret \ + -n ${LANGFUSE_NAMESPACE} --timeout=60s + kubectl wait --for=condition=Ready externalsecret/redis-auth-external-secret \ + -n ${LANGFUSE_NAMESPACE} --timeout=60s + kubectl wait --for=condition=Ready externalsecret/minio-auth-external-secret \ + -n ${LANGFUSE_NAMESPACE} --timeout=60s + + echo "ExternalSecrets created successfully" + else + echo "External Secrets Operator not found. Creating Kubernetes Secrets directly..." + + # Create PostgreSQL Secret + kubectl delete secret postgres-auth -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl create secret generic postgres-auth -n ${LANGFUSE_NAMESPACE} \ + --from-literal=username="${pg_user}" \ + --from-literal=password="${pg_password}" \ + --from-literal=url="${database_url}" + + # Create Keycloak OAuth Secret + kubectl delete secret keycloak-auth -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl create secret generic keycloak-auth -n ${LANGFUSE_NAMESPACE} \ + --from-literal=client_id="${oauth_client_id}" \ + --from-literal=client_secret="${oauth_client_secret}" + + # Create Redis Secret + kubectl delete secret redis-auth -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl create secret generic redis-auth -n ${LANGFUSE_NAMESPACE} \ + --from-literal=secret="${redis_password}" + + # Create MinIO Secret + minio_access_key=$(just vault::get langfuse/minio access_key) + minio_secret_key=$(just vault::get langfuse/minio secret_key) + kubectl delete secret minio-auth -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl create secret generic minio-auth -n ${LANGFUSE_NAMESPACE} \ + --from-literal=access_key="${minio_access_key}" \ + --from-literal=secret_key="${minio_secret_key}" + + # Store credentials in Vault if available (backup for admin credentials) + if helm status vault -n ${K8S_VAULT_NAMESPACE} &>/dev/null; then + just vault::put langfuse/postgres \ + username="${pg_user}" \ + password="${pg_password}" \ + url="${database_url}" + just vault::put keycloak/client/langfuse \ + client_id="${oauth_client_id}" \ + client_secret="${oauth_client_secret}" + fi + + echo "Kubernetes Secrets created successfully" + fi + + # Clean up temporary OAuth secret + kubectl delete secret langfuse-oauth-temp -n ${LANGFUSE_NAMESPACE} --ignore-not-found + +# Print Postgres password (from Kubernetes Secret) +postgres-password: + @kubectl get secret postgres-auth -n ${LANGFUSE_NAMESPACE} \ + -o jsonpath='{.data.password}' | base64 -d + @echo + +# Print Postgres password (from Vault) +postgres-password-from-vault: + @just vault::get postgres/user/langfuse password + +# Delete PostgreSQL user and database +delete-postgres-user-and-db: + #!/bin/bash + set -euo pipefail + if just postgres::user-exists langfuse &>/dev/null; then + just postgres::delete-user-and-db langfuse langfuse + else + echo "PostgreSQL user langfuse does not exist, skipping deletion" + fi + if just vault::exist postgres/user/langfuse &>/dev/null; then + just vault::delete postgres/user/langfuse + fi + +# Create ClickHouse user and database (for external ClickHouse) +create-clickhouse-user: + #!/bin/bash + set -euo pipefail + + # Create database if it doesn't exist + if ! just clickhouse::db-exists langfuse &>/dev/null; then + just clickhouse::create-db langfuse + echo "ClickHouse database 'langfuse' created" + else + echo "ClickHouse database 'langfuse' already exists" + fi + + # Create user if it doesn't exist + if just clickhouse::user-exists langfuse &>/dev/null; then + echo "ClickHouse user langfuse already exists" + # Ensure privileges are granted even if user already exists + just clickhouse::grant langfuse langfuse + exit + fi + + PASSWORD=$(just utils::random-password) + just vault::put clickhouse/user/langfuse username=langfuse password="${PASSWORD}" + + # Create user and grant privileges + just clickhouse::create-user langfuse "${PASSWORD}" + just clickhouse::grant langfuse langfuse + +# Delete ClickHouse user and database (for external ClickHouse) +delete-clickhouse-user: + #!/bin/bash + set -euo pipefail + if ! just clickhouse::user-exists langfuse &>/dev/null; then + echo "ClickHouse user langfuse does not exist, skipping deletion" + exit + fi + just clickhouse::delete-user langfuse + if just clickhouse::db-exists langfuse &>/dev/null; then + just clickhouse::delete-db langfuse + fi + just vault::delete clickhouse/user/langfuse + +# Create ClickHouse auth secret +create-clickhouse-secret: + #!/bin/bash + set -euo pipefail + if kubectl get secret clickhouse-auth -n ${LANGFUSE_NAMESPACE} &>/dev/null; then + echo "ClickHouse auth secret already exists" + exit + fi + + # for external ClickHouse + PASSWORD=$(just vault::get clickhouse/user/langfuse password) + + # for internal ClickHouse + # PASSWORD=$(just utils::random-password) + # just vault::put clickhouse/user/langfuse username=langfuse password="${PASSWORD}" + + kubectl create secret generic clickhouse-auth -n ${LANGFUSE_NAMESPACE} \ + --from-literal=password="${PASSWORD}" + +# Delete ClickHouse auth secret +delete-clickhouse-secret: + kubectl delete secret clickhouse-auth -n ${LANGFUSE_NAMESPACE} --ignore-not-found + +# Print ClickHouse password +clickhouse-password: + @kubectl get secret clickhouse-auth -n ${LANGFUSE_NAMESPACE} \ + -o jsonpath='{.data.password}' | base64 -d + @echo + +check-clickhouse-privilege: + kubectl exec -it clickhouse-pod -n clickhouse -- \ + clickhouse-client --user=langfuse --password=$(just clickhouse-password) \ + --query "SHOW GRANTS FOR langfuse" + +# Delete MinIO user and bucket +delete-minio-user: + #!/bin/bash + set -euo pipefail + if ! just minio::user-exists langfuse &>/dev/null; then + echo "MinIO user langfuse does not exist, skipping deletion" + exit + fi + just minio::delete-user langfuse + if just vault::exist langfuse/minio &>/dev/null; then + just vault::delete langfuse/minio + fi + +# Create Keycloak user +create-keycloak-user: + #!/bin/bash + set -euo pipefail + if just keycloak::user-exists langfuse &>/dev/null; then + echo "Keycloak user langfuse already exists, skipping creation" + exit + fi + PASSWORD=$(just utils::random-password) + just vault::put keycloak/user/langfuse username=langfuse password="${PASSWORD}" + just keycloak::create-system-user langfuse "${PASSWORD}" + +# Delete keycloak user +delete-keycloak-user: + #!/bin/bash + set -euo pipefail + if ! just keycloak::user-exists langfuse &>/dev/null; then + echo "Keycloak user langfuse does not exist, skipping deletion" + exit + fi + just keycloak::delete-user langfuse + just vault::delete keycloak/user/langfuse + +# Create Langfuse salt +create-salt: + #!/bin/bash + set -euo pipefail + if just vault::exist langfuse/salt &>/dev/null; then + echo "Salt for Langfuse already exists, skipping creation" + exit + fi + SALT=$(just utils::random-password) + just vault::put langfuse/salt value="${SALT}" + +# Delete Langfuse salt +delete-salt: + #!/bin/bash + set -euo pipefail + if ! just vault::exist langfuse/salt &>/dev/null; then + echo "Salt for Langfuse does not exist, skipping deletion" + exit + fi + just vault::delete langfuse/salt + +# Print Langfuse salt +salt: + #!/bin/bash + set -euo pipefail + if ! just vault::exist langfuse/salt &>/dev/null; then + echo "Salt for Langfuse does not exist" >&2 + exit 1 + fi + just vault::get langfuse/salt value + +# Create NextAuth secret +create-nextauth-secret: + #!/bin/bash + set -euo pipefail + if just vault::exist langfuse/nextauth &>/dev/null; then + echo "Langfuse NextAuth secret already exists, skipping creation" + exit + fi + SECRET=$(just utils::random-password) + just vault::put langfuse/nextauth secret="${SECRET}" + +# Delete NextAuth secret +delete-nextauth-secret: + #!/bin/bash + set -euo pipefail + if ! just vault::exist langfuse/nextauth &>/dev/null; then + echo "Langfuse NextAuth secret does not exist, skipping deletion" + exit + fi + just vault::delete langfuse/nextauth + +# Print NextAuth secret +nextauth-secret: + #!/bin/bash + set -euo pipefail + if ! just vault::exist langfuse/nextauth &>/dev/null; then + echo "Langfuse NextAuth secret does not exist" >&2 + exit 1 + fi + just vault::get langfuse/nextauth secret + +# Create Redis password +create-redis-password: + #!/bin/bash + set -euo pipefail + if just vault::exist langfuse/redis &>/dev/null; then + echo "Redis password already exists, skipping creation" + exit + fi + SECRET=$(just utils::random-password) + just vault::put langfuse/redis secret="${SECRET}" + +# Delete Redis password +delete-redis-password: + #!/bin/bash + set -euo pipefail + if ! just vault::exist langfuse/redis &>/dev/null; then + echo "Redis password does not exist, skipping deletion" + exit + fi + just vault::delete langfuse/redis + +# Print Redis password +redis-password: + #!/bin/bash + set -euo pipefail + if ! just vault::exist langfuse/redis &>/dev/null; then + echo "Redis password does not exist" >&2 + exit 1 + fi + just vault::get langfuse/redis secret + echo + +# Create Keycloak client for Langfuse +create-keycloak-client: + #!/bin/bash + set -euo pipefail + while [ -z "${LANGFUSE_HOST}" ]; do + LANGFUSE_HOST=$( + gum input --prompt="Langfuse host (FQDN): " --width=100 \ + --placeholder="e.g., langfuse.example.com" + ) + done + + echo "Creating Keycloak client for Langfuse..." + + just keycloak::delete-client ${KEYCLOAK_REALM} ${LANGFUSE_OIDC_CLIENT_ID} || true + + CLIENT_SECRET=$(just utils::random-password) + + just keycloak::create-client \ + realm=${KEYCLOAK_REALM} \ + client_id=${LANGFUSE_OIDC_CLIENT_ID} \ + redirect_url="https://${LANGFUSE_HOST}/api/auth/callback/keycloak" \ + client_secret="${CLIENT_SECRET}" + + kubectl delete secret langfuse-oauth-temp -n ${LANGFUSE_NAMESPACE} --ignore-not-found + kubectl create secret generic langfuse-oauth-temp -n ${LANGFUSE_NAMESPACE} \ + --from-literal=client_id="${LANGFUSE_OIDC_CLIENT_ID}" \ + --from-literal=client_secret="${CLIENT_SECRET}" + + echo "Keycloak client created successfully" + echo "Client ID: ${LANGFUSE_OIDC_CLIENT_ID}" + echo "Redirect URI: https://${LANGFUSE_HOST}/api/auth/callback/keycloak" + +# Delete Keycloak client for Langfuse +delete-keycloak-client: + #!/bin/bash + set -euo pipefail + echo "Deleting Keycloak client for Langfuse..." + just keycloak::delete-client ${KEYCLOAK_REALM} ${LANGFUSE_OIDC_CLIENT_ID} || true + kubectl delete secret langfuse-oauth-temp -n ${LANGFUSE_NAMESPACE} --ignore-not-found + if just vault::exist keycloak/client/langfuse &>/dev/null; then + just vault::delete keycloak/client/langfuse + fi diff --git a/langfuse/keycloak-auth-external-secret.yaml b/langfuse/keycloak-auth-external-secret.yaml new file mode 100644 index 0000000..14038fb --- /dev/null +++ b/langfuse/keycloak-auth-external-secret.yaml @@ -0,0 +1,21 @@ +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: keycloak-auth-external-secret +spec: + refreshInterval: 1h + secretStoreRef: + name: vault-secret-store + kind: ClusterSecretStore + target: + name: keycloak-auth + creationPolicy: Owner + data: + - secretKey: client_id + remoteRef: + key: keycloak/client/langfuse + property: client_id + - secretKey: client_secret + remoteRef: + key: keycloak/client/langfuse + property: client_secret diff --git a/langfuse/langfuse-values.gomplate.yaml b/langfuse/langfuse-values.gomplate.yaml new file mode 100644 index 0000000..9113677 --- /dev/null +++ b/langfuse/langfuse-values.gomplate.yaml @@ -0,0 +1,102 @@ +langfuse: + salt: + value: {{ .Env.LANGFUSE_SALT }} + features: + telemetryEnabled: false + # Allow SSO users to automatically create accounts on first login + # Username/password authentication is disabled via AUTH_DISABLE_USERNAME_PASSWORD + signUpDisabled: false + experimentalFeaturesEnabled: false + nextauth: + url: https://{{ .Env.LANGFUSE_HOST }} + secret: + value: {{ .Env.NEXTAUTH_SECRET }} + additionalEnv: + - name: DATABASE_URL + valueFrom: + secretKeyRef: + name: postgres-auth + key: url + # ClickHouse database name + - name: CLICKHOUSE_DB + value: "langfuse" + # https://langfuse.com/self-hosting/authentication-and-sso#keycloak + - name: AUTH_DISABLE_USERNAME_PASSWORD + value: "true" + - name: AUTH_KEYCLOAK_ALLOW_ACCOUNT_LINKING + value: "true" + - name: AUTH_KEYCLOAK_CLIENT_ID + valueFrom: + secretKeyRef: + name: keycloak-auth + key: client_id + - name: AUTH_KEYCLOAK_CLIENT_SECRET + valueFrom: + secretKeyRef: + name: keycloak-auth + key: client_secret + - name: AUTH_KEYCLOAK_ISSUER + value: "https://{{ .Env.KEYCLOAK_HOST }}/realms/{{ .Env.KEYCLOAK_REALM }}" + + ingress: + enabled: true + annotations: + kubernetes.io/ingress.class: traefik + traefik.ingress.kubernetes.io/router.entrypoints: websecure + className: traefik + hosts: + - host: {{ .Env.LANGFUSE_HOST }} + paths: + - path: / + pathType: ImplementationSpecific + tls: + enabled: true + +postgresql: + deploy: false + +redis: + deploy: true + architecture: standalone + auth: + username: "default" + existingSecret: redis-auth + existingSecretPasswordKey: secret + +clickhouse: + deploy: false + host: clickhouse-clickhouse.clickhouse + clusterEnabled: false + auth: + username: langfuse + existingSecret: clickhouse-auth + existingSecretKey: password + + # for internal ClickHouse + # # https://github.com/bitnami/charts/tree/main/bitnami/clickhouse + # deploy: true + # auth: + # existingSecret: clickhouse-auth + # existingSecretKey: password + # shards: 1 + # replicaCount: 1 + # zookeeper: + # enabled: true + # replicaCount: 1 + # # persistence: + # # storageClass: local-path + +s3: + deploy: false + bucket: langfuse + region: "auto" + endpoint: https://{{ .Env.MINIO_HOST }} + forcePathStyle: true + accessKeyId: + secretKeyRef: + name: minio-auth + key: access_key + secretAccessKey: + secretKeyRef: + name: minio-auth + key: secret_key diff --git a/langfuse/minio-auth-external-secret.yaml b/langfuse/minio-auth-external-secret.yaml new file mode 100644 index 0000000..d2e49ca --- /dev/null +++ b/langfuse/minio-auth-external-secret.yaml @@ -0,0 +1,21 @@ +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: minio-auth-external-secret +spec: + refreshInterval: 1h + secretStoreRef: + name: vault-secret-store + kind: ClusterSecretStore + target: + name: minio-auth + creationPolicy: Owner + data: + - secretKey: access_key + remoteRef: + key: langfuse/minio + property: access_key + - secretKey: secret_key + remoteRef: + key: langfuse/minio + property: secret_key diff --git a/langfuse/postgres-auth-external-secret.gomplate.yaml b/langfuse/postgres-auth-external-secret.gomplate.yaml new file mode 100644 index 0000000..21152df --- /dev/null +++ b/langfuse/postgres-auth-external-secret.gomplate.yaml @@ -0,0 +1,26 @@ +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: postgres-auth-external-secret + namespace: {{ .Env.LANGFUSE_NAMESPACE }} +spec: + refreshInterval: 1h + secretStoreRef: + name: vault-secret-store + kind: ClusterSecretStore + target: + name: postgres-auth + creationPolicy: Owner + data: + - secretKey: username + remoteRef: + key: langfuse/postgres + property: username + - secretKey: password + remoteRef: + key: langfuse/postgres + property: password + - secretKey: url + remoteRef: + key: langfuse/postgres + property: url diff --git a/langfuse/redis-auth-external-secret.yaml b/langfuse/redis-auth-external-secret.yaml new file mode 100644 index 0000000..6074441 --- /dev/null +++ b/langfuse/redis-auth-external-secret.yaml @@ -0,0 +1,17 @@ +apiVersion: external-secrets.io/v1 +kind: ExternalSecret +metadata: + name: redis-auth-external-secret +spec: + refreshInterval: 1h + secretStoreRef: + name: vault-secret-store + kind: ClusterSecretStore + target: + name: redis-auth + creationPolicy: Owner + data: + - secretKey: secret + remoteRef: + key: langfuse/redis + property: secret