buun-stack

A Kubernetes development stack for self-hosted environments, designed to run on a Linux machine in your home or office that you can access from anywhere via the internet.

📺 Watch the setup tutorial on YouTube | 📝 Read the detailed guide on Dev.to

Features

• Kubernetes Distribution: k3s lightweight Kubernetes
• Block Storage: Longhorn distributed block storage
• Object Storage: MinIO S3-compatible storage
• Identity & Access: Keycloak for OIDC authentication
• Secrets Management: HashiCorp Vault with External Secrets Operator
• Interactive Computing: JupyterHub for collaborative notebooks
• Business Intelligence: Metabase for business intelligence and data visualization
• Data Catalog: DataHub for metadata management and data discovery
• Database: PostgreSQL cluster
• Analytics Engine/Database: ClickHouse for high-performance analytics and data warehousing
• Workflow Orchestration: Apache Airflow for data pipeline automation and task scheduling
• Remote Access: Cloudflare Tunnel for secure internet connectivity
• Automation: Just task runner with templated configurations

Quick Start

For detailed step-by-step instructions, see the Installation Guide.

1. Clone and configure

   git clone https://github.com/buun-ch/buun-stack
   cd buun-stack
   mise install
   just env::setup
   

2. Deploy cluster and services

   just k8s::install
   just longhorn::install
   just vault::install
   just postgres::install
   just keycloak::install
   

3. Configure authentication

   just keycloak::create-realm
   just vault::setup-oidc-auth
   just keycloak::create-user
   just k8s::setup-oidc-auth
   

Core Components

k3s

Lightweight Kubernetes distribution optimized for edge computing and resource-constrained environments.

Longhorn

Enterprise-grade distributed storage system providing:

• Highly available block storage
• Backup and disaster recovery
• No single point of failure
• Support for NFS persistent volumes

HashiCorp Vault

Centralized secrets management offering:

• Secure secret storage
• Dynamic secrets generation
• Encryption as a service
• Integration with External Secrets Operator for automatic Kubernetes Secret synchronization

Keycloak

Open-source identity and access management providing:

• Single Sign-On (SSO)
• OIDC/OAuth2 authentication
• User federation and identity brokering

PostgreSQL

Production-ready relational database for:

• Keycloak data storage
• Application databases
• Vector similarity search with pgvector extension for AI/ML workloads

External Secrets Operator

Kubernetes operator for syncing secrets from external systems:

• Automatically syncs secrets from Vault to Kubernetes Secrets
• Supports multiple secret backends
• Provides secure secret rotation and lifecycle management

MinIO

S3-compatible object storage system providing:

• High-performance distributed object storage
• AWS S3 API compatibility
• Erasure coding for data protection
• Multi-tenancy support

JupyterHub

Multi-user platform for interactive computing:

• Collaborative Jupyter notebook environment
• Integrated with Keycloak for OIDC authentication
• Persistent storage for user workspaces
• Support for multiple kernels and environments
• Vault integration for secure secrets management

See JupyterHub Documentation for detailed setup and configuration.

Metabase

Business intelligence and data visualization platform:

• Open-source analytics and dashboards
• Interactive data exploration
• PostgreSQL integration for data storage
• Automated setup with Helm
• Session management through Vault/External Secrets
• Simplified deployment (no OIDC dependency)

Installation:

just metabase::install

Access Metabase at https://metabase.yourdomain.com and complete the initial setup wizard to create an admin account.

DataHub

Modern data catalog and metadata management platform:

• Centralized data discovery and documentation
• Data lineage tracking and impact analysis
• Schema evolution monitoring
• OIDC integration with Keycloak for secure access
• Elasticsearch-powered search and indexing
• Kafka-based real-time metadata streaming
• PostgreSQL backend for metadata storage

Installation:

just datahub::install

⚠️ Resource Requirements: DataHub is resource-intensive, requiring approximately 4-5GB of RAM and 1+ CPU cores across multiple components (Elasticsearch, Kafka, Zookeeper, and DataHub services). Deployment typically takes 15-20 minutes to complete. Ensure your cluster has sufficient resources before installation.

Access DataHub at https://datahub.yourdomain.com and use "Sign in with SSO" to authenticate via Keycloak.

ClickHouse

High-performance columnar OLAP database for analytics and data warehousing:

• Columnar storage for fast analytical queries
• Real-time data ingestion and processing
• Horizontal scaling for large datasets
• SQL interface with advanced analytics functions
• Integration with External Secrets for secure credential management
• Support for various data formats (CSV, JSON, Parquet, etc.)

Installation:

just clickhouse::install

Access ClickHouse at https://clickhouse.yourdomain.com using the admin credentials stored in Vault.

Apache Airflow

Modern workflow orchestration platform for data pipelines and task automation:

• Airflow 3 with modern SDK components and FastAPI integration
• DAG Development: Integrated with JupyterHub for seamless workflow creation and editing
• OIDC Authentication: Secure access through Keycloak integration
• Shared Storage: DAG files shared between JupyterHub and Airflow for direct editing
• Role-based Access Control: Multiple user roles (Admin, Operator, User, Viewer)
• REST API: Ful API access for programmatic DAG management

Installation:

just airflow::install

JupyterHub Integration: After installing both JupyterHub and Airflow, DAG files are automatically shared:

• Edit DAG files directly in JupyterHub: ~/airflow-dags/*.py
• Changes appear in Airflow UI within 1-2 minutes
• Full Python development environment with syntax checking
• Template files available for quick DAG creation

User Management:

# Assign roles to users
just airflow::assign-role <username> <role>

# Available roles: airflow_admin, airflow_op, airflow_user, airflow_viewer
just airflow::assign-role myuser airflow_admin

API Access: Create API users for programmatic access:

just airflow::create-api-user <username> <role>

💡 Development Workflow: Create DAGs in JupyterHub using ~/airflow-dags/dag_template.py as a starting point. Use .tmp extension during development to avoid import errors, then rename to .py when ready.

Access Airflow at https://airflow.yourdomain.com and authenticate via Keycloak.

Common Operations

User Management

Create additional users:

just keycloak::create-user

Add user to group:

just keycloak::add-user-to-group <username> <group>

Database Management

Create database:

just postgres::create-db <dbname>

Create database user:

just postgres::create-user <username>

Grant privileges:

just postgres::grant <dbname> <username>

Secret Management

Store secrets in Vault:

just vault::put <path> <key>=<value>

Retrieve secrets:

just vault::get <path> <field>

Remote Access

Once configured, you can access your cluster from anywhere:

# SSH access
ssh ssh.yourdomain.com

# Kubernetes API
kubectl --context yourpc-oidc get nodes

# Web interfaces
# Vault: https://vault.yourdomain.com
# Keycloak: https://auth.yourdomain.com
# Metabase: https://metabase.yourdomain.com
# Airflow: https://airflow.yourdomain.com
# JupyterHub: https://jupyter.yourdomain.com

Customization

Adding Custom Recipes

You can extend buun-stack with your own Just recipes and services:

1. Copy the example files:

   cp custom-example.just custom.just
   cp -r custom-example custom
   

2. Use the custom recipes:

   # Install reddit-rss
   just custom::reddit-rss::install
   
   # Install Miniflux feed reader
   just custom::miniflux::install
   

3. Create your own recipes:

Add new modules to the custom/ directory following the same pattern as the examples. Each module should have its own justfile with install, uninstall, and other relevant recipes.

The custom.just file is automatically imported by the main Justfile if it exists, allowing you to maintain your custom workflows separately from the core stack.

Troubleshooting

• Check logs: kubectl logs -n <namespace> <pod-name>

License

MIT License - See LICENSE file for details