diff --git a/INSTALLATION.md b/INSTALLATION.md new file mode 100644 index 0000000..914b280 --- /dev/null +++ b/INSTALLATION.md @@ -0,0 +1,381 @@ +# Installation Guide + +This guide provides detailed instructions for setting up buun-stack - a Kubernetes development environment accessible from anywhere via the internet. + +## Prerequisites + +### Linux Machine + +A Linux PC with the following requirements: + +- Docker support +- SSH daemon configured for remote access +- Passwordless sudo execution (required for k3sup) +- Low power consumption recommended for 24/7 operation + +### Cloud Services + +- **Domain Registrar**: For registering and managing domain names +- **DNS Provider**: DNS managed by Cloudflare (for tunnel setup) +- **Container Registry**: For storing container images (optional) + +### Local Development Machine + +- Linux or macOS preferred +- [mise](https://mise.jdx.dev/) installed + +## Setting Up the Linux Machine + +### Basic Configuration + +1. Install Linux with Docker support +2. Configure SSH daemon for remote access +3. Set up passwordless sudo execution + +### Arch Linux Specific Configuration + +For Arch Linux users, configure sshd to support keyboard-interactive authentication with PAM: + +Create `/etc/ssh/sshd_config.d/10-pamauth.conf`: + +``` +KbdInteractiveAuthentication yes +AuthenticationMethods publickey keyboard-interactive:pam +``` + +Restart the sshd service: + +```bash +sudo systemctl restart sshd +``` + +Create sudoers file for your account (replace `buun` with your username): + +`/etc/sudoers.d/buun`: + +``` +buun ALL=(ALL:ALL) NOPASSWD: ALL +``` + +## Installing Required Tools + +### Clone the Repository + +```bash +git clone https://github.com/buun-ch/buun-stack +cd buun-stack +``` + +### Install mise + +Follow the [Getting Started](https://mise.jdx.dev/getting-started.html) guide to install mise. + +### Install Project Tools + +```bash +mise install +mise ls -l # Verify installed tools +``` + +This installs the following tools: + +- **gomplate**: Template engine for configuration files +- **gum**: Interactive CLI for user input +- **helm**: Kubernetes package manager +- **just**: Task runner for installation commands +- **kubelogin**: kubectl authentication plugin +- **vault**: HashiCorp Vault CLI client + +## Creating the Kubernetes Cluster + +### Generate Configuration + +```bash +just env::setup # Creates .env.local with your configuration +``` + +This interactive command collects: + +- SSH hostname for local access +- External domain names for services +- Keycloak realm name +- Other configuration options + +### Deploy k3s Cluster + +```bash +just k8s::install +kubectl get nodes # Verify cluster status +``` + +The installation uses k3sup to deploy k3s on the remote machine and automatically configures kubeconfig on your local machine. + +## Configuring Cloudflare Tunnel + +### Create the Tunnel + +1. Navigate to Cloudflare Dashboard > Zero Trust > Network > Tunnels +2. Click "+ Create a tunnel" +3. Select "Cloudflared" +4. Enter tunnel name +5. Click "Save tunnel" + +### Install cloudflared + +#### Debian/Red Hat + +Follow the instructions displayed in the Cloudflare dashboard. + +#### Arch Linux + +```bash +paru cloudflared +``` + +Create the systemd unit file: + +```bash +sudo systemctl edit --force --full cloudflared.service +``` + +Add the following content (replace `` with your token): + +``` +[Unit] +Description=Cloudflare Tunnel +After=network.target + +[Service] +TimeoutStartSec=0 +Type=notify +ExecStart=/usr/bin/cloudflared tunnel --loglevel debug --logfile /var/log/cloudflared/cloudflared.log run --token +Restart=on-failure +RestartSec=5s + +[Install] +WantedBy=multi-user.target +``` + +Enable and start the service: + +```bash +sudo systemctl enable cloudflared +sudo systemctl start cloudflared +``` + +### Configure Public Hostnames + +In the Cloudflare tunnel configuration, add these public hostnames: + +- `ssh.yourdomain.com` → SSH localhost:22 +- `vault.yourdomain.com` → HTTPS localhost:443 (No TLS Verify) +- `auth.yourdomain.com` → HTTPS localhost:443 (No TLS Verify) +- `k8s.yourdomain.com` → HTTPS localhost:6443 (No TLS Verify) + +**Note**: Enable "No TLS Verify" for HTTPS services as they use self-signed certificates internally. + +### Configure SSH Access + +#### macOS + +Install cloudflared: + +```bash +brew install cloudflared +``` + +Configure SSH in `~/.ssh/config`: + +``` +Host yourdomain + Hostname ssh.yourdomain.com + ProxyCommand /opt/homebrew/bin/cloudflared access ssh --hostname %h +``` + +Test connection: + +```bash +ssh yourdomain +``` + +## Installing Core Components + +### Longhorn - Distributed Storage + +Longhorn provides distributed block storage with support for PersistentVolumes and NFS exports. + +#### Prerequisites for Longhorn + +Install open-iscsi on the Linux machine: + +```bash +# Arch Linux +sudo pacman -S open-iscsi +sudo systemctl enable iscsid +sudo systemctl start iscsid + +# Ubuntu/Debian +sudo apt-get install open-iscsi +sudo systemctl enable iscsid +sudo systemctl start iscsid +``` + +#### Install Longhorn + +```bash +just longhorn::install +``` + +### HashiCorp Vault - Secrets Management + +Vault provides centralized secrets management for the entire cluster. + +```bash +just vault::install +``` + +**Important**: Store the displayed root token securely. You'll need it for initial configuration. + +### PostgreSQL - Database Cluster + +PostgreSQL provides database services for Keycloak and applications. + +```bash +just postgres::install +``` + +### Keycloak - Identity Management + +Keycloak provides identity and access management with OIDC/OAuth2 support. + +```bash +just keycloak::install +``` + +## Configuring OIDC Authentication + +### Create Keycloak Realm + +```bash +just keycloak::create-realm +``` + +The default realm name is `buunstack`. To change it, edit `.env.local`: + +```bash +KEYCLOAK_REALM=your-realm +``` + +### Configure Vault OIDC Integration + +```bash +just vault::setup-oidc-auth +``` + +This enables Vault authentication via Keycloak OIDC. + +### Create Initial User + +```bash +just keycloak::create-user +``` + +Follow the prompts to create username and password. + +### Enable Kubernetes OIDC Authentication + +```bash +just k8s::setup-oidc-auth +``` + +This creates a new kubectl context with OIDC authentication. If your original context is `minipc1`, the OIDC context will be `minipc1-oidc`. + +## Testing the Setup + +### Verify OIDC Authentication + +```bash +kubectl config use-context minipc1-oidc +kubectl get nodes +``` + +### Test Pod Operations + +Create test resources: + +```bash +kubectl apply -f debug/debug-pod.yaml +kubectl apply -f debug/debug-svc.yaml +``` + +Test kubectl exec: + +```bash +kubectl exec debug-pod -it -- sh +# Inside the pod: +uname -a +ps x +exit +``` + +Test port forwarding: + +```bash +kubectl port-forward svc/debug-service 8080:8080 +# In another terminal: +curl localhost:8080 +``` + +### Test Vault OIDC + +```bash +export VAULT_ADDR=https://vault.yourdomain.com +vault login -method=oidc +vault kv get -mount=secret -field=password postgres/admin +``` + +## Troubleshooting + +### Check Pod Logs + +```bash +kubectl logs -n +``` + +### Check Service Status + +```bash +kubectl get pods -A +kubectl get svc -A +``` + +### Reset OIDC Configuration + +If OIDC authentication fails: + +1. Check Keycloak is running: `kubectl get pods -n keycloak` +2. Verify realm exists: Browse to `https://auth.yourdomain.com` +3. Re-run OIDC setup: `just k8s::setup-oidc-auth` + +### Connection Issues + +If unable to connect via Cloudflare Tunnel: + +1. Check tunnel status in Cloudflare dashboard +2. Verify cloudflared service: `sudo systemctl status cloudflared` +3. Check DNS resolution: `nslookup k8s.yourdomain.com` + +## Next Steps + +- Create custom recipes in `custom.just` for your workflows +- Deploy applications using Helm charts +- Set up CI/CD pipelines +- Configure monitoring and observability + +## Resources + +- [GitHub Repository](https://github.com/buun-ch/buun-stack) +- [k3s Documentation](https://k3s.io) +- [Cloudflare Tunnel Documentation](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/) +- [Video Tutorial](https://youtu.be/Ezv4dEjLeKo) +- [Dev.to Article](https://dev.to/buun-ch/building-a-remote-accessible-kubernetes-home-lab-with-k3s-5g05) diff --git a/README.md b/README.md index 4c97aae..11408bf 100644 --- a/README.md +++ b/README.md @@ -14,60 +14,30 @@ A Kubernetes development stack for self-hosted environments, designed to run on - **Remote Access**: Cloudflare Tunnel for secure internet connectivity - **Automation**: Just task runner with templated configurations -## Prerequisites - -- Linux PC (low power consumption recommended) -- DNS and tunnel managed by Cloudflare -- Local development machine (Linux or macOS preferred) - - Install [mise](https://mise.jdx.dev/) - ## Quick Start -1. **Clone the repository** +For detailed step-by-step instructions, see the [Installation Guide](./INSTALLATION.md). + +1. **Clone and configure** ```bash git clone https://github.com/buun-ch/buun-stack cd buun-stack - ``` - -2. **Install required tools** - - ```bash mise install - mise ls -l # Verify installation + just env::setup ``` -3. **Configure environment** - - ```bash - just env::setup # Creates .env.local with your configuration - ``` - -4. **Install Kubernetes cluster** +2. **Deploy cluster and services** ```bash just k8s::install - kubectl get nodes # Verify cluster is running + just longhorn::install + just vault::install + just postgres::install + just keycloak::install ``` -5. **Set up Cloudflare Tunnel** - - Create tunnel in Cloudflare dashboard - - Configure public hostnames: - - `ssh.yourdomain.com` → SSH localhost:22 - - `vault.yourdomain.com` → HTTPS localhost:443 (no TLS verify) - - `auth.yourdomain.com` → HTTPS localhost:443 (no TLS verify) - - `k8s.yourdomain.com` → HTTPS localhost:6443 (no TLS verify) - -6. **Install core components** - - ```bash - just longhorn::install # Storage layer - just vault::install # Secrets management - just postgres::install # Database - just keycloak::install # Identity provider - ``` - -7. **Configure authentication** +3. **Configure authentication** ```bash just keycloak::create-realm @@ -114,15 +84,54 @@ Production-ready relational database for: - Keycloak data storage - Application databases -## Task Management +## Common Operations -All operations are managed through `just` recipes. Key commands include: +### User Management + +Create additional users: ```bash -just # Show all available commands -just env::setup # Configure environment -just k8s::install # Install Kubernetes -just keycloak::create-user # Create a new user +just keycloak::create-user +``` + +Add user to group: + +```bash +just keycloak::add-user-to-group +``` + +### Database Management + +Create database: + +```bash +just postgres::create-db +``` + +Create database user: + +```bash +just postgres::create-user +``` + +Grant privileges: + +```bash +just postgres::grant +``` + +### Secret Management + +Store secrets in Vault: + +```bash +just vault::put = +``` + +Retrieve secrets: + +```bash +just vault::get ``` ## Remote Access