add support for Longhorn setup
This commit is contained in:
174
Longhorn/README.md
Normal file
174
Longhorn/README.md
Normal file
@@ -0,0 +1,174 @@
|
||||
Um ein Persistent Volume (PV) für einen Longhorn Storage Provider in einem Kubernetes-Cluster zu erstellen, musst du mehrere Schritte durchführen. Longhorn ist eine verteilte Blockspeicherlösung für Kubernetes, und du kannst damit Persistent Volumes provisionieren.
|
||||
|
||||
Hier sind die Schritte, um ein Persistent Volume für Longhorn zu erstellen:
|
||||
|
||||
### 1. Stelle sicher, dass Longhorn installiert ist
|
||||
Zuerst solltest du sicherstellen, dass Longhorn auf deinem Cluster installiert ist. Falls Longhorn noch nicht installiert ist, kannst du es mit Helm oder direkt aus den YAML-Dateien installieren.
|
||||
|
||||
|
||||
#### Node Labeling
|
||||
|
||||
In the case not all nodes should provide disk, e.g. certain nodes have special/fast disks.
|
||||
In this case the StorageClass needs to be adapted and added with a nodeselector [1].
|
||||
```
|
||||
k label nodes k3s-prod-worker-{1..3} node.longhorn.io/create-default-disk=true
|
||||
```
|
||||
[1] https://longhorn.io/kb/tip-only-use-storage-on-a-set-of-nodes/
|
||||
|
||||
|
||||
#### Mit Helm:
|
||||
```bash
|
||||
helm repo add longhorn https://charts.longhorn.io
|
||||
helm install longhorn longhorn/longhorn --namespace longhorn-system --create-namespace --values longhorn-values.yaml
|
||||
```
|
||||
|
||||
#### Adding additional disks
|
||||
https://medium.com/btech-engineering/longhorn-storage-solution-for-kubernetes-cluster-645bc1b98a5e
|
||||
|
||||
Add disk in Proxmox, which appears as:
|
||||
Run in worker node:
|
||||
|
||||
```
|
||||
$ lsblk
|
||||
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
|
||||
sda 8:0 0 30G 0 disk
|
||||
├─sda1 8:1 0 29G 0 part /
|
||||
├─sda14 8:14 0 4M 0 part
|
||||
├─sda15 8:15 0 106M 0 part /boot/efi
|
||||
└─sda16 259:0 0 913M 0 part /boot
|
||||
sdb 8:16 0 250G 0 disk
|
||||
sr0 11:0 1 4M 0 rom
|
||||
```
|
||||
|
||||
SDB...
|
||||
|
||||
```
|
||||
fdisk /dev/sdb
|
||||
|
||||
# Hit n(new), p(primary), Enter, Enter
|
||||
# w(write to disk and exit)
|
||||
|
||||
mkfs.ext4 /dev/sdb1
|
||||
|
||||
mkdir /mnt/nvmedisk1
|
||||
|
||||
nano /etc/fstab
|
||||
->
|
||||
/dev/sdb1 /mnt/nvmedisk1 ext4
|
||||
|
||||
systemctl daemon-reload
|
||||
|
||||
mount -a
|
||||
```
|
||||
|
||||
|
||||
### Check via UI
|
||||
```
|
||||
k port-forward svc/longhorn-frontend 8000:80 -n longhorn-system
|
||||
```
|
||||
|
||||
### 2. Erstelle ein PersistentVolume (PV) und ein PersistentVolumeClaim (PVC)
|
||||
|
||||
Nun kannst du ein Persistent Volume (PV) und einen Persistent Volume Claim (PVC) für den Longhorn-Storage erstellen.
|
||||
|
||||
#### PV für Longhorn:
|
||||
Erstelle eine YAML-Datei (z.B. `longhorn-pv.yaml`), um ein Persistent Volume zu definieren:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolume
|
||||
metadata:
|
||||
name: longhorn-pv
|
||||
spec:
|
||||
capacity:
|
||||
storage: 10Gi # Setze die gewünschte Speichergröße
|
||||
volumeMode: Filesystem
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
persistentVolumeReclaimPolicy: Retain # Optionally, 'Delete' oder 'Recycle'
|
||||
storageClassName: longhorn # Verwende den Longhorn-StorageClass-Namen
|
||||
csi:
|
||||
driver: driver.longhorn.io # Der Longhorn CSI-Treiber
|
||||
volumeHandle: longhorn-pv # Ein eindeutiger Bezeichner für das Volume
|
||||
fsType: ext4 # Dateisystemtyp
|
||||
```
|
||||
|
||||
#### PVC für Longhorn:
|
||||
Erstelle eine YAML-Datei für den Persistent Volume Claim (z.B. `longhorn-pvc.yaml`), um den PV zu beanspruchen:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: longhorn-pvc
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 10Gi # Die angeforderte Größe sollte mit der des PV übereinstimmen
|
||||
storageClassName: longhorn # Die gleiche StorageClass wie im PV
|
||||
volumeName: longhorn-pv # Der Name des PV, das für diesen PVC verwendet werden soll
|
||||
```
|
||||
|
||||
### 3. Anwenden der YAML-Dateien
|
||||
|
||||
Nachdem du die YAML-Dateien erstellt hast, kannst du sie auf deinem Kubernetes-Cluster anwenden:
|
||||
|
||||
```bash
|
||||
kubectl apply -f longhorn-pv.yaml
|
||||
kubectl apply -f longhorn-pvc.yaml
|
||||
```
|
||||
|
||||
### 4. Überprüfen des Status
|
||||
|
||||
Prüfe den Status der Persistent Volumes und Persistent Volume Claims:
|
||||
|
||||
```bash
|
||||
kubectl get pv
|
||||
kubectl get pvc
|
||||
```
|
||||
|
||||
Das PV sollte nun mit dem PVC verbunden sein, und du kannst den Longhorn-Storage verwenden.
|
||||
|
||||
### 5. Beispiel: Pod mit PVC verwenden
|
||||
|
||||
Nun kannst du einen Pod erstellen, der das PVC nutzt:
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: longhorn-demo
|
||||
spec:
|
||||
containers:
|
||||
- name: demo-container
|
||||
image: nginx
|
||||
volumeMounts:
|
||||
- mountPath: /usr/share/nginx/html
|
||||
name: longhorn-volume
|
||||
volumes:
|
||||
- name: longhorn-volume
|
||||
persistentVolumeClaim:
|
||||
claimName: longhorn-pvc
|
||||
```
|
||||
|
||||
Erstelle den Pod mit:
|
||||
|
||||
```bash
|
||||
kubectl apply -f pod.yaml
|
||||
```
|
||||
|
||||
Dieser Pod wird nun das Persistent Volume, das du mit Longhorn erstellt hast, als Speicher verwenden.
|
||||
|
||||
### Fazit:
|
||||
Mit diesen Schritten hast du ein Persistent Volume (PV) und einen Persistent Volume Claim (PVC) für den Longhorn-Storage Provider erstellt. Du kannst das Volume nun in deinen Pods nutzen.
|
||||
|
||||
|
||||
|
||||
## Disable Localpath as default
|
||||
```
|
||||
kubectl get storageclass
|
||||
|
||||
kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}'
|
||||
```
|
||||
1
Longhorn/auth
Normal file
1
Longhorn/auth
Normal file
@@ -0,0 +1 @@
|
||||
basti:$apr1$N23gJpBe$CYlDcwTfp8YsQMq0UcADQ0
|
||||
61
Longhorn/justfile
Normal file
61
Longhorn/justfile
Normal file
@@ -0,0 +1,61 @@
|
||||
set fallback:=true
|
||||
|
||||
export LONGHORN_NAMESPACE := env("LONGHORN_NAMESPACE","longhorn-system")
|
||||
|
||||
add-helm-repo:
|
||||
helm repo add longhorn https://charts.longhorn.io --force-update
|
||||
helm repo update
|
||||
|
||||
# Delete namespace
|
||||
delete-namespace:
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
if kubectl get namespace ${LONGHORN_NAMESPACE} &>/dev/null; then
|
||||
kubectl delete namespace ${LONGHORN_NAMESPACE} --ignore-not-found
|
||||
else
|
||||
echo "Namespace ${LONGHORN_NAMESPACE} does not exist."
|
||||
fi
|
||||
|
||||
|
||||
install:
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
just env::check
|
||||
|
||||
just add-helm-repo
|
||||
|
||||
helm upgrade longhorn longhorn/longhorn \
|
||||
--install \
|
||||
--cleanup-on-fail \
|
||||
--namespace ${LONGHORN_NAMESPACE} \
|
||||
--create-namespace
|
||||
|
||||
uninstall:
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
for crd in $(kubectl get crd -o name | grep longhorn); do
|
||||
kubectl patch $crd -p '{"metadata":{"finalizers":[]}}' --type=merge
|
||||
done
|
||||
|
||||
kubectl -n ${LONGHORN_NAMESPACE} patch -p '{"value": "true"}' --type=merge lhs deleting-confirmation-flag || true
|
||||
|
||||
helm uninstall longhorn --namespace ${LONGHORN_NAMESPACE} || true
|
||||
just delete-namespace
|
||||
|
||||
|
||||
install-dashboard-ingress:
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
just env::check
|
||||
|
||||
echo "Deploying Longhorn Dashboard Ingress with EXTERNAL_DOMAIN=${EXTERNAL_DOMAIN}"
|
||||
gomplate -f longhorn-certificate-gomplate.yaml | kubectl apply -f -
|
||||
gomplate -f longhorn-ingressroute-gomplate.yaml | kubectl apply -f -
|
||||
|
||||
uninstall-dashboard-ingress:
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
kubectl delete -f longhorn-ingressroute-gomplate.yaml || true
|
||||
kubectl delete -f longhorn-certificate-gomplate.yaml || true
|
||||
13
Longhorn/longhorn-certificate-gomplate.yaml
Normal file
13
Longhorn/longhorn-certificate-gomplate.yaml
Normal file
@@ -0,0 +1,13 @@
|
||||
---
|
||||
apiVersion: cert-manager.io/v1
|
||||
kind: Certificate
|
||||
metadata:
|
||||
name: longhorn-web-ui-cert
|
||||
namespace: longhorn-system
|
||||
spec:
|
||||
secretName: longhorn-web-ui-tls
|
||||
dnsNames:
|
||||
- longhorn-dashboard.{{.Env.EXTERNAL_DOMAIN}}
|
||||
issuerRef:
|
||||
name: cloudflare-cluster-issuer
|
||||
kind: ClusterIssuer
|
||||
16
Longhorn/longhorn-ingressroute-gomplate.yaml
Normal file
16
Longhorn/longhorn-ingressroute-gomplate.yaml
Normal file
@@ -0,0 +1,16 @@
|
||||
apiVersion: traefik.io/v1alpha1
|
||||
kind: IngressRoute
|
||||
metadata:
|
||||
name: longhorn-ingress-route
|
||||
namespace: longhorn-system
|
||||
spec:
|
||||
entryPoints:
|
||||
- websecure
|
||||
routes:
|
||||
- match: Host(`longhorn-dashboard.{{.Env.EXTERNAL_DOMAIN}}`)
|
||||
kind: Rule
|
||||
services:
|
||||
- name: longhorn-frontend
|
||||
port: 80
|
||||
tls:
|
||||
secretName: longhorn-web-ui-tls
|
||||
36
Longhorn/longhorn-values.yaml
Normal file
36
Longhorn/longhorn-values.yaml
Normal file
@@ -0,0 +1,36 @@
|
||||
global:
|
||||
nodeSelector:
|
||||
node.longhorn.io/create-default-disk: "true"
|
||||
|
||||
service:
|
||||
ui:
|
||||
type: NodePort
|
||||
nodePort: 30050
|
||||
manager:
|
||||
type: ClusterIP
|
||||
|
||||
# Replica count for the default Longhorn StorageClass.
|
||||
persistence:
|
||||
defaultClass: false
|
||||
defaultFsType: ext4
|
||||
defaultClassReplicaCount: 2
|
||||
reclaimPolicy: Delete
|
||||
|
||||
# Replica counts for CSI Attacher, Provisioner, Resizer, Snapshotter
|
||||
csi:
|
||||
attacherReplicaCount: 2
|
||||
provisionerReplicaCount: 2
|
||||
resizerReplicaCount: 2
|
||||
snapshotterReplicaCount: 2
|
||||
|
||||
# Default replica count and storage path
|
||||
defaultSettings:
|
||||
upgradeChecker: false
|
||||
kubernetesClusterAutoscalerEnabled: false
|
||||
allowCollectingLonghornUsageMetrics: false
|
||||
createDefaultDiskLabeledNodes: true
|
||||
defaultReplicaCount: 2
|
||||
defaultDataPath: "/k8s-data"
|
||||
|
||||
longhornUI:
|
||||
replicas: 1
|
||||
Reference in New Issue
Block a user