diff --git a/querybook/README.md b/querybook/README.md index d14efa7..1ad6000 100644 --- a/querybook/README.md +++ b/querybook/README.md @@ -101,7 +101,7 @@ just querybook::upgrade ### Building Custom Image -To build a custom Querybook image with `sqlalchemy-trino` support: +To build a custom Querybook image with bug fixes and `sqlalchemy-trino` support: 1. **Clone Querybook repository**: @@ -110,31 +110,60 @@ To build a custom Querybook image with `sqlalchemy-trino` support: cd querybook ``` -2. **Create requirements/local.txt**: +2. **Apply bug fix patch**: + + ```bash + # Copy patch file from buun-stack repository + # cp /path/to/buun-stack/querybook/querybook-fix-socketio-disconnect.diff . + + # Apply the patch + git apply querybook-fix-socketio-disconnect.diff + ``` + + **Patch includes**: + - Fix for WebSocket disconnect handler signature (python-socketio 5.12.0+ compatibility) + +3. **Create requirements/local.txt**: ```bash cat > requirements/local.txt <=0.3.0 requires trino>=0.310 (incompatible) + # - Both must be explicitly pinned to prevent pip from upgrading them when extra.txt is installed + trino==0.305.0 + sqlalchemy-trino==0.2.2 EOF ``` -3. **Build the Docker image**: + **Critical**: Both packages must be pinned: + - `trino==0.305.0` prevents pip from upgrading to 0.310+ when resolving dependencies + - `sqlalchemy-trino==0.2.2` is the only version compatible with trino 0.305 + - When `EXTRA_PIP_INSTALLS=extra.txt` is used, pip installs many packages which can trigger dependency upgrades + - Without explicitly pinning trino, pip may upgrade it to satisfy other package requirements, breaking query execution + +4. **Build the Docker image**: ```bash # For remote Docker host (e.g., k3s node) DOCKER_HOST=ssh://yourdomain.com docker build \ + --no-cache \ --build-arg EXTRA_PIP_INSTALLS=extra.txt \ -t localhost:30500/querybook:trino-metastore . # For local Docker docker build \ + --no-cache \ --build-arg EXTRA_PIP_INSTALLS=extra.txt \ -t localhost:30500/querybook:trino-metastore . ``` -4. **Push to registry**: + **Important**: Use `--no-cache` when changing `requirements/local.txt` to ensure pip installs the correct package versions. Docker layer caching can cause pip to reuse old dependency resolutions. + +5. **Push to registry**: ```bash DOCKER_HOST=ssh://yourdomain.com docker push localhost:30500/querybook:trino-metastore @@ -142,7 +171,7 @@ To build a custom Querybook image with `sqlalchemy-trino` support: docker push localhost:30500/querybook:trino-metastore ``` -5. **Deploy to Kubernetes**: +6. **Deploy to Kubernetes**: ```bash export QUERYBOOK_CUSTOM_IMAGE=localhost:30500/querybook @@ -150,6 +179,28 @@ To build a custom Querybook image with `sqlalchemy-trino` support: just querybook::upgrade ``` +7. **Restart Pods to use new image**: + + ```bash + # Delete all Querybook pods to force image pull + kubectl delete pod -n querybook -l app=querybook + + # Wait for pods to be ready + kubectl wait --for=condition=ready pod -l app=querybook -n querybook --timeout=120s + + # Verify correct package versions + kubectl exec -n querybook deployment/worker -- pip show trino sqlalchemy-trino | grep -E "Name:|Version:" + ``` + + Expected output: + + ``` + Name: trino + Version: 0.305.0 + Name: sqlalchemy-trino + Version: 0.2.2 + ``` + **Notes**: - The Dockerfile automatically includes `requirements/local.txt` if it exists (lines 40-42) diff --git a/querybook/querybook-fix-socketio-disconnect.diff b/querybook/querybook-fix-socketio-disconnect.diff new file mode 100644 index 0000000..b377f35 --- /dev/null +++ b/querybook/querybook-fix-socketio-disconnect.diff @@ -0,0 +1,26 @@ +diff --git a/querybook/server/datasources_socketio/datadoc.py b/querybook/server/datasources_socketio/datadoc.py +index d7455cd9..2f41e7a2 100644 +--- a/querybook/server/datasources_socketio/datadoc.py ++++ b/querybook/server/datasources_socketio/datadoc.py +@@ -165,7 +165,7 @@ def on_leave_room(data_doc_id): + + + @register_socket("disconnect", namespace=DATA_DOC_NAMESPACE) +-def disconnect(): ++def disconnect(*args, **kwargs): + data_doc_ids = rooms(request.sid, namespace=DATA_DOC_NAMESPACE) + for data_doc_id in data_doc_ids: + leave_room(data_doc_id) +diff --git a/querybook/server/datasources_socketio/query_execution.py b/querybook/server/datasources_socketio/query_execution.py +index 9c6a2f8a..7b3668db 100644 +--- a/querybook/server/datasources_socketio/query_execution.py ++++ b/querybook/server/datasources_socketio/query_execution.py +@@ -65,7 +65,7 @@ def on_leave_room(query_execution_id): + + + @register_socket("disconnect", namespace=QUERY_EXECUTION_NAMESPACE) +-def disconnect(): ++def disconnect(*args, **kwargs): + query_execution_ids = rooms(request.sid, namespace=QUERY_EXECUTION_NAMESPACE) + for query_execution_id in query_execution_ids: + leave_room(query_execution_id)