# Upgrading This document describes how to upgrade the Toucan Helm chart between breaking changes. If you wish to consult, how to upgrade in general, you can consult this document: {% content-ref url="/pages/OaYuokNnRkeXdlo1PQot" %} [Upgrades and rollbacks](/self-hosted-toucan/maintenance/updates.md) {% endcontent-ref %} ## v3 to v4 ### Newly added `oauth-encryption-secret` to dataexecution #### Context The data-execution service now supports HTTP API protected by OAuth2. Because of this, the data-execution service now requires the `oauth-encryption-secret` to be provided. It is a random 32 characters string. #### Conditions If `dataexecution.secrets.oauthEncryptionSecret` or `dataexecution.secrets.existingSecret` is not provided, the Helm Chart upgrade will fail. #### Procedure Here's your options: * **Set the secret in the `--set` parameter and forget it**: Generate the secret with `openssl rand -hex 16` and run `helm upgrade --set dataexecution.secrets.oauthEncryptionSecret=...`. * OR, **set the secret explicitely in the `values.override.yaml`**: ```yaml dataexecution: secrets: oauthEncryptionSecret: '' ``` * OR, **provide an existing secret**: ```yaml dataexecution: secrets: existingSecret: 'my-secret' ``` In this case, the secret will be picked up from the `my-secret` secret. It must contain the key `dataexecution-oauth-encryption-secret`. ## v2 to v3 ### Moving from NGINX to Canopee #### Context We've moved away from NGINX due to maintainability issues with the authentication module. Due to the complexity of the NGINX C library causing various bugs, we've built our own Rust-based reverse proxy called Canopee. This new reverse proxy has the same features as NGINX but it's much simpler and easier to maintain. #### Conditions Check the values to see if you are affected by the migration. If it contains `nginx`, you are affected and the `helm install` will fail. #### Procedure If you just have the ingress configured, the migration is simply replacing `nginx` with `canopee`: ```diff -nginx: +canopee: ingress: ... ``` If you've overridden the `.nginx.tucana.config.tcParams` configuration, the Helm Chart will now take priority over the value you've set. Meaning, setting: ```yaml canopee: tucana: config: tcParams: API_BASEROUTE: something ``` won't change anything. However, adding additional values to the `.tcParams` will indeed work: ```yaml canopee: tucana: config: tcParams: ADDITIONAL: something # This will be added ``` If you wish to totally override the configuration, you'll need to replace the default template: ```yaml canopee: tucana: mergeConfigmap: something-else-config # With "something-else-config" being a configmap that contains: # apiVersion: v1 # kind: ConfigMap # metadata: # name: something-else-config # data: # params.json: | # { # "API_BASEROUTE": "https://something-else.com/api/legacy", # ... # } ``` To know the required content of the configmap, check the Helm Chart's `templates/canopee/tucana-configmap.yaml` (you'll need to run `helm pull` to download the Helm Chart's source code). ## v1 to v2 ### Moving from Laputa to Garage #### Context Garage (Self-hosted S3 storage provider) is now deployed as part of the Helm Charts. The storage is still attached to Laputa, but, you'll need to migrate the data from Laputa to Garage. Instead of doing it manually, we provide a sidecar container to transfer the data automatically. **By default, the migration is disabled.** #### Conditions To check if you are affected by the migration, as an admin, run the following commands: ```shell kubectl exec -it -n toucan-stack-laputa-0 -c laputa -- bash find storage/ -type f ``` If there are any files in the storage directory, you are affected. #### Procedure Follow the following instructions: 1. Edit the `values.override.yaml` to add the new secrets: ```yaml # Run `openssl rand -hex 32` to generate a secret global: s3: keys: dataexecution: secret: '' # OR a secret: existingSecret: name: '' key: 'dataexecution-secret-key' toucan: secret: '' # OR a secret: existingSecret: name: '' key: 'toucan-secret-key' toucan_ro: secret: '' # OR a secret: existingSecret: name: '' key: 'toucan-ro-secret-key' garage: secrets: rpc: '' admin: '' # OR a secret that contains the keys: # - garage-rpc-secret # - garage-admin-secret existingSecret: '' laputa: s3migration: enabled: true ``` 2. Deploy the upgrade: ```shell helm upgrade -n toucan-stack oci://quay.io/toucantoco/charts/toucan-stack -f ./values.override.yaml ``` This will deploy the data-execution service, the s3 storage provider and change features flags on the services. 3. Check the logs of the sidecar container to see if the migration was successful. You can also check the sidecar logger container. ```shell # Sidecar handling the migration kubectl logs -n -c laputa-migration-to-hades # Sidecar logger kubectl logs -n -c tail-worker # Search for "copy_datasources_to_s3_job" ``` ### SpiceDB certificates are now handled internally #### Context The certificates for SpiceDB are now handled internally inside the Helm Charts instead of using `extraVolumes` and `extraVolumeMounts`. #### Conditions To check if you are affected by the migration, check the `values.override.yaml`, if you used `extraVolumes` and `extraVolumeMounts` with `spicedb-certs` for `dataset`, `layout` and `laputa`. #### Procedure Remove the following sections for `dataset`, `layout` and `laputa` in your `values.override.yaml`: ```diff dataset: extraVolumes: - - name: spicedb-certs - secret: - secretName: '{{ template "toucan-stack.spicedb.tls.secretName" . }}' - items: - - key: ca.crt - path: ca.crt - name: my-certs # ... extraVolumeMounts: - - name: spicedb-certs - mountPath: /spicedb-certs - name: my-certs # ... ``` ### Custom certificates for Laputa now requires to be mounted elsewhere #### Context Because the embedded S3 provider requires a custom certificate, the `ca-certificates.crt` must contain the CA certificate of the S3 provider. #### Conditions To check if you are affected by the migration, check the `values.override.yaml` for `extraVolumes` and `extraVolumeMounts` of `laputa`. If you have mounted a CA certificate for Laputa, you'll need to update the values. If you don't update the values, Laputa won't be able to connect to the S3 provider. #### Procedure Edit the `values.override.yaml` to move your CA certificate: ```diff laputa: extraVolumes: - name: ca-bundle secret: secretName: 'my-ca-secret' items: - key: ca.crt - path: ca-certificates.crt + path: my-ca.crt extraVolumeMounts: - name: ca-bundle - mountPath: /etc/ssl/certs + mountPath: /usr/local/share/ca-certificates/my-ca.crt + subPath: my-ca.crt ``` Laputa will naturally combine the certificates. You also need to add your CA certificates to the `dataexecution` service: ```yaml dataexecution: api: extraVolumes: - name: ca-bundle secret: secretName: 'my-ca-secret' items: - key: tls.crt path: my-ca.crt extraVolumeMounts: - name: ca-bundle mountPath: /usr/local/share/ca-certificates/my-ca.crt subPath: my-ca.crt worker: extraVolumes: - name: ca-bundle secret: secretName: 'my-ca-secret' items: - key: tls.crt path: my-ca.crt extraVolumeMounts: - name: ca-bundle mountPath: /usr/local/share/ca-certificates/my-ca.crt subPath: my-ca.crt ``` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs-v3.toucantoco.com/self-hosted-toucan/upgrading.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.