🔢Refresh stored datasets

Note

Dataset refresh operations only apply to stored datasets.

Live datasets are, by default, "fresh" datasets computed each time a user consults a visualization or a dataset depending on it.

On this page, you will learn how to:

  • Refresh stored datasets:

    • Manually (directly within the DataHub, in the "Datasets" tab)

    • Automatically (through automation)

  • Publish data

    • All datasets

    • Specific datasets

Refreshing datasets manually

Creating and refreshing a stored dataset

Each time you create a new stored dataset, you can "Save" it or "Save and Refresh" it.

Save modal with choice to "Save" or "Save and Refresh"
"Save" and "Save and Refresh" options

If you choose "Save," you will see a pending refresh icon in the Dataset tab of the DataHub. If you want to refresh it, simply click on that button.

Pending Refresh Icon
Pending Refresh

Hint

When you choose to "Save and Refresh," the dataset and all its parent datasets will be refreshed. If you have heavy parent datasets, you might be exposed to a long refresh time before being able to continue working on your app data.

To avoid that, you can simply use "Save" and Refresh the dataset or all datasets later.

Refreshing a dataset and its parent datasets

To refresh a dataset, and all its parent datasets, you can click on the refresh icon in the dataset list (if no refresh is pending, the button is grey)

Refreshing all datasets

To refresh all datasets in your app, click on the "Publish" dropdown and select "Refresh Data."

Refresh all dataset option
Refresh all datasets

Refreshing datasets automatically

Toucan provides automation tools, including a Refresh scheduler. To use it:

  • Go to the "Automations" interface (accessible in the Settings on top-right of your application)

  • Click on "Add an action"

  • Give a name to the automation

  • Choose "Refresh Data"

  • Set the automation frequency

  • Choose whether the creator and/or the admin should be notified about a refresh failure.

Data Scheduler interface
Data Scheduler interface

Refresh Status

The dot in the navigation panel indicates the status of the refresh

  • Blue dot: the refresh is in progress

  • Green dot: the refresh has succeeded, and data in staging is up to date

  • Red dot: there is an error. To get more details on the error, click the dot to see operation history details. Don't hesitate to reach out to our support team - [email protected] - to get assistance on errors you don't understand.

Green dot refresh status
Refresh status

Publishing all datasets

To publish your data so end users can see them in Production, you can either:

  • Use the "Publish" button (within the app menu): it will publish the data and layout design changes

  • Only publish data by using "Publish Data" in the the "Publish" dropdown button (-> publish Options)

Publish operations pop over
Publish operations pop over

Publishing a specific dataset

To publish a specific dataset into production mode, click on "Publish" on the action menu of your dataset within the data hub.

What happened if a refresh job failed?

Handling Refresh Job Failures

When a refresh job fails, the system implements two key notification mechanisms to promptly alert the necessary personnel and provide detailed information for troubleshooting.

Notifications to App Administrator

  • Email Alert: The administrator of the application is immediately notified via an email alert. This email contains information regarding the failed refresh job, including the time of failure and potential reasons, equipping the administrator with the necessary details to diagnose and rectify the issue.

Indications in the Operation Console

  • Error Message Display: Within the operation console, a failure indication is prominently displayed. This indication features a detailed error message within the error object. The message aims to clearly articulate the underlying reasons for the refresh failure, enabling swift actions for resolution by the technical team or administrator.

These measures ensure that refresh job failures are promptly communicated and addressed, minimizing downtime and maintaining system reliability.

What are the consequences if a refresh job failed?

In the event that the refresh job fails, the data remains unchanged from its previous state. Essentially, without the refresh process completing successfully, the data will not be updated, and as such, it retains the same status it had prior to the attempted refresh. Consequently, the data will not reflect any recent changes or updates that were intended to be applied during the refresh. This lack of update means that the data continues to operate or be viewed under its earlier conditions, without benefiting from any modifications or new inputs that would have been introduced had the refresh process occurred as planned.

Last updated

Was this helpful?