🚂New Data Execution System

This page explains how the new Data Execution system works

Toucan Highly Available Data Execution System (HADES) is the result of two years of R&D to provide the most efficient data execution layer for an embedded analytics tool relying on the latest technologies available. HADES improves data execution efficiency.

General

HADES is a multi-tenant service dedicated to executing data queries.

Its mission is to

  • receive data pipelines,

  • process data pipelines, and

  • deliver the results efficiently.

HADES : simplified schema of HADES interactions with other services

HADES is designed to address the limitations of our previous execution system.

It results in better rendering times for data-intensive dashboards (see Better than ever results in our blog post dedicated to HADES) and a smoother data preparation experience.

Our new Data Execution System enables more users to interact with their analytics dashboards simultaneously without experiencing slowdowns through improved handling of concurrent queries.

HADES ensures a faster experience for builders and viewers who have already accessed the same dashboard or underlying queries.

This system still maintains strict data isolation between customers with dedicated S3 buckets to each customer.

It also leverages on multi-tenant architecture benefits with a better peak management and high availability.

Data pipeline processing

HADES natively implements hybrid pipelines, to seamlessly blend on-source and in-memory data pipelines processing. We determine the optimal execution engine for each step of a data transformation pipeline prioritizing execution directly on the source database when possible, and switching to in-memory processing for unsupported operations.

How to test the new service

If you use a supported connector (see release note) want to test our new service, you can contact your CSM or our support.

IP Allowlisting

Our new service is accessible to an unique IP (EU area) : 51.15.128.70 Whitelist your backend IP as well.

Last updated

Was this helpful?