🧑‍🍳YouPrep™ Native SQL

Overview

YouPrep™ Native SQL is our powerful option that allows to leverage all the power of your Datawarehouse or database, by delegating YouPrep transformation steps directly to your datasource.

How does it work? We translate your transformation pipeline made with YouPrep directly into one SQL statement. It's very powerful, specially when you must deal with a lot of data within your tables.

The NativeSQL option is available for several connectors:

  • Athena

  • BigQuery

  • PostgreSQL

  • Redshift

  • Snowflake

Step coverage

Find below the availability of YouPrep steps under Native SQL:

StepPostgreSQLSnowflakeRedshiftBigQueryAthena

Add text column

Add total rows

Aggregate

Append

Compare text columns

Argmax

Argmin

Concatenate columns

Convert

Convert date to text

Convert text to date

Cumulated sum

Delete columns

Geographically Disolve

Duplicate

Duration

Evolution

Extract date information

Extract substring

Fill null

Filter

Formula

Geographical hierarchy

Get unique group(s) / value(s)

Hierarchical rollup

IfThenElse

Join

Keep column(s)

Moving average

Percentage

Pivot

Rank

Rename

Replace

Geographical Simplification

Sort

Split column

Columns statistics

To lowercase

To Uppercase

Top N rows

Unpivot

Waterfall

Hybrid Pipeline

Beta feature since 2024 Releasesv122 (need to be activated at the instance level)

To address the issues caused by the lack of support of some steps, we have implemented a hybrid pipeline approach.

This means you can have a unique pipeline mixing NativeSQL supported steps and unsupported steps. All transformation steps can be executed regardless of context, with the flexibility to execute the pipeline partially in NativeSQL and partially in Toucan, depending on the steps utilized.

NativeSQL steps are executed first whenever possible. An unsupported step causes the Toucan engine to take over the pipeline.

How to activate the NativeSQL option

This option is activated by default on Toucan.

Warning

This option is activated at the instance level, which means that it will be applied for all your Apps.

Some limitations

Note

Data transformation made with YouPrep on columns must be explicit (on a precise column). It's not possible to make a data transformation step on a non-predefined column (for example by referring to the column though the usage of a variable).

Last updated