Add missing dates

The Add missing dates step allows to add missing dates as new rows in a dates column. Exhaustive dates will range between the minimum and maximum date found in the dataset (or in each group if a group by logic is applied - see thereafter).

Added rows will be set to null in columns not referenced in the step configuration. You can use a “Fill null values” step afterwards if you need to fill those nulls.

You should make sure to use a group by logic if you want to add missing dates in independent groups of rows (e.g. you may need to add missing rows for every country found in a “COUNTRY” column). And you should ensure that every date is unique in every group of rows at the specified granularity, else you may get inconsistent results. You can specify “group by” columns in the Group by parameter as shown below. Please see example 2 below for illustration.

Step parameters

• Dates column column(string)*: the date column (must be of type date) with missing dates

• Dates granularity (string)*: the granularity of the date column (day, month or year)

• (Optional) Group bycolumn(array) (optional) : Use this option if you need to perform add missing dates in independent groups of rows. You should make sure that every date is unique inside each and every group. See example 2 below for illustration.

Example

Input

Date - Add missing dates

Configuration

{
  "dates_column": "DATE",
  "dates_granularity": "day",
  "group_by": []
}

Output

Date - Add missing dates output

The date of the January 6th, 2018 has been added to the dataset.