Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Datasets describe the data model of a project. Each dataset represents a database table. They also define the constraints and links between all other datasets in a project.

There are three types of datasets:

  • dwh (

    Image Removed

    (blue star)) type is a data warehouse service dataset type, it references a table in a relational dwh database, which may contain various kinds of data (e.g. orders, customers, administrative units...)

  • vt (

    Image Removed

    (blue star)) type is a vector tile dataset type, it references a vector tile service, which serves vector tiles that are displayed over the base map (CleverMaps vector tiles are hosted on Mapbox)

  • h3Grid (

    Image Removed

    (blue star)) type is a H3 grid dataset, it represents a grid visualization, where the geometries are generated on the fly by the application (see Tutorial 8 for more info, and H3 grid spatial index by Uber)

Datasets of dwh type also have a subtype, which defines the type of data they contain. There are five of them:

  • basic (

    Image Removed

    (blue star)) subtype is used to contain data which are not to be visualised on a map (e.g. orders, clients, or demography)

  • geometryPolygon (

    Image Removed

    (blue star)) subtype contains data which represent polygons on a map (e.g. administrative units, delivery zones)

  • geometryPoint (

    Image Removed

    (blue star)) subtype contains data which represent specific points on a map (e.g. shops, postcodes, or POIs)

  • geometryLine (

    Image Removed

    (blue star)) subtype contains data which represent lines on a map (e.g. routes, pipelines)

  • date (

    Image Removed

    (blue star)) subtype is a special subtype not to be used in a regular way, it is used to describe datasets the can-dim-dates data dimension

Datasets also subject to various validation checks. Datasets with ref.subtype other than basicgeometryPolygon, geometryPointgeometryLine and date must also contain properties.featureTitle key, which defines the format of visualisation of one row of the dataset (used in tooltips or the date picker). Datasets with geometry must have a link to a vt type dataset describing that geometry. More about these checks in the examples below.

...

The dataset also contains information about its data sources. The source data for UK districts was provided by Office for National Statistics under the Open Government license. For more info, see the syntax below.

Info

Note the reference to a vt dataset named districtgeojson in ref.properties.geometry key. This dataset must exist in the project before we add the geometryPolygon one.

...

These properties define the content of a tooltip shown on marker/area hover (example), and the details of a dataset (example).

The properties can be found on the common syntax level. The syntax example can be found in the "Example of a dwh dataset type with geometryPoint subtype" above.

...

Key

Type

Optionality

Description

Constraints

type

string

Status
colourRed
titlerequired

type identifier, always dwh

[dwh]

subtype

string

Status
colourRed
titlerequired

dwh subtype identifier

[basic, geometryPoint, geometryPolygon, geometryLine, date]

geometry

string

Status
colourYellow
titlevaries

geometry reference to a vector tile dataset

required only for geometryPolygon and geometryLine subtypes

{datasetName}

h3Geometries

array

Status
colourGreen
titleoptional

array of references to h3Grid dataset names (see example above)

enables the grid visualization


visualizations

array

Status
colourYellow
titlevaries

array of objects specifying the allowed visualizations of the dataset

required only for geometryPoint, geometryPolygon and geometryLine datasets


zoom

object

Status
colourGreen
titleoptional

map zoom object


table

string

Status
titleREAD-ONLY

name of the actual dwh table this dataset represents

it's derived from the name of the dataset, and _X postfix is added for each full load, where X is the number of the load

(a-z0-9_-)

primaryKey

string

Status
colourRed
titlerequired

primary key of the table - must be one of the dataset properties

should be unique

{datasetProperty}

categorizable

boolean

Status
colourGreen
titleoptional

indicates if the dataset is capable of being categorized in the Filters tool ()

default = true

should be true for fact datasets like "orders", "shops" or "clients", false for additional datasets like "postcodes", "dim_dates" or "price_categories"

[true, false]

fullTextIndex

boolean

Status
colourGreen
titleoptional

indicates if the dataset's data will be indexed for full text search in Search tool ()

default = true for geometry* subtypes, false for basic and date subtypes

[true, false]

properties

array

Status
colourRed
titlerequired

array of properties - database table column descriptions

dataset property order must be identical to the order of the data columns

size must be at least 1


...

For more info, see the additional syntax examples above.

Key

Type

Optionality

Description

Constraints

type

string

Status
colourRed
titlerequired

type of the visualization

[areas, grid, zones, dotmap, heatmap, line]

zoom

object

Status
colourRed
titlerequired

map zoom object


...

Key

Type

Optionality

Description

Constraints

name

string

Status
colourRed
titlerequired

name of the column as it will be referenced in other objects

(a-z0-9_-)

title

string

Status
colourRed
titlerequired

human-readable title of the column

(*)

column

string

Status
colourRed
titlerequired

name of the column as it is in the database table

(a-z0-9_-)

type

string

Status
colourRed
titlerequired

column data type

detailed description in the table below

[integer, bigint, decimal, string, boolean, date, latitude, longitude, h3_grid]

foreignKey

string

Status
colourYellow
titlevaries

name of the dataset, whose primaryKey is referenced by this foreignKey

if this key is mentioned, type must be string

(a-z0-9_-)

filterable

boolean

Status
colourGreen
titleoptional

indicates whether it's possible for this property to be added to a filter

default = true

[true, false]

description

string

Status
colourGreen
titleoptional

description of the property which appears in Filters ()

(*)

displayOptions

object

Status
colourGreen
titleoptional

options that specify how the property will be displayed


...

The displayOptions object allows you to specify how the property will be displayed. See the example above.

Key

Type

Optionality

Description

Constraints

valueOptions

array

Status
colourGreen
titleoptional

assigns colors to values of the property


...

Native datasets in this project are baskets, shops and clients. The other datasets were imported from different data dimensions. The datasets in orange were imported from the can-dim-dates dimension. Brown datasets are from a UK administrative units dimension. The dataset demography_postcode comes from a UK demography dimension. The h3Grid datasets are pink.

...

For geometryPoint and geometryPolygon datasets, a tooltip is shown on hover. The content is defined in dataset.properties.

...

"My Store: Grand Central" is the featureTitle.

...

The content of dataset details is defined in dataset.properties.featureAttributes. The details are shown on the dashboard, right under the indicator list.

...