Dump structure

Every time you make a dump using CleverMaps Shell, a dump directory structure is created.

All dumped metadata objects are saved to the file system in a wrapper, which contains detailed metadata about the object itself.

In addition, dump metadata (or "metadata about the metadata") file is created. It is used to store useful information, like dump time, or the original project ID. It is also used to store the dump content and to track local file changes.

These directory structures and files are generated by Shell and should not be modified.

Dump structure

Dump directory structure is created either in:

  1. the default directory you specified in the config file (recommended)

  2. working directory, e.g. the directory you run Shell from

Let's assume that in this case, /var/local/projects is our default directory. The dump structure is following:

1 /var/local/projects/{projectId}/

So, in practice:

1 /var/local/projects/q1zdp9d0ao78rdv5/

If we list the contents of this dump directory, we see data and metadata directories, and the dumpMetadata.json file (described below).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 ./ ../ data/ dumpMetadata.json metadata/ metadata/dashboards/ metadata/dataPermissions/ metadata/datasets/ metadata/exports/ metadata/indicatorDrills/ metadata/indicators/ metadata/markers/ metadata/markerSelectors/ metadata/metrics/ metadata/projectSettings/ metadata/views/

The data folder contains .csv files. The metadata folder contains subdirectories of the metadata object types. Each dumped file is wrapped in a dump object wrapper.

Dump object wrapper

Simple wrapper, which contains metadata about the dumped object.

Key

Type

Optionality

Description

Key

Type

Optionality

Description

url

string

generated

URL link uniquely identifying the object

dumpTime

string

generated

timestamp of when the object was dumped

version

string

generated

version of the object

starts at 1, is incremented everytime the object is pushed to the project

content

object

generated

the object itself

may contain valid content of one of metadata object types

Dump metadata file

The dumpMetadata.json file is located the dump directory, one level above the data and metadata directories. It contains metadata about the dump itself, and list of its contents.

Key

Type

Optionality

Description

Key

Type

Optionality

Description

server

string

generated

server, from where the project was dumped

projectId

string

generated

ID of the project

projectTitle

string

generated

human-readable title of the project

dumpTime

string

generated

timestamp of when the project was last dumped or fetched

user

string

generated

email of the user who performed the dump

metadataContent

object

optional

list of metadata object files contained in the dump, with their respective md5 hashes (used for local file modification detection)

dataContent

object

optional

like metadataContent, but with .csv files