Dump structure

Every time you make a dump using CleverMaps Shell, a dump directory structure is created. Every dump is uniquely identified by its dump ID - a time stamp of when the dump was created. Format of the dump ID is "yyyy-MM-dd_HH-mm-ss".

All dumped metadata objects are saved to the file system in a wrapper, which contains detailed metadata about the object itself.

In addition, dump metadata (or "metadata about the metadata") file is created. It is used to store useful information, like dump time, or the original project ID. It is also used to store the dump content and to track local file changes.

These directory structures and files are generated by Shell and should not be modified.

Dump structure

Dump directory structure is created either in:

  1. the default directory you specified in the config file (recommended)
  2. working directory, e.g. the directory you run Shell from
  3. the directory you specify as a value of --directory parameter of the dumpProject command

Let's assume that in this case, /var/local/metadata is our default directory. The dump structure is following:

/var/local/metadata/{projectId}/{dumpId}/

So, in practice:

/var/local/metadata/q1zdp9d0ao78rdv5/2017-08-28_20-17-51/

If we list the contents of this dump directory, we see data and metadata directories, and the dumpMetadata.json file (described below).

./
../
data/
dumpMetadata.json
metadata/
metadata/dashboards/
metadata/dataPermissions/
metadata/datasets/
metadata/exports/
metadata/indicatorDrills/
metadata/indicators/
metadata/markers/
metadata/markerSelectors/
metadata/metrics/
metadata/projectSettings/
metadata/views/

The data folder contains .csv files. The metadata folder contains subdirectories of the metadata object types. Each dumped file is wrapped in a dump object wrapper.

Dump object wrapper

Simple wrapper, which contains metadata about the dumped object.

KeyTypeOptionalityDescription
urlstring

GENERATED

URL link uniquely identifying the object

dumpTimestring

GENERATED

timestamp of when the object was dumped

versionstring

GENERATED

version of the object

starts at 1, is incremented everytime the object is pushed to the project

contentobject

GENERATED

the object itself

may contain valid content of one of metadata object types

Dump metadata file

The dumpMetadata.json file is located the dump directory, one level above the data and metadata directories. It contains metadata about the dump itself, and list of its contents.

KeyTypeOptionalityDescription
dumpIdstring

GENERATED

ID of the dump

serverstring

GENERATED

server, from where the project was dumped

projectIdstring

GENERATED

ID of the project

projectTitlestring

GENERATED

human-readable title of the project

dumpTimestringGENERATEDtimestamp of when the project was dumped
userstringGENERATEDemail of the user who performed the dump
metadataContentobjectOPTIONALlist of metadata object files contained in the dump, with their respective md5 hashes (used for local file modification detection)
dataContentobjectOPTIONALlike metadataContent, but with .csv files