Command list

CleverMaps Shell is controlled solely by a set of predefined commands. These commands can be divided in 4 categories, or workflow states. All commands and parameters are case sensitive.

Each command can be further specified by parameters, which can have default values. Each parameter also has a "--" prefix, which is a technicality, and is not mentioned in the tables below, for the sake of readability.

Parameters with string type take string input as a value, if they are mentioned. Parameters with enum type have predefined enumeration of strings, which can be passed as a value. Parameters with boolean type can be passed either true, false or no value (=true).

Workflow states

  • Started - you have started the tool
  • Connected to server - you have successfully logged in to your account on a specific server
  • Opened project - you have opened a project you have access to
  • Opened dump - you have created a new dump, or opened an existing one

_

_

Started state

login

Log in as a user with correct credentials.

Parameter nameTypeOptionalityDescriptionConstraints

email

string
your email accountstored in credentials file
passwordstring
your passwordstored in credentials file
accessTokenstring

generated CleverMaps access token

(see how to get one)

stored in config file
serverstring

server to connect to

default = https://secure.clevermaps.io

stored in config file
proxyHoststring

proxy server hostname

stored in config file
proxyPortintegerproxy server portstored in config file
s3AccessKeyIdstring

AWS S3 Access Key ID

required for S3 upload (loadCsv --s3Uri)

stored in config file
s3SecretAccessKeystring

AWS S3 Secret Access Key

required for S3 upload (loadCsv --s3Uri)

stored in config file

setup

Store your config and credentials in a file so you don't have to specify them each time you log in.

Parameter nameTypeOptionalityDescriptionConstraints

email

string
your email accountstored in credentials file
passwordstring
your passwordstored in credentials file
accessTokenstring

generated CleverMaps access token

(see how to get one)

stored in config file
serverstring

server to connect to

stored in config file
dumpDirectorystringdirectory where your dumps will be storedstored in config file
proxyHoststring

proxy server hostname

stored in config file
proxyPortintegerproxy server portstored in config file
s3AccessKeyIdstring

AWS S3 Access Key ID

required for S3 upload (loadCsv --s3Uri)

stored in config file
s3SecretAccessKeystring

AWS S3 Secret Access Key

required for S3 upload (loadCsv --s3Uri)

stored in config file

Connected to server

openProject

Open a specific project.

Parameter nameTypeOptionalityDescriptionConstraints

project

string
project ID

listProjects

List all your projects on the server.

Parameter nameTypeOptionalityDescriptionConstraints
verboseboolean
specifies if the output should be more verbose
shareenum
list projects by share[demo, dimension]
organizationstringlist projects by organization (organization ID)

createProject

Create a new project.

Parameter nameTypeOptionalityDescriptionConstraints
titlestring
title of the project
descriptionstring
description of the project
organizationstringID of the organization which is the owner of the project

editProject

Edit project properties.

Parameter nameTypeOptionalityDescriptionConstraints
projectstring
project ID
newTitlestringnew title of the project
newDescriptionstring
new description of the project
newStatusstringnew status of the project[enabled, disabled]
newOrganizationstringnew ID of the organization which is the owner of the project

deleteProject

Delete an existing project.

Parameter nameTypeOptionalityDescriptionConstraints
projectstring
project ID

Opened project

importProject

Allows you to import either:

You can also import a part of a project with one of the parameters (dashboards, datasets, indicators, indicatorDrills, markers, markerSelectors, metrics, views). If you specify none of these parameters, the whole project will be imported. Everytime you specify a datasets parameter, corresponding data will be imported.

Before each import, validate command is called in the background. If there are any model validation violations in the source project, the import is stopped, unless you also provide the --force parameter.

During the import, the origin key is set to all metadata objects. This key indicates the original location of the object (server and the project). This has a special use in case of datasets & data import. import first takes a look at what datasets currently are in the project and compares them with datasets that are about to be imported. Datasets that are not present in the destination project will be imported automatically. In case of datasets that are present in the destination project, 3 cases might occur:

  • if they have the same name and origin, the dataset will not be imported
  • if they have the same name but different origin, a warning is shown and the dataset will not be imported
  • if a prefix is specified, all source datasets will be imported
Parameter nameTypeOptionalityDescriptionConstraints
serverstring

specifies if the output should be more verbose

default = https://secure.clevermaps.io


projectstring

id of the project to import

(info) either project or dump must be specified


dumpstring

id of the dump to import

(info) either project or dump must be specified


cascadeFromstring

cascade import object and all objects it references

see usage examples below


prefixstringspecify a prefix for the metadata objects and data files
forcebooleanforce the import even if there are model violations in the source project
dashboardsbooleanimport dashboards only
datasetsbooleanimport datasets only
indicatorsbooleanimport indicators only
indicatorDrills
booleanimport indicator drills only
markersbooleanimport markers only
markerSelectorsbooleanimport marker selectors only
metricsbooleanimport metrics only
projectSettingsbooleanimport project settings only
sharesbooleanimport shares only
viewsbooleanimport views only
forceboolean

ignore source project validate errors and proceed with import anyway

skip failed dataset dumps (for projects with incomplete data)


skipDatabooleanskip data import
Usage examples:

importDatabase

Allows you to create datasets and import data from an external database.

This command reads the database metadata from which datasets are created, then imports the data and saves them as CSV files. You can choose to import either of which with --skipMetadata and --skipData parameters. Please note that this command does not create any other metadata objects than datasets. It's also possible to import only specific tables using the --tables parameter.

The database must be located on a running database server which is accessible under an URL. This can be on localhost, or anywhere on the internet. Valid credentials to the database are of course necessary.

So far, the command supports these database engines:

Parameter nameTypeOptionalityDescriptionConstraints
engineenum

name of the database engine

[postgresql]
hoststring

database server hostname

for local databases, use localhost


portintegerdatabase server port
schemastring

name of the database schema

leave out if your engine does not support schemas, or the schema is public


databasestringname of the database
userstringuser name for login to the database
passwordstringuser's password
tablesarray

list of tables to import

leave out if you want to import all tables from the database

example = "orders,clients,stores"


skipDatabooleanskip data import
skipMetadatabooleanskip metadata import
Usage examples:

loadCsv

Load data from a CSV file into a specified dataset.

loadCsv also offers various CSV input settings. Your CSV file may contain specific features, like custom quote or separator characters. The parameters with the csv prefix allow you to configure the data load to fit these features, instead of transforming the CSV file to one specific format. Special cases include the csvNull and csvForceNull parameters.

  • csvNull allows you to specify a value, which will be interpreted as a null value
    • e.g. "false" or "_"
  • csvForceNull then specifies on which columns the custom null replacement should be enforced
    • e.g. "name,title,description"
Parameter nameTypeOptionalityDescriptionConstraints
filestring

path to the CSV file

(info) one of file, s3Uri or url parameters must be specified


s3Uristring

URI of an object on AWS S3 to upload (see examples below)

(info) one of file, s3Uri or url parameters must be specified


urlstring

HTTPS URL which contains a CSV file to be loaded into the dataset

(info) one of file, s3Uri or url parameters must be specified


datasetstring
dataset to into which the data should be loaded
modeenum

incremental mode appends the data to the end of the table

full mode truncates the table and loads the table anew

[incremental, full]

csvHeader

boolean

specifies if the CSV file to upload has a header

default = true

[true, false]

csvSeparatorchar

specifies the CSV column separator character

default = ,


csvQuotechar

specifies the CSV quote character

default = "


csvEscapechar

specifies the CSV escape character

default = \


csvNullstringspecifies the replacement of custom CSV null values
csvForceNullenumspecifies which CSV columns should enforce the null replacement
verboseboolean

enables more verbose output

default = false

[true, false]
multipartboolean

enables multipart file upload (recommended for files larger than 2 GB)

default = true

[true, false]
gzipboolean

enables gzip compression

default = true

[true, false]
Usage examples:

Please note that your AWS S3 Access Key ID and Secret Access Key must be set using setup command first.

dumpCsv

Dump data from a specified dataset into a CSV file.

Parameter nameTypeOptionalityDescriptionConstraints
filestring
path to the CSV filestored in config file
datasetstring
dataset whose data will be dumped

dumpProject

Dump project data & metadata to a directory. If the dump if successfull, the current dump is opened.

Parameter nameTypeOptionalityDescriptionConstraints
directorystring
directory to which the dump will be saved

stored in config file

skipMetadataboolean
skip metadata dump
skipDatabooleanskip data dump
forcebooleanskip failed dataset dumps (for projects with incomplete data)

listDumps

List all local project dumps.

Parameter nameTypeOptionalityDescriptionConstraints
directorystring
parent directory of a specific project's dumps

stored in config file

openDump

Open a specific dump.

Parameter nameTypeOptionalityDescriptionConstraints
directorystring
parent directory of a specific project's dumps

stored in config file

dumpstring

open a specific dump

if not specified, latest dump is opened


truncateProject

Deletes all metadata and data from the project.

This command has no parameters.

validate

Validate the project's data model and data.

Parameter nameTypeOptionalityDescriptionConstraints
projectstringvalidate any other project than the currently opened one
skipModelstring
skip validations of the data model


skipDatastring

skip validations of the data itself


Opened dump

addMetadata

Add a metadata object to the project. The file must be located in a currently opened dump, and in the correct directory.

If the --objectName parameter is not specified, addMetadata will add all new objects in the current dump.

Parameter nameTypeOptionalityDescriptionConstraints
objectNamestring
name of the object (with or without .json extension)

createMetadata

Create a new metadata object.

(info) At this moment, only dataset type is supported. Datasets are generated from a provided CSV file.

Parameter nameTypeOptionalityDescriptionConstraints
typeenumtype of the object

[dataset]

objectNamestring
current name of the object (with or without .json extension)
subtypeenum

name of the object copy (with or without .json extension)

required only for dataset type

[basic, geometryPoint, geometryPolygon]
filestring

path to the CSV file (located either in dump, or anywhere in the file system)

required only for dataset type


primaryKeystring

name of the CSV column that will be marked as primary key

if not specified, first CSV column is selected


geometrystring

name of the geometry key

required only for geometryPolygon subtype


csvSeparatorchar

specifies custom CSV column separator character

default = ,


csvQuotechar

specifies custom CSV column quote character

default = "


csvEscapechar

specifies custom CSV column escape character

default = \


Usage examples:

removeMetadata

Remove a metadata object from the project and from the dump. The file must be located in a currently opened dump, and must not be new.

Parameter nameTypeOptionalityDescriptionConstraints
objectNamestring

name of the object (with or without .json extension)

(info) one of objectName, objectId or orphanObjects parameters must be specified


objectIdstringid of the object
orphanObjectsboolean

prints a list of removeMetadata commands to delete orphan metadata objects

orphan object is an object not referenced from any of the project's views, or visible anywhere else in the app


renameMetadata

Rename an object in a local dump and on the server. If the object is referenced in some other objects by URI (/rest/projects/$projectId/md/{objectType}?name=), the references will be renamed as well.

Parameter nameTypeOptionalityDescriptionConstraints
objectNamestring
current name of the object (with or without .json extension)
newNamestringnew name of the object (with or without .json extension)

copyMetadata

Create a copy of an object existing in a currently opened dump.

This command unwraps the object from a wrapper, renames it and removes generated common syntax keys. If the objectName and newName arguments are the same, the object is unwrapped only.

Parameter nameTypeOptionalityDescriptionConstraints
objectNamestring
current name of the object (with or without .json extension)
newNamestringname of the object copy (with or without .json extension)

pushProject

Upload all modified files (data & metadata) to the project.

This command basically wraps the functionality of loadCsv. It collects all modified metadata to upload, and performs full load of CSV files.

Parameter nameTypeOptionalityDescriptionConstraints
skipMetadataboolean
skip metadata push
skipDatabooleanskip data push
skipValidatebooleanskip the run of validate after push
forcebooleanforce metadata push when there's a share breaking change
verbose


boolean

enables more verbose output

default = false

[true, false]
multipartboolean

enables multipart file upload (recommended for files larger than 2 GB)

default = true

[true, false]
gzipboolean

enables gzip compression

default = true

[true, false]

status

Check the status of a currently opened dump against the project on the server.

This command detects files which have been locally or remotely changed, and also detects files which have a syntax error or constraint violations.

Parameter nameTypeOptionalityDescriptionConstraints
remote-
list metadata contents on the server

applyDiff

Create and apply metadata diff between two live projects.

This command compares all metadata objects of the currently opened project and project specified in the --sourceProject command, and applies changes to the currently opened dump. Metadata objects in dump can be either:

  • added (completely new objects that aren't present in currently opened project)
  • modified
  • deleted (deleted objects not present in the sourceProject)

When the command finishes, you can review the changes applied to the dump using either status or diff comands. The command then tells you to perform specific subsequent steps. This can be one of (or all) these commands:

  • addMetadata (to add the new objects)
  • pushProject (to push the changes in modified objects)
  • removeMetadata (to remove the deleted objects - a command list which must be copypasted to Shell is generated (info))
Parameter nameTypeOptionalityDescriptionConstraints
sourceProjectstring
id of the source project
objectTypesstring

list of object types to be compared

example = "views,indicators,metrics"


diff

Compare local metadata objects with those in project line by line.

If the --objectName parameter is not specified, all wrapped modifed objects are compared.

Parameter nameTypeOptionalityDescriptionConstraints
objectNamestring
name of a single object to compare (with or without .json extension)

The command outputs sets of changes - "deltas" done in each object. Each object can have multiple deltas, which are followed by header with syntax:

Where:
A1 = start delta line number in the dump file
A2 = end delta line number in the dump file
B1 = start delta line number in the remote file
B2 = end delta line number in the remote file

Specific output example:

Which means lines 82-85 in the dump object have been added (+) in favor of lines 82-85, which have been removed () from the remote object.