Nexus CLI Utility

A command line utility is included that allows for many Nexus operations to be run from the command line.  These operations include: listing entities on the server, importing entities from files on disk, deleting entities, copy entities between servers and modify the tags on entities.  These functions are implemented as a single command line tool, leveraging sub-commands to perform the individual operations.

Basic Command Usage

nexus [-h] {cp,htmlexport,import,ls,rm,tag} ...

The sub-commands are cp, import, ls, rm, htmlexport and tag.  Help on individual commands may be obtained by combining -h with the sub-command, so:

nexus cp -h

will display help related to the sub-command.  

Notes

Most commands require the URL of a server to be provided.  This is usually in the form:  http://{username}:{password}@{hostname}:{port}.  For example: http://nexus:cei@127.0.0.1:8000.  This URL syntax exposes the username and password of the database in cleartext as such command lines are often visible to all users running on the same system.  If no username or password is supplied (e.g. http://127.0.0.1:8000), the tool will look at the environmental variables:  NEXUS_TOOLS_USERNAME and NEXUS_TOOLS_PASSWORD for the username and password to use, avoiding command line exposure.

For templates, generally only top level templates can be manipulated for operations like deleting and copying and those operations will be applied to the entire tree of templates.

Copy Sub-command (cp)

usage: nexus cp [-h] [--test] [--verbose] [--type {item,template}]

                    [--query query_string]

                    source_server_url destination_server_url

Copy items, sessions, datasets and templates from one Nexus server to another.

positional arguments:

  source_server_url     Nexus server URL. Example:

                        http://nexus:cei@127.0.0.1:8000

  destination_server_url

                        Target Nexus server URL. Example:

                        http://nexus:cei@127.0.0.1:8000

optional arguments:

  -h, --help            show this help message and exit

  --test                Validate the operation, but do not perform it

  --verbose             Include progress and other information

  --type {item,template}

                        Select the object type to copy. Default: item

  --query query_string  Specify a query string. Default: no filter

Description

This tool copies a selected collection of objects (data items or templates) from one server to another.  GUIDs are preserved and any referenced session and dataset objects are copied as well.

Arguments

• source_server_url
  This argument specifies the hostname, port, username and password of the Nexus server to read items from.  For example: http://nexus:cei@127.0.0.1:8000 specifies a server running on the local machine (127.0.0.1) on port 8000.  The username is 'nexus' and the password is 'cei'.  
• destination_server_url
  This argument specifies the hostname, port, username and password of the Nexus server to copy items into.  For example: http://nexus:cei@127.0.0.1:8000 specifies a server running on the local machine (127.0.0.1) on port 8000.  The username is 'nexus' and the password is 'cei'.  

Options

Various options exist.

• -h, --help
  Show the usage message and exit
• --verbose
  Display additional status and progress information during the operation.
• --test
  Causes the tool to simulate the operation without actually performing it. This is helpful when debugging command lines, especially when used in conjunction with the -verbose option.
• --type {item,template}
  Specify the type of objects to copy.  This command can copy data items or templates.
• --query query_string
  Use the specified query string to select the specific objects of the specified type (--type option) to perform the operation on. The syntax is documented here.

Import Sub-command (import)

usage: nexus import [-h] [--disable_encoding_check]

                        [--session_guid [session_guid]]

                        [--session_date [date]]

                        [--session_hostname [hostname]]

                        [--session_platform [platform]]

                        [--session_application [application]]

                        [--session_version [version]]

                        [--session_tags [tag_string]]

                        [--dataset_guid [dataset_guid]]

                        [--dataset_filename [filename]]

                        [--dataset_dirname [directory]]

                        [--dataset_format [format]] [--dataset_parts [count]]

                        [--dataset_elements [count]]

                        [--dataset_tags [tag_string]]

                        [--data_guid [item_guid]] [--data_date [date]]

                        [--data_sequence [number]] [--data_name [name]]

                        [--data_source [name]] [--data_tags [tag_string]]

                        [--data_categories [categoryA [categoryB ...]]]

                        [--ext_image [ext [ext ...]]]

                        [--ext_movie [ext [ext ...]]]

                        [--ext_table [ext [ext ...]]]

                        [--ext_scene [ext [ext ...]]]

                        [--ext_file [ext [ext ...]]]

                        [--ext_text [ext [ext ...]]]

                        [--ext_html [ext [ext ...]]] [--table_col_labels]

                        [--table_row_labels] [--table_numeric]

                        [--table_delimiter [char]] [--table_quote_char [char]]

                        [--verbose] [--test] [--print_options]

                        nexus_server_url [file|directory [file|directory ...]]

Import data items from disk into the Nexus server.

positional arguments:

  nexus_server_url      Nexus server URL. Example:

                        http://nexus:cei@127.0.0.1:8000

  file|directory        File and directory names to import

optional arguments:

  -h, --help            show this help message and exit

  --disable_encoding_check

                        Disable the implicit check for UTF-8 encoding of

                        uploaded files

  --session_guid [session_guid]

                        GUID of an existing Nexus session to use for new data

                        items

  --session_date [date]

                        Creation date for the created session

  --session_hostname [hostname]

                        Hostname for the created session

  --session_platform [platform]

                        Platform name for the created session

  --session_application [application]

                        Application name for the created session

  --session_version [version]

                        Version for the created session

  --session_tags [tag_string]

                        Tags for the created session

  --dataset_guid [dataset_guid]

                        GUID of an existing Nexus dataset to use for new data

                        items

  --dataset_filename [filename]

                        Filename for the created dataset

  --dataset_dirname [directory]

                        Directory name for the created dataset

  --dataset_format [format]

                        Format for the created dataset

  --dataset_parts [count]

                        Number of parts for the created dataset

  --dataset_elements [count]

                        Number of elements for the created dataset

  --dataset_tags [tag_string]

                        Tags for the created dataset

  --data_guid [item_guid]

                        GUID for newly imported items (note only useful when

                        importing a single data item, use with caution)

  --data_date [date]    Creation date for created data items (__timestamp__ is

                        the file timestamp)

  --data_sequence [number]

                        Sequence number for the created data items

  --data_name [name]    Name for created data items (__file__ is the file

                        basename)

  --data_source [name]  Source for created data items (__file__ is the file

                        basename)

  --data_tags [tag_string]

                        Tags for created data items

  --data_categories [categoryA [categoryB ...]]

                        List of space-separated categories to assign the item

                        to.

  --ext_image [ext [ext ...]]

                        Filename extensions to be imported as image. Default:

                        ['png', 'jpg']

  --ext_movie [ext [ext ...]]

                        Filename extensions to be imported as movie. Default:

                        ['mp4']

  --ext_table [ext [ext ...]]

                        Filename extensions to be imported as table. Default:

                        ['csv']

  --ext_scene [ext [ext ...]]

                        Filename extensions to be imported as scene. Default:

                        ['stl', 'ply', 'csf', 'avz']

  --ext_file [ext [ext ...]]

                        Filename extensions to be imported as file. Default:

                        []

  --ext_text [ext [ext ...]]

                        Filename extensions to be imported as text. Default:

                        ['txt']

  --ext_html [ext [ext ...]]

                        Filename extensions to be imported as html. Default:

                        ['htm', 'html']

  --table_col_labels    Input csv files will have a column label row

  --table_row_labels    Input csv files will have a row label column

  --table_numeric       Table items generated from .csv files will have

                        strictly numeric content

  --table_delimiter [char]

                        The delimiter in csv files

  --table_quote_char [char]

                        The quote character in csv files

  --verbose             Enable verbose mode

  --test                Validate the operation, but do not perform it

  --print_options       Print import configuration option values

Description

The tool allows a user to upload data from files into a Nexus database. This is similar to the "Upload files into data items" application in the Nexus web interface. In addition to specifying files or directories to upload, many options for defining exactly how the data should be incorporated into the existing data at the Nexus server are provided. These include specification of session, dataset, tags, what data item type-specific file extensions should be uploaded as, and how tables should be handled.

Item file uploads for files of type 'csv, txt and html' now have to be strictly UTF-8 encoded. On upload, there is a check for UTF-8 encoding which is used to force this requirement. This check is enabled by default and uploads will fail if this check fails. You can disable this check using -disable_encoding_check if you feel that your file is correctly encoded.

The tool takes a list of files and/or directories as arguments. When a directory is specified, the tool will recursively visit its contents, uploading all files that match the file extensions specified with the -ext_* options. 

Arguments

• nexus_server_url
  This argument specifies the hostname, port, username and password of the Nexus server to import the data items into.  For example: http://nexus:cei@127.0.0.1:8000 specifies a server running on the local machine (127.0.0.1) on port 8000.  The username is 'nexus' and the password is 'cei'.  
• file | directory ...
  This argument specifies the name of a specific file to import or the name of a directory whose files should be imported.  This argument may be repeats as needed.  Note: the tool will recurse into the directory, looking for files with the matching extensions to import.

Options

If any of these options are specified multiple times on the command line, only the last one takes effect.

• -h, --help
  Show the usage message and exit
• --session_guid [GUID]
  The Globally Unique IDentifier (GUID) of the session to associated the data with. If omitted, a new session is created and assigned a new GUID. This new session GUID is then associated with the data. The additional -session_* command-line options are provided to specify attributes of the new session.
• --session_date [date]
  The creation date for the created session. This option is ignored if an existing session GUID is used instead of creating a new session. If omitted, the current time stamp is used. The format of the date is that parsed by the Python dateutil.parser.parse method, which is able to parse most known formats.
• --session_hostname [hostname]
  The host name to associate with the created session. This option is ignored if an existing session GUID is specified. The default is the host name of the system the  tool is run on.
• --session_platform [platform]
  The platform (operating system) name to associate with the created session. This option is ignored if an existing session GUID is specified. The default is the platform of the system the  tool is run on.
• --session_application [application]
  The application name to associate with the created session. This option is ignored if an existing session GUID is specified. The default is "nexus_data_import".
• --session_version [version]
  The version to associate with the created session. This option is ignored if an existing session GUID is specified. The default is the version number of the  tool.
• --session_tags [tag string]
  A space-separated list of tags to associate with the created session. This option is ignored if an existing session GUID is specified. The default is empty (no session tags). Note that session tags, dataset tags, and data item tags are distinct and separate. It's a common mistake to specify -session_tags when -data_tags is intended.
• --dataset_guid [GUID]
  The GUID of the dataset to associate the data with. If omitted, a new dataset is created and assigned a new GUID. This new dataset GUID is then associated with the data. The additional -dataset_* command-line options are provided to specify attributes of the new dataset.
• --dataset_filename [file name]
  The file name for the created dataset. This option is ignored if an existing dataset GUID is specified. The default file name is "unspecified".
• --dataset_dirname [directory name]
  The directory name for the created dataset. This option is ignored if an existing dataset GUID is specified. The default directory name is the current working directory when the nexus_data_import tool was run.
• --dataset_format [format]
  The format for the created dataset as a string. This option is ignored if an existing dataset GUID is specified. The default format string is "unspecified".
• --dataset_parts [count]
  The number of parts in the created dataset. This is only useful when 3D scene data is imported. This option is ignored if an existing dataset GUID is specified. The default is 0.
• --dataset_elements [count]
  The number of elements in the created dataset. This is only useful when 3D scene data is imported. This option is ignored if an existing dataset GUID is specified. The default is 0.
• --dataset_tags [tag string]
  A space-separated list of tags to associate with the created dataset. This option is ignored if an existing dataset GUID is specified. The default is empty (no dataset tags). Note that session tags, dataset tags, and data item tags are distinct and separate. It's a common mistake to specify -dataset_tags when -data_tags is intended.
• --data_date [date]
  The creation date for the created data items. The format of the date is that parsed by the Python dateutil.parser.parse method, which is able to parse most known formats. If the string "__timestamp__" is specified for the date, the creation date used will be that of each data file. The default is now, the time when the  tool was run.
• --data_sequence [sequence number]
  The sequence number for the created data items. This is used if data items can be considered in a specific ordered sequence. The default sequence number is 0.
• --data_name [name]
  The name for the created data items as a string. If "__file__" is specified, the name will be that of each data file. The default is "__file__".
• --data_source [source]
  The source of the data as a string. The default is "data_import". Set this if you wish to track where data comes from.
• --data_tags [tag string]
  A space-separated list of tags to associated with the data items. The default is empty (no data item tags). Note that session tags, dataset tags, and data item tags are distinct and separate. It's a common mistake to specify -session_tags or -dataset_tags when -data_tags is intended. See Tags discussion below.
• --disable_encoding_check
  Item file uploads for files of type 'csv, txt and html' have to be strictly UTF-8 encoded. On upload, there is a check for UTF-8 encoding which is used to force this requirement. This check is enabled by default and uploads will fail if this check fails. You can disable this check using -disable_encoding_check if you feel that your file is correctly encoded.
• --ext_files [ext [ext …]]
  --ext_tables [ext [ext …]]
  --ext_html [ext [ext …]]
  --ext_movies [ext [ext …]]
  --ext_images [ext [ext …]]
  --ext_text [ext [ext …]]
  --ext_scenes [ext [ext …]]
  The -ext_* options are used to map file extensions to a data item type. Each -ext_* option corresponds to a different data item type and specifies the filename extensions that should be identified as such. For example, using -ext_html html htm specifies that *.html and *.htm files should be imported as HTML data items.  By default, *.png and *.jpg map to Image items, *.mp4 maps to Animation items, *.csv maps to Table items, *.stl, *.ply, *.csf, and *.avz map to Scene items, and File items, String items, Tree items, and HTML items have no explicit mapping. If no arguments are given after a -ext_* option, that specifies that no file extensions should be mapped to that data item type.
• --table_col_labels
  --table_row_labels
  These two options specify that imported tables should consider the first column and first row, respectively, as labels for the table. If either of these options are omitted, it indicates that the column (or row) has no labels in the data file.
• --table_numeric
  Using this option indicates that table data should be coerced into numeric data types instead of strings (the default).
• --table_delimiter [char]
  The character that separates cell items from each other in an table input file. By default, the separator is a comma. Specifying nothing for -table_delimiter indicates that no separation should be done and each line is imported as a single cell item.
• --table_quote_char [char]
  The character that delimits quoted text in cell items in a table input file. By default, the quote character is a single quote ('). This allows for the table delimiter to be used within a single cell. Specifying nothing for -table_quote_char indicates that no special parsing should occur for quoted strings.
• --verbose
  Enables verbose output from the tool. When specified, this option will cause the tool to print many extra messages, including data types, files processed, etc.
• --print_options
  Prints out the selected command line options.
• --test
  Causes the tool to simulate the pushing of data items to a Nexus server without actually doing so. This is helpful when debugging data import, especially when used in conjunction with the -verbose option.

List Sub-command (ls)

usage: nexus ls [-h] [--full] [--long] [--guid] [--header] [--no-header]

                    [--type {item,session,dataset,template,category,group}]

                    [--query query_string]

                    nexus_server_url

List items, sessions, datasets and templates in the Nexus server.

positional arguments:

  nexus_server_url      Nexus server URL. Example:

                        http://nexus:cei@127.0.0.1:8000

optional arguments:

  -h, --help            show this help message and exit

  --full                Increase the level of output detail

  --long                One item per line

  --guid                Include the GUID in the output

  --header              Include headers in the output

  --no-header           Do not include headers in the output

  --type {item,session,dataset,template,category,group}

                        Select the object type to list. Default: item

  --query query_string  Specify a query string. Default: no filter

Description

This tool selects a collection of objects and prints the details of the objects to stdout.

Arguments

• nexus_server_url
  This argument specifies the hostname, port, username and password of the Nexus server to list items from.  For example: http://nexus:cei@127.0.0.1:8000 specifies a server running on the local machine (127.0.0.1) on port 8000.  The username is 'nexus' and the password is 'cei'.  

Options

Various options exist.

• -h, --help
  Show the usage message and exit
• --verbose
  Display additional status and progress information during the operation.
• --full
  Display additional information not included in a simple listing.  This option implies --long
• --long
  Display a single item on a each line.
• --header, --no-header
  Display If --header is specified, then some query details will be printed at the start of the operation.  --no-header suppresses the query details (the default).
• --guid
  Display include the GUID for each object in addition to other selected information.
• --verbose
  Display additional status and progress information during the operation.
• --type {item,template,dataset,template,category,group}
  Specify the type of objects to list.
• --query query_string
  Use the specified query string to select the specific objects of the specified type (--type option) to perform the operation on. The syntax is documented here.

Delete Sub-command (rm)

usage: nexus rm [-h] [--type {item,template}] [--query query_string]

                    [--verbose] [--test]

                    nexus_server_url

Delete items or report templates from the Nexus server. Only top level

templates can be deleted and all of their children will be deleted as well.

positional arguments:

  nexus_server_url      Nexus server URL. Example:

                        http://nexus:cei@127.0.0.1:8000

optional arguments:

  -h, --help            show this help message and exit

  --type {item,template}

                        Select the object type to remove. Default: item

  --query query_string  Specify a query string. Default: no filter

  --verbose             Enable verbose mode

  --test                Validate the operation, but do not perform it

Description

This tool deletes a selected collection of objects (data items or templates) from a server.

Arguments

• nexus_server_url
  This argument specifies the hostname, port, username and password of the Nexus server to delete items from.  For example: http://nexus:cei@127.0.0.1:8000 specifies a server running on the local machine (127.0.0.1) on port 8000.  The username is 'nexus' and the password is 'cei'.  

Options

Various options exist.

• -h, --help
  Show the usage message and exit
• --verbose
  Display additional status and progress information during the operation.
• --test
  Causes the tool to simulate the operation without actually performing it. This is helpful when debugging command lines, especially when used in conjunction with the -verbose option.
• --type {item,template}
  Specify the type of objects to delete.  This command can delete data items or templates.  It may delete sessions/datasets implicitly if they no are no longer referenced.
• --query query_string
  Use the specified query string to select the specific objects of the specified type (--type option) to perform the operation on. The syntax is documented here.

HTML export command (htmlexport)

usage: nexus htmlexport [-h] [--query query_string]

                            [--filename top_level_filename]

                            [--no_inline_files] [--verbose]

                            nexus_server_url destination_directory

Export a selected report as offline html from the Nexus server. Note: the

template query will always include a top level check. The report will be

exported into the named destination directory, which must not exist. The

default top level filename will be index.html stored in the destination

directory. The top level filename can be specified on the command line and

should include both filename and extension.

positional arguments:

  nexus_server_url      Nexus server URL. Example:

                        http://nexus:cei@127.0.0.1:8000

  destination_directory

                        Directory to save HTML report into. Must not exist.

optional arguments:

  -h, --help            show this help message and exit

  --query query_string  Query string used to select the report template to

                        export. Default: no filter

  --filename top_level_filename

                        The filename of the top-level file. Default:

                        index.html

  --no_inline_files     Disable the use of data URIs in the output

  --verbose             Enable verbose mode

Description

This tool generates a report on the nexus server and downloads the resulting html into a local directory structure.  Note: some item types will be converted into forms that do not require a running Nexus server.

Arguments

• nexus_server_url
  This argument specifies the hostname, port, username and password of the Nexus server to modify items on.  For example: http://nexus:cei@127.0.0.1:8000 specifies a server running on the local machine (127.0.0.1) on port 8000.  The username is 'nexus' and the password is 'cei'.  
• destination_directory
  The downloaded html/css/image and other files will be downloaded into this directory.  This directory cannot exist before the operation is performed.

Options

Various options exist.

• -h, --help
  Show the usage message and exit
• --query query_string
  This query string is used to select the report to be exported.  The result of the query must be a single top level report template.
• --filename top_level_filename
  By default, the top level HTML file will be named 'index.html' in the destination directory. This option can be used to specify a different filename.
• --no_inline_files
  By default, the export operation uses data URI strings embedded in the HTML to represent some files. If this option is specified, separate disk files will be used instead, but the local HTML cannot be displayed in a browser without a server or special browser flags.
• --verbose
  Display additional status and progress information during the operation.

PDF export command (pdfexport)

usage: nexus pdfexport [-h] [--query query_string] [--verbose] nexus_server_url pdf_filename

Export a selected report in pdf format from the Nexus server. Note: the template query will always include a top level

check. The report will be written into the named file, which must not exist.

positional arguments:

  nexus_server_url      Nexus server URL. Example: http://nexus:cei@127.0.0.1:8000

  pdf_filename          Filename to save the PDF file as. Must not exist.

optional arguments:

  -h, --help            show this help message and exit

  --query query_string  Query string used to select the report template to

                        export. Default: no filter

  --verbose             Enable verbose mode

Description

This tool generates a report on the nexus server and renders the selected report into a local PDF format file.

Arguments

• nexus_server_url
  This argument specifies the hostname, port, username and password of the Nexus server to modify items on.  For example: http://nexus:cei@127.0.0.1:8000 specifies a server running on the local machine (127.0.0.1) on port 8000.  The username is 'nexus' and the password is 'cei'.  
• pdf_filename
  The name to save the PDF file as.  Note: This file cannot exist before the operation is performed.

Options

Various options exist.

• -h, --help
  Show the usage message and exit
• --query query_string
  This query string is used to select the report to be exported.  The result of the query must be a single top level report template.
• --verbose
  Display additional status and progress information during the operation.

Tagging command (tag)

usage: nexus tag [-h] [--value VALUE]

                     [--type {item,session,dataset,template}]

                     [--query query_string] [--verbose] [--test]

                     nexus_server_url {add,remove,replace} tagname

Modify item, template, session, and dataset object tags in the Nexus server.

positional arguments:

  nexus_server_url      Nexus server URL. Example:

                        http://nexus:cei@127.0.0.1:8000

  {add,remove,replace}  Tag operation

  tagname               The tag to modify

optional arguments:

  -h, --help            show this help message and exit

  --value VALUE         Tag value if needed

  --type {item,session,dataset,template}

                        Select the object type to list. Default: item

  --query query_string  Specify a query string. Default: no filter

  --verbose             Enable verbose mode

  --test                Validate the operation, but do not perform it

Description

This tool walks a collection of objects (data items, sessions, datasets or templates) selected via a query string and manipulates the tag strings.  Consider a tag of the form tag_name=value, where '=value' is optional.  tag_name is specified via an argument and value may be specified via the --value option.  The resulting tag can be added to, removed from or modified on the selected objects. This tool will not add a tag multiple times to the same object.

Arguments

• nexus_server_url
  This argument specifies the hostname, port, username and password of the Nexus server to modify items on.  For example: http://nexus:cei@127.0.0.1:8000 specifies a server running on the local machine (127.0.0.1) on port 8000.  The username is 'nexus' and the password is 'cei'.  
• tag_operation {add,remove,replace}
  This argument selects the type of modification to apply to the named tag.  Options include:  add - add the tag to the items, remove - remove the tag from the items, replace - replace the tag. Replace will add the tag if is does not exist.  If it does exist, the value of the tag will be changed or removed, depending on the --value option. 
• tag_name
  The actual tag name to modify.

Options

Various options exist.

• -h, --help
  Show the usage message and exit
• --value [value]
  Specify the tag's value, forming the string:  {tag_name}={value} to be used as the tag.  If value is not specified, the tag will be simply {tag_name}.
• --verbose
  Display additional status and progress information during the operation.
• --test
  Causes the tool to simulate the operation without actually performing it. This is helpful when debugging command lines, especially when used in conjunction with the -verbose option.
• --type {item,template,session,dataset}
  Specify the type of objects to modify the tags on.  This command can work on data items, sessions, datasets or templates.
• --query query_string
  Use the specified query string to select the specific objects of the specified type (--type option) to perform the operation on. The syntax is documented here.