Nexus Data Import

Parent Previous Next

Nexus Data Import

Walk one or more directories or files on your local disk and create data items from the file contents. Those items are pushed into a running Nexus database.

Usage

nexus_data_import [-h] [-server [server]]

                  [-user [username]] [-password [password]]

                  [-session_guid [session_guid]]

                  [-session_date [date]]

                  [-session_hostname [hostname]]

                  [-session_platform [platform]]

                  [-session_application [application]]

                  [-session_version [version]]

                  [-session_tags [tag_string]]

                  [-dataset_guid [dataset_guid]]

                  [-dataset_filename [filename]]

                  [-dataset_dirname [directory]]

                  [-dataset_format [format]]

                  [-dataset_parts [count]]

                  [-dataset_elements [count]]

                  [-dataset_tags [tag_string]]

                  [-data_date [date]]

                  [-data_sequence [number]]

                  [-data_name [name]]

                  [-data_source [name]]

                  [-data_tags [tag_string]]

                  [-disable_encoding_check]

                  [-ext_files [ext [ext ...]]]

                  [-ext_tables [ext [ext ...]]]

                  [-ext_html [ext [ext ...]]]

                  [-ext_movies [ext [ext ...]]]

                  [-ext_images [ext [ext ...]]]

                  [-ext_text [ext [ext ...]]]

                  [-ext_scenes [ext [ext ...]]]

                  [-table_col_labels] [-table_row_labels] [-table_numeric]

                  [-table_delimiter [char]] [-table_quote_char [char]]

                  [-verbose] [-defaults] [-simulate]

                  [file|directory [file|directory ...]]

Description

The nexus_data_import tool allows a user to upload data from files into a Nexus database. This is similar to the "Upload files into data items" application in the Nexus web interface. In addition to specifying files or directories to upload, many options for defining exactly how the data should be incorporated into the existing data at the Nexus server are provided. These include specification of session, dataset, tags, what data item type-specific file extensions should be uploaded as, and how tables should be handled.

Item file uploads for files of type 'csv, txt and html' now have to be strictly UTF-8 encoded. On upload, there is a check for UTF-8 encoding which is used to force this requirement. This check is enabled by default and uploads will fail if this check fails. You can disable this check using -disable_encoding_check if you feel that your file is correctly encoded.

The tool takes a list of files and/or directories as arguments. When a directory is specified, the tool will recursively visit its contents, uploading all files that match the file extensions specified with the -ext_* options.

WARNING: The password command-line option is passed to nexus_data_import in cleartext. Such command lines are visible to all users running on the same system. Use with care.

Options

If any of these options are specified multiple times on the command line, only the last one takes effect.

Tags

Tags in Nexus are simple strings and can be applied to data items, sessions, and datasets. Tags are not required, but allow a wide variety of expression to define a data model. Without them, queries can be done only with the default data that Nexus provides (dates, names, connections). Best practice involves creating a data model through the judicious use of tags. Depending on the data model desired, the presence or absence of a tag is significant. For instance, one data model might attach significance to a tag "verified" applied to a data item. Such a tag might indicate that the data item has gone through a review process, and the absence of such tag indicates that the process has not been performed. Other tags take the form "key=value". Using this format, tags are more fine-grained, allowing them to take on different values. For instance, it's common to find tags "variable=pressure" or "variable=density" applied to image data that comes from physical simulations. Any number of tags may be specified and are separated by spaces when defined. So the tag string "verified variable=viscosity" is a valid list of two tags, the second one of which has a value.