Getting-Started-Upload

Note

The experiments executed on all E3SM supported supercomputers are automatically uploaded to PACE nightly.

These instructions are applicable for manually uploading experiment data. First, download the pace upload client and run the client to upload experiments to PACE.

Tool

Please download and use this tool: Python 3 version or Python 2 version to upload experiment data directly from supercomputers.

To download using command line:

Python 2 version

$ wget https://pace.ornl.gov/static/tools/pace-upload
$ chmod +x ./pace-upload

Python 3 version

$ wget https://pace.ornl.gov/static/tools/pace-upload3
$ chmod +x ./pace-upload3

This is a Python-based tool that can be executed directly (./pace-upload) once you download and make it executable. The system-wide Python 2.x installation on Summit/Theta/Cori/Compy has the requisite ‘requests’ module to enable this tool to work. It presently handles uploading user data in performance archive directories from all E3SM supported machines. The server checks for duplicates while processing uploaded data.

Please see the demo for reference. A log file is written at the end that details the upload progress and notes any errors.

Usage

Example:

$ ./pace-upload --perf-archive user-exps-set

Detailed usage:

$ ./pace-upload --help
usage: pace-upload [-h] [--exp-dir SOURCE] [--perf-archive SOURCE]
                [--application {e3sm}]
PACE upload tool.
optional arguments:
   -h, --help            show this help message and exit
   --exp-dir SOURCE, -ed SOURCE
                           Root directory containing experiment(s) results.
                           Handles multiple experiment directories under root
   --perf-archive SOURCE, -pa SOURCE
                           Root directory containing performance archive. Handles
                           multiple performance archive directories under root
   --application {e3sm}, -a {e3sm}
                           Application name