Getting-Started-Upload
Note
The experiments executed on all E3SM supported supercomputers are automatically uploaded to PACE nightly.
These instructions are applicable for manually uploading experiment data. First, download the pace upload client and run the client to upload experiments to PACE.
Tool
Please download and use this tool: Python 3 version or Python 2 version to upload experiment data directly from supercomputers.
To download using command line:
Python 2 version
$ wget https://pace.ornl.gov/static/tools/pace-upload
$ chmod +x ./pace-upload
Python 3 version
$ wget https://pace.ornl.gov/static/tools/pace-upload3
$ chmod +x ./pace-upload3
This is a Python-based tool that can be executed directly (./pace-upload) once you download and make it executable. The system-wide Python 2.x installation on Summit/Theta/Cori/Compy has the requisite ‘requests’ module to enable this tool to work. It presently handles uploading user data in performance archive directories from all E3SM supported machines. The server checks for duplicates while processing uploaded data.
Please see the demo for reference. A log file is written at the end that details the upload progress and notes any errors.
Usage
Example:
$ ./pace-upload --perf-archive user-exps-set
Detailed usage:
$ ./pace-upload --help
usage: pace-upload [-h] [--exp-dir SOURCE] [--perf-archive SOURCE]
[--application {e3sm}]
PACE upload tool.
optional arguments:
-h, --help show this help message and exit
--exp-dir SOURCE, -ed SOURCE
Root directory containing experiment(s) results.
Handles multiple experiment directories under root
--perf-archive SOURCE, -pa SOURCE
Root directory containing performance archive. Handles
multiple performance archive directories under root
--application {e3sm}, -a {e3sm}
Application name