Multi-Run Benchmark HAFS forecast using simple library

The master tarball is HAFS-BENCHMARK.tar
Follow the step below to:
  1. Build the dependencies library - simple-ufs.
  2. Use the dependency library to build Hafs forecast executable.
  3. Run the benchmark jobs and generate end-to-end forecasts (hours to F015). 
  4. Calculate and report MAX, Avg, and Min values on "Total runtime of mpiexec". 
  5. Calculate and report MAX, Avg, and Min values on "The total amount of wall time".
  6. Calculate and report MAX, Avg, and Min values on prep step time used (time between starting mpiexec and first atm.nc output file).
     Forecast preparation time used in seconds.
  7. Calculate and report MAX, Avg, and Min values on Average wall clock time in seconds between two output atmfhhh.nc files. 

Following these steps to run the HAFS forecast benchmark
Export base directory (change as needed):
  export benchmark_dir=LOCATION WHERE BENCHMARK WILL BE RUN
Prepare the package:
  mkdir -p ${benchmark_dir}
  cd ${benchmark_dir}
  cp ${LOCATION WHERE HAFS BENCHMARK MASTER TARBALL IS}/HAFS-BENCHMARK.tar .
  tar xvf HAFS-BENCHMARK.tar
  cd HAFS-BENCHMARK
Copy the control file from the default control file location:
  mkdir -p ${benchmark_dir}/HAFS-BENCHMARK/hafs-regression-test-control/T2O_2020092200_17L
  cp -Rp ${LOCATION WHERE HAFS BENCHMARK MASTER TARBALL IS}/hafs-regression-test-control/T2O_2020092200_17L ${benchmark_dir}/HAFS-BENCHMARK/hafs-regression-test-control
Modify hafs-benchmark.sh with account, queue, and multi-run information:
  vi hafs-benchmark.sh
  (Note, if the control files from the above step are at a different location, please update the $RCTL variable on line 24 accordingly.)
  (Note, the variable MULTI_RUN_COUNT control the amount of concurrent jobs submitted.)
To run benchmark:
  nohup ./hafs-benchmark.sh &> hafs-benchmark.log 2>&1 &
To exam the benchmark report output:
  sh hafs-benchmark-result.sh
  vi HAFS-multi-run-forecast-benchmart-report.info