Details for installing WARPXM on NERSC Cori

A functional (as of August 2020) bash setup file for NERSC Cori is below. Note that it is set up to use the Haswell processors; a different set up would be needed to run on the Knights Landing processors. Two particular settings are of special interest:

• The PETSC_DIR environment variable, set by loading the PETSc module, is used to point PKG_CONFIG_PATH properly.
• The METIS_ROOT environment variable must be set so that cmake finds the appropriate version of Metis, i.e., the one used by PETSc.

Various relevant NERSC documentation is linked here:

• https://docs.nersc.gov/getting-started
• https://docs.nersc.gov/systems/cori
• https://docs.nersc.gov/jobs/best-practices
• https://docs.nersc.gov/jobs/monitoring
• https://www.nersc.gov/assets/Uploads/Intro-to-Cori.pdf

# begin .bashrc.ext
 
[ -e $HOME/.dbgdot ] && echo "entering .bashrc.ext"
#
# User additions to .bashrc go in this file
#
#------------------------------------------------------------------------
# generic environment setup
#------------------------------------------------------------------------
alias ll='ls -al'
alias ltr='ls -ltrh'
alias ltra='ls -ltrha'
alias findp='find . -print | grep'
 
export PS1="\u@\h > "
 
unset color_prompt
 
alias ls='ls --color=auto'
alias grep='grep --color=auto'
alias fgrep='fgrep --color=auto'
alias egrep='egrep --color=auto'
 
# colored GCC warnings and errors
export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'
 
# your other favorite aliases here
 
#------------------------------------------------------------------------
# warpxm-specific environment (for nersc-cori)
#------------------------------------------------------------------------
 
# load modules
 
# a standard setup using built-in cray-petsc:
# --> works as of August 2020
module unload darshan
module load cray-petsc
module load python/3.7-anaconda-2019.10
module load cray-hdf5-parallel
module load cmake
 
# aliases, etc...
 
sq='sacct -X -s pd,r'
alias wq='watch -d -n 60 sacct -X -s pd,r'
# note that NERSC asks users to use the "-n 60" argument to prevent
# watch from using excessive resources
# they also ask users to use sacct instead of squeue
 
alias mi='make install -j10'
export wxg='warpxm_git'
#export wxg='warpxm_git_clean'
 
export code='code'
 
alias code='cd ~/codes/warpxm/'$code'; pwd'
alias wx='cd ~/codes/warpxm/'$code'/'${wxg}'; pwd'
alias build='cd ~/codes/warpxm/'$code'/build; pwd'
alias b='build'
alias bmi='echo "  "; echo "*** building warpxm ***"; echo "  "; (b; mi)'
 
# put your warpxm directory path here!
warpxm='/global/homes/e/etmeier/codes/warpxm'
 
export PYTHONPATH=$warpxm/${code}/build/tools/:$PYTHONPATH
export PYTHONPATH=$warpxm/${code}/build/tools/iman_plotting_modules:$PYTHONPATH
export PYTHONPATH=$warpxm/${code}/build/user_runs/eric_meier/post:$PYTHONPATH
export PATH=$warpxm/usr/bin:$PATH
export PATH=$warpxm/${code}/build/user_runs/eric_meier/post:$PATH
 
# set METIS_ROOT so that cmake finds the version used by PETSc:
export METIS_ROOT=/opt/cray/pe/tpsl/19.06.1/INTEL/19.0/haswell
 
export LD_LIBRARY_PATH=$warpxm/usr/lib:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=$warpxm/usr/lib:$PKG_CONFIG_PATH
export PKG_CONFIG_PATH=$warpxm/usr/lib/pkgconfig:$PKG_CONFIG_PATH
 
# the module load, above, has set PETSC_DIR ; now use it to set
# the PKG_CONFIG_PATH correctly:
export PKG_CONFIG_PATH=${PETSC_DIR}/lib/pkgconfig:$PKG_CONFIG_PATH
 
[ -e $HOME/.dbgdot ] && echo "exiting .bashrc.ext"
 
# end .bashrc.ext