tl;dr
Put the following in your .bashrc
use_build() {
if hostname | grep bebop &>/dev/null; then
echo "loading bebop spack"
module load gcc/10.2.0-z53hda3
module load intel-mpi/2019.10.317-qn674hj
module load intel-mkl/2020.4.304
source $HOME/git/spack-bebop/share/spack/setup-env.sh
source $HOME/git/spack-bebop/share/spack/spack-completion.bash
export clustername=bebop
fi
#other LCRC Machines
export SPACK_USER_CONFIG_PATH="$HOME/.spack/$clustername"
export SPACK_USER_CACHE_PATH="$SPACK_USER_CONFIG_PATH"
}
Then run
mkdir -p ~/git
git clone https://github.com/spack/spack git/spack-bebop
git clone https://github.com/robertu94/spack_packages git/robertu94_packages
source ~/.bashrc
spack compiler find
# for extra packages like libpressio
spack repo add ~/git/robertu94_packages
First create packages.yaml
at ~/.spack/bebop/packages.yaml
with:
packages:
all:
providers:
blas: [intel-mkl]
lapack: [intel-mkl]
fftw-api: [intel-mkl]
scalapack: [intel-mkl]
mkl: [intel-mkl]
mpi: [intel-mpi]
openssl:
externals:
- spec: openssl@1.0.2k-fips
prefix: /usr
buildable: false
After that create compilers.yaml
at ~/.spack/bebop/linux/compilers.yaml
with:
compilers:
- compiler:
spec: gcc@10.2.0
paths:
cc: /gpfs/fs1/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/gcc-6.5.0/gcc-10.2.0-z53hda3/bin/gcc
cxx: /gpfs/fs1/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/gcc-6.5.0/gcc-10.2.0-z53hda3/bin/g++
f77: /gpfs/fs1/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/gcc-6.5.0/gcc-10.2.0-z53hda3/bin/gfortran
fc: /gpfs/fs1/software/centos7/spack-latest/opt/spack/linux-centos7-x86_64/gcc-6.5.0/gcc-10.2.0-z53hda3/bin/gfortran
flags: {}
operating_system: centos7
target: x86_64
modules: [gcc/10.2.0-z53hda3]
environment: {}
extra_rpaths: []
And now spack should work. On your next login, just call use_build
to reload spack.
For the longer version see the guide on [configuring spack]({% link _guides/spack.markdown %})
What makes this machine special?
- We prefer intel-mkl for blas/lapack because of the Intel CPUs
- We use openssl and mpi from the system to avoid configuring them
- We load a new compiler from a module because the default is ancient
- Spack shares a home filesystem with other machines like
swing
these machines are completely different hardware wise and use different module systems. The load function loads a copy of spack specifically for Bebop and uses a separate spark instance for other machines. We use spack’sSPACK_USER_CONFIG_PATH
to keep these cleanly separate.
Changelog
- 2022-07-19 created this document
- 2022-08-11 updated to use Intel-MPI