User Tools

Site Tools


docs:trillian

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
docs:trillian [2015/06/15 09:52]
gibson [Installation]
docs:trillian [2015/07/02 13:28] (current)
gibson
Line 1: Line 1:
 ====== Channelflow on trillian ====== ====== Channelflow on trillian ======
 +
 +Some extra instructions for installing and using channelflow on 
 +[[http://​trillian-use.sr.unh.edu/​index.php/​Main_Page|trillian]],​
 +UNH's CRAY XE6m-200 supercomputer.
  
 ===== Installation ===== ===== Installation =====
Line 27: Line 31:
 mkdir ~/​channelflow/​build mkdir ~/​channelflow/​build
 cd ~/​channelflow/​build cd ~/​channelflow/​build
-cmake -DCMAKE_CXX_COMPILER=/​opt/​gcc/​4.7.2/​bin/​g++ -DWITH_FFTW=/​opt/​fftw/​3.3.0.2/​x86_64/​lib -DWITH_HDF5=~gibson/​packages/​hdf5-1.8.15/​lib -DWITH_EIGEN3=~gibson/​packages/​eigen-3.2.4 -DCMAKE_INSTALL_PREFIX=~/​channelflow ~/​svnrepos/​channelflow/​trunk/​+cmake -DCMAKE_C_COMPILER=/​opt/​gcc/​4.7.2/​bin/​gcc ​-DCMAKE_CXX_COMPILER=/​opt/​gcc/​4.7.2/​bin/​g++ -DWITH_FFTW=/​opt/​fftw/​3.3.0.2/​x86_64/​lib -DWITH_HDF5=~gibson/​packages/​hdf5-1.8.15/​lib -DWITH_EIGEN3=~gibson/​packages/​eigen-3.2.4 -DCMAKE_INSTALL_PREFIX=~/​channelflow ~/​svnrepos/​channelflow/​trunk/​
 make make
 make test make test
Line 37: Line 41:
 ~/​svnrepos/​channelflow ​  # pristine channelflow source code from subversion ~/​svnrepos/​channelflow ​  # pristine channelflow source code from subversion
 ~/​channelflow/​build ​     # build directory, where channelflow was configured and compiled ~/​channelflow/​build ​     # build directory, where channelflow was configured and compiled
-~/​channelflow/​include ​   # include directory, with all the channelflow header files (e.g. ''​flowfield.h''​+~/​channelflow/​include ​   # include directory, with all the channelflow header files (e.g. flowfield.h) 
-~/​channelflow/​lib ​       # library directory, with static and shared libs ''​libchflow.a'' ​and ''​libchflow.so'' ​            +~/​channelflow/​lib ​       # library directory, with static and shared libs libchflow.a and libchflow.so 
 +</​code> ​           ​
    
 +Note: Sometimes CMake can get confused if changes in the build system conflict with cached values of build system variables. If the above sequence doesn'​t work for you, try removing ''​CMakeCache.txt''​ and ''​CMakeFiles/''​ from your build directory and rerunning the ''​cmake ...''​ command above.
 +
 ===== Submitting jobs ===== ===== Submitting jobs =====
 +
 +If you start a computation by just typing in a command at the command prompt, it'll execute on trillian'​s login node. That's a no-no! (except for software builds and quick tests). Instead, you should run any long computation but submitting it to the [[https://​hpcc.usc.edu/​support/​documentation/​running-a-job-on-the-hpcc-cluster-using-pbs/​PBS|PBS]] (Portable Batch System) job control system. PBS will then farm the job out to one of the compute nodes. ​
 +
 +There are three main PBS commands:
 +  * **qsub** submit a job to the PBS queue
 +  * **qstat** list jobs running in the queue
 +  * **qdel** delete a job from the queue
 +
 +''​qsub''​ can be a bit tedious and complicated,​ so I wrote a some bash code for a simplified interface. Place the following in your ''​~/​.bashrc''​ file (and then run ''​source ~/​.bashrc''​) to make your current shell process and load this code).
 +
 +<​code>​
 +PATH=$PATH:​~/​channelflow/​bin
 +module unload cray-hdf5-parallel
 +module unload cce
 +module unload xt-asyncpe
 +module load gcc
 +module load fftw
 +module load cmake
 +module load pbs
 +
 +function qsubmit() {
 +  tag=$1
 +  shift
 +  echo "#PBS -N $tag" > tmp.pbs
 +  echo "#PBS -l nodes=1:​ppn=1,​walltime=48:​00:​00"​ >> tmp.pbs
 +  echo "#PBS -j oe" ​  >>​ tmp.pbs
 +  echo "#PBS -m ae" ​  >>​ tmp.pbs
 +  echo "cd $(pwd)" ​   >> tmp.pbs
 +  echo aprun $*       >>​ tmp.pbs
 +  qsub tmp.pbs
 +}
 +</​code>​
 +This bit of code tells bash where your channelflow programs are, configures the Cray programming environment correctly, and then defines a ''​qsubmit''​ function, which can be used as follows ​
 +
 +<​code>​
 +cd ~/​simulations/​test-qsubmit
 +qsubmit qtest couette -R 500 u0.h5
 +</​code>​
 +
 +This will submit the command ''​couette -R 500 u0.h5''​ to the PBS queue with jobname "​qtest",​ in working directory ''​~/​simulations/​test-qsubmit''​.
 +
  
docs/trillian.1434387148.txt.gz · Last modified: 2015/06/15 09:52 by gibson