changelog Updates

Did You Know ... Slack Edition

2020-02-13T20:19:00.001Z

Did you know we have a Slack channel for HPC/Research Computing?

Signup to our Carnegie Institution for Science workspace (click here) and then join the #hpc channel. Please use your Google login, "@carnegiescience.edu", email address.

This channel is useful for a couple of reasons (as well as other Carnegie Science channels!):
- Anyone can share photos, Googe Drive documents, articles, issues and best practices.
- Anyone can send me an informal message to talk about HPC/Research Computing/Memex issues, software questions, usage statistics, and much more. If I can’t answer or address your concerns right away, a formal ticket can be created for a faster response ***.
- Messages can be directed to a particular user or posted to the channel for everyone to see. General issues can be shared to #hpc and searched by other users with similar questions or issues.

Other things you can do are create a new private channel, have a group call (type "/call @user1 @user2"), set reminders (type "/remind help" for options), connect with a Github (type "/github help" for options), and much more (formatting messages, built-in commands)!

*** Slack does not replace our ticketing system. It is a way for us to discuss research computing without the formality of a ticketing system (by emailing [email protected]" target="_blank" rel="noopener">[email protected] or typing "/freshservice-ticket" from Slack’s direct messaging system).

Did You Know ... Python Edition

2020-01-23T16:56:00.001Z

Did you know official support for Python 2 is over?

That said, we have Python 3 available on Memex by loading the module, "python/3.6.7". This Python version includes conda, R, Jupyter, IntelMPI, and many other packages. Most Python packages can be installed by sending your request to [email protected]" target="_blank" rel="noopener">[email protected].
Did you know you can install your own Python packages using conda?

For example, if you’d like the latest python version available, 3.8.1, first search for it here first (mainly conda-forge channel) or directly from the command line with:
```
 module purge \
#only needed to ensure a clean environment
module load python/3.6.7
conda search python \
#latest should be 3.8.1 as of last week
```
and use it to set up your own conda environment (both anaconda.org and the command line shows the proper channel, "-c", to use below).
```
 conda create -p /lustre/scratch/$USER/.envs/py38 \
-c conda-forge python=3.8.1
```
The above command assumes /lustre/scratch/$USER/.envs is where you want the conda environment, py38, to reside. It has to be in a directory you own.

To activate any conda environment (using example, py38, above):
```
 source activate py38
```
To deactivate any conda environment:
```
 source deactivate
```
To remove an entire conda environment:
```
 conda-env remove -n py38
```
Did you know you can list existing conda environments already on Memex?

To list the available conda environments (still using the module, python/3.6.7), simply type:
```
 conda env list
```
if you can see them, you can activate them but if you don’t have write permission, you cannot install, remove, or modify them. You can however, clone them (make the new name unique–> "foo", not the clone name --> “foo2”):

conda create -p /lustre/scratch/$USER/.envs/foo --clone foo2
Did you know you can list any installed Python package, in the base conda environment (activated by module load python/3.6.7) or a custom conda environment (activated by source activate foo)?

conda list packagename #replace packagename with real package name

where “packagename” is a placeholder for a real package (like python, or r-base). Of course,
```
 conda list
```
lists all packages installed in your current environment.
Did you know there are other useful commands (#replace packagename and X with real values):

conda search packagename=X.X.X --info #where X.X.X=version_number of packagename
conda clean #removes downloaded packages and caches
conda remove packagename #to remove package
conda list --revisions #to identify a revision to rollback to
conda install --revision X #rollback to revision number, X

Type conda -h and/or conda-env -h for more options.

Note: To manage where your cache directory is located (includes tar balls and downloaded packages), you can set the following in your bashrc:

echo CONDA_PKGS_DIRS=/lustre/scratch/$USER/.envs/pkgs >> \
$HOME/.bashrc
source $HOME/.bashrc

Or in your current environment only with:

export CONDA_PKGS_DIRS=/lustre/scratch/$USER/.envs/pkgs

Did You Know ... Storage Edition

2020-01-15T15:57:00.001Z

Did you know there’s a 256GB quota for all /home directories?

This policy will be fully enforced in the coming weeks. This requirement is needed to manage space and load for /home. Home directories should primarily be used to set up scripts, small software applications, and environment variables across the cluster. To that end, we will be enforcing a hard quota of 256GB. A consolidation of all /home directories to one storage device will take place this year. The exact date will be advertised but please contact me or [email protected]" target="_blank" rel="noopener">[email protected] with any other questions.

Note: If you are currently over the limit, you will have time to move data but start moving (or removing) data now.

Did you know we have about 950TB of storage right now?

This includes the Lustre filesystem (/lustre, 698TB) and the MemexNFS filesystem (/work, /scratch, /share/apps, and /home ~ 252TB). Check out our login banner (see "Mountpoint Information") to see how each filesystem should be used.

Did you know we now have a Memex Globus endpoint, "cisuser#carnegiescience"?

Globus transfers are typically much faster than our command line option. Instructions on how to setup and use Globus on Memex is in our FreshService ticketing system (Solutions --> Computation).
Did you know you can check disk usage in the following ways?

For most users, the new command
```
 $ zquota
```
will show your /home usage and your group’s usage for /work/DEPT. Shared directories such as /share/apps, /scratch, and /lustre usage can seen by the ‘df’ command. For example:
```
 $ df -h /lustre /share/apps /scratch
```
shows usage for all three filesystems. Of course, other commands like $ du -sh /home/username/directory/ for directories or $ ls -Shl filename for files can be used as well.

Did You Know ... SLURM Edition

2020-01-13T21:58:00.001Z

Did you know there’s a gui to view SLURM jobs? Inside a VNC or “ssh -XY …” session, type sview -a from the command line (check out the menu selection as well):

Did you know you can view the maximum resources of each node with:

sinfo -e -o "%20N %10c %10m %25f %10G"
Did you know you can view the maximum memory used for running jobs with (see “/bin/sacct -e” for format options):

/bin/sacct --format="JobID,CPUTime,MaxRSS" -j JOBID

better yet, try sstat --format="jobid,maxrss,avecpu" -j JOBID (“sstat -e” for format options)?
Did you know you can email yourself job status changes?

#SBATCH [email protected]
#SBATCH --mail-type=FAIL,BEGIN,END,SUSPEND

Here are a few more tips and you can contact me directly with any questions.