Institute of Formal and Applied Linguistics

slurm-template.sh

I use the following script as an entry point for most of my experiments. I contains the most important (for me) Slurm parameters and commands to run jobs on the ÚFAL cluster. Check NPFL118, ÚFAL wiki, sbatch docs for more info.

#!/bin/bash
#SBATCH -J MyJob                            # name of this job
#SBATCH -o ./logs/%x.%j.out                 # name of output file (./logs/MyJob.AllocationNumber.out)
#SBATCH -p gpu-ms,gpu-troja                 # name of GPU partition
#SBATCH -N 1                                # number of computation nodes to request
#SBATCH -c 1                                # number of CPUs to request (default 1)
#SBATCH -G 1                                # number of GPUs to request (default 0)
#SBATCH -C "gpuram16G|gpuram24G"            # request GPU with 16G OR 24G
#SBATCH --mem=16G                           # request 16G memory per node (rule of thumb: max 32G per GPU)
#SBATCH --mail-type=END,FAIL                # send a mail when the job completes or crashes
#SBATCH --mail-user=<name>@ufal.mff.cuni.cz

# Stop script in case of error.
set -e                 

# If not at least 1 argument was provided, send some output to stderr (>&2).
[ "$#" -ge 1 ] || { echo Usage: "$0 [args]" >&2; exit 1; }  

# Execute any bash commands. "$@" = "$1" "$2" "$3"...
source ./venv/bin/activate
python your_code.py "$@"

ufalmount

I use the following script to quickly mount directories from the ÚFAL cluster on my laptop. This can be more comfortable for remote development than, e.g., VSCode SSH, especially if you need to browse/transfer files and prefer to use a file manager GUI for that, or simply feel more comfortably using your own installed software than what is available on the cluster or your workstation. In this example, I mount all remote directories to a single directory ~/mounted/. remote cannot contain symlinks. My <destination> is my workstation at the office. Add the script to PATH (cf. NPFL118) to run it as a shell command.

#!/usr/bin/env bash

set -e

[ "$#" -eq 1 ] || { echo Usage: "$0 [home|dir1|dir2]" >&2; exit 1; }

case $1 in
  home)
    remote="" ;;

  dir1)
    remote="path to dir1" ;;

  dir2)
    remote="path to dir2" ;;

  *)
    echo Valid options: home dir1 dir2
    exit 1 ;;
esac

# unmount the local directory if it is already mounted
[[ $(findmnt -M "$HOME"/mounted/) ]] && umount "$HOME"/mounted/

sshfs -F "$HOME/.ssh/config" -o follow_symlinks <destination>:"$remote" "$HOME"/mounted/