Specifying SLURM Resources When Executing Multiple

问题:

According to the answers here What does the --ntasks or -n tasks does in SLURM? one can run multiple jobs in parallel via ntasks parameter for sbatch followed by srun. To ask a follow up question - how would one specify the amount of memory needed when running jobs in parallel like so?

If say 3 jobs are running in parallel each needing 8G of memory, would one specify 24G of memory in sbatch(i.e. the sum of memory from all jobs) or not give memory parameters in sbatch but instead specify 8G of memory for each srun?

回答1:

You need to specify the memory requirement in the script submitted with sbatch, otherwise you will end up with the default memory allocation, which might not correspond to your needs. If you then specify the 8GB memory in the srun call, you might end up with no jobs being able to start if the default memory is lower than that, or having only one or two jobs running in parallel if the default memory is between 16 and 24GB.

You can request --mem=24GB, but that offer less flexibility than specifying --mem-per-cpu=8G.