Limiting the number of Matlab workers using the distributed computing toolbox.

This is a modification of how to run a matlab job on pegasus2 using the distributed computing toolbox.

By default, matlab will use 12 workers (as of R2013a/b) when opening a matlabpool. To use fewer workers, specify the number when calling matlabpool, for example modify "dct_example.m" as:

%=====================================================================
% DCT Example: Do nothing on Nworkers. Print datestamp to show
% how it's going.
%
% Art Gleason July 14, 2014
%=====================================================================
Nworkers = 4; % make sure this matches the number in dct_bsub.job
              % on the line #BSUB -n 
              % e.g. if Nworkers = 4 then use #BSUB -n 4
              % if Nworkers = 0 or 1 then use #BSUB -n 1

N = 20;       % bump this up if you want a longer job with many workers

%---open Nworkers, 0=serial, 12=max number possible---
if( Nworkers > 12 )
  Nworkers = 12;
elseif( Nworkers < 0 )
  Nworkers = 0;
end
if( Nworkers > 0 )
  matlabpool('open',Nworkers);
end

parfor(ix=1:N, Nworkers)
  ixstamp = sprintf('Iteration %d at %s\n', ix, datestr(now));
  disp(ixstamp);
  pause(5);
end

if( Nworkers > 0 )
  matlabpool('close');
end

Make sure #BSUB -n uses the correct number of workers, 4 in this case:

#!/bin/bash
#BSUB -J DCTexample
#BSUB -o %J.out
#BSUB -e %J.err
#BSUB -W 00:30
#BSUB -q general
#BSUB -n 4
#
### Run job
# cd not needed if CWD is the right one when this is submitted
# In other words, cd to the dir
# Note: it IS needed to module load matlab before submitting this
matlab < dct_example.m >& dct_example.log

You can also limit the number of workers when using the DCE. In fact, the DCE example already showed how to do that. In that example, we used 20 workers, you can just adjust the numbers to use fewer if needed to stay within memory limits.