1 [[!meta title="Introducing dqsub"]]
3 I've been using qsub for a while now on the cluster here at the
4 [IGB at UofI](http://www.igb.illinois.edu). qsub is a command line
5 program which is used to submit jobs to a scheduler to eventually be
6 run on one (or more) nodes of a cluster.
8 Unfortunately, qsub's interface is horrible. It requires that you
9 write a shell script for every single little thing you run, and
10 doesn't do simple things like providing defaults or running multiple
11 jobs at once with slightly different arguments. I've dealt with this
12 for a while using some rudimentary shell scripting, but I finally had
15 So instead, I wrote a wrapper around qsub called dqsub.
17 What used to require a complicated invocation like:
19 echo -e '#!/bin/bash\nmake foo'| \
20 qsub -q default -S /bin/bash -d $(pwd) \
21 -l mem=8G,nodes=1:ppn=4 -;
25 dqsub --mem 8G --ppn 4 make foo;
27 Want to run some command in every single directory which starts with
30 ls -1 SRX*|dqsub --mem 8G --ppn 4 --array chdir make bar;
32 Want instead to behave like xargs but do the same thing?
34 ls -1 SRX*|dqsub --mem 8G --ppn 4 --array xargs make bar -C;
36 Now, this wrapper isn't complete yet, but it's already more than
37 enough to do what I require, and has saved me quite a bit of time
40 [You can steal dqsub for yourself](http://git.donarmstrong.com/uiuc_igb_scripts.git/f/dqsub)
42 Feel free to request specific features, too.
45 [[!tag debian tech biology cluster]]