From: Don Armstrong <don@donarmstrong.com>
Date: Wed, 5 Aug 2015 21:48:26 +0000 (-0700)
Subject: add dqsub post
X-Git-Url: https://git.donarmstrong.com/?p=don.git;a=commitdiff_plain;h=90f19e3f091fe8a0e3eaafe3cd7eaf544eb85980

add dqsub post
---

diff --git a/posts/introducing_dqsub.mdwn b/posts/introducing_dqsub.mdwn
new file mode 100644
index 0000000..55b569b
--- /dev/null
+++ b/posts/introducing_dqsub.mdwn
@@ -0,0 +1,45 @@
+[[!meta title="Introducing dqsub"]]
+
+I've been using qsub for a while now on the cluster here at the
+[IGB at UofI](http://www.igb.illinois.edu). qsub is a command line
+program which is used to submit jobs to a scheduler to eventually be
+run on one (or more) nodes of a cluster.
+
+Unfortunately, qsub's interface is horrible. It requires that you
+write a shell script for every single little thing you run, and
+doesn't do simple things like providing defaults or running multiple
+jobs at once with slightly different arguments. I've dealt with this
+for a while using some rudimentary shell scripting, but I finally had
+enough.
+
+So instead, I wrote a wrapper around qsub called dqsub.
+
+What used to require a complicated invocation like:
+
+    echo -e '#!/bin/bash\nmake foo'| \
+     qsub -q default -S /bin/bash -d $(pwd) \
+      -l mem=8G,nodes=1:ppn=4 -;
+
+can now be run with
+
+    dqsub --mem 8G --ppn 4 make foo;
+
+Want to run some command in every single directory which starts with
+SRX? That's easy:
+
+    ls -1 SRX*|dqsub --mem 8G --ppn 4 --array chdir make bar;
+
+Want instead to behave like xargs but do the same thing?
+
+    ls -1 SRX*|dqsub --mem 8G --ppn 4 --array xargs make bar -C;
+
+Now, this wrapper isn't complete yet, but it's already more than
+enough to do what I require, and has saved me quite a bit of time
+already.
+
+You can steal it for yourself: [http://git.donarmstrong.com/uiuc_igb_scripts.git/f/dqsub]
+
+Feel free to request specific features, too.
+
+
+[[!tag debian tech biology cluster]]