+++ /dev/null
-Author: Martin Asser Hansen - Copyright (C) - All rights reserved
-
-Contact: mail@maasha.dk
-
-Date: May 2008
-
-License: GNU General Public License version 2 (http://www.gnu.org/copyleft/gpl.html)
-
-Description: Adds a unique identifier to each record in stream.
-
-Usage: ... | $script [options]
-
-Options: [-k <string> | --key=<string>] - Identifier key - Default=ID
-Options: [-p <string> | --prefix=<string>] - Identifier prefix - Default=ID
-Options: [-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-Options: [-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-
-Examples: ... | $script - Add identifier to all records.
-Examples: ... | $script -k SEQ_NAME - Change the identifier key to SEQ_NAME.
-Examples: ... | $script -p ID_ - Change identifier prefix from ID00000000 to ID_00000000.
-
-Keys out: <ID> - The specified key.
-
--- /dev/null
+=Biopiece: add_ident=
+
+==Synopsis==
+
+Adds a unique identifier to each record in the stream.
+
+==Description==
+
+For records without an identifier, or where the identifier is non-unique, a new identifier can be added with *add_ident* - or existing identifiers can be replaced.
+
+==Usage==
+
+{{{
+... | add_ident [options]
+}}}
+
+==Options==
+
+{{{
+[-k <string> | --key=<string>] - Identifier key - Default=ID
+[-p <string> | --prefix=<string>] - Identifier prefix - Default=ID
+[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
+[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
+}}}
+
+==Examples==
+
+Consider the following table:
+
+{{{
+Organism Sequence Count
+Human ATACGTCAG 23524
+Dog AGCATGAC 2442
+Mouse GACTG 234
+Cat AAATGCA 2342
+}}}
+
+We use [read_tab] to get the Sequence column, and the *add_ident* to generate a unique identifier for each record:
+
+{{{
+read_tab -i test.tab -s 1 -c 1 -k SEQ | add_ident
+
+ID: ID00000000
+SEQ: ATACGTCAG
+---
+ID: ID00000001
+SEQ: AGCATGAC
+---
+ID: ID00000002
+SEQ: GACTG
+---
+ID: ID00000003
+SEQ: AAATGCA
+---
+}}}
+
+However, if you want to output the sequence with [write_fasta] then the new key ID can be replaced with the required SEQ_NAME using the `-k`
+switch:
+
+{{{
+read_tab -i test.tab -s 1 -c 1 -k SEQ | add_ident -k SEQ_NAME
+
+SEQ: ATACGTCAG
+SEQ_NAME: ID00000000
+---
+SEQ: AGCATGAC
+SEQ_NAME: ID00000001
+---
+SEQ: GACTG
+SEQ_NAME: ID00000002
+---
+SEQ: AAATGCA
+SEQ_NAME: ID00000003
+---
+}}}
+
+Finally, if you want to change the format of the identifier, the prefix can be changed with the `-p` switch:
+
+{{{
+read_tab -i test.tab -s 1 -c 1 -k SEQ | add_ident -k SEQ_NAME -p ID_
+
+SEQ: ATACGTCAG
+SEQ_NAME: ID_00000000
+---
+SEQ: AGCATGAC
+SEQ_NAME: ID_00000001
+---
+SEQ: GACTG
+SEQ_NAME: ID_00000002
+---
+SEQ: AAATGCA
+SEQ_NAME: ID_00000003
+---
+}}}
+
+==See also==
+
+[read_tab]
+
+[write_fasta]
+
+==Author==
+
+Martin Asser Hansen - Copyright (C) - All rights reserved.
+mail@maasha.dk
+August 2007
+
+==License==
+
+GNU General Public License version 2
+
+http://www.gnu.org/copyleft/gpl.html
+
+==Help==
+
+*add_ident* is part of the Biopieces framework.
+
+http://code.google.com/p/biopieces/
+++ /dev/null
-Author: Martin Asser Hansen - Copyright (C) - All rights reserved
-
-Contact: mail@maasha.dk
-
-Date: August 2007
-
-License: GNU General Public License version 2 (http://www.gnu.org/copyleft/gpl.html)
-
-Description: Grab records in stream.
-
-Usage: ... | $script [options]
-
-Options: [-p <string> | --patterns=<string>] - Grab using comma separated list of patterns.
-Options: [-P <file> | --patterns_in=<file>] - Grab using patterns from file - one pattern per line.
-Options: [-r <string> | --regex=<string>] - Grab using Perl regex.
-Options: [-e <string> | --eval=<string> - Grab 'key,operator,value'. Operators: '>,<,>=,<=,=,!=,eq,ne'.
-Options: [-E <file> | --exact_in=<file> - Grab using exact expressions from file - one expression per line.
-Options: [-i | --invert] - Display non-mathing results.
-Options: [-c | --case_insensitive] - Turn regex matching case insensitive.
-Options: [-k <string> | --keys=<string>] - Comma separated list of keys to grab the value for.
-Options: [-K | --keys_only] - Only grab for keys.
-Options: [-V | --vals_only] - Only grab for vals.
-Options: [-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-Options: [-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-
-Examples: ... | $script -p SEQ - Grab all records matching SEQ in keys or vals.
-Examples: ... | $script -p 110 -k SEQ_LEN - Grab all records where SEQ_LEN matching 110.
-Examples: ... | $script -p 110 -k SEQ_LEN -i - Grab all records where SEQ_LEN is not 110.
-Examples: ... | $script -r 'A[TUG]C' - Grab all records matching ATC, AUC, or AGC.
-Examples: ... | $script -p SEQ -K - Grab all records with a key matching SEQ.
-Examples: ... | $script -p SEQ -V - Grab all records with a value matching SEQ.
-Examples: ... | $script -e 'SEQ_LEN<30' - Grab all records with a SEQ_LEN less than 30.
-Examples: ... | $script -e 'OS eq D.mel' - Grab all records with OS equal to 'D.mel'.
-Examples: ... | $script -E names.txt - Grab all records with exact match to names in file.
-Examples: ... | $script -E seq.txt -i - Grab all records with no exact match to sequences in file.
--- /dev/null
+=Biopiece: grab=
+
+==Synopsis==
+
+Grab records in stream.
+
+==Description==
+
+*grab* selects records from the stream by matching keys or values using a pattern, a regular expression, or a numerical evaluation. *grab* is biopieces' equivalent of Unix' grep, however, *grab* is much more versatile.
+
+==Usage==
+
+{{{
+... | grab [options]
+}}}
+
+==Options==
+
+{{{
+[-p <string> | --patterns=<string>] - Grab using comma separated list of patterns.
+[-P <file> | --patterns_in=<file>] - Grab using patterns from file - one pattern per line.
+[-r <string> | --regex=<string>] - Grab using Perl regex.
+[-e <string> | --eval=<string> - Grab 'key,operator,value'. Operators: '>,<,>=,<=,=,!=,eq,ne'.
+[-E <file> | --exact_in=<file> - Grab using exact expressions from file - one expression per line.
+[-i | --invert] - Display non-mathing results.
+[-c | --case_insensitive] - Turn regex matching case insensitive.
+[-k <string> | --keys=<string>] - Comma separated list of keys to grab the value for.
+[-K | --keys_only] - Only grab for keys.
+[-V | --vals_only] - Only grab for vals.
+[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
+[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
+}}}
+
+==Examples==
+
+To easily *grab* all records in the stream that has any mentioning of the pattern 'human' just pipe the data stream through *grab* like this:
+
+{{{
+... | grab -p human
+}}}
+
+This will search for the pattern 'human' in all keys and all values. The `-p` switch takes a comma separated list of patterns, so in order to match multiple patterns do:
+
+{{{
+... | grab -p human,mouse
+}}}
+
+It is also possible to use the `-P` switch instead of `-p`. `-P` is used to read a file with one pattern per line:
+
+{{{
+... | grab -P patterns.txt
+}}}
+
+If you want the opposite result - to find all records that does not match the patterns, add the `-i` switch, which not only works with the `-p` and `-P` switch, but also with `-r` and `-e`:
+
+{{{
+... | grab -p human -i
+}}}
+
+If you want to search the record keys only, e.g. to find all records containing the key SEQ you can add the `-K` switch. This will prevent matching of SEQ in any record value, and in fact SEQ is a not uncommon peptide sequence you could get an unwanted record. Also, this will give an increase in speed since only the keys are searched:
+
+{{{
+... | grab -p SEQ -K
+}}}
+
+However, if you are interested in finding the peptide sequence SEQ and not the SEQ key, just add the `-V` switch instead:
+
+{{{
+... | grab -p SEQ -V
+}}}
+
+Also, if you want to *grab* for certain key/value pairs you can supply a comma separated list of keys whos values will then be searched using the `-k` switch. This is handy if your records contain large genomic sequences and you don't want to search the entire sequence for e.g. the organism name - it is much faster to tell *grab* which keys to search the value for:
+
+{{{
+... | grab -p human -k SEQ_NAME
+}}}
+
+It is also possible to invoke flexible matching using regex (regular expressions) instead of simple pattern matching. In *grab* the regex engine is Perl based, and allows use of different type of wild cards, alternatives, etc. If you want to *grab* records withs the sequence ATCG or GCTA you can do this:
+
+{{{
+... | grab -r 'ATCG|GCTA'
+}}}
+
+Or if you want to find sequences beginning with ATCG:
+
+{{{
+... | grab -r '^ATCG'
+}}}
+
+You can also use *grab* to locate records that fulfill a numerical property using the `-e` switch witch takes an expression in three parts. The first part is the key that holds the value we want to evaluate, the second part holds one if these eight operators:
+
+ # Greater than: >
+ # Greater than or equal to: >=
+ # Less than: <
+ # Less than or equal to: <=
+ # Equal to: =
+ # Not equal to: !=
+ # String wise equal to: eq
+ # String wise not equal to: ne
+
+And finally comes the number used in the evaluation. So to *grab* all records with a sequence length greater than 30:
+
+{{{
+... | grab -e 'SEQ_LEN > 30'
+}}}
+
+If you want to locate all records containing the pattern 'human' and where the sequence length is greater that 30, you do this by running the stream through *grab* twice:
+
+{{{
+... | grab -p 'human' | grab -e 'SEQ_LEN > 30'
+}}}
+
+Finally, it is possible to do fast matching of expressions from a file using the `-E` switch. Each of these expressions has to be matched exactly over the entrie length, which if useful if you e.g. have a file with accession numbers, that you want to locate in the stream:
+
+{{{
+... | grab -E acc_no.txt
+}}}
+
+Using `-E` is much faster than using `-P`, because with `-E` the expression has to be complete matches, where `-P` looks for subpatterns.
+
+NB! To get the best speed performance, use the most restrictive *grab* first.
+
+==See also==
+
+
+==Author==
+
+Martin Asser Hansen - Copyright (C) - All rights reserved.
+mail@maasha.dk
+August 2007
+
+==License==
+
+GNU General Public License version 2
+
+http://www.gnu.org/copyleft/gpl.html
+
+==Help==
+
+*grab* is part of the Biopieces framework.
+
+http://code.google.com/p/biopieces/
+++ /dev/null
-Author: Martin Asser Hansen - Copyright (C) - All rights reserved
-
-Contact: mail@maasha.dk
-
-Date: August 2007
-
-License: GNU General Public License version 2 (http://www.gnu.org/copyleft/gpl.html)
-
-Description: Write records from stream as tab separated table.
-
-Usage: ... | $script [options]
-
-Options: [-x | --no_stream] - Do not emit records.
-Options: [-o <file> | --data_out=<file>] - Write result to file.
-Options: [-c | --comment] - Print comment line - Default=no
-Options: [-d <string> | --delimit=<string>] - Changes delimiter - Default='\t'
-Options: [-k <string> | --keys=<string>] - Comma separated list of keys to print in that order.
-Options: [-K <string> | --no_keys=<string>] - Comma separated list of keys to ignore.
-Options: [-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-Options: [-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-Options: [-Z | --compress] - Compress output using gzip.
-
-Examples: ... | $script -c -o table.csv - Output tabular data to file 'table.csv' with comment line.
-Examples: ... | $script -k SEQ_NAME,SEQ -x - Output tabular data for columns SEQ_NAME and SEQ only.
-Examples: ... | $script -d ',' -K SEQ - Output comma separated data ignoring SEQ column.
-Examples: ... | $script -Z -o test.tab.gz - Output zipped tabular data to file 'test.tab.gz'.
--- /dev/null
+=Biopiece: write_tab=
+
+==Synopsis==
+
+Write tabular output from stream.
+
+==Description==
+
+Outputting the data stream as a table can be done with *write_tab*, which will write generate one row per record with the values as columns.
+
+==Usage==
+
+{{{
+... | write_tab [options]
+}}}
+
+==Options==
+
+{{{
+[-x | --no_stream] - Do not emit records.
+[-o <file> | --data_out=<file>] - Write result to file.
+[-c | --comment] - Print comment line - Default=no
+[-d <string> | --delimit=<string>] - Changes delimiter - Default='\t'
+[-k <string> | --keys=<string>] - Comma separated list of keys to print in that order.
+[-K <string> | --no_keys=<string>] - Comma separated list of keys to ignore.
+[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
+[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
+[-Z | --compress] - Compress output using gzip.
+}}}
+
+==Examples==
+
+To write all records from the stream as a table, do:
+
+{{{
+... | write_tab
+}}}
+
+If you supply the optional `-c` switch, when the first row in the table will be a 'comment' line prefixed with a '#':
+
+{{{
+... | write_tab -c
+}}}
+
+You can also change the delimiter from the default (tab) to e.g. ',':
+
+{{{
+... | write_tab -d ','
+}}}
+
+If you want the values output in a specific order you have to supply a comma separated list using the `-k` switch that will print only those keys in that order:
+
+{{{
+... | write_tab -k SEQ_NAME,COUNT
+}}}
+
+Keys from e.g. [read_tab] V0, V1, V2 ... Vn, is automagically sorted numerically.
+
+Alternatively, if you have some keys that you don't want in the tabular output, use the `-K` switch. So to print all keys except SEQ and SEQ_TYPE do:
+
+{{{
+... | write_tab -K SEQ,SEQ_TYPE
+}}}
+
+Finally, if you have a stream containing a mix of different records types, e.g. records with sequences and records with matches, then you can use *write_tab* to output all the records in tabluar format, however, the `-c`, `-k`, and `-K` switches will only respond to records of the first type encountered. The reason is that outputting mixed records is probably not what you want anyway, and you should remove all the unwanted records from the stream before outputting the table: [grab] is your friend.
+
+==See also==
+
+[read_tab]
+
+[grab]
+
+==Author==
+
+Martin Asser Hansen - Copyright (C) - All rights reserved.
+mail@maasha.dk
+August 2007
+
+==License==
+
+GNU General Public License version 2
+
+http://www.gnu.org/copyleft/gpl.html
+
+==Help==
+
+*write_tab* is part of the Biopieces framework.
+
+http://code.google.com/p/biopieces/
projects / biopieces.git / commitdiff