+++ /dev/null
-This directory contains the 'usage' file for all biopieces.
-They are written in Google's wiki format descripted here:
-
-http://code.google.com/p/support/wiki/WikiSyntax
-
-Since the format is quite imcomplete, only some of the syntax
-is supported by the biopieces until Google upgrades it. The
-reason for using this format is that it will render nicely
-on the Google biopieces wiki:
-
-http://code.google.com/p/biopieces/w/list
-
-... and at the same time, the biopiece *read_gwiki*
-
-will deal with simple ASCII markup.
-
-The below features are supported, and if more is needed, look at
-the ~/code_perl/Maasha/Gwiki.pm module.
-
-= heading =
-== subheading ==
-=== Level 3 ===
-
-{{{
-verbatim block
-}}}
-
-
-*inline bold*
-
-_inline emphasis_
-
-`inline verbatim`
-
----- (horizontal line)
-
-
-
-Martin A. Hansen, June 2008
+++ /dev/null
-=Biopiece: add_ident=
-
-==Synopsis==
-
-Adds a unique identifier to each record in the stream.
-
-==Description==
-
-For records without an identifier, or where the identifier is non-unique,
-a new identifier can be added with [add_ident] - or existing identifiers can be replaced.
-
-==Usage==
-
-{{{
-... | add_ident [options]
-}}}
-
-==Options==
-
-{{{
-[-k <string> | --key=<string>] - Identifier key - Default=ID
-[-p <string> | --prefix=<string>] - Identifier prefix - Default=ID
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the following table:
-
-{{{
-Organism Sequence Count
-Human ATACGTCAG 23524
-Dog AGCATGAC 2442
-Mouse GACTG 234
-Cat AAATGCA 2342
-}}}
-
-We use [read_tab] to get the Sequence column, and the [add_ident] to generate a unique identifier for each record:
-
-{{{
-read_tab -i test.tab -s 1 -c 1 -k SEQ | add_ident
-
-ID: ID00000000
-SEQ: ATACGTCAG
----
-ID: ID00000001
-SEQ: AGCATGAC
----
-ID: ID00000002
-SEQ: GACTG
----
-ID: ID00000003
-SEQ: AAATGCA
----
-}}}
-
-However, if you want to output the sequence with [write_fasta] then the new key ID can be replaced with the required SEQ_NAME using the `-k`
-switch:
-
-{{{
-read_tab -i test.tab -s 1 -c 1 -k SEQ | add_ident -k SEQ_NAME
-
-SEQ: ATACGTCAG
-SEQ_NAME: ID00000000
----
-SEQ: AGCATGAC
-SEQ_NAME: ID00000001
----
-SEQ: GACTG
-SEQ_NAME: ID00000002
----
-SEQ: AAATGCA
-SEQ_NAME: ID00000003
----
-}}}
-
-Finally, if you want to change the format of the identifier, the prefix can be changed with the `-p` switch:
-
-{{{
-read_tab -i test.tab -s 1 -c 1 -k SEQ | add_ident -k SEQ_NAME -p ID_
-
-SEQ: ATACGTCAG
-SEQ_NAME: ID_00000000
----
-SEQ: AGCATGAC
-SEQ_NAME: ID_00000001
----
-SEQ: GACTG
-SEQ_NAME: ID_00000002
----
-SEQ: AAATGCA
-SEQ_NAME: ID_00000003
----
-}}}
-
-==See also==
-
-[read_tab]
-
-[write_fasta]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[add_ident] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: analyze_seq=
-
-==Synopsis==
-
-Analyzes the sequence composition of sequences in the stream.
-
-==Description==
-
-[analyze_seq] analyzes the sequence specified by the SEQ key in each record. This is
-done by automagically guessing the sequence type based on the first 100 residues, and
-then determining the frequency of relevant residues and indels. Futhermore, GC% is
-determined for nucleotide sequence as well as SOFT_MASK% (soft masked sequence is
-indicated by lower case letters), HARD_MASK% (hard masked sequence consists of N's).
-Also, a MIX_INDEX is calculated that indicates the complexity of the sequence defined as
-the most common residue over the sequence length (that [analyze_seq] also determines).
-Finally, for nucleotide sequences, a melting temperature is calculated (4 degrees per GC-pair,
-2 degrees per AT/U-pair)
-
-==Usage==
-
-{{{
-... | analyze_seq [options]
-}}}
-
-==Options==
-
-{{{
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the file `test.fna` containing the single entry:
-
-{{{
->test
-ACGACGCATNNNNNNactgatcga
-}}}
-
-To analyze this sequence, read the file using [read_fasta] and then pipe the stream through [analyze_vals]:
-
-{{{
-read_fasta -i test.fna | analyze_seq
-
-RES:D: 0
-MIX_INDEX: 0.25
-RES:W: 0
-RES:G: 4
-RES:B: 0
-SOFT_MASK%: 37.50
-RES:V: 0
-SEQ_NAME: test
-RES_SUM: 24
-HARD_MASK%: 25.00
-RES:H: 0
-RES:S: 0
-RES:.: 0
-RES:N: 6
-RES:A: 6
-GC%: 37.50
-MELT_TEMP: 54.00
-RES:Y: 0
-RES:M: 0
-RES:T: 3
-RES:_: 0
-RES:K: 0
-SEQ_TYPE: DNA
-RES:~: 0
-SEQ: ACGACGCATNNNNNNactgatcga
-RES:R: 0
-SEQ_LEN: 24
-RES:C: 5
-RES:-: 0
-RES:U: 0
----
-}}}
-
-If you have a stack of sequences in one file and you want to determine the mean GC content
-of all the sequences, you can do it using the [mean_vals] biopiece:
-
-{{{
-read_fasta -i test.fna | analyze_seq | mean_vals -k GC% -x
-
-GC%_MEAN: 37.50
----
-}}}
-
-Similarly, if you want the total count of Ns in all sequences use the biopiece [sum_vals]:
-
-{{{
-read_fasta -i test.fna | analyze_seq | sum_vals -k RES:N
-
-RES:N_SUM: 6
----
-}}}
-
-Finally, if you want to remove low complexity sequence from the stream, [grab] is your friend:
-
-{{{
-read_fasta -i test.fna | analyze_seq | grab -e 'MIX_INDEX<0.85'
-}}}
-
-==See also==
-
-[read_fasta]
-
-[grab]
-
-[mean_vals]
-
-[sum_vals]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[analyze_seq] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: complement_seq=
-
-==Synopsis==
-
-Complement nucleotide sequences in the stream.
-
-==Description==
-
-[complement_seq] complements nucleotide sequences in the stream which along with [reverse_seq]
-results in reverse/complemented sequence.
-
-==Usage==
-
-{{{
-... | complement_seq [options]
-}}}
-
-==Options==
-
-{{{
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-To reverse complement sequence first reverse the sequence with [reverse_seq] and then
-complement it using [complement_seq] (or visa versa):
-
-{{{
-... | complement_seq | reverse_seq
-}}}
-
-==See also==
-
-[reverse_seq]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[complement_seq] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: count_records=
-
-==Synopsis==
-
-Count the number of records in the stream.
-
-==Description==
-
-[count_records] count the number of records in the stream and outputs the count as a
-record who's count is _not_ included. Using [count_recrods] with the `-x` switch is
-a lot faster because this prevents the need for parsing and emitting the records.
-
-==Usage==
-
-{{{
-... | count_records [options]
-}}}
-
-==Options==
-
-{{{
-[-x | --no_stream] - Do not emit records.
-[-o <file> | --data_out=<file>] - Write result to file.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-To count the records in the stream and saving the result to file, do:
-
-{{{
-... | count_record -o <file>
-}}}
-
-The resulting file will then contain some thing like this:
-
-{{{
-count_records: 1234
----
-}}}
-
-==See also==
-
-[count_vals]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[count_records] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: count_vals=
-
-==Synopsis==
-
-Count the number of times values of given keys exists in stream
-
-==Description==
-
-Given a comma seperated list of keys [count_vals] for each of these keys counts the
-number of identical values. Since the count basically is dependant on one hash per key,
-[count_vals] have the potential to blow the memory quite easily. This is countered by
-caching the count to disk for every 5 million records, however, the disk caching may be slow.
-
-==Usage==
-
-{{{
-... | count_vals [options]
-}}}
-
-==Options==
-
-{{{
-[-k <string> | --keys=<string>] - Comma separeted list of keys.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the following two column table in the file `test.tab`:
-
-{{{
-Human H2
-Human H3
-Dog D1
-Dog D2
-Mouse M1
-}}}
-
-To count the values of both columns we first read the table with [read_tab] and then pipe the result to [count_vals]:
-
-{{{
-read_tab -i test.tab | count_vals -k V0,V1
-
-V0: Human
-V1_COUNT: 1
-V1: H1
-V0_COUNT: 3
----
-V0: Human
-V1_COUNT: 1
-V1: H2
-V0_COUNT: 3
----
-V0: Human
-V1_COUNT: 1
-V1: H3
-V0_COUNT: 3
----
-V0: Dog
-V1_COUNT: 1
-V1: D1
-V0_COUNT: 2
----
-V0: Dog
-V1_COUNT: 1
-V1: D2
-V0_COUNT: 2
----
-V0: Mouse
-V1_COUNT: 1
-V1: M1
-V0_COUNT: 1
----
-}}}
-
-The result is that for each of the specified keys (V0 and V1) a new key with the suffix _COUNT
-is added where the value is the global count. The result is better displayed after piping through [write_tab]:
-
-{{{
-read_tab -i test.tab | count_vals -k V0,V1 | write_tab -xck V0,V0_COUNT,V1,V1_COUNT
-
-#V0 V0_COUNT V1 V1_COUNT
-Human 3 H1 1
-Human 3 H2 1
-Human 3 H3 1
-Dog 2 D1 1
-Dog 2 D2 1
-Mouse 1 M1 1
-}}}
-
-==See also==
-
-[read_tab]
-
-[write_tab]
-
-[uniq_vals]
-
-[grab]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[count_vals] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: create_weight_matrix=
-
-==Synopsis==
-
-Create a residue composition weight matrix of an alignment in the stream.
-
-==Description==
-
-[create_weight_matrix] calculates the frequency of all residues per column in aligned
-sequences from the stream - either as exact residue counts or percentages.
-
-==Usage==
-
-{{{
-... | create_weight_matrix [options]
-}}}
-
-==Options==
-
-{{{
-[-p | --percent] - Output the result in percent - Default=absolute
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the following alignment in the file `aln.fna` in FASTA format:
-
-{{{
->test5
----TAACAGGCACT
->test2
------GAATCGACT
->test1
---CTAGCTTCGACT
->test3
-ACGAAACTAGCATC
->test4
-----AGCATCGACT
-}}}
-
-To create a weight matrix from the above alignment, read it in with [read_fasta] and pipe the
-stream through [create_weight_matrix]:
-
-{{{
-read_fasta -i aln.fna | create_weight_matrix
-}}}
-
-The resulting five records will look the first one below, which is not really understandable:
-
-{{{
-V13: 0
-V11: 0
-V7: 0
-V4: 2
-V3: 3
-V9: 0
-V0: -
-V2: 4
-V8: 0
-V12: 0
-V5: 1
-V10: 0
-V1: 4
-V6: 0
-V14: 0
----
-}}}
-
-To make sense pipe the result through [write_tab] like this:
-
-{{{
-read_fasta -i aln.fna | create_weight_matrix | write_tab -x
-
-- 4 4 3 2 1 0 0 0 0 0 0 0 0 0
-A 1 0 0 1 4 2 1 3 1 0 0 5 0 0
-C 0 1 1 0 0 0 4 0 0 3 2 0 4 1
-G 0 0 1 0 0 3 0 0 1 2 3 0 0 0
-T 0 0 0 2 0 0 0 2 3 0 0 0 1 4
-}}}
-
-The above weight matrix shows the frequencies of all residue types (1st column) found at
-all positions throughout the alignment.
-
-To obtain the percentwise frequencies use the `-p` switch to [create_weight_matrix]:
-
-{{{
-read_fasta -i aln.fna | create_weight_matrix -p | write_tab -x
-
-- 80 80 60 40 20 0 0 0 0 0 0 0 0 0
-A 20 0 0 20 80 40 20 60 20 0 0 100 0 0
-C 0 20 20 0 0 0 80 0 0 60 40 0 80 20
-G 0 0 20 0 0 60 0 0 20 40 60 0 0 0
-T 0 0 0 40 0 0 0 40 60 0 0 0 20 80
-}}}
-
-==See also==
-
-[read_fasta]
-
-[write_tab]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[create_weight_matrix] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: extract_seq=
-
-==Synopsis==
-
-Extract subsequence from sequences in stream.
-
-==Description==
-
-[extract_seq] extracts a subsequence from a sequence in all records in the stream.
-The sequence is then _replaced_ with this subsequence.
-
-==Usage==
-
-{{{
-... | extract_seq [options]
-}}}
-
-==Options==
-
-{{{
-[-b <int> | --beg=<int>] - Begin position of subsequence (first residue=1)
-[-e <int> | --end=<int>] - End position of subsequence
-[-l <int> | --len=<int>] - Length of subsequence
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the following FASTA entry in the file `test.fna`:
-
-{{{
->test
-ACGACGCATNNNNNNactgatcga
-}}}
-
-To obtains the subsequence from position 5 (first residue is 1) to postion 10 we
-first read in the sequence using [read_fasta] and then we pipe the stream to
-[extract_seq]:
-
-{{{
-read_fasta -i test.fna | extract_seq -b 5 -e 10
-
-SEQ: CGCATN
-SEQ_LEN: 6
-SEQ_NAME: test
----
-}}}
-
-Note the positions (first position is 1 ) and the returned sequence:
-
-{{{
-1 10 20
-| | |
-123456789012345678901234
-ACGACGCATNNNNNNactgatcga
-}}}
-
-We could also have specified a length with `-l` instead of end postion with `-e`:
-
-{{{
-read_fasta -i test.fna | extract_seq -b 5 -l 5
-
-SEQ: CGCAT
-SEQ_LEN: 5
-SEQ_NAME: test
----
-}}}
-
-Now, if we only specify the begin position, what happens?
-
-{{{
-read_fasta -i test.fna | extract_seq -b 5
-
-SEQ: CGCATNNNNNNactgatcga
-SEQ_LEN: 20
-SEQ_NAME: test
----
-}}}
-
-Or if we only speficy the end postion?
-
-{{{
-read_fasta -i test.fna | extract_seq -b 5 -e 10
-
-SEQ: ACGACGCATN
-SEQ_LEN: 10
-SEQ_NAME: test
----
-}}}
-
-Or what about if we only specify the length?
-
-{{{
-read_fasta -i test.fna | extract_seq -l 5
-
-SEQ: ACGAC
-SEQ_LEN: 5
-SEQ_NAME: test
----
-}}}
-
-That is quite practical if we want the first five residues of all the sequences,
-but what if we want the five _last_ residues? Easy! We use [reverse_seq] to reverse
-the sequences, and then we get the first 5 residues (which in fact are the last
-five residues), and the we reverse the sequence again with [reverse_seq]:
-
-{{{
-read_fasta -i test.fna | reverse_seq | extract_seq -l 5 | reverse_seq
-
-SEQ: atcga
-SEQ_LEN: 5
-SEQ_NAME: test
----
-}}}
-
-==See also==
-
-[read_fasta]
-
-[reverse_seq]
-
-[get_genome_seq]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[extract_seq] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: fold_seq=
-
-==Synopsis==
-
-Fold sequences in stream.
-
-==Description==
-
-[fold_seq] fold nucleotide sequence into secodary structure, which is useful to illustrate
-e.g. miRNA hairpins. The resulting folding information may be uploaded to a custom UCSC Genome
-Browser track (see [upload_to_ucsc]).
-
-[fold_seq] currently uses RNAfold from the Vienna package as folding engine. RNAfold must be
-installed for [fold_seq] to work. Read more about RNAfold here:
-
-http://www.tbi.univie.ac.at/RNA/
-
-==Usage==
-
-{{{
-... | fold_seq [options]
-}}}
-
-==Options==
-
-{{{
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the following RNA sequence entry in FASTA format from the file `test.fna`:
-
-{{{
->MI0000116
-UUCAGCCUUUGAGAGUUCCAUGCUUCCUUGCAUUCAAUAGUUAUAUUCAAGCAUAUGGAAUGUAAAGAAGUAUGGAGCGAAAUCUGGCGAG
-}}}
-
-We can read that file with [read_fasta] and then fold the sequence with [fold_seq]:
-
-{{{
-read_fasta -i test.fna | fold_seq
-
-SIZE: 91
-FREE_ENERGY: -36.20
-SCORE: 36
-SEQ: UUCAGCCUUUGAGAGUUCCAUGCUUCCUUGCAUUCAAUAGUUAUAUUCAAGCAUAUGGAAUGUAAAGAAGUAUGGAGCGAAAUCUGGCGAG
-SEQ_NAME: MI0000116
-SEQ_LEN: 91
-SEC_STRUCT: ....(((...((..((((((((((((.((((((((.((((((.......))).))).)))))))).))))))))))))....)).)))...
-CONF: 1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1 <truncated>
----
-}}}
-
-The resulting record contains the secondary structure in Stockholm format as the value to SEC_STRUCT.
-Also, the FREE_ENERGY key is useful for selecting good structures. [grab] is your friend. The CONF key
-holds confidence information for the folding at any given position (not used, so set to 1s).
-
-Now, if you have a fully configure UCSC Genome Browser system installed, it is possible to upload secondary
-structure information. To do this, you must have a BED entry, and the folded sequence. The reasonable way to
-do this is to have a BED file with the coordinates of the sequence you wish to fold and display and then the
-magic:
-
-{{{
-read_bed -i <BED file> | get_genome_seq -g <genome> | fold_seq | upload_to_ucsc -d <genome> -t <my_table_rnaSecStr> -x
-}}}
-
-Notice that the table name _must_ contain the string 'rnaSecStr' to display the folding information in the
-Genome Browser.
-
-==See also==
-
-[read_fasta]
-
-[upload_to_ucsc]
-
-[grab]
-
-[read_bed]
-
-[get_genome_seq]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[fold_seq] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: get_genome_seq=
-
-==Synopsis==
-
-Extract subsequences from a genome sequence.
-
-==Description==
-
-[get_genome_seq] can be used to get subsequences from a specified genome that have been
-indexed with [index_genome_seq]. The subsequence can be obtained explicitly or from
-BED/PSL/BLAST entries in the stream.
-
-Use [list_genomes] to see available genome sequences.
-
-==Usage==
-
-{{{
-... | get_genome_seq [options]
-}}}
-
-==Options==
-
-{{{
-[-g <genome> | --genome=<genome>] - Genome to get subsequence from.
-[-c <string> | --chr=<string>] - Chromosome with requested subsequence.
-[-b <int> | --beg=<int>] - Begin position of subsequence (first residue=1).
-[-e <int> | --end=<int>] - End position of subsequence.
-[-l <int> | --len=<int>] - Length of subsequence.
-[-f <int> | --flank=<int>] - Include flanking sequence.
-[-m | --mask] - Softmask non-exonic sequence.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-To get an explicit subsequence from the human genome (currently hg18) do:
-
-{{{
-get_genome_seq -g hg18 -c chrX -b 12000 -l 20
-
-CHR_END: 12018
-SEQ: GGTGCAGTAACACCTGCCGT
-CHR_BEG: 11999
-SEQ_LEN: 20
-CHR: chrX
----
-}}}
-
-If you have a stream with BED, PSL or BLAST records, you obtain the subsequence
-simple by piping the stream through [get_genome_seq] and the sequence will be
-added to the record. Below is an example for a BED entry:
-
-{{{
-read_bed -i <BED file> -n 1 | get_genome_seq -g hg18
-
-STRAND: +
-CHR_END: 95127728
-Q_ID: gi|108087685|gb|DQ594132.1|_Homo_sapiens_piRNA_piR-60244,_complete_sequence
-CHR_BEG: 95127700
-SCORE: 1
-REC_TYPE: BED
-BED_LEN: 29
-CHR: chr15
-SEQ: TTCACTTCTCCCATGTAGTTCCTGAGTGC
-BED_COLS: 6
----
-}}}
-
-Note that the sequence is based on the CHR_BEG and CHR_END positions. There is currently
-no switch that obtains subsequence based on record blocks.
-
-==See also==
-
-[index_genome_seq]
-
-[list_genomes]
-
-[read_bed]
-
-[extract_seq]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[get_genome_seq] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: grab=
-
-==Synopsis==
-
-Grab records in stream.
-
-==Description==
-
-[grab] selects records from the stream by matching keys or values using a pattern,
-a regular expression, or a numerical evaluation. [grab] is biopieces' equivalent of
-Unix' grep, however, [grab] is much more versatile.
-
-==Usage==
-
-{{{
-... | grab [options]
-}}}
-
-==Options==
-
-{{{
-[-p <string> | --patterns=<string>] - Grab using comma separated list of patterns.
-[-P <file> | --patterns_in=<file>] - Grab using patterns from file - one pattern per line.
-[-r <string> | --regex=<string>] - Grab using Perl regex.
-[-e <string> | --eval=<string> - Grab 'key,operator,value'. Operators: '>,<,>=,<=,=,!=,eq,ne'.
-[-E <file> | --exact_in=<file> - Grab using exact expressions from file - one expression per line.
-[-i | --invert] - Display non-mathing results.
-[-c | --case_insensitive] - Turn regex matching case insensitive.
-[-k <string> | --keys=<string>] - Comma separated list of keys to grab the value for.
-[-K | --keys_only] - Only grab for keys.
-[-V | --vals_only] - Only grab for vals.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-To easily [grab] all records in the stream that has any mentioning of the pattern 'human'
-just pipe the data stream through [grab] like this:
-
-{{{
-... | grab -p human
-}}}
-
-This will search for the pattern 'human' in all keys and all values. The `-p` switch takes
-a comma separated list of patterns, so in order to match multiple patterns do:
-
-{{{
-... | grab -p human,mouse
-}}}
-
-It is also possible to use the `-P` switch instead of `-p`. `-P` is used to read a file with one pattern per line:
-
-{{{
-... | grab -P patterns.txt
-}}}
-
-If you want the opposite result - to find all records that does not match the patterns,
-add the `-i` switch, which not only works with the `-p` and `-P` switch, but also with `-r` and `-e`:
-
-{{{
-... | grab -p human -i
-}}}
-
-If you want to search the record keys only, e.g. to find all records containing the key SEQ
-you can add the `-K` switch. This will prevent matching of SEQ in any record value, and in
-fact SEQ is a not uncommon peptide sequence you could get an unwanted record. Also, this will
-give an increase in speed since only the keys are searched:
-
-{{{
-... | grab -p SEQ -K
-}}}
-
-However, if you are interested in finding the peptide sequence SEQ and not the SEQ key, just
-add the `-V` switch instead:
-
-{{{
-... | grab -p SEQ -V
-}}}
-
-Also, if you want to [grab] for certain key/value pairs you can supply a comma separated list
-of keys whos values will then be searched using the `-k` switch. This is handy if your records
-contain large genomic sequences and you don't want to search the entire sequence for e.g. the
-organism name - it is much faster to tell [grab] which keys to search the value for:
-
-{{{
-... | grab -p human -k SEQ_NAME
-}}}
-
-It is also possible to invoke flexible matching using regex (regular expressions) instead of
-simple pattern matching. In [grab] the regex engine is Perl based, and allows use of different
-type of wild cards, alternatives, etc. If you want to [grab] records withs the sequence ATCG
-or GCTA you can do this:
-
-{{{
-... | grab -r 'ATCG|GCTA'
-}}}
-
-Or if you want to find sequences beginning with ATCG:
-
-{{{
-... | grab -r '^ATCG'
-}}}
-
-You can also use [grab] to locate records that fulfill a numerical property using the `-e` switch
-witch takes an expression in three parts. The first part is the key that holds the value we want
-to evaluate, the second part holds one if these eight operators:
-
- # Greater than: >
- # Greater than or equal to: >=
- # Less than: <
- # Less than or equal to: <=
- # Equal to: =
- # Not equal to: !=
- # String wise equal to: eq
- # String wise not equal to: ne
-
-And finally comes the number used in the evaluation. So to [grab] all records with a sequence
-length greater than 30:
-
-{{{
-... | grab -e 'SEQ_LEN > 30'
-}}}
-
-If you want to locate all records containing the pattern 'human' and where the sequence length
-is greater that 30, you do this by running the stream through [grab] twice:
-
-{{{
-... | grab -p 'human' | grab -e 'SEQ_LEN > 30'
-}}}
-
-Finally, it is possible to do fast matching of expressions from a file using the `-E` switch.
-Each of these expressions has to be matched exactly over the entrie length, which if useful if
-you e.g. have a file with accession numbers, that you want to locate in the stream:
-
-{{{
-... | grab -E acc_no.txt
-}}}
-
-Using `-E` is much faster than using `-P`, because with `-E` the expression has to be complete
-matches, where `-P` looks for subpatterns.
-
-NB! To get the best speed performance, use the most restrictive [grab] first.
-
-==See also==
-
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[grab] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: invert_align=
-
-==Synopsis==
-
-Inverts an alignment showing only non-mathing residues using the first sequence
-as reference.
-
-==Description==
-
-[invert_align] is useful to locate mismatches or other differences in an alignment
-between the reference sequence (the first sequence in the alignment) and the remaining
-sequences. Invertion can be 'hard' where matching residues are shown as `-` or 'soft'
-where matching residues are shown in lower case. In both cases, mismatches are shown as
-capital letters and gaps or missing sequence is shown as `_`.
-
-==Usage==
-
-{{{
-... | invert_align [options]
-}}}
-
-==Options==
-
-{{{
-[-s | --soft] - Use soft inversion instead of hard inversion.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the alignment in the file `aln.fna` in FASTA format:
-
-{{{
->test1
-CTAGC-TTCGACT
->test2
---AGC-TTCGA--
->test3
---AGCTTTCGA--
->test4
---AG--CTCGA--
->test5
---AG--TTCGAC-
-}}}
-
-Reading the alignment using [read_fasta] and writing it using [write_align] results in:
-
-{{{
-read_fasta -i aln.fna | write_align -x
-
- .
-test1 CTAGC-TTCGACT
-test2 --AGC-TTCGA--
-test3 --AGCTTTCGA--
-test4 --AG--CTCGA--
-test5 --AG--TTCGAC-
-Consensus: 50% --AG--TTCGA--
-}}}
-
-However, if we insert an instance of [invert_align] it is clear where the sequence differences are:
-
-{{{
-read_fasta -i aln.fna | invert_align | write_align -x
-
- .
-test1 CTAGC_TTCGACT
-test2 __---------__
-test3 __---T-----__
-test4 __--_-C----__
-test5 __--_-------_
-Consensus: 50% -------------
-}}}
-
-And if we instead of hard inverting the sequence uses the `-s` switch of [invert_align] to obtain soft
-inverted alignment, where the matching residues are in lower case letters instead of represented as `-`, we get:
-
-{{{
-read_fasta -i aln.fna | invert_align -s | write_align -x
-
- .
-test1 CTAGC_TTCGACT
-test2 __agc_ttcga__
-test3 __agcTttcga__
-test4 __ag__Ctcga__
-test5 __ag__ttcgac_
-Consensus: 50% --AG--TTCGA--
-}}}
-
-==See also==
-
-[read_fasta]
-
-[write_align]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[invert_align] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: print_usage=
-
-==Synopsis==
-
-Print a biopiece' 'usage' information from a usage file.
-
-==Description==
-
-Biopiece usage is kept in seperate files, one for each biopiece. The usage files are written in Google's wiki format and are thus independent of programming language. The usage files are kept in `~/biopieces/bp_usage/`
-
-For more about Google's wiki format:
-
-http://code.google.com/p/support/wiki/WikiSyntax
-
-==Usage==
-
-{{{
-print_usage [options] -i <usage file>
-}}}
-
-==Options==
-
-{{{
-[-i <file> | --data_in=<file>] - Read tabular data from file.
-[-h | --help] - Print verbose usage.
-}}}
-
-==Examples==
-
-{{{
-print_usage -i ~/biopieces/bp_usage/print_usage -h
-}}}
-
-==See also==
-
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[print_usage] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: read_bed=
-
-==Synopsis==
-
-Read BED data (Browser Extensible Data).
-
-==Description==
-
-The BED (Browser Extensible Data) format is a tabular format for data pertaining to one of the Eukaryotic genomes in the UCSC genome brower. The BED format consists
-of up to 12 columns, where the first three are mandatory.
-
- # CHR - the name of the chromosome.
- # CHR_BEG - the chromosome begin position.
- # CHR_END - the chromosome end position.
- # Q_ID - the name of the feature.
- # SCORE - a score between 0 and 1000.
- # STRAND - the orientation of the feature.
- # THICK_BEG - begin position of 'thick' drawing used for UTRs.
- # THICK_END - end position of 'thick' drawing used for UTRs.
- # ITEMRGB - RGB color code for feature.
- # BLOCKCOUNT - number of exon blocks.
- # BLOCKSIZES - list of block sizes.
- # Q_BEGS - list of block begins.
-
-Furthermore, an extra three helper columns are added to the record by [read_bed]:
-
- # REC_TYPE - the type of record, here BED.
- # BED_LEN - the length of the entire feature.
- # BED_COLS - the number of BED columns (for speed).
-
-So a typical 12 column BED record looks like this:
-
-{{{
-STRAND: -
-Q_ID: AA695812
-CHR_END: 31601
-THICK_END: 31601
-SCORE: 0
-CHR_BEG: 31176
-BED_LEN: 426
-REC_TYPE: BED
-BLOCKCOUNT: 1
-CHR: chr4
-THICK_BEG: 31176
-Q_BEGS: 0,
-BLOCKSIZES: 426,
-ITEMRGB: 0
-BED_COLS: 12
----
-}}}
-
-For more about the BED format:
-
-http://genome.ucsc.edu/FAQ/FAQformat#format1
-
-==Usage==
-
-{{{
-read_bed [options] -i <BED file(s)>
-}}}
-
-==Options==
-
-{{{
-[-i <file(s)> | --data_in=<file(s)>] - Comma separated list of files or glob expression to read.
-[-n <int> | --num=<int>] - Limit number of records to read.
-[-I <file> | --stream_in=<file>] - Read input stream from file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output stream to file - Default=STDOUT
-}}}
-
-==Examples==
-
-To read all BED entries from a file:
-
-{{{
-read_bed -i test.bed
-}}}
-
-To read in only 10 records from a BED file:
-
-{{{
-read_bed -n 10 -i test.bed
-}}}
-
-To read all BED entries from multiple files:
-
-{{{
-read_bed -i test1.bed,test2.bed
-}}}
-
-To read BED entries from multiple files using a glob expression:
-
-{{{
-read_bed -i '*.bed'
-}}}
-
-
-==See also==
-
-[write_bed]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[read_bed] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
-
+++ /dev/null
-=Biopiece: read_fasta=
-
-==Synopsis==
-
-Read FASTA entries from one or more files.
-
-==Description==
-
-[read_fasta] read in sequence entries from FASTA files. Each sequence entry consists of a
-sequence name prefixed by a '>' followed by the sequence name on a line of its own, followed
-by one or my lines of sequence until the next entry or the end of the file. The resulting
-biopiece record consists of the following record type:
-
-{{{
-SEQ_NAME: test
-SEQ_LEN: 10
-SEQ: ATCGATCGAC
----
-}}}
-
-For more about the FASTA format:
-
-http://en.wikipedia.org/wiki/Fasta_format
-
-==Usage==
-
-{{{
-read_fasta [options] -i <FASTA file(s)>
-}}}
-
-==Options==
-
-{{{
-[-i <file(s)> | --data_in=<file(s)>] - Comma separated list of files or glob expression to read.
-[-n <int> | --num=<int>] - Limit number of records to read.
-[-I <file> | --stream_in=<file>] - Read input stream from file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output stream to file - Default=STDOUT
-}}}
-
-==Examples==
-
-To read all FASTA entries from a file:
-
-{{{
-read_fasta -i test.fna
-}}}
-
-To read in only 10 records from a FASTA file:
-
-{{{
-read_fasta -n 10 -i test.fna
-}}}
-
-To read all FASTA entries from multiple files:
-
-{{{
-read_fasta -i test1.fna,test2.fna
-}}}
-
-To read FASTA entries from multiple files using a glob expression:
-
-{{{
-read_fasta -i '*.fna'
-}}}
-
-
-==See also==
-
-[read_align]
-
-[write_fasta]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[read_fasta] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
-
+++ /dev/null
-=Biopiece: read_tab=
-
-==Synopsis==
-
-Read tabular data.
-
-==Description==
-
-Tabular input can be read with [read_tab] which will read in chosen rows and
-chosen columns (separated by a given delimiter) from a table in ASCII text format.
-
-==Usage==
-
-{{{
-read_tab [options] -i <table file(s)>
-}}}
-
-==Options==
-
-{{{
-[-i <file(s)> | --data_in=<file(s)>] - Read tabular data from file.
-[-d <regex> | --delimit=<regex>] - Changes delimiter - Default='\s+'
-[-c <string> | --cols=<string>] - Comma separated list of cols to read in that order.
-[-k <string> | --keys]=<string>] - Comma separated list of keys to use for each column.
-[-s <int> | --skip=<int>] - Skip number of initial records.
-[-n <int> | --num=<int>] - Limit number of records to read.
-[-I <file> | --stream_in=<file>] - Read input stream from file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output stream to file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the following table from the file from the file `test.tab`:
-
-{{{
-Organism Sequence Count
-Human ATACGTCAG 23524
-Dog AGCATGAC 2442
-Mouse GACTG 234
-Cat AAATGCA 2342
-}}}
-
-Reading the entire table:
-
-{{{
-read_tab -i test.tab
-}}}
-
-The above command will result in 5 records, one for each row, where the keys V0, V1, V2 are the default keys for the columns:
-
-{{{
-V0: Organism
-V2: Count
-V1: Sequence
----
-V0: Human
-V2: 23524
-V1: ATACGTCAG
----
-V0: Dog
-V2: 2442
-V1: AGCATGAC
----
-V0: Mouse
-V2: 234
-V1: GACTG
----
-V0: Cat
-V2: 2342
-V1: AAATGCA
----
-}}}
-
-However, the first line is a comment line that can be skipped using the `-s` switch which
-will skip a specified number of lines before reading. So to get the rows with data do:
-
-{{{
-read_tab -i test.tab -s 1
-
-V0: Human
-V2: 23524
-V1: ATACGTCAG
----
-V0: Dog
-V2: 2442
-V1: AGCATGAC
----
-V0: Mouse
-V2: 234
-V1: GACTG
----
-V0: Cat
-V2: 2342
-V1: AAATGCA
----
-}}}
-
-It is possible to select a subset of columns to read by using the `-c` switch which takes
-a comma separated list of columns numbers (first column is designated 0) as argument.
-So to read in only the sequence and the count so that the count comes before the sequence do:
-
-{{{
-read_tab -i test.tab -s 1 -c 2,1
-
-V0: 23524
-V1: ATACGTCAG
----
-V0: 2442
-V1: AGCATGAC
----
-V0: 234
-V1: GACTG
----
-V0: 2342
-V1: AAATGCA
----
-}}}
-
-It is also possible to rename the columns with the `-k` switch:
-
-{{{
-read_tab -i test.tab -s 1 -c 2,1 -k COUNT,SEQ
-
-SEQ: ATACGTCAG
-COUNT: 23524
----
-SEQ: AGCATGAC
-COUNT: 2442
----
-SEQ: GACTG
-COUNT: 234
----
-SEQ: AAATGCA
-COUNT: 2342
----
-}}}
-
-==See also==
-
-[write_tab]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[read_tab] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: rename_keys=
-
-==Synopsis==
-
-Rename keys of records in stream.
-
-==Description==
-
-Sometimes it is necessary to rename record keys to allow biopieces, who require
-specific record keys, to operate on the records. This could e.g. be BLAST records
-where there is both a subject ID (S_ID) and a subject sequence (S_SEQ) as well as
-a query ID (Q_ID) and and a query sequence (Q_SEQ). If you want to write either the
-query sequence or the subject sequence as FASTA output, you will have to rename the
-record keys accordingly.
-
-==Usage==
-
-{{{
-... | rename_keys [options]
-}}}
-
-==Options==
-
-{{{
-[-k <search,replace> | --keys=<search,replace>] - Keys to find and replace.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-To rename all record key Q_ID to SEQ_NAME do:
-
-{{{
-... | rename_keys -k Q_ID,SEQ_NAME
-}}}
-
-If you need to rename more than one key, then pipe the stream though [rename_keys] twice:
-
-{{{
-... | rename_keys -k Q_ID,SEQ_NAME | rename_keys -k Q_SEQ,SEQ
-}}}
-
-==See also==
-
-[blast_seq]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[rename_keys] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: reverse_seq=
-
-==Synopsis==
-
-Reverse sequences in the stream.
-
-==Description==
-
-[reverse_seq] reverses sequences in the stream which is useful for extracting subsequences
-[extract_seq] or reverse complementing sequence [complement_seq].
-
-==Usage==
-
-{{{
-... | reverse_seq [options]
-}}}
-
-==Options==
-
-{{{
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-To reverse sequences simply pipe the stream through [reverse_seq]:
-
-{{{
-... | reverse_seq
-}}}
-
-To reverse complement sequence first reverse the sequence with [reverse_seq] and then
-complement it using [complement_seq] (or visa versa):
-
-{{{
-... | reverse_seq | complement_seq
-}}}
-
-To obtain the last e.g. five residues from all sequences, first use [reverse_seq], then [extract_seq],
-and then re-reverse with [reverse_seq]:
-
-{{{
-... | reverse_seq | extract_seq -l 5 | reverse_seq
-}}}
-
-==See also==
-
-[complement_seq]
-
-[extract_seq]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[reverse_seq] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: sort_records=
-
-==Synopsis==
-
-Sort records in the stream.
-
-==Description==
-
-[sort_records] sort records in the data stream using the values of a list of given keys.
-If values from the first key collides, then the next key's value is used, and so on.
-
-==Usage==
-
-{{{
-... | sort_records [options]
-}}}
-
-==Options==
-
-{{{
-[-k <string> | --keys=<string>] - Comma separated list of keys to sort by. Append n for numeric sorting instead of alphabetic.
-[-r | --reverse] - Reverse sort order.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-Sort records alphabetically according to SEQ:
-
-{{{
-... | sort_records -k SEQ
-}}}
-
-Sort records numerically according to SEQ_LEN:
-
-{{{
-... | sort_records -k SEQ_LENn
-}}}
-
-Output records in reverse order sorted according to SEQ_LEN and SEQ:
-
-{{{
-... | sort_records -k SEQ_LENn,SEQ -r
-}}}
-
-==See also==
-
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[sort_records] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: tile_seq=
-
-==Synopsis==
-
-Using the first sequence in the stream as reference, tile all subsequent sequences
-based on pairwise alignments.
-
-==Description==
-
-[tile_seq] can create an alignment of several sequences based on pairwise alignments.
-This is useful for e.g. matching short sequences such as ESTs or deep sequencing reads
-against a reference sequence. [tile_seq] is more precise than a multiple alignment, where
-the introduction of indels in the reference sequence will most likely ruin the alignment.
-Also, [tile_seq] is capable of dealing with thousands of sequences.
-
-[tile_seq] currently uses Muscle as alignment engine, and Muscle must be installed in
-order for [tile_seq] to work.
-
-For more about Muscle:
-
-http://www.drive5.com/muscle/
-
-==Usage==
-
-{{{
-... | tile_seq [options]
-}}}
-
-==Options==
-
-{{{
-[-i <int> | --identity=<int>] - Minimum identity (%) for pairwise alignment - Default=70
-[-s | --supress_indels] - Supress insertions in query sequence.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the following file `test.fna` containing these FASTA entries:
-
-{{{
->ref
-ACGACTAGCATCGACTGACA
->test1
-CTAGCTTCGACT
->test2
-GAATCGACT
->test3
-ACGAAACTAGCATC
->test4
-AGCATCGACT
->test5TAACAGGCACT
-}}}
-
-In order to tile the test1, test2 ... test5 sequences against the reference sequence,
-first read in the sequence using [read_fasta] and then pipe through [tile_seq]:
-
-{{{
-read_fasta -i test.fna | tile_seq
-
-SEQ: ACGACTAGCATCGACTGACA
-SEQ_NAME: ref
----
-SEQ: ACGAAACTAGCATC------
-SEQ_NAME: test3_+_85.71
----
-SEQ: ----CTAGCTTCGACT----
-SEQ_NAME: test1_+_91.67
----
-SEQ: ------AGCATCGACT----
-SEQ_NAME: test4_+_100.00
----
-SEQ: -------GAATCGACT----
-SEQ_NAME: test2_+_88.89
----
-}}}
-
-The resulting tiled sequences show the reference sequence as the first sequence, and then
-the subsequence sequences sorted alphabetically by the sequence itself, thus giving the
-tiled output. To pieces of information is added to the SEQ_NAME key, namely the orientation
-of the pairwise alignment that gave the highest similarity, and a global identity score that
-is calculated as the number of matches over the length of the shortest sequence in the pairwise
-alignment. Use the `-i` switch to change the identity cutoff for the inclusion of alignments:
-
-{{{
-read_fasta -i test.fna | tile_seq -i 60
-
-SEQ: ACGACTAGCATCGACTGACA
-SEQ_NAME: ref
----
-SEQ: ACGAAACTAGCATC------
-SEQ_NAME: test3_+_85.71
----
-SEQ: ----CTAGCTTCGACT----
-SEQ_NAME: test1_+_91.67
----
-SEQ: -----TAACAGGCACT----
-SEQ_NAME: test5_+_63.64
----
-SEQ: ------AGCATCGACT----
-SEQ_NAME: test4_+_100.00
----
-SEQ: -------GAATCGACT----
-SEQ_NAME: test2_+_88.89
----
-}}}
-
-Now test5 is part of the alignment, and the tiled sequences can be written using [write_align]:
-
-{{{
-read_fasta -i test.fna | tile_seq -i 60 | write_align -x
-
- . .
-ref ACGACTAGCATCGACTGACA
-test3_+_85.71 ACGAAACTAGCATC------
-test1_+_91.67 ----CTAGCTTCGACT----
-test5_+_63.64 -----TAACAGGCACT----
-test4_+_100.00 ------AGCATCGACT----
-test2_+_88.89 -------GAATCGACT----
-Consensus: 50% -------------ACT----
-}}}
-
-To better illustrate mismatches in the alignment use [invert_align]:
-
-{{{
-read_fasta -i test.fna | tile_seq -i 60 | invert_align | write_align -x
-
- . .
-ref ACGACTAGCATCGACTGACA
-test3_+_85.71 ----AACTAGCATC______
-test1_+_91.67 ____-----T------____
-test5_+_63.64 _____--A--GGC---____
-test4_+_100.00 ______----------____
-test2_+_88.89 _______-A-------____
-Consensus: 50% --------------------
-}}}
-
-Now we clearly see that an insertion in test3 offsets the alignment. This can behaviour can be
-suppressed using the `-s` switch to [tile_seq]:
-
-{{{
-read_fasta -i test.fna | tile_seq -i 60 -s | invert_align | write_align -x
-
- . .
-ref ACGACTAGCATCGACTGACA
-test3_+_100.00 ------------________
-test1_+_91.67 ____-----T------____
-test5_+_63.64 _____--A--GGC---____
-test4_+_100.00 ______----------____
-test2_+_88.89 _______-A-------____
-Consensus: 50% --------------------
-}}}
-
-Note that the identity score of test3 changes dramatically with the use of the `-s` switch.
-
-==See also==
-
-[read_fasta]
-
-[invert_align]
-
-[write_align]
-
-[align_seq]
-
-[write_fasta]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[tile_seq] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: transliterate_seq=
-
-==Synopsis==
-
-Transliterate chars from sequences in stream.
-
-==Description==
-
-Transliteration is ultra fast search and replace (or search and delete) of characters in sequence
-and is useful for things as lowercasing sequence, converting sequence from RNA to DNA or convering
-soft-masked sequence to hard-masked.
-
-==Usage==
-
-{{{
-... | transliterate_seq [options]
-}}}
-
-==Options==
-
-{{{
-[-s <chars> | --search=<chars>] - String of chars to locate and replace
-[-r <chars> | --replace=<chars>] - String of chars for replacing
-[-d <chars> | --delete=<chars>] - String of chars to delete
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-To lowercase sequence:
-
-{{{
-transliterate_seq -s ATCGUN -r atcgun
-}}}
-
-To convert RNA to DNA:
-
-{{{
-transliterate_seq -s Uu -r Tt
-}}}
-
-To convert soft-masked sequence to hard-masked:
-
-{{{
-transliterate_seq -s atcgu -r NNNNN
-}}}
-
-To remove all N's from a sequence:
-
-{{{
-transliterate_seq -s nN -d
-}}}
-
-==See also==
-
-[transliterate_vals]
-
-[uppercase_seq]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[transliterate_seq] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: transliterate_vals=
-
-==Synopsis==
-
-Transliterate chars from values in stream.
-
-==Description==
-
-Transliteration is ultra fast search and replace (or search and delete) of characters in values
-and is useful for things as converting sequence from RNA to DNA or removing indels from patterns.
-
-==Usage==
-
-{{{
-... | transliterate_vals [options]
-}}}
-
-==Options==
-
-{{{
-[-k <string> | --keys=<string>] - List of values to transliterate
-[-s <chars> | --search=<chars>] - String of chars to locate and replace
-[-r <chars> | --replace=<chars>] - String of chars for replacing
-[-d <chars> | --delete=<chars>] - String of chars to delete
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-To convert RNA sequence to DNA:
-
-{{{
-transliterate_vals -k SEQ -s Uu -r Tt
-}}}
-
-To remove indels from patterns:
-
-{{{
-transliterate_vals -k PATTERN -s '._-~' -d
-}}}
-
-==See also==
-
-[transliterate_seq]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[transliterate_vals] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: uniq_vals=
-
-==Synopsis==
-
-Select unique or non-unique records from the stream based on the value of a given key.
-
-==Description==
-
-[uniq_vals] selects records from the stream by checking values of a given key. If a duplicate
-record exists, it will only be output once. Thus, [uniq_vals] does _not_ locate records
-where the value to the specified key is located only once (see [count_vals]). If the `-i` switch
-is used, then non-unique records are located.
-
-==Usage==
-
-{{{
-... | uniq_vals [options]
-}}}
-
-==Options==
-
-{{{
-[-k <string> | --key=<string>] - Key for which the value is checked for uniqueness.
-[-i | --invert] - Display non-unique records.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the following two column table in the file `test.tab`:
-
-{{{
-Human H2
-Human H3
-Dog D1
-Dog D2
-Mouse M1
-}}}
-
-To locate all unique values of the first columen we use [read_tab] and pipe the result to [uniq_vals]:
-
-{{{
-read_tab -i test.tab | uniq_vals -k V0
-
-V0: Human
-V1: H1
----
-V0: Dog
-V1: D1
----
-V0: Mouse
-V1: M1
----
-}}}
-
-The result is three records, one _unique_ for each V0.
-
-If we instead want the non-unique records we use the `-i` switch with [uniq_vals]:
-
-{{{
-read_tab -i test.tab | uniq_vals -k V0 -i
-
-V0: Human
-V1: H2
----
-V0: Human
-V1: H3
----
-V0: Dog
-V1: D2
----
-}}}
-
-... and the result shows those records which duplicate values to V0.
-
-So, how do we get the non-duplicated record with the `Mouse`? That is in fact not a job
-for [uniq_vals], but rather for [count_vals] and [grab].
-
-{{{
-read_tab -i test.tab | count_vals -k V0 | grab -e 'V0_COUNT=1'
-
-V0: Mouse
-V1: M1
-V0_COUNT: 1
----
-}}}
-
-However, if we use both [count_vals] and [uniq_vals] we can obtain a list of how many times
-each of the records were duplicated based on the first column:
-
-{{{
-read_tab -i test.tab | count_vals -k V0 | uniq_vals -k 'V0_COUNT'
-
-V0: Human
-V1: H1
-V0_COUNT: 3
----
-V0: Dog
-V1: D1
-V0_COUNT: 2
----
-V0: Mouse
-V1: M1
-V0_COUNT: 1
----
-}}}
-
-==See also==
-
-[read_tab]
-
-[count_vals]
-
-[grab]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[uniq_vals] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: uppercase_seq=
-
-==Synopsis==
-
-Uppercases sequences in stream.
-
-==Description==
-
-[upppercase_seq] changes all lowercase letter in sequences to uppercase letters.
-
-==Usage==
-
-{{{
-... | uppercase_seq [options]
-}}}
-
-==Options==
-
-{{{
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-}}}
-
-==Examples==
-
-Consider the following sequence entry in FASTA format in the file `test.fna`:
-
-{{{
->test
-ACGACGCATNNNNNNactgatcga
-}}}
-
-To read in this sequence use [read_fasta] and then [uppercase_seq] to uppercase all letters:
-
-{{{
-read_fasta -i test.fna | uppercase_seq
-
-SEQ: ACGACGCATNNNNNNACTGATCGA
-SEQ_NAME: test
-SEQ_LEN: 24
----
-}}}
-
-==See also==
-
-[read_fasta]
-
-[transliterate_seq]
-
-[transliterate_vals]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[uppercase_seq] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: write_bed=
-
-==Synopsis==
-
-Write BED entries from stream.
-
-==Description==
-
-[write_bed] outputs BED (Browser Extensible Data) records from the stream.
-The BED format consists of up to 12 columns, where the first three are mandatory.
-
- # CHR - the name of the chromosome.
- # CHR_BEG - the chromosome begin position.
- # CHR_END - the chromosome end position.
- # Q_ID - the name of the feature.
- # SCORE - a score between 0 and 1000.
- # STRAND - the orientation of the feature.
- # THICK_BEG - begin position of 'thick' drawing used for UTRs.
- # THICK_END - end position of 'thick' drawing used for UTRs.
- # ITEMRGB - RGB color code for feature.
- # BLOCKCOUNT - number of exon blocks.
- # BLOCKSIZES - list of block sizes.
- # Q_BEGS - list of block begins.
-
-Records can be written to BED format if the REC_TYPE is either:
-
- * BED - from [read_bed]
- * PSL - from [read_psl] or [blat_seq]
- * PATSCAN - from [patscan_seq]
- * BLAST - from [read_blast_tab] or [blast_seq]
- * VMATCH - from [vmatch_seq]
-
-... and if S_ID key begins with 'chr'.
-
-Also, generic records may be output as BED format if the record contains
-CHR, CHR_BEG, and CHR_END keys and if the CHR value begins with 'chr'.
-
-For more about the BED format:
-
-http://genome.ucsc.edu/FAQ/FAQformat#format1
-
-==Usage==
-
-{{{
-... | write_bed [options]
-}}}
-
-==Options==
-
-{{{
-[-x | --no_stream] - Do not emit records.
-[-o <file> | --data_out=<file>] - Write result to file.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-[-Z | --compress] - Compress output using gzip.
-}}}
-
-==Examples==
-
-{{{
-... | write_bed -xo test.bed
-}}}
-
-==See also==
-
-[read_bed]
-
-[read_psl]
-
-[blat_seq]
-
-[patscan_seq]
-
-[read_blast_tab]
-
-[blast_seq]
-
-[vmatch_seq]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[write_bed] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
+++ /dev/null
-=Biopiece: write_fasta=
-
-==Synopsis==
-
-Write sequences from stream in FASTA format.
-
-==Description==
-
-[write_fasta] writes sequence from the data stream in FASTA format. However,
-a FASTA entry will only be written if a SEQ key and a SEQ_NAME key is present.
-If a SEQ key is present and not SEQ_NAME, then the Q_ID key will be used as
-SEQ_NAME - if such a key is found.
-
-For more about the FASTA format:
-
-http://en.wikipedia.org/wiki/Fasta_format
-
-==Usage==
-
-{{{
-... | write_fasta [options]
-}}}
-
-==Options==
-
-{{{
-[-x | --no_stream] - Do not emit records.
-[-o <file> | --data_out=<file>] - Write result to file.
-[-w <int> | --wrap=<int>] - Wrap sequences to a given width.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-[-Z | --compress] - Compress output using gzip.
-}}}
-
-==Examples==
-
-Writing entries wrapped to blocks of 80 to STDOUT.
-
-{{{
-... | write_fasta -x -w 80
-}}}
-
-Writing entries to file 'test.fna'.
-
-{{{
-... | write_fasta -x -o test.fna
-}}}
-
-Write zipped entries to file 'test.fna.gz'.
-
-{{{
-... | write_fasta -x -Z -o test.fna.gz
-}}}
-
-==See also==
-
-[read_fasta]
-
-[read_align]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[write_fasta] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
-
+++ /dev/null
-=Biopiece: write_tab=
-
-==Synopsis==
-
-Write tabular output from stream.
-
-==Description==
-
-Outputting the data stream as a table can be done with [write_tab]
-which will write generate one row per record with the values as columns.
-
-==Usage==
-
-{{{
-... | write_tab [options]
-}}}
-
-==Options==
-
-{{{
-[-x | --no_stream] - Do not emit records.
-[-o <file> | --data_out=<file>] - Write result to file.
-[-c | --comment] - Print comment line - Default=no
-[-d <string> | --delimit=<string>] - Changes delimiter - Default='\t'
-[-k <string> | --keys=<string>] - Comma separated list of keys to print in that order.
-[-K <string> | --no_keys=<string>] - Comma separated list of keys to ignore.
-[-I <file> | --stream_in=<file>] - Read input from stream file - Default=STDIN
-[-O <file> | --stream_out=<file>] - Write output to stream file - Default=STDOUT
-[-Z | --compress] - Compress output using gzip.
-}}}
-
-==Examples==
-
-To write all records from the stream as a table, do:
-
-{{{
-... | write_tab
-}}}
-
-If you supply the optional `-c` switch, when the first row in the table will be a 'comment' line prefixed with a '#':
-
-{{{
-... | write_tab -c
-}}}
-
-You can also change the delimiter from the default (tab) to e.g. ',':
-
-{{{
-... | write_tab -d ','
-}}}
-
-If you want the values output in a specific order you have to supply a comma
-separated list using the `-k` switch that will print only those keys in that order:
-
-{{{
-... | write_tab -k SEQ_NAME,COUNT
-}}}
-
-Keys from e.g. [read_tab] V0, V1, V2 ... Vn, is automagically sorted numerically.
-
-Alternatively, if you have some keys that you don't want in the tabular output,
-use the `-K` switch. So to print all keys except SEQ and SEQ_TYPE do:
-
-{{{
-... | write_tab -K SEQ,SEQ_TYPE
-}}}
-
-Finally, if you have a stream containing a mix of different records types, e.g.
-records with sequences and records with matches, then you can use [write_tab] to
-output all the records in tabluar format, however, the `-c`, `-k`, and `-K` switches
-will only respond to records of the first type encountered. The reason is that outputting
-mixed records is probably not what you want anyway, and you should remove all the unwanted
-records from the stream before outputting the table: [grab] is your friend.
-
-==See also==
-
-[read_tab]
-
-[grab]
-
-==Author==
-
-Martin Asser Hansen - Copyright (C) - All rights reserved.
-
-mail@maasha.dk
-
-August 2007
-
-==License==
-
-GNU General Public License version 2
-
-http://www.gnu.org/copyleft/gpl.html
-
-==Help==
-
-[write_tab] is part of the Biopieces framework.
-
-http://code.google.com/p/biopieces/
projects / biopieces.git / commitdiff