1 --------------------------------------------------------------------------------
3 --------------------------------------------------------------------------------
5 BamTools: a C++ API & toolkit for reading/writing/manipulating BAM files.
23 --------------------------------------------------------------------------------
25 --------------------------------------------------------------------------------
27 BamTools provides both a programmer's API and an end-user's toolkit for handling
30 ----------------------------------------
32 ----------------------------------------
34 The API consists of 2 main modules: BamReader and BamWriter. As you would
35 expect, BamReader provides read-access to BAM files, while BamWriter handles
36 writing data to BAM files. BamReader provides the interface for random-access
37 (jumping) in a BAM file, as well as generating BAM index files.
39 BamMultiReader is an extra module that allows you to manage multiple open BAM
40 files for reading. It provides some validation & bookkeeping under the hood to
41 keep all files sync'ed up for you.
43 Additional files used by the API:
45 - BamAlignment.* : implements the BamAlignment data structure
47 - BamAux.h : contains various constants, data structures and utility
48 methods used throught the API.
50 - BamIndex.* : implements both the standard BAM format index (".bai") as
51 well as a new BamTools-specific index (".bti").
53 - BGZF.* : contains our implementation of the Broad Institute's BGZF
56 ----------------------------------------
58 ----------------------------------------
60 If you've been using the BamTools since the early days, you'll notice that our
61 'toy' API examples (BamConversion, BamDump, BamTrim,...) are now gone. We have
62 dumped these in favor of a suite of small utilities that we hope both
63 developers and end-users find useful:
65 usage: bamtools [--help] COMMAND [ARGS]
67 Available bamtools commands:
69 convert Converts between BAM and a number of other formats
70 count Prints number of alignments in BAM file(s)
71 coverage Prints coverage statistics from the input BAM file
72 filter Filters BAM file(s) by user-specified criteria
73 header Prints BAM header information
74 index Generates index for BAM file
75 merge Merge multiple BAM files into single file
76 random Select random alignments from existing BAM file(s)
77 sort Sorts the BAM file according to some criteria
78 split Splits a BAM file on user-specifed property, creating a
79 new BAM output file for each value found
80 stats Prints some basic statistics from input BAM file(s)
82 See 'bamtools help COMMAND' for more information on a specific command.
84 --------------------------------------------------------------------------------
86 --------------------------------------------------------------------------------
88 ----------------------------------------
90 ----------------------------------------
92 BamTools has been migrated to a CMake-based build system. We believe that this
93 should simplify the build process across all platforms, especially as the
94 BamTools API moves into a shared library (that you link to instead of compiling
95 lots of source files directly into your application). CMake is available on all
96 major platforms, and indeed comes *out-of-the-box* with many Linux distributions.
98 To see if you have CMake (and which version), try this command:
102 BamTools requires CMake version >= 2.6.4. If you are missing CMake or have an
103 older version, check your OS package manager (for Linux) or download it here:
104 http://www.cmake.org/cmake/resources/software.html .
106 ----------------------------------------
108 ----------------------------------------
110 Ok, now that you have CMake ready to go, let's build BamTools. A good
111 practice in building applications is to do an out-of-source build, meaning
112 that we're going to set up an isolated place to hold all the intermediate
115 In the top-level directory of BamTools, type the following commands:
122 This creates a Visual Studio solution file, which can then be built to create
123 the toolkit executable and API DLL's.
126 After running cmake, just run:
130 Then go back up to the BamTools root directory.
134 ----------------------------------------
136 ----------------------------------------
138 Assuming the build process finished correctly, you should be able to find the
139 toolkit executable here:
143 The BamTools-associated libraries will be found here:
147 --------------------------------------------------------------------------------
149 --------------------------------------------------------------------------------
151 ** General usage information - perhaps explain common terms, point to SAM/BAM
154 ----------------------------------------
156 ----------------------------------------
158 The API, as noted above, contains 2 main modules - BamReader & BamWriter - for
159 dealing with BAM files. Alignment data is made available through the
160 BamAlignment data structure.
162 A simple (read-only) scenario for accessing BAM data would look like the
165 // open our BamReader
167 reader.Open("someData.bam", "someData.bam.bai");
169 // define our region of interest
170 // in this example: bases 0-500 on the reference "chrX"
171 int id = reader.GetReferenceID("chrX");
172 BamRegion region(id, 0, id, 500);
173 reader.SetRegion(region);
175 // iterate through alignments in this region,
176 // ignoring alignments with a MQ below some cutoff
178 while ( reader.GetNextAlignment(al) ) {
179 if ( al.MapQuality >= 50 )
186 To use this API in your application, you simply need to do 3 things:
188 1 - Build the BamTools library (see Installation steps above).
190 2 - Import BamTools API with the following lines of code
191 #include "BamReader.h" // (or "BamMultiReader.h") as needed
192 #include "BamWriter.h" // as needed
193 using namespace BamTools; // all of BamTools classes/methods live in
196 3 - Link with '-lbamtools' ('l' as in Lima).
198 You may need to modify the -L flag (library path) as well to help your linker
199 find the (BAMTOOLS_ROOT)/lib directory.
201 See any included programs for more detailed usage examples. See comments in the
202 header files for more detailed API documentation.
204 Note - For users that don't want to bother with the new BamTools shared library
205 scheme: you are certainly free to just compile the API source code directly into
206 your application, but be aware that the files involved are subject to change.
207 Meaning that filenames, number of files, etc. are not fixed. You will also need
208 to be sure to link with '-lz' for ZLIB functionality (linking with '-lbamtools'
209 gives you this automatically).
211 ----------------------------------------
213 ----------------------------------------
215 BamTools provides a small, but powerful suite of command-line utility programs
216 for manipulating and querying BAM files for data.
222 All BamTools utilities handle I/O operations using a common set of arguments.
227 The input BAM files(s).
229 If a tool accepts multiple BAM files as input, each file gets its own "-in"
230 option on the command line. If no "-in" is provided, the tool will attempt
231 to read BAM data from stdin.
233 To read a single BAM file, use a single "-in" option:
234 > bamtools *tool* -in myData1.bam ...ARGS...
236 To read multiple BAM files, use multiple "-in" options:
237 > bamtools *tool* -in myData1.bam -in myData2.bam ...ARGS...
239 To read from stdin (if supported), omit the "-in" option:
240 > bamtools *tool* ...ARGS...
246 If a tool outputs a result BAM file, specify the filename using this option.
247 If none is provided, the tool will typically write to stdout.
249 *Note: Not all tools output BAM data (e.g. count, header, etc.)
253 A region of interest. See below for accepted 'REGION string' formats.
255 Many of the tools accept this option, which allows a user to only consider
256 alignments that overlap this region (whether counting, filtering, merging,
259 An alignment is considered to overlap a region if any part of the alignments
260 intersects the left/right boundaries. Thus, a 50bp alignment at position 70
261 will overlap a region beginning at position 100.
264 ----------------------
265 A proper REGION string can be formatted like any of the following examples:
266 where 'chr1' is the name of a reference (not its ID)and '' is any valid
267 integer position within that reference.
270 chr1 - only alignments on (entire) reference 'chr1'
271 chr1:500 - only alignments overlapping the region starting at
272 chr1:500 and continuing to the end of chr1
273 chr1:500..1000 - only alignments overlapping the region starting at
274 chr1:500 and continuing to chr1:1000
275 chr1:500..chr3:750 - only alignments overlapping the region starting at
276 chr1:500 and continuing to chr3:750. This 'spanning'
277 region assumes that the reference specified as the
278 right boundary will occur somewhere in the file after
279 the left boundary. On a sorted BAM, a REGION of
280 'chr4:500..chr2:1500' will produce undefined
281 (incorrect) results. So don't do it. :)
283 *Note: Most of the tools that accept a REGION string will perform without an
284 index file, but typically at great cost to performance (having to
285 plow through the entire file until the region of interest is found).
286 For optimum speed, be sure that index files are available for your
291 Force compression of BAM output.
293 When tools are piped together (see details below), the default behavior is
294 to turn off compression. This can greatly increase performance when the data
295 does not have to be constantly decompressed and recompressed. This is
296 ignored any time an output BAM file is specified using "-out".
302 Many of the tools in BamTools can be chained together by piping. Any tool that
303 accepts stdin can be piped into, and any that can output stdout can be piped
306 > bamtools filter -in data1.bam -in data2.bam -mapQuality ">50" | bamtools count
308 will give a count of all alignments in your 2 BAM files with a mapQuality of
309 greater than 50. And of course, any tool writing to stdout can be piped into
316 convert Converts between BAM and a number of other formats
317 count Prints number of alignments in BAM file(s)
318 coverage Prints coverage statistics from the input BAM file
319 filter Filters BAM file(s) by user-specified criteria
320 header Prints BAM header information
321 index Generates index for BAM file
322 merge Merge multiple BAM files into single file
323 random Select random alignments from existing BAM file(s)
324 sort Sorts the BAM file according to some criteria
325 split Splits a BAM file on user-specifed property, creating a new
326 BAM output file for each value found
327 stats Prints some basic statistics from input BAM file(s)
333 Description: converts BAM to a number of other formats
335 Usage: bamtools convert -format <FORMAT> [-in <filename> -in <filename> ...]
336 [-out <filename>] [other options]
339 -in <BAM filename> the input BAM file(s) [stdin]
340 -out <BAM filename> the output BAM file [stdout]
341 -format <FORMAT> the output file format - see below for
345 -region <REGION> genomic region. Index file is recommended for
346 better performance, and is read
347 automatically if it exists. See 'bamtools
348 help index' for more details on creating
352 -fasta <FASTA filename> FASTA reference file
353 -mapqual print the mapping qualities
356 -noheader omit the SAM header from output
359 --help, -h shows this help text
363 - Currently supported output formats ( BAM -> X )
365 Format type FORMAT (command-line argument)
366 ------------ -------------------------------
376 > bamtools convert -format json -in myData.bam -out myData.json
378 - Pileup Options have no effect on formats other than "pileup"
379 SAM Options have no effect on formats other than "sam"
385 Description: prints number of alignments in BAM file(s).
387 Usage: bamtools count [-in <filename> -in <filename> ...] [-region <REGION>]
390 -in <BAM filename> the input BAM file(s) [stdin]
391 -region <REGION> genomic region. Index file is recommended
392 for better performance, and is used
393 automatically if it exists. See
394 'bamtools help index' for more details
398 --help, -h shows this help text
404 Description: prints coverage data for a single BAM file.
406 Usage: bamtools coverage [-in <filename>] [-out <filename>]
409 -in <BAM filename> the input BAM file [stdin]
410 -out <filename> the output file [stdout]
413 --help, -h shows this help text
419 Description: filters BAM file(s).
421 Usage: bamtools filter [-in <filename> -in <filename> ...]
422 [-out <filename> | [-forceCompression]]
424 [ [-script <filename] | [filterOptions] ]
427 -in <BAM filename> the input BAM file(s) [stdin]
428 -out <BAM filename> the output BAM file [stdout]
429 -region <REGION> only read data from this genomic region (see
430 README for more details)
431 -script <filename> the filter script file (see README for more
433 -forceCompression if results are sent to stdout (like when
434 piping to another tool), default behavior
435 is to leave output uncompressed. Use this
436 flag to override and force compression
439 -alignmentFlag <int> keep reads with this *exact* alignment flag
440 (for more detailed queries, see below)
441 -insertSize <int> keep reads with insert size that matches
443 -mapQuality <[0-255]> keep reads with map quality that matches
445 -name <string> keep reads with name that matches pattern
446 -queryBases <string> keep reads with motif that matches pattern
447 -tag <TAG:VALUE> keep reads with this key=>value pair
449 Alignment Flag Filters:
450 -isDuplicate <true/false> keep only alignments that are marked as
452 -isFailedQC <true/false> keep only alignments that failed QC [true]
453 -isFirstMate <true/false> keep only alignments marked as first mate
455 -isMapped <true/false> keep only alignments that were mapped [true]
456 -isMateMapped <true/false> keep only alignments with mates that mapped
458 -isMateReverseStrand <true/false> keep only alignments with mate on reverse
460 -isPaired <true/false> keep only alignments that were sequenced as
462 -isPrimaryAlignment <true/false> keep only alignments marked as primary
464 -isProperPair <true/false> keep only alignments that passed paired-end
466 -isReverseStrand <true/false> keep only alignments on reverse strand
468 -isSecondMate <true/false> keep only alignments marked as second mate
472 --help, -h shows this help text
478 The BamTools filter tool allows you to use an external filter script to define
479 complex filtering behavior. This script uses what I'm calling properties,
480 filters, and a rule - all implemented in a JSON syntax.
484 A 'property' is a typical JSON entry of the form:
486 "propertyName" : "value"
488 Here are the property names that BamTools will recognize:
513 For properties with boolean values, use the words "true" or "false".
518 will keep only alignments that are flagged as 'mapped'.
520 For properties with numeric values, use the desired number with optional
521 comparison operators ( >, >=, <, <=, !). For example,
523 "mapQuality" : ">=75"
525 will keep only alignments with mapQuality greater than or equal to 75.
527 If you're familiar with JSON, you know that integers can be bare (without
528 quotes). However, if you a comparison operator, be sure to enclose in quotes.
530 For string-based properties, the above operators are available. In addition,
531 you can also use some basic pattern-matching operators. For example,
533 "reference" : "ALU*" // reference starts with 'ALU'
534 "name" : "*foo" // name ends with 'foo'
535 "cigar" : "*D*" // cigar contains a 'D' anywhere
538 The reference property refers to the reference name, not the BAM reference
541 The tag property has an extra layer, so that the syntax will look like this:
545 where XX is the 2-letter SAM/BAM tag and value is, well, the value.
546 Comparison operators can still apply to values, so tag properties of:
555 A 'filter' is a JSON container of properties that will be AND-ed together. For
559 "reference" : "chr1",
560 "mapQuality" : ">50",
564 would result in an output BAM file containing only alignments from chr1 with a
565 mapQuality >50 and edit distance of less than 4.
567 A single, unnamed filter like this is the minimum necessary for a complete
568 filter script. Save this file and use as the -script parameter and you should
571 Moving on to more potent filtering...
573 You can also define multiple filters.
574 To do so, you just need to use the "filters" keyword along with JSON array
581 "reference" : "chr1",
585 "reference" : "chr1",
586 "isReverseStrand" : "true"
591 These filters will be (inclusive) OR-ed together by default. So you'd get a
592 resulting BAM with only alignments from chr1 that had either mapQuality >50 or
593 on the reverse strand (or both).
597 Alternatively to anonymous OR-ed filters, you can also provide what I've called
598 a "rule". By giving each filter an "id", using this "rule" keyword you can
599 describe boolean relationships between your filter sets.
601 Available rule operators:
607 This might sound a little fuzzy at this point, so let's get back to an example:
614 "reference" : "chr1",
619 "reference" : "chr1",
620 "isReverseStrand" : "true"
624 "reference" : "chr1",
625 "queryBases" : "AGCT*"
629 "rule" : " (filter1 | filter2) & !filter3 "
632 In this case, we would only retain aligments that passed filter 1 OR filter 2,
633 AND also NOT filter 3.
635 These are dummy examples, and don't make much sense as an actual query case. But
636 hopefully this serves an adequate primer to get you started and discover the
637 potential flexibility here.
643 Description: prints header from BAM file(s).
645 Usage: bamtools header [-in <filename> -in <filename> ...]
648 -in <BAM filename> the input BAM file(s) [stdin]
651 --help, -h shows this help text
657 Description: creates index for BAM file.
659 Usage: bamtools index [-in <filename>] [-bti]
662 -in <BAM filename> the input BAM file [stdin]
663 -bti create (non-standard) BamTools index file
664 (*.bti). Default behavior is to create
665 standard BAM index (*.bai)
668 --help, -h shows this help tex
674 Description: merges multiple BAM files into one.
676 Usage: bamtools merge [-in <filename> -in <filename> ...]
677 [-out <filename> | [-forceCompression]] [-region <REGION>]
680 -in <BAM filename> the input BAM file(s)
681 -out <BAM filename> the output BAM file
682 -forceCompression if results are sent to stdout (like when
683 piping to another tool), default behavior
684 is to leave output uncompressed. Use this
685 flag to override and force compression
686 -region <REGION> genomic region. See README for more details
689 --help, -h shows this help text
695 Description: grab a random subset of alignments.
697 Usage: bamtools random [-in <filename> -in <filename> ...]
698 [-out <filename>] [-forceCompression] [-n]
702 -in <BAM filename> the input BAM file [stdin]
703 -out <BAM filename> the output BAM file [stdout]
704 -forceCompression if results are sent to stdout (like when
705 piping to another tool), default behavior
706 is to leave output uncompressed. Use this
707 flag to override and force compression
708 -region <REGION> only pull random alignments from within this
709 genomic region. Index file is
710 recommended for better performance, and
711 is used automatically if it exists. See
712 'bamtools help index' for more details
716 -n <count> number of alignments to grab. Note that no
717 duplicate checking is performed [10000]
720 --help, -h shows this help text
726 Description: sorts a BAM file.
728 Usage: bamtools sort [-in <filename>] [-out <filename>] [sortOptions]
731 -in <BAM filename> the input BAM file [stdin]
732 -out <BAM filename> the output BAM file [stdout]
735 -byname sort by alignment name
738 -n <count> max number of alignments per tempfile
740 -mem <Mb> max memory to use [1024]
743 --help, -h shows this help text
749 Description: splits a BAM file on user-specified property, creating a new BAM
750 output file for each value found.
752 Usage: bamtools split [-in <filename>] [-stub <filename stub>]
753 < -mapped | -paired | -reference | -tag <TAG> >
756 -in <BAM filename> the input BAM file [stdin]
757 -stub <filename stub> prefix stub for output BAM files (default
758 behavior is to use input filename,
759 without .bam extension, as stub). If
760 input is stdin and no stub provided, a
761 timestamp is generated as the stub.
764 -mapped split mapped/unmapped alignments
765 -paired split single-end/paired-end alignments
766 -reference split alignments by reference
767 -tag <tag name> splits alignments based on all values of TAG
768 encountered (i.e. -tag RG creates a BAM
769 file for each read group in original
773 --help, -h shows this help text
779 Description: prints general alignment statistics.
781 Usage: bamtools stats [-in <filename> -in <filename> ...] [statsOptions]
784 -in <BAM filename> the input BAM file [stdin]
787 -insert summarize insert size data
790 --help, -h shows this help text
792 --------------------------------------------------------------------------------
794 --------------------------------------------------------------------------------
796 Both the BamTools API and toolkit are released under the MIT License.
797 Copyright (c) 2009-2010 Derek Barnett, Erik Garrison, Gabor Marth,
800 See included file LICENSE for details.
802 --------------------------------------------------------------------------------
803 V. Acknowledgements :
804 --------------------------------------------------------------------------------
806 * Aaron Quinlan for several key feature ideas and bug fix contributions
807 * Baptiste Lepilleur for the public-domain JSON parser (JsonCPP)
808 * Heng Li, author of SAMtools - the original C-language BAM API/toolkit.
810 --------------------------------------------------------------------------------
812 --------------------------------------------------------------------------------
814 Feel free to contact me with any questions, comments, suggestions, bug reports,
819 Biology Dept., Boston College
821 Email: barnetde@bc.edu
822 Project Websites: http://github.com/pezmaster31/bamtools (ACTIVE SUPPORT)
823 http://sourceforge.net/projects/bamtools (major updates only)