1 --------------------------------------------------------------------------------
3 --------------------------------------------------------------------------------
5 BamTools: a C++ API & toolkit for reading/writing/manipulating BAM files.
23 --------------------------------------------------------------------------------
25 --------------------------------------------------------------------------------
27 BamTools provides both a programmer's API and an end-user's toolkit for handling
30 ----------------------------------------
32 ----------------------------------------
34 The API consists of 2 main modules: BamReader and BamWriter. As you would
35 expect, BamReader provides read-access to BAM files, while BamWriter handles
36 writing data to BAM files. BamReader provides the interface for random-access
37 (jumping) in a BAM file, as well as generating BAM index files.
39 BamMultiReader is an extra module that allows you to manage multiple open BAM
40 files for reading. It provides some validation & bookkeeping under the hood to
41 keep all files sync'ed up for you.
43 Additional files used by the API:
45 - BamAlignment.* : implements the BamAlignment data structure
47 - BamAux.h : contains various constants, data structures and utility
48 methods used throught the API.
50 - BamIndex.* : implements both the standard BAM format index (".bai") as
51 well as a new BamTools-specific index (".bti").
53 - BGZF.* : contains our implementation of the Broad Institute's BGZF
56 ----------------------------------------
58 ----------------------------------------
60 If you've been using the BamTools since the early days, you'll notice that our
61 'toy' API examples (BamConversion, BamDump, BamTrim,...) are now gone. We have
62 dumped these in favor of a suite of small utilities that we hope both
63 developers and end-users find useful:
65 usage: bamtools [--help] COMMAND [ARGS]
67 Available bamtools commands:
69 convert Converts between BAM and a number of other formats
70 count Prints number of alignments in BAM file(s)
71 coverage Prints coverage statistics from the input BAM file
72 filter Filters BAM file(s) by user-specified criteria
73 header Prints BAM header information
74 index Generates index for BAM file
75 merge Merge multiple BAM files into single file
76 random Select random alignments from existing BAM file(s)
77 sort Sorts the BAM file according to some criteria
78 split Splits a BAM file on user-specifed property, creating a
79 new BAM output file for each value found
80 stats Prints some basic statistics from input BAM file(s)
82 See 'bamtools help COMMAND' for more information on a specific command.
84 --------------------------------------------------------------------------------
86 --------------------------------------------------------------------------------
88 ----------------------------------------
90 ----------------------------------------
92 BamTools has been migrated to a CMake-based build system. We believe that this
93 should simplify the build process across all platforms, especially as the
94 BamTools API moves into a shared library (that you link to instead of compiling
95 lots of source files directly into your application). CMake is available on all
96 major platforms, and indeed comes *out-of-the-box* with many Linux distributions.
98 To see if you have CMake (and which version), try this command:
102 BamTools requires CMake version >= 2.6.4. If you are missing CMake or have an
103 older version, check your OS package manager (for Linux) or download it here:
104 http://www.cmake.org/cmake/resources/software.html .
106 ----------------------------------------
108 ----------------------------------------
110 Ok, now that you have CMake ready to go, let's build BamTools. A good
111 practice in building applications is to do an out-of-source build, meaning
112 that we're going to set up an isolated place to hold all the intermediate
115 In the top-level directory of BamTools, type the following commands:
122 This creates a Visual Studio solution file, which can then be built to create
123 the toolkit executable and API DLL's.
126 After running cmake, just run:
130 Then go back up to the BamTools root directory.
134 ----------------------------------------
136 ----------------------------------------
138 Assuming the build process finished correctly, you should be able to find the
139 toolkit executable here:
143 The BamTools-associated libraries will be found here:
147 The BamTools API headers will be found here:
151 --------------------------------------------------------------------------------
153 --------------------------------------------------------------------------------
155 ** General usage information - perhaps explain common terms, point to SAM/BAM
158 ----------------------------------------
160 ----------------------------------------
162 The API, as noted above, contains 2 main modules - BamReader & BamWriter - for
163 dealing with BAM files. Alignment data is made available through the
164 BamAlignment data structure.
166 A simple (read-only) scenario for accessing BAM data would look like the
169 // open our BamReader
171 reader.Open("someData.bam", "someData.bam.bai");
173 // define our region of interest
174 // in this example: bases 0-500 on the reference "chrX"
175 int id = reader.GetReferenceID("chrX");
176 BamRegion region(id, 0, id, 500);
177 reader.SetRegion(region);
179 // iterate through alignments in this region,
180 // ignoring alignments with a MQ below some cutoff
182 while ( reader.GetNextAlignment(al) ) {
183 if ( al.MapQuality >= 50 )
190 To use this API in your application, you simply need to do the following:
192 1 - Build the BamTools library (see Installation steps above).
194 2 - Import BamTools API functionality as needed, for example:
196 #include "api/BamReader.h"
197 #include "api/BamWriter.h"
198 using namespace BamTools; // all BamTools classes/methods live within
201 3 - In your own build step, point your include path to the
202 (BAMTOOLS_ROOT)/include directory. Link your app with '-lbamtools' ('l'
205 * You may need to modify the -L flag (library path) as well to help your linker
206 find the (BAMTOOLS_ROOT)/lib directory.
208 * Depending on your platform and where you install the BamTools API library, you
209 may also need to adjust how your app locates the library at runtime. For
210 Windows users, this can be as simple as dropping the DLL in the same folder as
211 your executable. For *nix users (using gcc at least), you can add the following
212 to your app's CXXFLAGS:
214 -Wl,-rpath,$(BAMTOOLS_LIB_DIR)
216 where BAMTOOLS_LIB_DIR is, as you would guess, the directory containing the libs.
217 An alternative is to set your local LD_LIBRARY_PATH environment variable.
219 See any included programs for more detailed usage examples. See comments in the
220 header files for more detailed API documentation.
222 Note - For users that don't want to bother with the new BamTools shared library
223 scheme: you are certainly free to just compile the API source code directly into
224 your application, but be aware that the files involved are subject to change.
225 Meaning that filenames, number of files, etc. are not fixed. You will also need
226 to be sure to link with '-lz' for ZLIB functionality (linking with '-lbamtools'
227 gives you this automatically).
229 ----------------------------------------
231 ----------------------------------------
233 BamTools provides a small, but powerful suite of command-line utility programs
234 for manipulating and querying BAM files for data.
240 All BamTools utilities handle I/O operations using a common set of arguments.
245 The input BAM files(s).
247 If a tool accepts multiple BAM files as input, each file gets its own "-in"
248 option on the command line. If no "-in" is provided, the tool will attempt
249 to read BAM data from stdin.
251 To read a single BAM file, use a single "-in" option:
252 > bamtools *tool* -in myData1.bam ...ARGS...
254 To read multiple BAM files, use multiple "-in" options:
255 > bamtools *tool* -in myData1.bam -in myData2.bam ...ARGS...
257 To read from stdin (if supported), omit the "-in" option:
258 > bamtools *tool* ...ARGS...
264 If a tool outputs a result BAM file, specify the filename using this option.
265 If none is provided, the tool will typically write to stdout.
267 *Note: Not all tools output BAM data (e.g. count, header, etc.)
271 A region of interest. See below for accepted 'REGION string' formats.
273 Many of the tools accept this option, which allows a user to only consider
274 alignments that overlap this region (whether counting, filtering, merging,
277 An alignment is considered to overlap a region if any part of the alignments
278 intersects the left/right boundaries. Thus, a 50bp alignment at position 70
279 will overlap a region beginning at position 100.
282 ----------------------
283 A proper REGION string can be formatted like any of the following examples:
284 where 'chr1' is the name of a reference (not its ID)and '' is any valid
285 integer position within that reference.
288 chr1 - only alignments on (entire) reference 'chr1'
289 chr1:500 - only alignments overlapping the region starting at
290 chr1:500 and continuing to the end of chr1
291 chr1:500..1000 - only alignments overlapping the region starting at
292 chr1:500 and continuing to chr1:1000
293 chr1:500..chr3:750 - only alignments overlapping the region starting at
294 chr1:500 and continuing to chr3:750. This 'spanning'
295 region assumes that the reference specified as the
296 right boundary will occur somewhere in the file after
297 the left boundary. On a sorted BAM, a REGION of
298 'chr4:500..chr2:1500' will produce undefined
299 (incorrect) results. So don't do it. :)
301 *Note: Most of the tools that accept a REGION string will perform without an
302 index file, but typically at great cost to performance (having to
303 plow through the entire file until the region of interest is found).
304 For optimum speed, be sure that index files are available for your
309 Force compression of BAM output.
311 When tools are piped together (see details below), the default behavior is
312 to turn off compression. This can greatly increase performance when the data
313 does not have to be constantly decompressed and recompressed. This is
314 ignored any time an output BAM file is specified using "-out".
320 Many of the tools in BamTools can be chained together by piping. Any tool that
321 accepts stdin can be piped into, and any that can output stdout can be piped
324 > bamtools filter -in data1.bam -in data2.bam -mapQuality ">50" | bamtools count
326 will give a count of all alignments in your 2 BAM files with a mapQuality of
327 greater than 50. And of course, any tool writing to stdout can be piped into
334 convert Converts between BAM and a number of other formats
335 count Prints number of alignments in BAM file(s)
336 coverage Prints coverage statistics from the input BAM file
337 filter Filters BAM file(s) by user-specified criteria
338 header Prints BAM header information
339 index Generates index for BAM file
340 merge Merge multiple BAM files into single file
341 random Select random alignments from existing BAM file(s)
342 sort Sorts the BAM file according to some criteria
343 split Splits a BAM file on user-specifed property, creating a new
344 BAM output file for each value found
345 stats Prints some basic statistics from input BAM file(s)
351 Description: converts BAM to a number of other formats
353 Usage: bamtools convert -format <FORMAT> [-in <filename> -in <filename> ...]
354 [-out <filename>] [other options]
357 -in <BAM filename> the input BAM file(s) [stdin]
358 -out <BAM filename> the output BAM file [stdout]
359 -format <FORMAT> the output file format - see below for
363 -region <REGION> genomic region. Index file is recommended for
364 better performance, and is read
365 automatically if it exists. See 'bamtools
366 help index' for more details on creating
370 -fasta <FASTA filename> FASTA reference file
371 -mapqual print the mapping qualities
374 -noheader omit the SAM header from output
377 --help, -h shows this help text
381 - Currently supported output formats ( BAM -> X )
383 Format type FORMAT (command-line argument)
384 ------------ -------------------------------
394 > bamtools convert -format json -in myData.bam -out myData.json
396 - Pileup Options have no effect on formats other than "pileup"
397 SAM Options have no effect on formats other than "sam"
403 Description: prints number of alignments in BAM file(s).
405 Usage: bamtools count [-in <filename> -in <filename> ...] [-region <REGION>]
408 -in <BAM filename> the input BAM file(s) [stdin]
409 -region <REGION> genomic region. Index file is recommended
410 for better performance, and is used
411 automatically if it exists. See
412 'bamtools help index' for more details
416 --help, -h shows this help text
422 Description: prints coverage data for a single BAM file.
424 Usage: bamtools coverage [-in <filename>] [-out <filename>]
427 -in <BAM filename> the input BAM file [stdin]
428 -out <filename> the output file [stdout]
431 --help, -h shows this help text
437 Description: filters BAM file(s).
439 Usage: bamtools filter [-in <filename> -in <filename> ...]
440 [-out <filename> | [-forceCompression]]
442 [ [-script <filename] | [filterOptions] ]
445 -in <BAM filename> the input BAM file(s) [stdin]
446 -out <BAM filename> the output BAM file [stdout]
447 -region <REGION> only read data from this genomic region (see
448 README for more details)
449 -script <filename> the filter script file (see README for more
451 -forceCompression if results are sent to stdout (like when
452 piping to another tool), default behavior
453 is to leave output uncompressed. Use this
454 flag to override and force compression
457 -alignmentFlag <int> keep reads with this *exact* alignment flag
458 (for more detailed queries, see below)
459 -insertSize <int> keep reads with insert size that matches
461 -mapQuality <[0-255]> keep reads with map quality that matches
463 -name <string> keep reads with name that matches pattern
464 -queryBases <string> keep reads with motif that matches pattern
465 -tag <TAG:VALUE> keep reads with this key=>value pair
467 Alignment Flag Filters:
468 -isDuplicate <true/false> keep only alignments that are marked as
470 -isFailedQC <true/false> keep only alignments that failed QC [true]
471 -isFirstMate <true/false> keep only alignments marked as first mate
473 -isMapped <true/false> keep only alignments that were mapped [true]
474 -isMateMapped <true/false> keep only alignments with mates that mapped
476 -isMateReverseStrand <true/false> keep only alignments with mate on reverse
478 -isPaired <true/false> keep only alignments that were sequenced as
480 -isPrimaryAlignment <true/false> keep only alignments marked as primary
482 -isProperPair <true/false> keep only alignments that passed paired-end
484 -isReverseStrand <true/false> keep only alignments on reverse strand
486 -isSecondMate <true/false> keep only alignments marked as second mate
490 --help, -h shows this help text
496 The BamTools filter tool allows you to use an external filter script to define
497 complex filtering behavior. This script uses what I'm calling properties,
498 filters, and a rule - all implemented in a JSON syntax.
502 A 'property' is a typical JSON entry of the form:
504 "propertyName" : "value"
506 Here are the property names that BamTools will recognize:
531 For properties with boolean values, use the words "true" or "false".
536 will keep only alignments that are flagged as 'mapped'.
538 For properties with numeric values, use the desired number with optional
539 comparison operators ( >, >=, <, <=, !). For example,
541 "mapQuality" : ">=75"
543 will keep only alignments with mapQuality greater than or equal to 75.
545 If you're familiar with JSON, you know that integers can be bare (without
546 quotes). However, if you a comparison operator, be sure to enclose in quotes.
548 For string-based properties, the above operators are available. In addition,
549 you can also use some basic pattern-matching operators. For example,
551 "reference" : "ALU*" // reference starts with 'ALU'
552 "name" : "*foo" // name ends with 'foo'
553 "cigar" : "*D*" // cigar contains a 'D' anywhere
556 The reference property refers to the reference name, not the BAM reference
559 The tag property has an extra layer, so that the syntax will look like this:
563 where XX is the 2-letter SAM/BAM tag and value is, well, the value.
564 Comparison operators can still apply to values, so tag properties of:
573 A 'filter' is a JSON container of properties that will be AND-ed together. For
577 "reference" : "chr1",
578 "mapQuality" : ">50",
582 would result in an output BAM file containing only alignments from chr1 with a
583 mapQuality >50 and edit distance of less than 4.
585 A single, unnamed filter like this is the minimum necessary for a complete
586 filter script. Save this file and use as the -script parameter and you should
589 Moving on to more potent filtering...
591 You can also define multiple filters.
592 To do so, you just need to use the "filters" keyword along with JSON array
599 "reference" : "chr1",
603 "reference" : "chr1",
604 "isReverseStrand" : "true"
609 These filters will be (inclusive) OR-ed together by default. So you'd get a
610 resulting BAM with only alignments from chr1 that had either mapQuality >50 or
611 on the reverse strand (or both).
615 Alternatively to anonymous OR-ed filters, you can also provide what I've called
616 a "rule". By giving each filter an "id", using this "rule" keyword you can
617 describe boolean relationships between your filter sets.
619 Available rule operators:
625 This might sound a little fuzzy at this point, so let's get back to an example:
632 "reference" : "chr1",
637 "reference" : "chr1",
638 "isReverseStrand" : "true"
642 "reference" : "chr1",
643 "queryBases" : "AGCT*"
647 "rule" : " (filter1 | filter2) & !filter3 "
650 In this case, we would only retain aligments that passed filter 1 OR filter 2,
651 AND also NOT filter 3.
653 These are dummy examples, and don't make much sense as an actual query case. But
654 hopefully this serves an adequate primer to get you started and discover the
655 potential flexibility here.
661 Description: prints header from BAM file(s).
663 Usage: bamtools header [-in <filename> -in <filename> ...]
666 -in <BAM filename> the input BAM file(s) [stdin]
669 --help, -h shows this help text
675 Description: creates index for BAM file.
677 Usage: bamtools index [-in <filename>] [-bti]
680 -in <BAM filename> the input BAM file [stdin]
681 -bti create (non-standard) BamTools index file
682 (*.bti). Default behavior is to create
683 standard BAM index (*.bai)
686 --help, -h shows this help tex
692 Description: merges multiple BAM files into one.
694 Usage: bamtools merge [-in <filename> -in <filename> ...]
695 [-out <filename> | [-forceCompression]] [-region <REGION>]
698 -in <BAM filename> the input BAM file(s)
699 -out <BAM filename> the output BAM file
700 -forceCompression if results are sent to stdout (like when
701 piping to another tool), default behavior
702 is to leave output uncompressed. Use this
703 flag to override and force compression
704 -region <REGION> genomic region. See README for more details
707 --help, -h shows this help text
713 Description: grab a random subset of alignments.
715 Usage: bamtools random [-in <filename> -in <filename> ...]
716 [-out <filename>] [-forceCompression] [-n]
720 -in <BAM filename> the input BAM file [stdin]
721 -out <BAM filename> the output BAM file [stdout]
722 -forceCompression if results are sent to stdout (like when
723 piping to another tool), default behavior
724 is to leave output uncompressed. Use this
725 flag to override and force compression
726 -region <REGION> only pull random alignments from within this
727 genomic region. Index file is
728 recommended for better performance, and
729 is used automatically if it exists. See
730 'bamtools help index' for more details
734 -n <count> number of alignments to grab. Note that no
735 duplicate checking is performed [10000]
738 --help, -h shows this help text
744 Description: sorts a BAM file.
746 Usage: bamtools sort [-in <filename>] [-out <filename>] [sortOptions]
749 -in <BAM filename> the input BAM file [stdin]
750 -out <BAM filename> the output BAM file [stdout]
753 -byname sort by alignment name
756 -n <count> max number of alignments per tempfile
758 -mem <Mb> max memory to use [1024]
761 --help, -h shows this help text
767 Description: splits a BAM file on user-specified property, creating a new BAM
768 output file for each value found.
770 Usage: bamtools split [-in <filename>] [-stub <filename stub>]
771 < -mapped | -paired | -reference | -tag <TAG> >
774 -in <BAM filename> the input BAM file [stdin]
775 -stub <filename stub> prefix stub for output BAM files (default
776 behavior is to use input filename,
777 without .bam extension, as stub). If
778 input is stdin and no stub provided, a
779 timestamp is generated as the stub.
782 -mapped split mapped/unmapped alignments
783 -paired split single-end/paired-end alignments
784 -reference split alignments by reference
785 -tag <tag name> splits alignments based on all values of TAG
786 encountered (i.e. -tag RG creates a BAM
787 file for each read group in original
791 --help, -h shows this help text
797 Description: prints general alignment statistics.
799 Usage: bamtools stats [-in <filename> -in <filename> ...] [statsOptions]
802 -in <BAM filename> the input BAM file [stdin]
805 -insert summarize insert size data
808 --help, -h shows this help text
810 --------------------------------------------------------------------------------
812 --------------------------------------------------------------------------------
814 Both the BamTools API and toolkit are released under the MIT License.
815 Copyright (c) 2009-2010 Derek Barnett, Erik Garrison, Gabor Marth,
818 See included file LICENSE for details.
820 --------------------------------------------------------------------------------
821 V. Acknowledgements :
822 --------------------------------------------------------------------------------
824 * Aaron Quinlan for several key feature ideas and bug fix contributions
825 * Baptiste Lepilleur for the public-domain JSON parser (JsonCPP)
826 * Heng Li, author of SAMtools - the original C-language BAM API/toolkit.
828 --------------------------------------------------------------------------------
830 --------------------------------------------------------------------------------
832 Feel free to contact me with any questions, comments, suggestions, bug reports,
837 Biology Dept., Boston College
839 Email: barnetde@bc.edu
840 Project Websites: http://github.com/pezmaster31/bamtools (ACTIVE SUPPORT)
841 http://sourceforge.net/projects/bamtools (major updates only)