derek [Thu, 10 Nov 2011 04:58:20 +0000 (23:58 -0500)]
Added generic I/O device to BamIndex side of things
* Remote BAM access (now w/ random access) seems to be working with the
simple test cases so far
* Major TODO: not yet implemented for Windows
derek [Mon, 7 Nov 2011 17:50:10 +0000 (12:50 -0500)]
Implemented basic TCP support layer
* buffered I/O
* design should support future expansion of protocols, proxies, etc
* so far, HTTP range requests working well (on plain HTML text tests,
not yet BAM-tested)
derek [Wed, 12 Oct 2011 20:30:59 +0000 (16:30 -0400)]
Major speedup in SamSequenceDictionary & SamReadGroupDictionary classes
* Please note that this does introduce a minor source-incompatibility,
only affecting those working directly with the provided Sam*Iterator
typedefs. The short answer is that the iterator now references a
std::pair instead of the 'plain old' data. Use the pair's "second" field
to access the desired SamSequence or SamReadGroup.
* Doxygen docs have been updated to reflect this and provide a bit more
explanation/examples (in docs folder run 'doxygen Doxyfile' to get the
updated API pages).
derek [Tue, 11 Oct 2011 20:52:51 +0000 (16:52 -0400)]
Cleanup in SortTool
* Now using the new BamTools::Algorithms::Sort function objects
* Special handling of unmapped alignments should no longer be necessary
as the sorting function objects (also used by multi-reader merging
strategy) handles those cases.
derek [Fri, 7 Oct 2011 20:11:43 +0000 (16:11 -0400)]
Merge with earlier IODevice work
* This commit still has some console pollution. I need to work in the
recent Exception/ErrorString approach, but wanted to go ahead and do the
merge-conflict resolution now before diving into remote file support.
derek [Fri, 7 Oct 2011 19:12:57 +0000 (15:12 -0400)]
Removed STDERR pollution by API
* Accomplished this by introducing a GetErrorString() on most API
objects. When a method returns false, you can ignore it, parse the error
string to decide what to do next, prompt the user, make a sandwich,
whatever. But nothing should leak out to the console.
* Internally the error messages are passed by a new BamException class.
This new exception should not cross the library boundary. The exception
should be caught "under the hood" and its what() string should be
(possibly formatted and) stored as the error string in one of the high-
In standard indexed BAM files with with sparce coverage (our test case was a roughly 1M read RNAseq BAM file), queries made to intervals may not have any of the candidate offesets present in the index as the BAM index only contains bins that have reads.
Without this bail out, we would get a crash. Returning false silently is the preferred behavior in our view as it allows our read logic to go to the next query and does not add noise to stderr.
Removed 'core mode' concept from BamMultiReader internals
* Now char data is only generated if needed by multi-merger
implementation or on-demand by client call to
BamMultiReader::GetNextAlignment()
Basic internal implementation of BamFile & BamPipe
* BgzfStream now working on IBamIODevice instead of FILE*
* BamReaderPrivate now queries stream's IsOpen() method instead of
touching member variable directly
* Empty implementations of BamHttp & BamFtp
* Added global BT_ASSERT_X macro for convenience
Bug discovered. The chunkStop was not being read from the correct offset (rather always being read as the first chunkStart value for the # alignment chunks in that bin of the index.
The result of this is that chunkStop will never be >= minOffset (or maybe rarely, since it always equals the first chunkStart for the first chunk) and thus the linear index doesn't really help in reducing the number of seeks performed.
derek [Tue, 28 Jun 2011 16:31:25 +0000 (12:31 -0400)]
Added unique-alignment checks for ResolveTool
* Unique-ness determined by comparing MapQuality to 0
* Only pairs with both mates unique are used for the 'makeStats' median
fragment size calculation.
Alec Chapman [Tue, 28 Jun 2011 01:58:30 +0000 (21:58 -0400)]
Fix Visual Studio compiler errors.
Don't use dynamic stack allocation (variable length arrays).
Rename bamtools target to bamtools_cmd to not conflict with BamTools target (they differ only in case).
bamtools_cmd only compiles if I remove bamtools_filter.cpp, which I haven't committed.
I also had to manually configure the include directory for zlib,
but that's probably due to having multiple copies floating around my machine.
derek [Thu, 23 Jun 2011 19:35:35 +0000 (15:35 -0400)]
Fixed -fPIC issue for CentOS users.
* Forced compiler flag that was not being automatically set by CMake on
that OS. Had previously set this on API library. Got feedback that it
worked there, so I added the flag to Utils & JsonCPP libs as well.
derek [Fri, 17 Jun 2011 04:09:49 +0000 (00:09 -0400)]
Removed pessimistic warnings when jumping to regions with no data, using
the standard index format (not actually an error case, so no need to
alarm users with scary messages)
derek [Fri, 17 Jun 2011 02:13:26 +0000 (22:13 -0400)]
Added re-calculation of BamAlignment's BinID during
BamWriter::SaveAlignment() in all cases
* Previously, the bin IDs of purely "core-only" alignments were simply
written directly out to output BAM. However, in cases where alignment
Position is changed (re-alignment), the original bin ID may no longer be
correct.
derek [Tue, 14 Jun 2011 17:41:56 +0000 (13:41 -0400)]
Implemented better coupling of unmapped reads with mates during sorting
(assuming assigned same coordinates)
* Used std::stable_sort instead of std::sort, to preserve order
* Add checks at buffer boundary to keep mates from being split into
different temp files. This makes the buffer boundary "softer", but in
practice, shouldn't differ much if at all.
derek [Sat, 11 Jun 2011 21:05:43 +0000 (17:05 -0400)]
Created 3 modes for ResolveTool: makeStats, markPairs, & twoPass
* "TwoPass" mode (the initial implementation of the tool) effectively
eliminates piped BAMs as an input option, since you can't exactly rewind
stdin and start reading from the beginning.
* To get around this, I separated the two passes into separate "modes"
(-makeStats & -markPairs), that communicate via a simple, human-readable
stats summary file. Data can then be merged, filtered, etc and piped
into each mode if you don't mind the runtime of preprocessing twice but
don't want to physically store the unresolved intermediate BAM file.
Brought API up to compliance with recent SAM Format Spec (v1.4-r962)
* Added support for new "binary array" tag type
* Added support for '=' and 'X' CIGAR ops
* Added support for multiple PG entries in header
* Added support for new RG fields
Major performance boost to startup & random-access - especially for the
use cases involving multiple (hundreds) of BAMs with BAI index files.
* This did require some changes to the BamIndex interface. I doubt man
y people are writing custom index format classes, but if you are one of
them and have any problems, feel free to contact me with questions.
projects / bamtools.git / log