1 Beta Release 0.1.16 (21 April, 2011)
2 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4 Notable changes in samtools:
6 * Support the new SAM/BAM type `B' in the latest SAM spec v1.4.
8 * When the output file of `samtools merge' exists, do not overwrite it unless
9 a new command-line option `-f' is applied.
11 * Bugfix: BED support is not working when the input BED is not sorted.
13 * Bugfix: some reads without coordinates but given on the reverse strand are
16 Notable changes in bcftools:
18 * Code cleanup: separated max-likelihood inference and Bayesian inference.
20 * Test Hardy-Weinberg equilibrium with a likelihood-ratio test.
22 * Provided another association test P-value by likelihood-ratio test.
24 * Use Brent's method to estimate the site allele frequency when EM converges
25 slowly. The resulting ML estimate of allele frequnecy is more accurate.
27 * Added the `ldpair' command, which computes r^2 between SNP pairs given in
30 Also, the `pileup' command, which has been deprecated by `mpileup' since
31 version 0.1.10, will be dropped in the next release. The old `pileup' command
32 is substandard and causing a lot of confusion.
34 (0.1.16: 21 April 2011, r963:234)
38 Beta Release 0.1.15 (10 April, 2011)
39 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
43 * Allow to perform variant calling or to extract information in multiple
44 regions specified by a BED file (`samtools mpileup -l', `samtools view -L'
45 and `bcftools view -l').
47 * Added the `depth' command to samtools to compute the per-base depth with a
48 simpler interface. File `bam2depth.c', which implements this command, is the
49 recommended example on how to use the mpileup APIs.
51 * Estimate genotype frequencies with ML; perform chi^2 based Hardy-Weinberg
52 test using this estimate.
54 * For `samtools view', when `-R' is specified, drop read groups in the header
55 that are not contained in the specified file.
57 * For `samtools flagstat', separate QC-pass and QC-fail reads.
59 * Improved the command line help of `samtools mpileup' and `bcftools view'.
61 * Use a global variable to control the verbose level of samtools stderr
62 output. Nonetheless, it has not been full utilized.
64 * Fixed an issue in association test which may report false associations,
65 possibly due to floating point underflow.
67 (0.1.15: 10 April 2011, r949:203)
71 Beta release 0.1.14 (21 March, 2011)
72 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
74 This release implements a method for testing associations for case-control
75 data. The method does not call genotypes but instead sums over all genotype
76 configurations to compute a chi^2 based test statistics. It can be potentially
77 applied to comparing a pair of samples (e.g. a tumor-normal pair), but this
78 has not been evaluated on real data.
80 Another new feature is to make X chromosome variant calls when female and male
81 samples are both present. The user needs to provide a file indicating the
82 ploidy of each sample (see also manual bcftools/bcftools.1).
84 Other notable changes:
86 * Added `bcftools view -F' to parse BCF files generated by samtools r921 or
87 older which encodes PL in a different way.
89 * Changed the behavior of `bcftools view -s'. Now when a list of samples is
90 provided, the samples in the output will be reordered to match the ordering
91 in the sample list. This change is mainly designed for association test.
93 * Sped up `bcftools view -v' for target sequencing given thousands of samples.
94 Also added a new option `view -d' to skip loci where only a few samples are
97 * Dropped HWE test. This feature has never been implemented properly. An EM
98 should be much better. To be implemented in future.
100 * Added the `cat' command to samtools. This command concatenate BAMs with
101 identical sequence dictionaries in an efficient way. Modified from bam_cat.c
102 written by Chris Saunders.
104 * Added `samtools view -1' to write BAMs at a low compression level but twice
105 faster to create. The `sort' command generates temporary files at a low
106 compression level as well.
108 * Added `samtools mpileup -6' to accept "BAM" with Illumina 1.3+ quality
109 strings (strictly speaking, such a file is not BAM).
111 * Added `samtools mpileup -L' to skip INDEL calling in regions with
112 excessively high coverage. Such regions dramatically slow down mpileup.
114 * Updated `misc/export2sam.pl', provided by Chris Saunders from Illumina Inc.
116 (0.1.14: 21 March 2011, r933:170)
120 Beta release 0.1.13 (1 March, 2011)
121 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
123 The most important though largely invisible modification is the change of the
124 order of genotypes in the PL VCF/BCF tag. This is to conform the upcoming VCF
125 spec v4.1. The change means that 0.1.13 is not backward compatible with VCF/BCF
126 generated by samtools older than r921 inclusive. VCF/BCF generated by the new
127 samtools will contain a line `##fileformat=VCFv4.1' as well as the samtools
130 Single Individual Haplotyping (SIH) is added as an experimental feature. It
131 originally aims to produce haploid consensus from fosmid pool sequencing, but
132 also works with short-read data. For short reads, phased blocks are usually too
133 short to be useful in many applications, but they can help to rule out part of
134 SNPs close to INDELs or between copies of CNVs.
137 Other notable changes in samtools:
139 * Construct per-sample consensus to reduce the effect of nearby SNPs in INDEL
140 calling. This reduces the power but improves specificity.
142 * Improved sorting order checking in indexing. Now indexing is the preferred way
143 to check if a BAM is sorted.
145 * Added a switch `-E' to mpileup and calmd. This option uses an alternative way
146 to apply BAQ, which increases sensistivity, especially to MNPs, at the cost of
147 a little loss in specificity.
149 * Added `mpileup -A' to allow to use reads in anomalous pairs in SNP calling.
151 * Added `mpileup -m' to allow fine control of the collection of INDEL candidates.
153 * Added `mpileup -S' to compute per-sample strand bias P-value.
155 * Added `mpileup -G' to exclude read groups in variant calling.
157 * Fixed segfault in indel calling related to unmapped and refskip reads.
159 * Fixed an integer overflow in INDEL calling. This bug produces wrong INDEL
160 genotypes for longer short INDELs, typically over 10bp.
162 * Fixed a bug in tview on big-endian machines.
164 * Fixed a very rare memory issue in bam_md.c
166 * Fixed an out-of-boundary bug in mpileup when the read base is `N'.
168 * Fixed a compiling error when the knetfile library is not used. Fixed a
169 library compiling error due to the lack of bam_nt16_nt4_table[] table.
170 Suppress a compiling warning related to the latest zlib.
173 Other notable changes in bcftools:
175 * Updated the BCF spec.
177 * Added the `FQ' VCF INFO field, which gives the phred-scaled probability
178 of all samples being the same (identical to the reference or all homozygous
179 variants). Option `view -f' has been dropped.
181 * Implementated of "vcfutils.pl vcf2fq" to generate a consensus sequence
182 similar to "samtools.pl pileup2fq".
184 * Make sure the GT FORMAT field is always the first FORMAT to conform the VCF
185 spec. Drop bcf-fix.pl.
187 * Output bcftools specific INFO and FORMAT in the VCF header.
189 * Added `view -s' to call variants from a subset of samples.
191 * Properly convert VCF to BCF with a user provided sequence dictionary. Nonetheless,
192 custom fields are still unparsed and will be stored as a missing value.
194 * Fixed a minor bug in Fisher's exact test; the results are rarely changed.
197 (0.1.13: 1 March 2011, r926:134)
201 Beta release 0.1.12a (2 December, 2010)
202 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
204 This is another bug fix release:
206 * Fixed a memory violation in mpileup, which causes segfault. Release
207 0.1.9 and above are affected.
209 * Fixed a memory violation in the indel caller, which does not causes
210 segfault, but may potentially affect deletion calls in an unexpected
211 way. Release 0.1.10 and above are affected.
213 * Fixed a bug in computing r-square in bcftools. Few are using this
214 functionality and it only has minor effect.
216 * Fixed a memory leak in bam_fetch().
218 * Fixed a bug in writing meta information to the BAM index for the last
219 sequence. This bug is invisible to most users, but it is a bug anyway.
221 * Fixed a bug in bcftools which causes false "DP4=0,0,0,0" annotations.
223 (0.1.12: 2 December 2010, r862)
227 Beta release 0.1.11 (21 November, 2010)
228 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
230 This is mainly a bug fix release:
232 * Fixed a bug in random retrieval (since 0.1.8). It occurs when reads
233 are retrieved from a small region containing no reads.
235 * Fixed a bug in pileup (since 0.1.9). The bug causes an assertion
236 failure when the first CIGAR operation is a deletion.
238 * Improved fault tolerence in remote access.
240 One minor feature has been implemented in bcftools:
242 * Added a reference-free variant calling mode. In this mode, a site is
243 regarded as a variat iff the sample(s) contains two or more alleles;
244 the meaning of the QUAL field in the VCF output is changed
245 accordingly. Effectively, the reference allele is irrelevant to the
246 result in the new mode, although the reference sequence has to be
247 used in realignment when SAMtools computes genotype likelihoods.
249 In addition, since 0.1.10, the `pileup' command has been deprecated by
250 `mpileup' which is more powerful and more accurate. The `pileup' command
251 will not be removed in the next few releases, but new features will not
254 (0.1.11: 21 November 2010, r851)
258 Beta Release 0.1.10 (16 November, 2010)
259 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
261 This release is featured as the first major improvement to the indel
262 caller. The method is similar to the old one implemented in the pileup
263 command, but the details are handled more carefully both in theory and
264 in practice. As a result, the new indel caller usually gives more
265 accurate indel calls, though at the cost of sensitivity. The caller is
266 implemented in the mpileup command and is invoked by default. It works
267 with multiple samples.
269 Other notable changes:
271 * With the -r option, the calmd command writes the difference between
272 the original base quality and the BAQ capped base quality at the BQ
273 tag but does not modify the base quality. Please use -Ar to overwrite
274 the original base quality (the 0.1.9 behavior).
276 * Allow to set a maximum per-sample read depth to reduce memory. In
277 0.1.9, most of memory is wasted for the ultra high read depth in some
278 regions (e.g. the chr1 centromere).
280 * Optionally write per-sample read depth and per-sample strand bias
283 * Compute equal-tail (Bayesian) credible interval of site allele
284 frequency at the CI95 VCF annotation.
286 * Merged the vcfutils.pl varFilter and filter4vcf for better SNP/indel
289 (0.1.10: 16 November 2010, r829)
293 Beta Release 0.1.9 (27 October, 2010)
294 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
296 This release is featured as the first major improvement to the samtools'
297 SNP caller. It comes with a revised MAQ error model, the support of
298 multi-sample SNP calling and the computation of base alignment quality
301 The revised MAQ error model is based on the original model. It solves an
302 issue of miscalling SNPs in repetitive regions. Althought such SNPs can
303 usually be filtered at a later step, they mess up unfiltered calls. This
304 is a theoretical flaw in the original model. The revised MAQ model
305 deprecates the orginal MAQ model and the simplified SOAPsnp model.
307 Multi-sample SNP calling is separated in two steps. The first is done by
308 samtools mpileup and the second by a new program, bcftools, which is
309 included in the samtools source code tree. Multi-sample SNP calling also
310 works for single sample and has the advantage of enabling more powerful
311 filtration. It is likely to deprecate pileup in future once a proper
312 indel calling method is implemented.
314 BAQ is the Phred-scaled probability of a read base being wrongly
315 aligned. Capping base quality by BAQ has been shown to be very effective
316 in suppressing false SNPs caused by misalignments around indels or in
317 low-complexity regions with acceptable compromise on computation
318 time. This strategy is highly recommended and can be used with other SNP
321 In addition to the three major improvements, other notable changes are:
323 * Changes to the pileup format. A reference skip (the N CIGAR operator)
324 is shown as '<' or '>' depending on the strand. Tview is also changed
327 * Accelerated pileup. The plain pileup is about 50% faster.
329 * Regional merge. The merge command now accepts a new option to merge
330 files in a specified region.
332 * Fixed a bug in bgzip and razip which causes source files to be
333 deleted even if option -c is applied.
335 * In APIs, propogate errors to downstream callers and make samtools
336 return non-zero values once errors occur.
338 (0.1.9: 27 October 2010, r783)
342 Beta Release 0.1.8 (11 July, 2010)
343 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
345 Notable functional changes:
347 * Added the `reheader' command which replaces a BAM header with a new
348 header. This command is much faster than replacing header by
349 BAM->SAM->BAM conversions.
351 * Added the `mpileup' command which computes the pileup of multiple
354 * The `index' command now stores the number of mapped and unmapped
355 reads in the index file. This information can be retrieved quickly by
356 the new `idxstats' command.
358 * By default, pileup used the SOAPsnp model for SNP calling. This
359 avoids the floating overflow in the MAQ model which leads to spurious
360 calls in repetitive regions, although these calls will be immediately
361 filtered by varFilter.
363 * The `tview' command now correctly handles CIGARs like 7I10M and
364 10M1P1I10M which cause assertion failure in earlier versions.
366 * Tview accepts a region like `=10,000' where `=' stands for the
367 current sequence name. This saves typing for long sequence names.
369 * Added the `-d' option to `pileup' which avoids slow indel calling
370 in ultradeep regions by subsampling reads locally.
372 * Added the `-R' option to `view' which retrieves alignments in read
373 groups listed in the specified file.
375 Performance improvements:
377 * The BAM->SAM conversion is up to twice faster, depending on the
378 characteristic of the input.
380 * Parsing SAM headers with a lot of reference sequences is now much
383 * The number of lseek() calls per query is reduced when the query
384 region contains no read alignments.
388 * Fixed an issue in the indel caller that leads to miscall of indels.
389 Note that this solution may not work well when the sequencing indel
390 error rate is higher than the rate of SNPs.
392 * Fixed another issue in the indel caller which may lead to incorrect
395 * Fixed a bug in `sort' when option `-o' is applied.
397 * Fixed a bug in `view -r'.
399 APIs and other changes:
401 * Added iterator interfaces to random access and pileup. The callback
402 interfaces directly call the iterator interfaces.
404 * The BGZF blocks holding the BAM header are indepedent of alignment
405 BGZF blocks. Alignment records shorter than 64kB is guaranteed to be
406 fully contained in one BGZF block. This change is fully compatible
407 with the old version of samtools/picard.
409 Changes in other utilities:
411 * Updated export2sam.pl by Chris Saunders.
413 * Improved the sam2vcf.pl script.
415 * Added a Python version of varfilter.py by Aylwyn Scally.
417 (0.1.8: 11 July 2010, r613)
421 Beta Release 0.1.7 (10 November, 2009)
422 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
426 * Improved the indel caller in complex scenariors, in particular for
427 long reads. The indel caller is now able to make reasonable indel
428 calls from Craig Venter capillary reads.
430 * Rewrote single-end duplicate removal with improved
431 performance. Paired-end reads are not touched.
433 * Duplicate removal is now library aware. Samtools remove potential
434 PCR/optical dupliates inside a library rather than across libraries.
436 * SAM header is now fully parsed, although this functionality is not
437 used in merging and so on.
439 * In samtools merge, optionally take the input file name as RG-ID and
440 attach the RG tag to each alignment.
442 * Added FTP support in the RAZF library. RAZF-compressed reference
443 sequence can be retrieved remotely.
445 * Improved network support for Win32.
447 * Samtools sort and merge are now stable.
449 Changes in other utilities:
451 * Implemented sam2vcf.pl that converts the pileup format to the VCF
454 * This release of samtools is known to work with the latest
455 Bio-Samtools Perl module.
457 (0.1.7: 10 November 2009, r510)
461 Beta Release 0.1.6 (2 September, 2009)
462 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
466 * In tview, do not show a blank screen when no reads mapped to the
467 corresponding region.
469 * Implemented native HTTP support in the BGZF library. Samtools is now
470 able to directly open a BAM file on HTTP. HTTP proxy is also
471 supported via the "http_proxy" environmental variable.
473 * Samtools is now compitable with the MinGW (win32) compiler and the
476 * The calmd (or fillmd) command now calculates the NM tag and replaces
477 MD tags if they are wrong.
479 * The view command now recognizes and optionally prints FLAG in HEXs or
480 strings to make a SAM file more friendly to human eyes. This is a
481 samtools-C extension, not implemented in Picard for the time
482 being. Please type `samtools view -?' for more information.
484 * BAM files now have an end-of-file (EOF) marker to facilitate
485 truncation detection. A warning will be given if an on-disk BAM file
486 does not have this marker. The warning will be seen on BAM files
487 generated by an older version of samtools. It does NO harm.
489 * New key bindings in tview: `r' to show read names and `s' to show
490 reference skip (N operation) as deletions.
492 * Fixed a bug in `samtools merge -n'.
494 * Samtools merge now optionally copies the header of a user specified
495 SAM file to the resultant BAM output.
497 * Samtools pileup/tview works with a CIGAR with the first or the last
498 operation is an indel.
500 * Fixed a bug in bam_aux_get().
503 Changes in other utilies:
505 * Fixed wrong FLAG in maq2sam.
508 (0.1.6: 2 September 2009, r453)
512 Beta Release 0.1.5 (7 July, 2009)
513 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
517 * Support opening a BAM alignment on FTP. Users can now use "tview" to
518 view alignments at the NCBI ftp site. Please read manual for more
521 * In library, propagate errors rather than exit or complain assertion
524 * Simplified the building system and fixed compiling errors caused by
527 * Fixed an issue about lost header information when a SAM is imported
530 * Implemented "samtool.pl varFilter" which filters both SNPs and short
531 indels. This command replaces "indelFilter".
533 * Implemented "samtools.pl pileup2fq" to generate FASTQ consensus from
536 * In pileup, cap mapping quality at 60. This helps filtering when
537 different aligners are in use.
539 * In pileup, allow to output variant sites only.
541 * Made pileup generate correct calls in repetitive region. At the same
542 time, I am considering to implement a simplified model in SOAPsnp,
543 although this has not happened yet.
545 * In view, added '-u' option to output BAM without compression. This
546 option is preferred when the output is piped to other commands.
548 * In view, added '-l' and '-r' to get the alignments for one library or
549 read group. The "@RG" header lines are now partially parsed.
551 * Do not include command line utilities to libbam.a.
553 * Fixed memory leaks in pileup and bam_view1().
555 * Made faidx more tolerant to empty lines right before or after FASTA >
559 Changes in other utilities:
561 * Updated novo2sam.pl by Colin Hercus, the key developer of novoalign.
564 This release involves several modifications to the key code base which
565 may potentially introduce new bugs even though we have tried to minimize
566 this by testing on several examples. Please let us know if you catch
569 (0.1.5: 7 July 2009, r373)
573 Beta Release 0.1.4 (21 May, 2009)
574 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
578 * Added the 'rmdupse' command: removing duplicates for SE reads.
580 * Fixed a critical bug in the indel caller: clipped alignments are not
583 * Fixed a bug in the tview: gapped alignment may be incorrectly
586 * Unified the interface to BAM and SAM I/O. This is done by
587 implementing a wrapper on top of the old APIs and therefore old APIs
588 are still valid. The new I/O APIs also recognize the @SQ header
591 * Generate the MD tag.
593 * Generate "=" bases. However, the indel caller will not work when "="
596 * Enhanced support of color-read display (by Nils Homer).
598 * Implemented the GNU building system. However, currently the building
599 system does not generate libbam.a. We will improve this later. For
600 the time being, `make -f Makefile.generic' is preferred.
602 * Fixed a minor bug in pileup: the first read in a chromosome may be
605 * Fixed bugs in bam_aux.c. These bugs do not affect other components as
606 they were not used previously.
608 * Output the 'SM' tag from maq2sam.
610 (0.1.4: 21 May 2009, r297)
614 Beta Release 0.1.3 (15 April, 2009)
615 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
617 Notable changes in SAMtools:
619 * SAMtools is more consistent with the specification: a) '*' in the
620 QUAL field is allowed; b) the field separator is TAB only and SPACE
621 is treated as a character in a field; c) empty header is allowed.
623 * Implemented GLFv3 support in pileup.
625 * Fixed a severe bug in fixmate: strand information is wrongly
628 * Fixed a bug in alignment retrieval: alignments bridging n*16384bp are
629 not correctly retrieved sometimes.
631 * Fixed a bug in rmdup: segfault if unmapped reads are present.
633 * Move indel_filter.pl to samtools.pl and improved the filtering by
634 checking the actual number of alignments containing indels. The indel
635 pileup line is also changed a little to make this filtration easier.
637 * Fixed a minor bug in indexing: the bin number of an unmapped read is
640 * Added `flagstat' command to show statistics on the FLAG field.
642 * Improved indel caller by setting the maximum window size in local
645 Changes in other utilities:
647 * Fixed a bug in maq2sam: a tag name is obsolete.
649 * Improvement to wgsim: a) added support for SOLiD read simulation; b)
650 show the number of substitutions/indels/errors in read name; c)
651 considerable code clean up.
653 * Various converters: improved functionality in general.
655 * Updated the example SAM due to the previous bug in fixmate.
657 (0.1.3: 15 April 2009, r227)
661 Beta Release 0.1.2 (28 January, 2008)
662 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
664 Notable changes in SAMtools:
666 * Implemented a Bayesian indel caller. The new caller generate scores
667 and genotype and is potentially more accurate than Maq's indel
668 caller. The pileup format is also changed accordingly.
670 * Implemented rmdup command: remove potential PCR duplicates. Note that
671 this command ONLY works for FR orientation and requires ISIZE is
674 * Added fixmate command: fill in mate coordinates, ISIZE and mate
675 related flags from a name-sorted alignment.
677 * Fixed a bug in indexing: reads bridging 16x kbp were not retrieved.
679 * Allow to select reads shown in the pileup output with a mask.
681 * Generate GLFv2 from pileup.
683 * Added two more flags for flagging PCR/optical duplicates and for QC
686 * Fixed a bug in sort command: name sorting for large alignment did not
689 * Allow to completely disable RAZF (using Makefile.lite) as some people
690 have problem to compile it.
692 * Fixed a bug in import command when there are reads without
695 * Fixed a bug in tview: clipping broke the alignment viewer.
697 * Fixed a compiling error when _NO_CURSES is applied.
699 * Fixed a bug in merge command.
701 Changes in other utilities:
703 * Added wgsim, a paired-end reads simulator. Wgsim was adapted from
704 maq's reads simulator. Colin Hercus further improved it to allow
707 * Added wgsim_eval.pl, a script that evaluates the accuracy of
708 alignment on reads generated by wgsim.
710 * Added soap2sam.pl, a SOAP2->SAM converter. This converter does not
711 work properly when multiple hits are output.
713 * Added bowtie2sam.pl, a Bowtie->SAM converter. Only the top hit will
714 be retained when multiple hits are present.
716 * Fixed a bug in export2sam.pl for QC reads.
718 * Support RG tag at MAQ->SAM converter.
720 * Added novo2sam.pl, a NovoAlign->SAM converter. Multiple hits and
721 indel are not properly handled, though.
723 * Added zoom2sam.pl, a ZOOM->SAM converter. It only works with the
724 default Illumina output.
726 (0.1.2: 28 January 2008; r116)
730 Beta Release 0.1.1 (22 December, 2008)
731 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
733 The is the first public release of samtools. For more information,
734 please check the manual page `samtools.1' and the samtools website
735 http://samtools.sourceforge.net