4 [Bo Li](http://pages.cs.wisc.edu/~bli) \(bli at cs dot wisc dot edu\)
11 * [Introduction](#introduction)
12 * [Compilation & Installation](#compilation)
15 * [Simulation](#simulation)
16 * [Generate Transcript-to-Gene-Map from Trinity Output](#gen_trinity)
17 * [Differential Expression Analysis](#de)
19 * [Acknowledgements](#acknowledgements)
24 ## <a name="introduction"></a> Introduction
26 RSEM is a software package for estimating gene and isoform expression
27 levels from RNA-Seq data. The RSEM package provides an user-friendly
28 interface, supports threads for parallel computation of the EM
29 algorithm, single-end and paired-end read data, quality scores,
30 variable-length reads and RSPD estimation. In addition, it provides
31 posterior mean and 95% credibility interval estimates for expression
32 levels. For visualization, It can generate BAM and Wiggle files in
33 both transcript-coordinate and genomic-coordinate. Genomic-coordinate
34 files can be visualized by both UCSC Genome browser and Broad
35 Institute's Integrative Genomics Viewer (IGV). Transcript-coordinate
36 files can be visualized by IGV. RSEM also has its own scripts to
37 generate transcript read depth plots in pdf format. The unique feature
38 of RSEM is, the read depth plots can be stacked, with read depth
39 contributed to unique reads shown in black and contributed to
40 multi-reads shown in red. In addition, models learned from data can
41 also be visualized. Last but not least, RSEM contains a simulator.
43 ## <a name="compilation"></a> Compilation & Installation
45 To compile RSEM, simply run
49 To install, simply put the rsem directory in your environment's PATH
54 C++, Perl and R are required to be installed.
56 To take advantage of RSEM's built-in support for the Bowtie alignment
57 program, you must have [Bowtie](http://bowtie-bio.sourceforge.net) installed.
59 ## <a name="usage"></a> Usage
61 ### I. Preparing Reference Sequences
63 RSEM can extract reference transcripts from a genome if you provide it
64 with gene annotations in a GTF file. Alternatively, you can provide
65 RSEM with transcript sequences directly.
67 Please note that GTF files generated from the UCSC Table Browser do not
68 contain isoform-gene relationship information. However, if you use the
69 UCSC Genes annotation track, this information can be recovered by
70 downloading the knownIsoforms.txt file for the appropriate genome.
72 To prepare the reference sequences, you should run the
73 'rsem-prepare-reference' program. Run
75 rsem-prepare-reference --help
77 to get usage information or visit the [rsem-prepare-reference
78 documentation page](http://deweylab.biostat.wisc.edu/rsem/rsem-prepare-reference.html).
80 ### II. Calculating Expression Values
82 To calculate expression values, you should run the
83 'rsem-calculate-expression' program. Run
85 rsem-calculate-expression --help
87 to get usage information or visit the [rsem-calculate-expression
88 documentation page](http://deweylab.biostat.wisc.edu/rsem/rsem-calculate-expression.html).
90 #### Calculating expression values from single-end data
92 For single-end models, users have the option of providing a fragment
93 length distribution via the '--fragment-length-mean' and
94 '--fragment-length-sd' options. The specification of an accurate fragment
95 length distribution is important for the accuracy of expression level
96 estimates from single-end data. If the fragment length mean and sd are
97 not provided, RSEM will not take a fragment length distribution into
100 #### Using an alternative aligner
102 By default, RSEM automates the alignment of reads to reference
103 transcripts using the Bowtie alignment program. To use an alternative
104 alignment program, align the input reads against the file
105 'reference_name.idx.fa' generated by 'rsem-prepare-reference', and
106 format the alignment output in SAM or BAM format. Then, instead of
107 providing reads to 'rsem-calculate-expression', specify the '--sam' or
108 '--bam' option and provide the SAM or BAM file as an argument. When
109 using an alternative aligner, you may also want to provide the
110 '--no-bowtie' option to 'rsem-prepare-reference' so that the Bowtie
111 indices are not built.
113 RSEM requires all alignments of the same read group together. For
114 paired-end reads, RSEM also requires the two mates of any alignment be
115 adjacent. To check if your SAM/BAM file satisfy the requirements,
118 rsem-sam-validator <input.sam/input.bam>
120 If your file does not satisfy the requirements, you can use
121 'convert-sam-for-rsem' to convert it into a BAM file which RSEM can
124 convert-sam-for-rsem --help
126 to get usage information or visit the [convert-sam-for-rsem
128 page](http://deweylab.biostat.wisc.edu/rsem/convert-sam-for-rsem.html).
130 However, please note that RSEM does ** not ** support gapped
131 alignments. So make sure that your aligner does not produce alignments
132 with intersions/deletions. Also, please make sure that you use
133 'reference_name.idx.fa' , which is generated by RSEM, to build your
136 ### III. Visualization
138 RSEM contains a version of samtools in the 'sam' subdirectory. RSEM
139 will always produce three files:'sample_name.transcript.bam', the
140 unsorted BAM file, 'sample_name.transcript.sorted.bam' and
141 'sample_name.transcript.sorted.bam.bai' the sorted BAM file and
142 indices generated by the samtools included. All three files are in
143 transcript coordinates. When users specify the --output-genome-bam
144 option RSEM will produce three files: 'sample_name.genome.bam', the
145 unsorted BAM file, 'sample_name.genome.sorted.bam' and
146 'sample_name.genome.sorted.bam.bai' the sorted BAM file and indices
147 generated by the samtools included. All these files are in genomic
150 #### a) Converting transcript BAM file into genome BAM file
152 Normally, RSEM will do this for you via '--output-genome-bam' option
153 of 'rsem-calculate-expression'. However, if you have run
154 'rsem-prepare-reference' and use 'reference_name.idx.fa' to build
155 indices for your aligner, you can use 'rsem-tbam2gbam' to convert your
156 transcript coordinate BAM alignments file into a genomic coordinate
157 BAM alignments file without the need to run the whole RSEM
158 pipeline. Please note that 'rsem-prepare-reference' will convert all
159 'N' into 'G' by default for 'reference_name.idx.fa'. If you do not
160 want this to happen, please use '--no-ntog' option.
164 rsem-tbam2gbam reference_name unsorted_transcript_bam_input genome_bam_output
166 reference_name : The name of reference built by 'rsem-prepare-reference'
167 unsorted_transcript_bam_input : This file should satisfy: 1) the alignments of a same read are grouped together, 2) for any paired-end alignment, the two mates should be adjacent to each other, 3) this file should not be sorted by samtools
168 genome_bam_output : The output genomic coordinate BAM file's name
170 #### b) Generating a Wiggle file
172 A wiggle plot representing the expected number of reads overlapping
173 each position in the genome/transcript set can be generated from the
174 sorted genome/transcript BAM file output. To generate the wiggle
175 plot, run the 'rsem-bam2wig' program on the
176 'sample_name.genome.sorted.bam'/'sample_name.transcript.sorted.bam' file.
180 rsem-bam2wig sorted_bam_input wig_output wiggle_name [--no-fractional-weight]
182 sorted_bam_input : Input BAM format file, must be sorted
183 wig_output : Output wiggle file's name, e.g. output.wig
184 wiggle_name : the name of this wiggle plot
185 --no-fractional-weight : If this is set, RSEM will not look for "ZW" tag and each alignment appeared in the BAM file has weight 1. Set this if your BAM file is not generated by RSEM. Please note that this option must be at the end of the command line
187 #### c) Loading a BAM and/or Wiggle file into the UCSC Genome Browser or Integrative Genomics Viewer(IGV)
189 For UCSC genome browser, please refer to the [UCSC custom track help page](http://genome.ucsc.edu/goldenPath/help/customTrack.html).
191 For integrative genomics viewer, please refer to the [IGV home page](http://www.broadinstitute.org/software/igv/home). Note: Although IGV can generate read depth plot from the BAM file given, it cannot recognize "ZW" tag RSEM puts. Therefore IGV counts each alignment as weight 1 instead of the expected weight for the plot it generates. So we recommend to use the wiggle file generated by RSEM for read depth visualization.
193 Here are some guidance for visualizing transcript coordinate files using IGV:
195 1) Import the transcript sequences as a genome
197 Select File -> Import Genome, then fill in ID, Name and Fasta file. Fasta file should be 'reference_name.transcripts.fa'. After that, click Save button. Suppose ID is filled as 'reference_name', a file called 'reference_name.genome' will be generated. Next time, we can use: File -> Load Genome, then select 'reference_name.genome'.
199 2) Load visualization files
201 Select File -> Load from File, then choose one transcript coordinate visualization file generated by RSEM. IGV might require you to convert wiggle file to tdf file. You should use igvtools to perform this task. One way to perform the conversion is to use the following command:
203 igvtools tile reference_name.transcript.wig reference_name.transcript.tdf reference_name.genome
205 #### d) Generating Transcript Wiggle Plots
207 To generate transcript wiggle plots, you should run the
208 'rsem-plot-transcript-wiggles' program. Run
210 rsem-plot-transcript-wiggles --help
212 to get usage information or visit the [rsem-plot-transcript-wiggles
213 documentation page](http://deweylab.biostat.wisc.edu/rsem/rsem-plot-transcript-wiggles.html).
215 #### e) Visualize the model learned by RSEM
217 RSEM provides an R script, 'rsem-plot-model', for visulazing the model learned.
221 rsem-plot-model sample_name output_plot_file
223 sample_name: the name of the sample analyzed
224 output_plot_file: the file name for plots generated from the model. It is a pdf file
226 The plots generated depends on read type and user configuration. It
227 may include fragment length distribution, mate length distribution,
228 read start position distribution (RSPD), quality score vs observed
229 quality given a reference base, position vs percentage of sequencing
230 error given a reference base and histogram of reads with different
231 number of alignments.
233 fragment length distribution and mate length distribution: x-axis is fragment/mate length, y axis is the probability of generating a fragment/mate with the associated length
235 RSPD: Read Start Position Distribution. x-axis is bin number, y-axis is the probability of each bin. RSPD can be used as an indicator of 3' bias
237 Quality score vs. observed quality given a reference base: x-axis is Phred quality scores associated with data, y-axis is the "observed quality", Phred quality scores learned by RSEM from the data. Q = -10log_10(P), where Q is Phred quality score and P is the probability of sequencing error for a particular base
239 Position vs. percentage sequencing error given a reference base: x-axis is position and y-axis is percentage sequencing error
241 Histogram of reads with different number of alignments: x-axis is the number of alignments a read has and y-axis is the number of such reads. The inf in x-axis means number of reads filtered due to too many alignments
243 ## <a name="example"></a> Example
245 Suppose we download the mouse genome from UCSC Genome Browser. We
246 will use a reference_name of 'mm9'. We have a FASTQ-formatted file,
247 'mmliver.fq', containing single-end reads from one sample, which we
248 call 'mmliver_single_quals'. We want to estimate expression values by
249 using the single-end model with a fragment length distribution. We
250 know that the fragment length distribution is approximated by a normal
251 distribution with a mean of 150 and a standard deviation of 35. We
252 wish to generate 95% credibility intervals in addition to maximum
253 likelihood estimates. RSEM will be allowed 1G of memory for the
254 credibility interval calculation. We will visualize the probabilistic
255 read mappings generated by RSEM on UCSC genome browser. We will
256 generate a list of genes' transcript wiggle plots in 'output.pdf'. The
257 list is 'gene_ids.txt'. We will visualize the models learned in
258 'mmliver_single_quals.models.pdf'
260 The commands for this scenario are as follows:
262 rsem-prepare-reference --gtf mm9.gtf --mapping knownIsoforms.txt --bowtie-path /sw/bowtie /data/mm9 /ref/mm9
263 rsem-calculate-expression --bowtie-path /sw/bowtie --phred64-quals --fragment-length-mean 150.0 --fragment-length-sd 35.0 -p 8 --output-genome-bam --calc-ci --memory-allocate 1024 /data/mmliver.fq /ref/mm9 mmliver_single_quals
264 rsem-bam2wig mmliver_single_quals.sorted.bam mmliver_single_quals.sorted.wig mmliver_single_quals
265 rsem-plot-transcript-wiggles --gene-list --show-unique mmliver_single_quals gene_ids.txt output.pdf
266 rsem-plot-model mmliver_single_quals mmliver_single_quals.models.pdf
268 ## <a name="simulation"></a> Simulation
272 rsem-simulate-reads reference_name estimated_model_file estimated_isoform_results theta0 N output_name [-q]
274 estimated_model_file: file containing model parameters. Generated by
275 rsem-calculate-expression.
276 estimated_isoform_results: file containing isoform expression levels.
277 Generated by rsem-calculate-expression.
278 theta0: fraction of reads that are "noise" (not derived from a transcript).
279 N: number of reads to simulate.
280 output_name: prefix for all output files.
281 [-q] : set it will stop outputting intermediate information.
285 output_name.fa if single-end without quality score;
286 output_name.fq if single-end with quality score;
287 output_name_1.fa & output_name_2.fa if paired-end without quality
289 output_name_1.fq & output_name_2.fq if paired-end with quality score.
291 output_name.sim.isoforms.results, output_name.sim.genes.results : Results estimated based on sample values.
293 ## <a name="gen_trinity"></a> Generate Transcript-to-Gene-Map from Trinity Output
295 For Trinity users, RSEM provides a perl script to generate transcript-to-gene-map file from the fasta file produced by Trinity.
299 extract-transcript-to-gene-map-from-trinity trinity_fasta_file map_file
301 trinity_fasta_file: the fasta file produced by trinity, which contains all transcripts assembled.
302 map_file: transcript-to-gene-map file's name.
304 ## <a name="de"></a> Differential Expression Analysis
306 Popular differential expression (DE) analysis tools such as edgeR and
307 DESeq do not take variance due to read mapping uncertainty into
308 consideration. Because read mapping ambiguity is prevalent among
309 isoforms and de novo assembled transcripts, these tools are not ideal
310 for DE detection in such conditions.
312 EBSeq, an empirical Bayesian DE analysis tool developed in UW-Madison,
313 can take variance due to read mapping ambiguity into consideration by
314 grouping isoforms with parent gene's number of isoforms. In addition,
315 it is more robust to outliers. For more information about EBSeq
316 (including the paper describing their method), please visit [EBSeq's
317 website](http://www.biostat.wisc.edu/~ningleng/EBSeq_Package).
320 RSEM includes EBSeq in its folder named 'EBSeq'. To use it, first type
324 to compile the EBSeq related codes.
326 EBSeq requires gene-isoform relationship for its isoform DE
327 detection. However, for de novo assembled transcriptome, it is hard to
328 obtain an accurate gene-isoform relationship. Instead, RSEM provides a
329 script 'rsem-generate-ngvector', which clusters transcripts based on
330 measures directly relating to read mappaing ambiguity. First, it
331 calcualtes the 'unmappability' of each transcript. The 'unmappability'
332 of a transcript is the ratio between the number of k mers with at
333 least one perfect match to other transcripts and the total number of k
334 mers of this transcript, where k is a parameter. Then, Ng vector is
335 generated by applying Kmeans algorithm to the 'unmappability' values
336 with number of clusters set as 3. This program will make sure the mean
337 'unmappability' scores for clusters are in ascending order. All
338 transcripts whose lengths are less than k are assigned to cluster
341 rsem-generate-ngvector --help
343 to get usage information or visit the [rsem-generate-ngvector
345 page](http://deweylab.biostat.wisc.edu/rsem/rsem-generate-ngvector.html).
347 If your reference is a de novo assembled transcript set, you should
348 run 'rsem-generate-ngvector' first. Then load the resulting
349 'output_name.ngvec' into R. For example, you can use
351 NgVec <- scan(file="output_name.ngvec", what=0, sep="\n")
353 . After that, set "NgVector = NgVec" for your differential expression
354 test (either 'EBTest' or 'EBMultiTest').
357 For users' convenience, RSEM also provides a script
358 'rsem-generate-data-matrix' to extract input matrix from expression
361 rsem-generate-data-matrix sampleA.[genes/isoforms].results sampleB.[genes/isoforms].results ... > output_name.counts.matrix
363 The results files are required to be either all gene level results or
364 all isoform level results. You can load the matrix into R by
366 IsoMat <- data.matrix(read.table(file="output_name.counts.matrix"))
368 before running either 'EBTest' or 'EBMultiTest'.
370 Lastly, RSEM provides two scripts, 'rsem-run-ebseq' and
371 'rsem-control-fdr', to help users find differential expressed
372 genes/transcripts. First, 'rsem-run-ebseq' calls EBSeq to calculate related statistics
373 for all genes/transcripts. Run
375 rsem-run-ebseq --help
377 to get usage information or visit the [rsem-run-ebseq documentation
378 page](http://deweylab.biostat.wisc.edu/rsem/rsem-run-ebseq.html). Second,
379 'rsem-control-fdr' takes 'rsem-run-ebseq' 's result and reports called
380 differentially expressed genes/transcripts by controlling the false
383 rsem-control-fdr --help
385 to get usage information or visit the [rsem-control-fdr documentation
386 page](http://deweylab.biostat.wisc.edu/rsem/rsem-control-fdr.html). These
387 two scripts can perform DE analysis on either 2 conditions or multiple
390 Please note that 'rsem-run-ebseq' and 'rsem-control-fdr' use EBSeq's
391 default parameters. For advanced use of EBSeq or information about how
392 EBSeq works, please refer to [EBSeq's
393 manual](http://www.bioconductor.org/packages/devel/bioc/vignettes/EBSeq/inst/doc/EBSeq_Vignette.pdf).
395 Questions related to EBSeq should
396 be sent to <a href="mailto:nleng@wisc.edu">Ning Leng</a>.
398 ## <a name="authors"></a> Authors
400 RSEM is developed by Bo Li, with substaintial technical input from Colin Dewey.
402 ## <a name="acknowledgements"></a> Acknowledgements
404 RSEM uses the [Boost C++](http://www.boost.org) and
405 [samtools](http://samtools.sourceforge.net) libraries. RSEM includes
406 [EBSeq](http://www.biostat.wisc.edu/~ningleng/EBSeq_Package/) for
407 differential expression analysis.
409 We thank earonesty for contributing patches.
411 We thank Han Lin for suggesting possible fixes.
413 ## <a name="license"></a> License
415 RSEM is licensed under the [GNU General Public License
416 v3](http://www.gnu.org/licenses/gpl-3.0.html).