make
+For cygwin users, please uncomment the 3rd and 7th line in
+'sam/Makefile' before you run 'make'.
+
+To compile EBSeq, which is included in the RSEM package, run
+
+ make ebseq
+
To install, simply put the rsem directory in your environment's PATH
variable.
+If you prefer to put all RSEM executables to a bin directory, please
+also remember to put 'rsem_perl_utils.pm' and 'WHAT_IS_NEW' to the
+same bin directory. 'rsem_perl_utils.pm' is required for most RSEM's
+perl scripts and 'WHAT_IS_NEW' contains the RSEM version information.
+
### Prerequisites
C++, Perl and R are required to be installed.
-To take advantage of RSEM's built-in support for the Bowtie alignment
-program, you must have [Bowtie](http://bowtie-bio.sourceforge.net) installed.
+To take advantage of RSEM's built-in support for the Bowtie/Bowtie 2
+alignment program, you must have
+[Bowtie](http://bowtie-bio.sourceforge.net) and/or [Bowtie
+2](http://bowtie-bio.sourceforge.net/bowtie2) installed.
## <a name="usage"></a> Usage
#### Using an alternative aligner
By default, RSEM automates the alignment of reads to reference
-transcripts using the Bowtie alignment program. To use an alternative
+transcripts using the Bowtie alignment program. Turn on '--bowtie2'
+for 'rsem-prepare-reference' and 'rsem-calculate-expression' will
+allow RSEM to use the Bowtie 2 alignment program instead. Please note
+that indel alignments, local alignments and discordant alignments are
+disallowed when RSEM uses Bowtie 2 since RSEM currently cannot handle
+them. See the description of '--bowtie2' option in
+'rsem-calculate-expression' for more details. To use an alternative
alignment program, align the input reads against the file
'reference_name.idx.fa' generated by 'rsem-prepare-reference', and
format the alignment output in SAM or BAM format. Then, instead of
generated by the samtools included. All these files are in genomic
coordinates.
-#### a) Generating a Wiggle file
+#### a) Converting transcript BAM file into genome BAM file
+
+Normally, RSEM will do this for you via '--output-genome-bam' option
+of 'rsem-calculate-expression'. However, if you have run
+'rsem-prepare-reference' and use 'reference_name.idx.fa' to build
+indices for your aligner, you can use 'rsem-tbam2gbam' to convert your
+transcript coordinate BAM alignments file into a genomic coordinate
+BAM alignments file without the need to run the whole RSEM
+pipeline. Please note that 'rsem-prepare-reference' will convert all
+'N' into 'G' by default for 'reference_name.idx.fa'. If you do not
+want this to happen, please use '--no-ntog' option.
+
+Usage:
+
+ rsem-tbam2gbam reference_name unsorted_transcript_bam_input genome_bam_output
+
+reference_name : The name of reference built by 'rsem-prepare-reference'
+unsorted_transcript_bam_input : This file should satisfy: 1) the alignments of a same read are grouped together, 2) for any paired-end alignment, the two mates should be adjacent to each other, 3) this file should not be sorted by samtools
+genome_bam_output : The output genomic coordinate BAM file's name
+
+#### b) Generating a Wiggle file
A wiggle plot representing the expected number of reads overlapping
each position in the genome/transcript set can be generated from the
sorted_bam_input : Input BAM format file, must be sorted
wig_output : Output wiggle file's name, e.g. output.wig
-wiggle_name : the name of this wiggle plot
---no-fractional-weight : If this is set, RSEM will not look for "ZW" tag and each alignment appeared in the BAM file has weight 1. Set this if your BAM file is not generated by RSEM. Please note that this option must be at the end of the command line.
+wiggle_name : The name of this wiggle plot
+--no-fractional-weight : If this is set, RSEM will not look for "ZW" tag and each alignment appeared in the BAM file has weight 1. Set this if your BAM file is not generated by RSEM. Please note that this option must be at the end of the command line
-#### b) Loading a BAM and/or Wiggle file into the UCSC Genome Browser or Integrative Genomics Viewer(IGV)
+#### c) Loading a BAM and/or Wiggle file into the UCSC Genome Browser or Integrative Genomics Viewer(IGV)
For UCSC genome browser, please refer to the [UCSC custom track help page](http://genome.ucsc.edu/goldenPath/help/customTrack.html).
1) Import the transcript sequences as a genome
-Select File -> Import Genome, then fill in ID, Name and Fasta file. Fasta file should be 'reference_name.transcripts.fa'. After that, click Save button. Suppose ID is filled as 'reference_name', a file called 'reference_name.genome' will be generated. Next time, we can use: File -> Load Genome, then select 'reference_name.genome'.
+Select File -> Import Genome, then fill in ID, Name and Fasta file. Fasta file should be 'reference_name.idx.fa'. After that, click Save button. Suppose ID is filled as 'reference_name', a file called 'reference_name.genome' will be generated. Next time, we can use: File -> Load Genome, then select 'reference_name.genome'.
2) Load visualization files
igvtools tile reference_name.transcript.wig reference_name.transcript.tdf reference_name.genome
-#### c) Generating Transcript Wiggle Plots
+#### d) Generating Transcript Wiggle Plots
To generate transcript wiggle plots, you should run the
'rsem-plot-transcript-wiggles' program. Run
to get usage information or visit the [rsem-plot-transcript-wiggles
documentation page](http://deweylab.biostat.wisc.edu/rsem/rsem-plot-transcript-wiggles.html).
-#### d) Visualize the model learned by RSEM
+#### e) Visualize the model learned by RSEM
RSEM provides an R script, 'rsem-plot-model', for visulazing the model learned.
## <a name="example"></a> Example
Suppose we download the mouse genome from UCSC Genome Browser. We
-will use a reference_name of 'mm9'. We have a FASTQ-formatted file,
+will use a reference_name of 'mouse_125'. We have a FASTQ-formatted file,
'mmliver.fq', containing single-end reads from one sample, which we
call 'mmliver_single_quals'. We want to estimate expression values by
using the single-end model with a fragment length distribution. We
The commands for this scenario are as follows:
- rsem-prepare-reference --gtf mm9.gtf --mapping knownIsoforms.txt --bowtie-path /sw/bowtie /data/mm9 /ref/mm9
- rsem-calculate-expression --bowtie-path /sw/bowtie --phred64-quals --fragment-length-mean 150.0 --fragment-length-sd 35.0 -p 8 --output-genome-bam --calc-ci --memory-allocate 1024 /data/mmliver.fq /ref/mm9 mmliver_single_quals
+ rsem-prepare-reference --gtf mm9.gtf --mapping knownIsoforms.txt --bowtie-path /sw/bowtie /data/mm9 /ref/mouse_125
+ rsem-calculate-expression --bowtie-path /sw/bowtie --phred64-quals --fragment-length-mean 150.0 --fragment-length-sd 35.0 -p 8 --output-genome-bam --calc-ci --memory-allocate 1024 /data/mmliver.fq /ref/mouse_125 mmliver_single_quals
rsem-bam2wig mmliver_single_quals.sorted.bam mmliver_single_quals.sorted.wig mmliver_single_quals
rsem-plot-transcript-wiggles --gene-list --show-unique mmliver_single_quals gene_ids.txt output.pdf
rsem-plot-model mmliver_single_quals mmliver_single_quals.models.pdf
## <a name="simulation"></a> Simulation
+RSEM provides users the 'rsem-simulate-reads' program to simulate RNA-Seq data based on parameters learned from real data sets. Run
+
+ rsem-simulate-reads
+
+to get usage information or read the following subsections.
+
### Usage:
rsem-simulate-reads reference_name estimated_model_file estimated_isoform_results theta0 N output_name [-q]
-estimated_model_file: file containing model parameters. Generated by
-rsem-calculate-expression.
-estimated_isoform_results: file containing isoform expression levels.
-Generated by rsem-calculate-expression.
-theta0: fraction of reads that are "noise" (not derived from a transcript).
-N: number of reads to simulate.
-output_name: prefix for all output files.
-[-q] : set it will stop outputting intermediate information.
+__reference_name:__ The name of RSEM references, which should be already generated by 'rsem-prepare-reference'
+
+__estimated_model_file:__ This file describes how the RNA-Seq reads will be sequenced given the expression levels. It determines what kind of reads will be simulated (single-end/paired-end, w/o quality score) and includes parameters for fragment length distribution, read start position distribution, sequencing error models, etc. Normally, this file should be learned from real data using 'rsem-calculate-expression'. The file can be found under the 'sample_name.stat' folder with the name of 'sample_name.model'. 'model_file_description.txt' provides the format and meanings of this file.
+
+__estimated_isoform_results:__ This file contains expression levels for all isoforms recorded in the reference. It can be learned using 'rsem-calculate-expression' from real data. The corresponding file users want to use is 'sample_name.isoforms.results'. If simulating from user-designed expression profile is desired, start from a learned 'sample_name.isoforms.results' file and only modify the 'TPM' column. The simulator only reads the TPM column. But keeping the file format the same is required. If the RSEM references built are aware of allele-specific transcripts, 'sample_name.alleles.results' should be used instead.
+
+__theta0:__ This parameter determines the fraction of reads that are coming from background "noise" (instead of from a transcript). It can also be estimated using 'rsem-calculate-expression' from real data. Users can find it as the first value of the third line of the file 'sample_name.stat/sample_name.theta'.
+
+__N:__ The total number of reads to be simulated. If 'rsem-calculate-expression' is executed on a real data set, the total number of reads can be found as the 4th number of the first line of the file 'sample_name.stat/sample_name.cnt'.
+
+__output_name:__ Prefix for all output files.
+
+__--seed seed:__ Set seed for the random number generator used in simulation. The seed should be a 32-bit unsigned integer.
+
+__-q:__ Set it will stop outputting intermediate information.
### Outputs:
+output_name.sim.isoforms.results, output_name.sim.genes.results: Expression levels estimated by counting where each simulated read comes from.
+output_name.sim.alleles.results: Allele-specific expression levels estimated by counting where each simulated read comes from.
+
output_name.fa if single-end without quality score;
output_name.fq if single-end with quality score;
output_name_1.fa & output_name_2.fa if paired-end without quality
score;
output_name_1.fq & output_name_2.fq if paired-end with quality score.
-output_name.sim.isoforms.results, output_name.sim.genes.results : Results estimated based on sample values.
+**Format of the header line**: Each simulated read's header line encodes where it comes from. The header line has the format:
+
+ {>/@}_rid_dir_sid_pos[_insertL]
+
+__{>/@}:__ Either '>' or '@' must appear. '>' appears if FASTA files are generated and '@' appears if FASTQ files are generated
+
+__rid:__ Simulated read's index, numbered from 0
+
+__dir:__ The direction of the simulated read. 0 refers to forward strand ('+') and 1 refers to reverse strand ('-')
+
+__sid:__ Represent which transcript this read is simulated from. It ranges between 0 and M, where M is the total number of transcripts. If sid=0, the read is simulated from the background noise. Otherwise, the read is simulated from a transcript with index sid. Transcript sid's transcript name can be found in the 'transcript_id' column of the 'sample_name.isoforms.results' file (at line sid + 1, line 1 is for column names)
+
+__pos:__ The start position of the simulated read in strand dir of transcript sid. It is numbered from 0
+
+__insertL:__ Only appear for paired-end reads. It gives the insert length of the simulated read.
+
+### Example:
+
+Suppose we want to simulate 50 millon single-end reads with quality scores and use the parameters learned from [Example](#example). In addition, we set theta0 as 0.2 and output_name as 'simulated_reads'. The command is:
+
+ rsem-simulate-reads /ref/mouse_125 mmliver_single_quals.stat/mmliver_single_quals.model mmliver_single_quals.isoforms.results 0.2 50000000 simulated_reads
## <a name="gen_trinity"></a> Generate Transcript-to-Gene-Map from Trinity Output
DESeq do not take variance due to read mapping uncertainty into
consideration. Because read mapping ambiguity is prevalent among
isoforms and de novo assembled transcripts, these tools are not ideal
-for DE detection in such conditions.
+for DE detection in such conditions.
-**EBSeq**, an empirical Bayesian DE analysis tool developed in
-UW-Madison, can take variance due to read mapping ambiguity into
-consideration by grouping isoforms with parent gene's number of
-isoforms. In addition, it is more robust to outliers. For more
-information about EBSeq (including the paper describing their method),
-please visit <a
-href="http://www.biostat.wisc.edu/~ningleng/EBSeq_Package">EBSeq
-website</a>.
+EBSeq, an empirical Bayesian DE analysis tool developed in UW-Madison,
+can take variance due to read mapping ambiguity into consideration by
+grouping isoforms with parent gene's number of isoforms. In addition,
+it is more robust to outliers. For more information about EBSeq
+(including the paper describing their method), please visit [EBSeq's
+website](http://www.biostat.wisc.edu/~ningleng/EBSeq_Package).
-RSEM includes the newest version of EBSeq in its folder
-named 'EBSeq'. To use it, first type
+
+RSEM includes EBSeq in its folder named 'EBSeq'. To use it, first type
make ebseq
NgVec <- scan(file="output_name.ngvec", what=0, sep="\n")
-. After that, replace 'IsoNgTrun' with 'NgVec' in the second line of
-section 3.2.5 (Page 10) of EBSeq's vignette:
+. After that, set "NgVector = NgVec" for your differential expression
+test (either 'EBTest' or 'EBMultiTest').
- IsoEBres=EBTest(Data=IsoMat, NgVector=NgVec, ...)
For users' convenience, RSEM also provides a script
'rsem-generate-data-matrix' to extract input matrix from expression
IsoMat <- data.matrix(read.table(file="output_name.counts.matrix"))
-before running function 'EBTest'.
-
-At last, RSEM provides a R script, 'rsem-find-DE', which run EBSeq for
-you.
+before running either 'EBTest' or 'EBMultiTest'.
-Usage:
+Lastly, RSEM provides two scripts, 'rsem-run-ebseq' and
+'rsem-control-fdr', to help users find differential expressed
+genes/transcripts. First, 'rsem-run-ebseq' calls EBSeq to calculate related statistics
+for all genes/transcripts. Run
- rsem-find-DE data_matrix_file [--ngvector ngvector_file] number_sample_condition1 FDR_rate output_file
+ rsem-run-ebseq --help
-This script calls EBSeq to find differentially expressed genes/transcripts in two conditions.
+to get usage information or visit the [rsem-run-ebseq documentation
+page](http://deweylab.biostat.wisc.edu/rsem/rsem-run-ebseq.html). Second,
+'rsem-control-fdr' takes 'rsem-run-ebseq' 's result and reports called
+differentially expressed genes/transcripts by controlling the false
+discovery rate. Run
-data_matrix_file: m by n matrix containing expected counts, m is the number of transcripts/genes, n is the number of total samples.
-[--ngvector ngvector_file]: optional field. 'ngvector_file' is calculated by 'rsem-generate-ngvector'. Having this field is recommended for transcript data.
-number_sample_condition1: the number of samples in condition 1. A condition's samples must be adjacent. The left group of samples are defined as condition 1.
-FDR_rate: false discovery rate.
-output_file: the output file.
+ rsem-control-fdr --help
-The results are written as a matrix with row and column names. The row names are the differentially expressed transcripts'/genes' ids. The column names are 'PPEE', 'PPDE', 'PostFC' and 'RealFC'.
+to get usage information or visit the [rsem-control-fdr documentation
+page](http://deweylab.biostat.wisc.edu/rsem/rsem-control-fdr.html). These
+two scripts can perform DE analysis on either 2 conditions or multiple
+conditions.
-PPEE: posterior probability of being equally expressed.
-PPDE: posterior probability of being differentially expressed.
-PostFC: posterior fold change (condition 1 over condition2).
-RealFC: real fold change (condition 1 over condition2).
-
-To get the above usage information, type
-
- rsem-find-DE
-
-Note: any wrong parameter setting will lead 'rsem-find-DE' to output
-usage information and halt.
+Please note that 'rsem-run-ebseq' and 'rsem-control-fdr' use EBSeq's
+default parameters. For advanced use of EBSeq or information about how
+EBSeq works, please refer to [EBSeq's
+manual](http://www.bioconductor.org/packages/devel/bioc/vignettes/EBSeq/inst/doc/EBSeq_Vignette.pdf).
Questions related to EBSeq should
be sent to <a href="mailto:nleng@wisc.edu">Ning Leng</a>.
## <a name="authors"></a> Authors
-RSEM is developed by Bo Li, with substaintial technical input from Colin Dewey.
+The RSEM algorithm is developed by Bo Li and Colin Dewey. The RSEM software is mainly implemented by Bo Li.
## <a name="acknowledgements"></a> Acknowledgements
[EBSeq](http://www.biostat.wisc.edu/~ningleng/EBSeq_Package/) for
differential expression analysis.
-We thank earonesty for contributing patches.
+We thank earonesty and Dr. Samuel Arvidsson for contributing patches.
+
+We thank Han Lin, j.miller, Joël Fillon, Dr. Samuel G. Younkin and Malcolm Cook for suggesting possible fixes.
## <a name="license"></a> License
projects / rsem.git / blobdiff