################################################################################## #Andy Rampersaud, 03.21.16 #This Pipeline_Version_History.txt file is used to record different versions of the pipeline #Continuous pipeline updates either major or minor should be recorded #The pipeline version numbers start with v2.0 #Version numbers follow the nomenclature decribed here: #https://en.wikipedia.org/wiki/Software_versioning #major.minor[.build[.revision]] ################################################################################## #--------------------------------------------------------------------------------- Version: v2.0 Updates: Initial pipeline release Previous versions of the pipeline were available and under development #--------------------------------------------------------------------------------- Version: v2.0.1 Updates: Moved Sample_Labels.txt to 00_Setup_Pipeline folder Sample_Labels.txt now has a color column (no need for Sample_Labels_Color.txt) README file updates: 1. Explain file organization 2. Instructions for copy commands without typing password every time 3. Instructions for using tmux terminal 02_Review_Pipeline_Parameters.sh updates: 1. Added if statements for checking if VM folders exists, if not then create them 2. Added print statement to indicate pipeline version number #--------------------------------------------------------------------------------- Version: v2.0.2 Updates: 01_Pipeline_Setup.sh: 1. Added/updated variable descriptions #--------------------------------------------------------------------------------- Version: v2.0.3 Updates: 02_Review_Pipeline_Parameters.sh: 1. Ran the 02_Review_Pipeline_Parameters.sh script to show the pipeline version in the *_Pipeline_Parameters.txt file #--------------------------------------------------------------------------------- Version: v2.0.4 Updates: 01_Pipeline_Setup.sh: 1. Increased job time limit to: TIME_LIMIT="96:00:00". Some TopHat mapping jobs need longer than the default 12-hour time limit to complete. #--------------------------------------------------------------------------------- Version: v2.0.5 Updates: 02_FASTQC -> Summarize_Jobs.sh: 1. Added URL link after the script completes (the waxmanlabvm stores our FASTQC reports) #--------------------------------------------------------------------------------- Version: v2.0.6 Updates: Generate_Tracks -> Generate_Tracks.sh: 1. Changed "${BU_User}'/Lab_Files/" to be my directory location (aramp10/Lab_Files) #--------------------------------------------------------------------------------- v2.0.7 Updates: TopHat_Paired_End.qsub -> TopHat_Paired_End.qsub: 1. Update TopHat version being used to be v2.1.1 (most current on the SCC) 2. Updated the TopHat command help message in the script (see full list of options) 3. Hard-coded the (--no-coverage-search) option in the TopHat command I'll want to eventually add another pipeline variable to give the user the option to turn on/off the (coverage-search). We decided by default it should be turned off (reduce job run-time). #--------------------------------------------------------------------------------- v2.1.0 Updates: Tisha added steps for lincRNA analysis List of changes Tisha made: 1. I copied Andy's GTF and added lncRNA GTFS to a new GTF directory /unprotected/projects/waxmanlab/routines/GTF_Files 2. I modified the Run_Jobs.sh and Summarize_Jobs.sh in HTSeq counting step to ignore lncRNA GTFs 3. modified these files to allow lncRNA GTF to be counted Folder: *b_Extract_Counts_featureCounts Run_jobs.sh Extract_Counts.qsub Summarize_Jobs.sh 4. Added lncRNA folder to add a step for lncRNA differential expression folder name: *DiffExp_1_lncRNA_featureCounts 5. Added notification in the README files and Run_Jobs.sh of *DiffExp_1_lncRNA_featureCounts that lncRNA differential expression only make sense for liver samples Also in the README files, I also added the following notes for lncRNA #--------------------------------------------------------------------------------- 1. Since segex limits the gene name to 18 characters, the *SEGEXUpload* files will have a shortened version of lncRNA name compared to the DiffExp_v2* files. For example full lncRNA name: ncRNA_inter_chrXX_XXXXX, becomes nc_inter_cXX_XXXXX (so ncRNA becomes nc and chr becomes c) 2. Here's a ucsc track for lncRNA gene models http://waxmanlabvm.bu.edu/~tisha/gtf/ucsc_ncRNA_output_filtered_final_gene_putativeStrand.gtf #--------------------------------------------------------------------------------- #--------------------------------------------------------------------------------- v2.1.1 Updates: Generate_Tracks -> Generate_Tracks.sh: 1. Added Tisha's lncRNA gene models GTF file (trackline) to the UCSC track line files This way when users want to visualize DE lncRNAs they can see the lncRNA gene model track #--------------------------------------------------------------------------------- v2.1.2 Updates: Additional instructions for DiffExp folders: README_*.txt 01_Pipeline_Setup.sh #--------------------------------------------------------------------------------- Version: v2.1.3 Updates: 1. Clean template pipeline folders: The lab directory: /restricted/projectnb/waxmanlab/routines Now has pipeline folders without previous job log files and summary folders Users will only see job log files and summary folders from their data #--------------------------------------------------------------------------------- Version: v2.1.4 Updates: 09*_DiffExp_* jobs: Tisha updated the differentialAnalysis.R script for RefSeq and LincRNA analysis The number of reads per sample is now included in the normalization process Note the addition of the "numMappedReadsList" variable in differentialAnalysis.R #--------------------------------------------------------------------------------- Version: v2.1.5 Updates: 1. *_Generate_Tracks job: Updated chromatin state tracks (updated files provided by Gracia) Updated tracks reflect a color change in the chromatin state maps BigBed_Name_Color.txt: Change "segments_colored" to "segments_recolored" Also added the "chrom_state_legend.pdf" in the *_Generate_Tracks folder #--------------------------------------------------------------------------------- Version: v2.2.0 Updates: 00_Setup_Pipeline: 1. 01_Pipeline_Setup.sh: VM_DIR_FASTQC: remove "public_html" from the directory path We are now using the waxman-server mount point on the SCC: /net/waxman-server/mnt/data/waxmanlabvm_home/ Move VM_DIR_FASTQC variable to bottom of 01_Pipeline_Setup.sh VM_DIR_public_html: rename variable to VM_DIR_UCSC, remove "public_html" (We are now using the waxman-server mount point on the SCC) 2. 02_Review_Pipeline_Parameters.sh: Check if VM_DIR_FASTQC exists: remove ssh, only check the location Check if VM_DIR_UCSC exists: remove ssh, only check the location 02_FASTQC: 1. Summarize_Jobs.sh: change scp command to copy command, change echo statement (no longer using the tilde for file/folder paths) 11_UCSC_BigWig: 1. Summarize_Jobs.sh: change scp command to copy command 12_Generate_Tracks: 1. Generate_Tracks.sh: remove tilde, use full file/folder paths #--------------------------------------------------------------------------------- Version: v2.3.0 Updates: 11_UCSC_BigWig: VM_DIR_public_html: rename variable to VM_DIR_UCSC, remove "public_html" Modified files: Run_Jobs.sh normbigWig.qsub Confirmed that the Summarize_Jobs.sh has VM_DIR_UCSC Recheck the variable names for 'VM_DIR_public_html': Only match: Pipeline_Version_History.txt (this is correct) New steps (contributed by Kritika Karri): 13a_Pearson_Correlation_HTSeq 13b_Pearson_Correlation_featureCounts I have added a new module in the RNAseq pipeline to assure the quality of the sample replicates by calculating the Pearson correlation values within each sample. The quality can be tested at two levels: a) By calculating the Pearson values for all the genes b) By filtering out genes based on the mean RPKM (>1) and calculate the correlation. See the README document in the step-specific folder for more details #--------------------------------------------------------------------------------- ##################################################################################