----------------------- Need shift 5 command (have more than 9 arguments): ----------------------- ----------------------- Start of variable list: ----------------------- SCRIPT_DIR: /restricted/projectnb/waxmanlab/kkarri/samples/fastqfiles/Water_Saline/Scripts/09a_DiffExp_3_HTSeq Dataset_DIR: /restricted/projectnb/waxmanlab/kkarri/samples/fastqfiles/Water_Saline Dataset_Label: Water_Saline GTF_Files_DIR: /restricted/projectnb/waxmanlab/kkarri/GTF_Files1 ANNOTATION_FILE: RefSeq_GeneBody.gtf CONDITION_1_NAME: Co CONDITION_2_NAME: PHENO Lengths_DIR: /restricted/projectnb/waxmanlab/kkarri/GTF_Files1/lengths GENE_LENGTHS_FILE: Exon_Regions_Lengths.txt COUNT_DIR: RefSeq_Exon_GTF OUTPUT_PREFIX: DiffExp_v2_GeneBody DiffExp_Index: DiffExp_3a COL_SUFFIX: GeneBody COUNT_PROGRAM: HTSeq ----------------------- End of variable list ----------------------- ========================================================== Starting on : Fri Jun 23 15:49:48 EDT 2017 Running on node : scc-kb6 Current directory : /restricted/projectnb/waxmanlab/kkarri/samples/fastqfiles/Water_Saline/Scripts/09a_DiffExp_3_HTSeq Current job ID : 9534915 Current job name : Step_09a_DiffExp_3a Task index number : undefined Parameter for multiple cores : 1 ========================================================== Change dir to scratch directory Print scratch directory location: /scratch/9534915.1.linga Loading required modules... ------------------------------------------ Sample_DIR: C_M1 Sample_ID: C_M1 Description: C_M1 M_Num: M1 Copy Condition_1 sample count files to Condition_1 folder calculate mapped reads: C_M1 M_Num_Cond1_List: M1 ------------------------------------------ ------------------------------------------ Sample_DIR: PB_M1 Sample_ID: PB_M1 Description: PB_M1 M_Num: M1 Copy Condition_2 sample count files to Condition_2 folder calculate mapped reads: PB_M1 Sample_DIR: PB_M2 Sample_ID: PB_M2 Description: PB_M2 M_Num: M2 Copy Condition_2 sample count files to Condition_2 folder calculate mapped reads: PB_M2 Sample_DIR: PB_M3 Sample_ID: PB_M3 Description: PB_M3 M_Num: M3 Copy Condition_2 sample count files to Condition_2 folder calculate mapped reads: PB_M3 M_Num_Cond2_List: M1M2M3 ------------------------------------------ ========================================================== Number of replicates in each condition: NUM_REP_CONDITION1: 1 NUM_REP_CONDITION1: 1 WARNING: Either your Condition_1 or Condition_2 has only 1 sample (no replicates) The differential expression analysis is designed for cases where there are at least 2 replicates/biological condition. The resulting differential expression analysis will be converted to a stringent version due to absence of replicate samples. Samples from this comparison will be treated as "replicates" to estimate the gene level variance in edgeR and, separately in, DESeq2. Expect very few to zero DE genes from DESeq2, EdgeR should yield a DE gene count. Interpreting the results: use the edgeR fold change or DESeq ratio (not DESeq fold change) Note regarding the SEGEX upload file(s): Since DESeq2 fold changes are not the same as the ratio of the normalized counts between the conditions, so the output of the segex upload file will be different between "DESeq ratio" and "DESeq foldChange". The former is truly the ratio between the normalized counts between the two conditions (plus pseudocounts) whereas the latter is the fold change estimated by DESeq. ========================================================== Renaming input count files Counting program: HTSeq Removing last 5 lines (remove special counters) Counting program: HTSeq Removing last 5 lines (remove special counters) Counting program: HTSeq Removing last 5 lines (remove special counters) Counting program: HTSeq Removing last 5 lines (remove special counters) number of mapped reads 103979138 29008801 27802491 34434534 ========================================================== List files in Input -rw-r--r-- 1 kkarri montilab-p 175377 Jun 23 15:49 /scratch/9534915.1.linga/Input/Co0.out -rw-r--r-- 1 kkarri montilab-p 10 Jun 23 15:49 /scratch/9534915.1.linga/Input/Co0_num_mapped_reads.txt -rwx------ 1 kkarri montilab-p 197610 Jun 23 15:49 /scratch/9534915.1.linga/Input/Exon_Regions_Lengths.txt -rw-r--r-- 1 kkarri montilab-p 168607 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO0.out -rw-r--r-- 1 kkarri montilab-p 9 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO0_num_mapped_reads.txt -rw-r--r-- 1 kkarri montilab-p 168577 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO1.out -rw-r--r-- 1 kkarri montilab-p 9 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO1_num_mapped_reads.txt -rw-r--r-- 1 kkarri montilab-p 169406 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO2.out -rw-r--r-- 1 kkarri montilab-p 9 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO2_num_mapped_reads.txt /scratch/9534915.1.linga/Input/Co: total 8 drwxr-xr-x 2 kkarri montilab-p 4096 Jun 23 15:49 . drwxr-xr-x 4 kkarri montilab-p 4096 Jun 23 15:49 .. /scratch/9534915.1.linga/Input/PHENO: total 8 drwxr-xr-x 2 kkarri montilab-p 4096 Jun 23 15:49 . drwxr-xr-x 4 kkarri montilab-p 4096 Jun 23 15:49 .. ========================================================== Starting to run my commands Printing Rscript command: Rscript differentialAnalysis.R Co PHENO 1 3 RefSeq_GeneBody.gtf /scratch/9534915.1.linga/Input DiffExp_v2_GeneBody Exon_Regions_Lengths.txt [1] "Arguments for differentialAnalysisDESeq.R:" [1] "Co" [1] "PHENO" [1] 1 [1] 3 [1] "RefSeq_GeneBody.gtf" [1] "/scratch/9534915.1.linga/Input" [1] "DiffExp_v2_GeneBody" [1] "Exon_Regions_Lengths.txt" load GTF file ... parse attributes ... [1] "output file is in: /scratch/9534915.1.linga/Input/DiffExp_v2_GeneBody_Co_PHENO.txt" ========================================================== Create SEGEX formatted file: Printing Rscript command: Rscript formatForSegex_ver3.R DiffExp_v2_GeneBody_Co_PHENO.txt PHENO_Water_Saline_M1M2M3_vs_Co_Water_Saline_M1_DiffExp_v2_GeneBody_forSEGEXUpload 1 GeneBody ========================================================== Comparison_Info: PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1 #---------------------------------------------------------------------------------- Running Diff_Genes.R #---------------------------------------------------------------------------------- Printing Rscript command: Rscript Diff_Genes.R PHENO_Water_Saline_M1M2M3_vs_Co_Water_Saline_M1_DiffExp_v2_GeneBody_forSEGEXUpload_DESeq.txt 2 0.05 DESeq GeneBody'_'PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1 [1] "Print arguments:" [1] "-----------------" [1] "Differential expression_Output:" [1] "PHENO_Water_Saline_M1M2M3_vs_Co_Water_Saline_M1_DiffExp_v2_GeneBody_forSEGEXUpload_DESeq.txt" [1] "fold_change_cutoff:" [1] "2" [1] "padj_cutoff:" [1] "0.05" [1] "postfix:" [1] "DESeq" [1] "count_method:" [1] "GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1" [1] "-----------------" [1] "Differential Gene Counts" [1] "Number of significant differential genes with positive fold change (Up Genes):" [1] 26 7 [1] "Number of significant differential genes with negative fold change (Down Genes):" [1] 106 7 [1] "Check out Up_Genes_DESeq_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt!" [1] "Check out Down_Genes_DESeq_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt!" Rscript Diff_Genes.R PHENO_Water_Saline_M1M2M3_vs_Co_Water_Saline_M1_DiffExp_v2_GeneBody_forSEGEXUpload_EdgeR.txt 2 0.05 EdgeR GeneBody'_'PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1 [1] "Print arguments:" [1] "-----------------" [1] "Differential expression_Output:" [1] "PHENO_Water_Saline_M1M2M3_vs_Co_Water_Saline_M1_DiffExp_v2_GeneBody_forSEGEXUpload_EdgeR.txt" [1] "fold_change_cutoff:" [1] "2" [1] "padj_cutoff:" [1] "0.05" [1] "postfix:" [1] "EdgeR" [1] "count_method:" [1] "GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1" [1] "-----------------" [1] "Differential Gene Counts" [1] "Number of significant differential genes with positive fold change (Up Genes):" [1] 71 7 [1] "Number of significant differential genes with negative fold change (Down Genes):" [1] 342 7 [1] "Check out Up_Genes_EdgeR_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt!" [1] "Check out Down_Genes_EdgeR_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt!" #---------------------------------------------------------------------------------- Running Venn_Diff_Genes.R #---------------------------------------------------------------------------------- Printing Rscript command: #---------------------------------------------------------------------------------- Rscript Venn_Diff_Genes.R Down_Genes_DESeq_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt Down_Genes_EdgeR_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt GeneBody_Counting DiffExp_3a [1] "Print arguments:" [1] "-----------------" [1] "File1:" [1] "Down_Genes_DESeq_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt" [1] "File2:" [1] "Down_Genes_EdgeR_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt" [1] "Subtitle:" [1] "GeneBody_Counting" [1] "DiffExp_Index:" [1] "DiffExp_3a" [1] "-----------------" [1] "Down.DESeq.GeneBody.HTSeq" [1] "Down.EdgeR.GeneBody.HTSeq" null device 1 [1] "Removing VennDiagram*.log files" [1] "Check out Venn diagram and Count.Table!" #---------------------------------------------------------------------------------- Rscript Venn_Diff_Genes.R Up_Genes_DESeq_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt Up_Genes_EdgeR_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt GeneBody_Counting DiffExp_3a [1] "Print arguments:" [1] "-----------------" [1] "File1:" [1] "Up_Genes_DESeq_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt" [1] "File2:" [1] "Up_Genes_EdgeR_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt" [1] "Subtitle:" [1] "GeneBody_Counting" [1] "DiffExp_Index:" [1] "DiffExp_3a" [1] "-----------------" [1] "Up.DESeq.GeneBody.HTSeq" [1] "Up.EdgeR.GeneBody.HTSeq" null device 1 [1] "Removing VennDiagram*.log files" [1] "Check out Venn diagram and Count.Table!" #---------------------------------------------------------------------------------- #---------------------------------------------------------------------------------- Merging Count.Table(s) into one text file #---------------------------------------------------------------------------------- ========================================================== Renaming the Differential_Expression_File ========================================================== List files in Input -rw-r--r-- 1 kkarri montilab-p 175377 Jun 23 15:49 /scratch/9534915.1.linga/Input/Co0.out -rw-r--r-- 1 kkarri montilab-p 10 Jun 23 15:49 /scratch/9534915.1.linga/Input/Co0_num_mapped_reads.txt -rw-r--r-- 1 kkarri montilab-p 293 Jun 23 15:53 /scratch/9534915.1.linga/Input/DiffExp_3a_Venn_Tables_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt -rw-r--r-- 1 kkarri montilab-p 5112566 Jun 23 15:52 /scratch/9534915.1.linga/Input/DiffExp_v2_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt -rwxr-xr-x 1 kkarri montilab-p 7394 Jun 23 15:49 /scratch/9534915.1.linga/Input/Diff_Genes.R -rw-r--r-- 1 kkarri montilab-p 26765 Jun 23 15:53 /scratch/9534915.1.linga/Input/Down_Genes_DESeq_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt -rw-r--r-- 1 kkarri montilab-p 84373 Jun 23 15:53 /scratch/9534915.1.linga/Input/Down_Genes_EdgeR_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt -rwx------ 1 kkarri montilab-p 197610 Jun 23 15:49 /scratch/9534915.1.linga/Input/Exon_Regions_Lengths.txt -rw-r--r-- 1 kkarri montilab-p 168607 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO0.out -rw-r--r-- 1 kkarri montilab-p 9 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO0_num_mapped_reads.txt -rw-r--r-- 1 kkarri montilab-p 168577 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO1.out -rw-r--r-- 1 kkarri montilab-p 9 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO1_num_mapped_reads.txt -rw-r--r-- 1 kkarri montilab-p 169406 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO2.out -rw-r--r-- 1 kkarri montilab-p 9 Jun 23 15:49 /scratch/9534915.1.linga/Input/PHENO2_num_mapped_reads.txt -rw-r--r-- 1 kkarri montilab-p 1692553 Jun 23 15:53 /scratch/9534915.1.linga/Input/PHENO_Water_Saline_M1M2M3_vs_Co_Water_Saline_M1_DiffExp_v2_GeneBody_forSEGEXUpload_DESeq.txt -rw-r--r-- 1 kkarri montilab-p 1727227 Jun 23 15:53 /scratch/9534915.1.linga/Input/PHENO_Water_Saline_M1M2M3_vs_Co_Water_Saline_M1_DiffExp_v2_GeneBody_forSEGEXUpload_EdgeR.txt -rw-r--r-- 1 kkarri montilab-p 2002 Jun 23 15:53 /scratch/9534915.1.linga/Input/Up_Genes_DESeq_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt -rw-r--r-- 1 kkarri montilab-p 5222 Jun 23 15:53 /scratch/9534915.1.linga/Input/Up_Genes_EdgeR_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1.txt -rwxr-xr-x 1 kkarri montilab-p 9553 Jun 23 15:49 /scratch/9534915.1.linga/Input/Venn_Diff_Genes.R -rw-r--r-- 1 kkarri montilab-p 38685 Jun 23 15:53 /scratch/9534915.1.linga/Input/Venn_Down.DESeq.GeneBody.HTSeq.Down.EdgeR.GeneBody.HTSeq.png -rw-r--r-- 1 kkarri montilab-p 36913 Jun 23 15:53 /scratch/9534915.1.linga/Input/Venn_Up.DESeq.GeneBody.HTSeq.Up.EdgeR.GeneBody.HTSeq.png -rwxr-xr-x 1 kkarri montilab-p 3925 Jun 23 15:49 /scratch/9534915.1.linga/Input/formatForSegex_ver3.R /scratch/9534915.1.linga/Input/Co: total 8 drwxr-xr-x 2 kkarri montilab-p 4096 Jun 23 15:49 . drwxr-xr-x 4 kkarri montilab-p 4096 Jun 23 15:53 .. /scratch/9534915.1.linga/Input/PHENO: total 8 drwxr-xr-x 2 kkarri montilab-p 4096 Jun 23 15:49 . drwxr-xr-x 4 kkarri montilab-p 4096 Jun 23 15:53 .. ========================================================== List files in scratch total 41M drwxr-xr-x 3 kkarri montilab-p 4.0K Jun 23 15:53 . drwxrwxrwt. 103 root root 12K Jun 23 15:51 .. -rw-r--r-- 1 kkarri montilab-p 48 Jun 23 15:49 Condition_1.txt -rw-r--r-- 1 kkarri montilab-p 87 Jun 23 15:49 Condition_2.txt drwxr-xr-x 4 kkarri montilab-p 4.0K Jun 23 15:53 Input -rwx------ 1 kkarri montilab-p 41M Jun 23 15:49 RefSeq_GeneBody.gtf -rwxr-xr-x 1 kkarri montilab-p 16K Jun 23 15:49 differentialAnalysis.R ========================================================== Re-naming files in OUTPUT_DIR Need to append the COUNT_PROGRAM name to all output files List files in OUTPUT_DIR total 6.1M drwxr-sr-x 2 kkarri waxmanlab 32K Jun 23 15:53 . drwxr-sr-x 6 kkarri waxmanlab 32K Jun 23 15:49 .. -rw-r--r-- 1 kkarri waxmanlab 293 Jun 23 15:53 DiffExp_3a_Venn_Tables_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1_HTSeq.txt -rw-r--r-- 1 kkarri waxmanlab 4.9M Jun 23 15:53 DiffExp_v2_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1_HTSeq.txt -rw-r--r-- 1 kkarri waxmanlab 27K Jun 23 15:53 Down_Genes_DESeq_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1_HTSeq.txt -rw-r--r-- 1 kkarri waxmanlab 83K Jun 23 15:53 Down_Genes_EdgeR_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1_HTSeq.txt -rw-r--r-- 1 kkarri waxmanlab 1.7M Jun 23 15:53 PHENO_Water_Saline_M1M2M3_vs_Co_Water_Saline_M1_DiffExp_v2_GeneBody_forSEGEXUpload_DESeq_HTSeq.txt -rw-r--r-- 1 kkarri waxmanlab 1.7M Jun 23 15:53 PHENO_Water_Saline_M1M2M3_vs_Co_Water_Saline_M1_DiffExp_v2_GeneBody_forSEGEXUpload_EdgeR_HTSeq.txt -rw-r--r-- 1 kkarri waxmanlab 2.0K Jun 23 15:53 Up_Genes_DESeq_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1_HTSeq.txt -rw-r--r-- 1 kkarri waxmanlab 5.1K Jun 23 15:53 Up_Genes_EdgeR_GeneBody_PHENO_Water_Saline_M1M2M3_Co_Water_Saline_M1_HTSeq.txt -rw-r--r-- 1 kkarri waxmanlab 38K Jun 23 15:53 Venn_Down.DESeq.GeneBody.HTSeq.Down.EdgeR.GeneBody.HTSeq_HTSeq.png -rw-r--r-- 1 kkarri waxmanlab 37K Jun 23 15:53 Venn_Up.DESeq.GeneBody.HTSeq.Up.EdgeR.GeneBody.HTSeq_HTSeq.png ========================================================== ========================================================== Finished on : Fri Jun 23 15:53:04 EDT 2017 3 minutes and 16 seconds elapsed. ==========================================================