----------------------- Need shift 5 command (have more than 9 arguments): ----------------------- ----------------------- Start of variable list: ----------------------- SCRIPT_DIR: /net/waxman-server/mnt/data/waxmanlabvm_home/kkarri/G166_PCA_ENV/Scripts/09b_DiffExp_2_featureCounts Dataset_DIR: /net/waxman-server/mnt/data/waxmanlabvm_home/kkarri/G166_PCA_ENV Dataset_Label: G166_Env GTF_Files_DIR: /restricted/projectnb/waxmanlab/kkarri/mouse_gtf_files/final-RNAseq-Pipeline/GTF_Files ANNOTATION_FILE: RefSeq_GeneBody.gtf CONDITION_1_NAME: M_control CONDITION_2_NAME: M_TCPO_2d Lengths_DIR: /restricted/projectnb/waxmanlab/kkarri/mouse_gtf_files/final-RNAseq-Pipeline/GTF_Files/lengths GENE_LENGTHS_FILE: Exon_Regions_Lengths.txt COUNT_DIR: RefSeq_Exon_GTF OUTPUT_PREFIX: DiffExp_v2_GeneBody DiffExp_Index: DiffExp_2a COL_SUFFIX: GeneBody COUNT_PROGRAM: featureCounts ----------------------- End of variable list ----------------------- ========================================================== Starting on : Tue Apr 23 11:21:55 EDT 2019 Running on node : scc-ka2 Current directory : /net/waxman-server/mnt/data/waxmanlabvm_home/kkarri/G166_PCA_ENV/Scripts/09b_DiffExp_2_featureCounts Current job ID : 5542725 Current job name : Step_09b_DiffExp_2a Task index number : undefined Parameter for multiple cores : 1 ========================================================== Change dir to scratch directory Print scratch directory location: /scratch/5542725.1.linga Loading required modules... ------------------------------------------ Sample_DIR: G166_M1 Sample_ID: G166_M1 Description: M1_control M_Num: M1 Copy Condition_1 sample count files to Condition_1 folder calculate mapped reads: G166_M1 Sample_DIR: G166_M2 Sample_ID: G166_M2 Description: M2_control M_Num: M2 Copy Condition_1 sample count files to Condition_1 folder calculate mapped reads: G166_M2 M_Num_Cond1_List: M1M2 ------------------------------------------ ------------------------------------------ Sample_DIR: G166_M7 Sample_ID: G166_M7 Description: M_TCPO_2d M_Num: M7 Copy Condition_2 sample count files to Condition_2 folder calculate mapped reads: G166_M7 Sample_DIR: G166_M8 Sample_ID: G166_M8 Description: M_TCPO_2d M_Num: M8 Copy Condition_2 sample count files to Condition_2 folder calculate mapped reads: G166_M8 Sample_DIR: G166_M9 Sample_ID: G166_M9 Description: M_TCPO_2d M_Num: M9 Copy Condition_2 sample count files to Condition_2 folder calculate mapped reads: G166_M9 M_Num_Cond2_List: M7M8M9 ------------------------------------------ ========================================================== Number of replicates in each condition: NUM_REP_CONDITION1: 1 NUM_REP_CONDITION1: 1 WARNING: Either your Condition_1 or Condition_2 has only 1 sample (no replicates) The differential expression analysis is designed for cases where there are at least 2 replicates/biological condition. The resulting differential expression analysis will be converted to a stringent version due to absence of replicate samples. Samples from this comparison will be treated as "replicates" to estimate the gene level variance in edgeR and, separately in, DESeq2. Expect very few to zero DE genes from DESeq2, EdgeR should yield a DE gene count. Interpreting the results: use the edgeR fold change or DESeq ratio (not DESeq fold change) Note regarding the SEGEX upload file(s): Since DESeq2 fold changes are not the same as the ratio of the normalized counts between the conditions, so the output of the segex upload file will be different between "DESeq ratio" and "DESeq foldChange". The former is truly the ratio between the normalized counts between the two conditions (plus pseudocounts) whereas the latter is the fold change estimated by DESeq. ========================================================== Renaming input count files Counting program: Not HTSeq No lines removed. Counting program: Not HTSeq No lines removed. Counting program: Not HTSeq No lines removed. Counting program: Not HTSeq No lines removed. number of mapped reads 59800590 33413858 25862416 9260240 ========================================================== List files in Input -rw-r--r-- 1 kkarri waxmanlab 291522 Apr 23 11:21 /scratch/5542725.1.linga/Input/Exon_Regions_Lengths.txt -rw-r--r-- 1 kkarri waxmanlab 249822 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d0.out -rw-r--r-- 1 kkarri waxmanlab 9 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d0_num_mapped_reads.txt -rw-r--r-- 1 kkarri waxmanlab 245096 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d1.out -rw-r--r-- 1 kkarri waxmanlab 9 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d1_num_mapped_reads.txt -rw-r--r-- 1 kkarri waxmanlab 242178 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d2.out -rw-r--r-- 1 kkarri waxmanlab 9 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d2_num_mapped_reads.txt -rw-r--r-- 1 kkarri waxmanlab 237806 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_control0.out -rw-r--r-- 1 kkarri waxmanlab 8 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_control0_num_mapped_reads.txt /scratch/5542725.1.linga/Input/M_TCPO_2d: total 8 drwxr-xr-x 2 kkarri waxmanlab 4096 Apr 23 11:21 . drwxr-xr-x 4 kkarri waxmanlab 4096 Apr 23 11:21 .. /scratch/5542725.1.linga/Input/M_control: total 12 drwxr-xr-x 2 kkarri waxmanlab 4096 Apr 23 11:21 . drwxr-xr-x 4 kkarri waxmanlab 4096 Apr 23 11:21 .. -rw-r--r-- 1 kkarri waxmanlab 9 Apr 23 11:21 G166_M2_num_mapped_reads.txt ========================================================== Starting to run my commands Printing Rscript command: Rscript differentialAnalysis.R M_control M_TCPO_2d 1 3 RefSeq_GeneBody.gtf /scratch/5542725.1.linga/Input DiffExp_v2_GeneBody Exon_Regions_Lengths.txt [1] "Arguments for differentialAnalysisDESeq.R:" [1] "M_control" [1] "M_TCPO_2d" [1] 1 [1] 3 [1] "RefSeq_GeneBody.gtf" [1] "/scratch/5542725.1.linga/Input" [1] "DiffExp_v2_GeneBody" [1] "Exon_Regions_Lengths.txt" [1] "sum1:80417.7356383661" "sum1:80417.7356383661" [1] "sum2:60163.602758817" "sum2:80633.6565089061" "sum2:107606.910871921" [4] "sum2:82801.3900465482" load GTF file ... parse attributes ... [1] "output file is in: /scratch/5542725.1.linga/Input/DiffExp_v2_GeneBody_M_control_M_TCPO_2d.txt" ========================================================== Create SEGEX formatted file: Printing Rscript command: Rscript formatForSegex_ver3.R DiffExp_v2_GeneBody_M_control_M_TCPO_2d.txt M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload 1 GeneBody Printing Rscript command: Rscript formatForSegex_ver4.R DiffExp_v2_GeneBody_M_control_M_TCPO_2d.txt M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload 1 GeneBody ========================================================== Comparison_Info: M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2 #---------------------------------------------------------------------------------- Running Diff_Genes.R #---------------------------------------------------------------------------------- Printing Rscript command: Rscript Diff_Genes.R M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_DESeq.txt 2 0.05 DESeq GeneBody'_'M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2 [1] "Print arguments:" [1] "-----------------" [1] "Differential expression_Output:" [1] "M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_DESeq.txt" [1] "fold_change_cutoff:" [1] "2" [1] "padj_cutoff:" [1] "0.05" [1] "postfix:" [1] "DESeq" [1] "count_method:" [1] "GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2" [1] "-----------------" [1] "Differential Gene Counts" [1] "Number of significant differential genes with positive fold change (Up Genes):" [1] 32 7 [1] "Number of significant differential genes with negative fold change (Down Genes):" [1] 11 7 [1] "Check out Up_Genes_DESeq_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt!" [1] "Check out Down_Genes_DESeq_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt!" Rscript Diff_Genes.R M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_EdgeR.txt 2 0.05 EdgeR GeneBody'_'M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2 [1] "Print arguments:" [1] "-----------------" [1] "Differential expression_Output:" [1] "M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_EdgeR.txt" [1] "fold_change_cutoff:" [1] "2" [1] "padj_cutoff:" [1] "0.05" [1] "postfix:" [1] "EdgeR" [1] "count_method:" [1] "GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2" [1] "-----------------" [1] "Differential Gene Counts" [1] "Number of significant differential genes with positive fold change (Up Genes):" [1] 15 7 [1] "Number of significant differential genes with negative fold change (Down Genes):" [1] 9 7 [1] "Check out Up_Genes_EdgeR_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt!" [1] "Check out Down_Genes_EdgeR_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt!" #---------------------------------------------------------------------------------- Running Venn_Diff_Genes.R #---------------------------------------------------------------------------------- Printing Rscript command: #---------------------------------------------------------------------------------- Rscript Venn_Diff_Genes.R Down_Genes_DESeq_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt Down_Genes_EdgeR_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt GeneBody_Counting DiffExp_2a [1] "Print arguments:" [1] "-----------------" [1] "File1:" [1] "Down_Genes_DESeq_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt" [1] "File2:" [1] "Down_Genes_EdgeR_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt" [1] "Subtitle:" [1] "GeneBody_Counting" [1] "DiffExp_Index:" [1] "DiffExp_2a" [1] "-----------------" [1] "Down.DESeq.GeneBody.HTSeq" [1] "Down.EdgeR.GeneBody.HTSeq" null device 1 [1] "Removing VennDiagram*.log files" [1] "Check out Venn diagram and Count.Table!" #---------------------------------------------------------------------------------- Rscript Venn_Diff_Genes.R Up_Genes_DESeq_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt Up_Genes_EdgeR_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt GeneBody_Counting DiffExp_2a [1] "Print arguments:" [1] "-----------------" [1] "File1:" [1] "Up_Genes_DESeq_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt" [1] "File2:" [1] "Up_Genes_EdgeR_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt" [1] "Subtitle:" [1] "GeneBody_Counting" [1] "DiffExp_Index:" [1] "DiffExp_2a" [1] "-----------------" [1] "Up.DESeq.GeneBody.HTSeq" [1] "Up.EdgeR.GeneBody.HTSeq" null device 1 [1] "Removing VennDiagram*.log files" [1] "Check out Venn diagram and Count.Table!" #---------------------------------------------------------------------------------- #---------------------------------------------------------------------------------- Merging Count.Table(s) into one text file #---------------------------------------------------------------------------------- Comparison_Info: M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2 #---------------------------------------------------------------------------------- ========================================================== Renaming the Differential_Expression_File ========================================================== List files in Input -rw-r--r-- 1 kkarri waxmanlab 288 Apr 23 11:24 /scratch/5542725.1.linga/Input/DiffExp_2a_Venn_Tables_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt -rw-r--r-- 1 kkarri waxmanlab 9346926 Apr 23 11:24 /scratch/5542725.1.linga/Input/DiffExp_v2_GeneBody_M_control_M_TCPO_2d.txt -rwxr-xr-x 1 kkarri waxmanlab 7394 Apr 23 11:21 /scratch/5542725.1.linga/Input/Diff_Genes.R -rw-r--r-- 1 kkarri waxmanlab 944 Apr 23 11:24 /scratch/5542725.1.linga/Input/Down_Genes_DESeq_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt -rw-r--r-- 1 kkarri waxmanlab 816 Apr 23 11:24 /scratch/5542725.1.linga/Input/Down_Genes_EdgeR_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt -rw-r--r-- 1 kkarri waxmanlab 291522 Apr 23 11:21 /scratch/5542725.1.linga/Input/Exon_Regions_Lengths.txt -rw-r--r-- 1 kkarri waxmanlab 249822 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d0.out -rw-r--r-- 1 kkarri waxmanlab 9 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d0_num_mapped_reads.txt -rw-r--r-- 1 kkarri waxmanlab 245096 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d1.out -rw-r--r-- 1 kkarri waxmanlab 9 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d1_num_mapped_reads.txt -rw-r--r-- 1 kkarri waxmanlab 242178 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d2.out -rw-r--r-- 1 kkarri waxmanlab 9 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_TCPO_2d2_num_mapped_reads.txt -rw-r--r-- 1 kkarri waxmanlab 3322042 Apr 23 11:24 /scratch/5542725.1.linga/Input/M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_DESeq.txt -rw-r--r-- 1 kkarri waxmanlab 2741314 Apr 23 11:24 /scratch/5542725.1.linga/Input/M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_EdgeR.txt -rw-r--r-- 1 kkarri waxmanlab 3322040 Apr 23 11:24 /scratch/5542725.1.linga/Input/M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_TPM_DESeq.txt -rw-r--r-- 1 kkarri waxmanlab 2741312 Apr 23 11:24 /scratch/5542725.1.linga/Input/M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_TPM_EdgeR.txt -rw-r--r-- 1 kkarri waxmanlab 237806 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_control0.out -rw-r--r-- 1 kkarri waxmanlab 8 Apr 23 11:21 /scratch/5542725.1.linga/Input/M_control0_num_mapped_reads.txt -rw-r--r-- 1 kkarri waxmanlab 3529 Apr 23 11:24 /scratch/5542725.1.linga/Input/Up_Genes_DESeq_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt -rw-r--r-- 1 kkarri waxmanlab 1339 Apr 23 11:24 /scratch/5542725.1.linga/Input/Up_Genes_EdgeR_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2.txt -rwxr-xr-x 1 kkarri waxmanlab 9553 Apr 23 11:21 /scratch/5542725.1.linga/Input/Venn_Diff_Genes.R -rw-r--r-- 1 kkarri waxmanlab 39037 Apr 23 11:24 /scratch/5542725.1.linga/Input/Venn_Down.DESeq.GeneBody.HTSeq.Down.EdgeR.GeneBody.HTSeq.png -rw-r--r-- 1 kkarri waxmanlab 37888 Apr 23 11:24 /scratch/5542725.1.linga/Input/Venn_Up.DESeq.GeneBody.HTSeq.Up.EdgeR.GeneBody.HTSeq.png -rwxr-xr-x 1 kkarri waxmanlab 3925 Apr 23 11:21 /scratch/5542725.1.linga/Input/formatForSegex_ver3.R -rwxr-xr-x 1 kkarri waxmanlab 3929 Apr 23 11:21 /scratch/5542725.1.linga/Input/formatForSegex_ver4.R /scratch/5542725.1.linga/Input/M_TCPO_2d: total 8 drwxr-xr-x 2 kkarri waxmanlab 4096 Apr 23 11:21 . drwxr-xr-x 4 kkarri waxmanlab 4096 Apr 23 11:24 .. /scratch/5542725.1.linga/Input/M_control: total 12 drwxr-xr-x 2 kkarri waxmanlab 4096 Apr 23 11:21 . drwxr-xr-x 4 kkarri waxmanlab 4096 Apr 23 11:24 .. -rw-r--r-- 1 kkarri waxmanlab 9 Apr 23 11:21 G166_M2_num_mapped_reads.txt ========================================================== List files in scratch total 72M drwx------ 3 kkarri waxmanlab 4.0K Apr 23 11:24 . drwxrwxrwt. 68 root root 116K Apr 23 11:21 .. -rw-r--r-- 1 kkarri waxmanlab 87 Apr 23 11:21 Condition_1.txt -rw-r--r-- 1 kkarri waxmanlab 111 Apr 23 11:21 Condition_2.txt drwxr-xr-x 4 kkarri waxmanlab 4.0K Apr 23 11:24 Input -rw-r--r-- 1 kkarri waxmanlab 72M Apr 23 11:21 RefSeq_GeneBody.gtf -rwxr-xr-x 1 kkarri waxmanlab 17K Apr 23 11:21 differentialAnalysis.R ========================================================== Re-naming files in OUTPUT_DIR Need to append the COUNT_PROGRAM name to all output files List files in OUTPUT_DIR total 30M drwxr-xr-x 2 kkarri waxmanlab 4.0K Apr 23 11:19 . drwxr-xr-x 7 kkarri waxmanlab 4.0K Apr 23 11:16 .. -rw-r--r-- 1 kkarri waxmanlab 288 Apr 23 11:19 DiffExp_2a_Venn_Tables_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 9.0M Apr 23 11:19 DiffExp_v2_GeneBody_M_control_M_TCPO_2d_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 944 Apr 23 11:19 Down_Genes_DESeq_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 816 Apr 23 11:19 Down_Genes_EdgeR_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 3.2M Apr 23 11:19 M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_DESeq_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 2.7M Apr 23 11:19 M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_EdgeR_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 3.2M Apr 23 11:19 M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_TPM_DESeq_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 2.7M Apr 23 11:19 M_TCPO_2d_G166_Env_M7M8M9_vs_M_control_G166_Env_M1M2_DiffExp_v2_GeneBody_forSEGEXUpload_TPM_EdgeR_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 3.5K Apr 23 11:19 Up_Genes_DESeq_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 1.4K Apr 23 11:19 Up_Genes_EdgeR_GeneBody_M_TCPO_2d_G166_Env_M7M8M9_M_control_G166_Env_M1M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 39K Apr 23 11:19 Venn_Down.DESeq.GeneBody.HTSeq.Down.EdgeR.GeneBody.HTSeq_featureCounts.png -rw-r--r-- 1 kkarri waxmanlab 37K Apr 23 11:19 Venn_Up.DESeq.GeneBody.HTSeq.Up.EdgeR.GeneBody.HTSeq_featureCounts.png ========================================================== ========================================================== Finished on : Tue Apr 23 11:24:55 EDT 2019 3 minutes and 0 seconds elapsed. ==========================================================