----------------------- Need shift 5 command (have more than 9 arguments): ----------------------- ----------------------- Start of variable list: ----------------------- SCRIPT_DIR: /restricted/projectnb/waxmanlab/kkarri/G160_Samples/Scripts/09b_DiffExp_8_featureCounts Dataset_DIR: /restricted/projectnb/waxmanlab/kkarri/G160_Samples Dataset_Label: G160 GTF_Files_DIR: /unprotected/projects/waxmanlab/routines/GTF_Files ANNOTATION_FILE: Exon_Only_Regions.gtf CONDITION_1_NAME: F_Chromatin_Pellet CONDITION_2_NAME: F_Nuclear Lengths_DIR: /unprotected/projects/waxmanlab/routines/GTF_Files/lengths GENE_LENGTHS_FILE: Exon_Only_Regions_Lengths.txt COUNT_DIR: RefSeq_Exon_Only_GTF OUTPUT_PREFIX: DiffExp_v2_Exonic_Only DiffExp_Index: DiffExp_8b COL_SUFFIX: Exonic_Only COUNT_PROGRAM: featureCounts ----------------------- End of variable list ----------------------- ========================================================== Starting on : Thu Aug 24 05:30:05 EDT 2017 Running on node : scc-kb8 Current directory : /restricted/projectnb/waxmanlab/kkarri/G160_Samples/Scripts/09b_DiffExp_8_featureCounts Current job ID : 383465 Current job name : Step_09b_DiffExp_8b Task index number : undefined Parameter for multiple cores : 1 ========================================================== Change dir to scratch directory Print scratch directory location: /scratch/383465.1.linga Loading required modules... ------------------------------------------ Sample_DIR: G160_M2 Sample_ID: G160_M2 Description: Female M_Num: M2 Copy Condition_1 sample count files to Condition_1 folder calculate mapped reads: G160_M2 M_Num_Cond1_List: M2 ------------------------------------------ ------------------------------------------ Sample_DIR: G160_M3 Sample_ID: G160_M3 Description: Female M_Num: M3 Copy Condition_2 sample count files to Condition_2 folder calculate mapped reads: G160_M3 M_Num_Cond2_List: M3 ------------------------------------------ ========================================================== Number of replicates in each condition: NUM_REP_CONDITION1: 1 NUM_REP_CONDITION1: 1 WARNING: Either your Condition_1 or Condition_2 has only 1 sample (no replicates) The differential expression analysis is designed for cases where there are at least 2 replicates/biological condition. The resulting differential expression analysis will be converted to a stringent version due to absence of replicate samples. Samples from this comparison will be treated as "replicates" to estimate the gene level variance in edgeR and, separately in, DESeq2. Expect very few to zero DE genes from DESeq2, EdgeR should yield a DE gene count. Interpreting the results: use the edgeR fold change or DESeq ratio (not DESeq fold change) Note regarding the SEGEX upload file(s): Since DESeq2 fold changes are not the same as the ratio of the normalized counts between the conditions, so the output of the segex upload file will be different between "DESeq ratio" and "DESeq foldChange". The former is truly the ratio between the normalized counts between the two conditions (plus pseudocounts) whereas the latter is the fold change estimated by DESeq. ========================================================== Renaming input count files Counting program: Not HTSeq No lines removed. Counting program: Not HTSeq No lines removed. number of mapped reads 46209433 46585452 ========================================================== List files in Input -rw-r--r-- 1 kkarri pulmseq 291285 Aug 24 05:30 /scratch/383465.1.linga/Input/Exon_Only_Regions_Lengths.txt -rw-r--r-- 1 kkarri pulmseq 245900 Aug 24 05:30 /scratch/383465.1.linga/Input/F_Chromatin_Pellet0.out -rw-r--r-- 1 kkarri pulmseq 9 Aug 24 05:30 /scratch/383465.1.linga/Input/F_Chromatin_Pellet0_num_mapped_reads.txt -rw-r--r-- 1 kkarri pulmseq 244919 Aug 24 05:30 /scratch/383465.1.linga/Input/F_Nuclear0.out -rw-r--r-- 1 kkarri pulmseq 9 Aug 24 05:30 /scratch/383465.1.linga/Input/F_Nuclear0_num_mapped_reads.txt /scratch/383465.1.linga/Input/F_Chromatin_Pellet: total 8 drwxr-xr-x 2 kkarri pulmseq 4096 Aug 24 05:30 . drwxr-xr-x 4 kkarri pulmseq 4096 Aug 24 05:30 .. /scratch/383465.1.linga/Input/F_Nuclear: total 8 drwxr-xr-x 2 kkarri pulmseq 4096 Aug 24 05:30 . drwxr-xr-x 4 kkarri pulmseq 4096 Aug 24 05:30 .. ========================================================== Starting to run my commands Printing Rscript command: Rscript differentialAnalysis.R F_Chromatin_Pellet F_Nuclear 1 1 Exon_Only_Regions.gtf /scratch/383465.1.linga/Input DiffExp_v2_Exonic_Only Exon_Only_Regions_Lengths.txt [1] "Arguments for differentialAnalysisDESeq.R:" [1] "F_Chromatin_Pellet" [1] "F_Nuclear" [1] 1 [1] 1 [1] "Exon_Only_Regions.gtf" [1] "/scratch/383465.1.linga/Input" [1] "DiffExp_v2_Exonic_Only" [1] "Exon_Only_Regions_Lengths.txt" load GTF file ... parse attributes ... [1] "output file is in: /scratch/383465.1.linga/Input/DiffExp_v2_Exonic_Only_F_Chromatin_Pellet_F_Nuclear.txt" ========================================================== Create SEGEX formatted file: Printing Rscript command: Rscript formatForSegex_ver3.R DiffExp_v2_Exonic_Only_F_Chromatin_Pellet_F_Nuclear.txt F_Nuclear_G160_M3_vs_F_Chromatin_Pellet_G160_M2_DiffExp_v2_Exonic_Only_forSEGEXUpload 1 Exonic_Only ========================================================== Comparison_Info: F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2 #---------------------------------------------------------------------------------- Running Diff_Genes.R #---------------------------------------------------------------------------------- Printing Rscript command: Rscript Diff_Genes.R F_Nuclear_G160_M3_vs_F_Chromatin_Pellet_G160_M2_DiffExp_v2_Exonic_Only_forSEGEXUpload_DESeq.txt 2 0.05 DESeq Exonic_Only'_'F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2 [1] "Print arguments:" [1] "-----------------" [1] "Differential expression_Output:" [1] "F_Nuclear_G160_M3_vs_F_Chromatin_Pellet_G160_M2_DiffExp_v2_Exonic_Only_forSEGEXUpload_DESeq.txt" [1] "fold_change_cutoff:" [1] "2" [1] "padj_cutoff:" [1] "0.05" [1] "postfix:" [1] "DESeq" [1] "count_method:" [1] "Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2" [1] "-----------------" [1] "Differential Gene Counts" [1] "Number of significant differential genes with positive fold change (Up Genes):" [1] 0 7 [1] "Number of significant differential genes with negative fold change (Down Genes):" [1] 0 7 [1] "Check out Up_Genes_DESeq_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt!" [1] "Check out Down_Genes_DESeq_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt!" Rscript Diff_Genes.R F_Nuclear_G160_M3_vs_F_Chromatin_Pellet_G160_M2_DiffExp_v2_Exonic_Only_forSEGEXUpload_EdgeR.txt 2 0.05 EdgeR Exonic_Only'_'F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2 [1] "Print arguments:" [1] "-----------------" [1] "Differential expression_Output:" [1] "F_Nuclear_G160_M3_vs_F_Chromatin_Pellet_G160_M2_DiffExp_v2_Exonic_Only_forSEGEXUpload_EdgeR.txt" [1] "fold_change_cutoff:" [1] "2" [1] "padj_cutoff:" [1] "0.05" [1] "postfix:" [1] "EdgeR" [1] "count_method:" [1] "Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2" [1] "-----------------" [1] "Differential Gene Counts" [1] "Number of significant differential genes with positive fold change (Up Genes):" [1] 3 7 [1] "Number of significant differential genes with negative fold change (Down Genes):" [1] 4 7 [1] "Check out Up_Genes_EdgeR_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt!" [1] "Check out Down_Genes_EdgeR_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt!" #---------------------------------------------------------------------------------- Running Venn_Diff_Genes.R #---------------------------------------------------------------------------------- Printing Rscript command: #---------------------------------------------------------------------------------- Rscript Venn_Diff_Genes.R Down_Genes_DESeq_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt Down_Genes_EdgeR_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt Exonic_Only_Counting DiffExp_8b [1] "Print arguments:" [1] "-----------------" [1] "File1:" [1] "Down_Genes_DESeq_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt" [1] "File2:" [1] "Down_Genes_EdgeR_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt" [1] "Subtitle:" [1] "Exonic_Only_Counting" [1] "DiffExp_Index:" [1] "DiffExp_8b" [1] "-----------------" [1] "Down.DESeq.Exonic_Only.HTSeq" [1] "Down.EdgeR.Exonic_Only.HTSeq" null device 1 [1] "Removing VennDiagram*.log files" [1] "Check out Venn diagram and Count.Table!" #---------------------------------------------------------------------------------- Rscript Venn_Diff_Genes.R Up_Genes_DESeq_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt Up_Genes_EdgeR_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt Exonic_Only_Counting DiffExp_8b [1] "Print arguments:" [1] "-----------------" [1] "File1:" [1] "Up_Genes_DESeq_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt" [1] "File2:" [1] "Up_Genes_EdgeR_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt" [1] "Subtitle:" [1] "Exonic_Only_Counting" [1] "DiffExp_Index:" [1] "DiffExp_8b" [1] "-----------------" [1] "Up.DESeq.Exonic_Only.HTSeq" [1] "Up.EdgeR.Exonic_Only.HTSeq" null device 1 [1] "Removing VennDiagram*.log files" [1] "Check out Venn diagram and Count.Table!" #---------------------------------------------------------------------------------- #---------------------------------------------------------------------------------- Merging Count.Table(s) into one text file #---------------------------------------------------------------------------------- Comparison_Info: F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2 #---------------------------------------------------------------------------------- ========================================================== Renaming the Differential_Expression_File ========================================================== List files in Input -rw-r--r-- 1 kkarri pulmseq 302 Aug 24 05:32 /scratch/383465.1.linga/Input/DiffExp_8b_Venn_Tables_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt -rw-r--r-- 1 kkarri pulmseq 4920269 Aug 24 05:32 /scratch/383465.1.linga/Input/DiffExp_v2_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt -rwxr-xr-x 1 kkarri pulmseq 7394 Aug 24 05:30 /scratch/383465.1.linga/Input/Diff_Genes.R -rw-r--r-- 1 kkarri pulmseq 173 Aug 24 05:32 /scratch/383465.1.linga/Input/Down_Genes_DESeq_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt -rw-r--r-- 1 kkarri pulmseq 478 Aug 24 05:32 /scratch/383465.1.linga/Input/Down_Genes_EdgeR_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt -rw-r--r-- 1 kkarri pulmseq 291285 Aug 24 05:30 /scratch/383465.1.linga/Input/Exon_Only_Regions_Lengths.txt -rw-r--r-- 1 kkarri pulmseq 245900 Aug 24 05:30 /scratch/383465.1.linga/Input/F_Chromatin_Pellet0.out -rw-r--r-- 1 kkarri pulmseq 9 Aug 24 05:30 /scratch/383465.1.linga/Input/F_Chromatin_Pellet0_num_mapped_reads.txt -rw-r--r-- 1 kkarri pulmseq 244919 Aug 24 05:30 /scratch/383465.1.linga/Input/F_Nuclear0.out -rw-r--r-- 1 kkarri pulmseq 9 Aug 24 05:30 /scratch/383465.1.linga/Input/F_Nuclear0_num_mapped_reads.txt -rw-r--r-- 1 kkarri pulmseq 2063828 Aug 24 05:32 /scratch/383465.1.linga/Input/F_Nuclear_G160_M3_vs_F_Chromatin_Pellet_G160_M2_DiffExp_v2_Exonic_Only_forSEGEXUpload_DESeq.txt -rw-r--r-- 1 kkarri pulmseq 6249909 Aug 24 05:32 /scratch/383465.1.linga/Input/F_Nuclear_G160_M3_vs_F_Chromatin_Pellet_G160_M2_DiffExp_v2_Exonic_Only_forSEGEXUpload_EdgeR.txt -rw-r--r-- 1 kkarri pulmseq 173 Aug 24 05:32 /scratch/383465.1.linga/Input/Up_Genes_DESeq_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt -rw-r--r-- 1 kkarri pulmseq 846 Aug 24 05:32 /scratch/383465.1.linga/Input/Up_Genes_EdgeR_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2.txt -rwxr-xr-x 1 kkarri pulmseq 9553 Aug 24 05:30 /scratch/383465.1.linga/Input/Venn_Diff_Genes.R -rw-r--r-- 1 kkarri pulmseq 31006 Aug 24 05:32 /scratch/383465.1.linga/Input/Venn_Down.DESeq.Exonic_Only.HTSeq.Down.EdgeR.Exonic_Only.HTSeq.png -rw-r--r-- 1 kkarri pulmseq 30187 Aug 24 05:32 /scratch/383465.1.linga/Input/Venn_Up.DESeq.Exonic_Only.HTSeq.Up.EdgeR.Exonic_Only.HTSeq.png -rwxr-xr-x 1 kkarri pulmseq 3925 Aug 24 05:30 /scratch/383465.1.linga/Input/formatForSegex_ver3.R /scratch/383465.1.linga/Input/F_Chromatin_Pellet: total 8 drwxr-xr-x 2 kkarri pulmseq 4096 Aug 24 05:30 . drwxr-xr-x 4 kkarri pulmseq 4096 Aug 24 05:32 .. /scratch/383465.1.linga/Input/F_Nuclear: total 8 drwxr-xr-x 2 kkarri pulmseq 4096 Aug 24 05:30 . drwxr-xr-x 4 kkarri pulmseq 4096 Aug 24 05:32 .. ========================================================== List files in scratch total 19M drwxr-xr-x 3 kkarri pulmseq 4.0K Aug 24 05:32 . drwxrwxrwt. 62 root root 112K Aug 24 05:30 .. -rw-r--r-- 1 kkarri pulmseq 56 Aug 24 05:30 Condition_1.txt -rw-r--r-- 1 kkarri pulmseq 56 Aug 24 05:30 Condition_2.txt -rw-r--r-- 1 kkarri pulmseq 19M Aug 24 05:30 Exon_Only_Regions.gtf drwxr-xr-x 4 kkarri pulmseq 4.0K Aug 24 05:32 Input -rwxr-xr-x 1 kkarri pulmseq 16K Aug 24 05:30 differentialAnalysis.R ========================================================== Re-naming files in OUTPUT_DIR Need to append the COUNT_PROGRAM name to all output files List files in OUTPUT_DIR total 11M drwxr-sr-x 2 kkarri waxmanlab 32K Aug 24 05:32 . drwxr-sr-x 6 kkarri waxmanlab 32K Aug 24 05:30 .. -rw-r--r-- 1 kkarri waxmanlab 302 Aug 24 05:32 DiffExp_8b_Venn_Tables_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 4.7M Aug 24 05:32 DiffExp_v2_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 173 Aug 24 05:32 Down_Genes_DESeq_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 478 Aug 24 05:32 Down_Genes_EdgeR_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 2.0M Aug 24 05:32 F_Nuclear_G160_M3_vs_F_Chromatin_Pellet_G160_M2_DiffExp_v2_Exonic_Only_forSEGEXUpload_DESeq_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 6.0M Aug 24 05:32 F_Nuclear_G160_M3_vs_F_Chromatin_Pellet_G160_M2_DiffExp_v2_Exonic_Only_forSEGEXUpload_EdgeR_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 173 Aug 24 05:32 Up_Genes_DESeq_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 846 Aug 24 05:32 Up_Genes_EdgeR_Exonic_Only_F_Nuclear_G160_M3_F_Chromatin_Pellet_G160_M2_featureCounts.txt -rw-r--r-- 1 kkarri waxmanlab 31K Aug 24 05:32 Venn_Down.DESeq.Exonic_Only.HTSeq.Down.EdgeR.Exonic_Only.HTSeq_featureCounts.png -rw-r--r-- 1 kkarri waxmanlab 30K Aug 24 05:32 Venn_Up.DESeq.Exonic_Only.HTSeq.Up.EdgeR.Exonic_Only.HTSeq_featureCounts.png ========================================================== ========================================================== Finished on : Thu Aug 24 05:32:40 EDT 2017 2 minutes and 35 seconds elapsed. ==========================================================