Hello!
@shayanhoss and I are running into errors with the v1.3 pipeline on AWS Batch. There seems to be some issue with nxf-scratch-dir in our latest error, though we've also had errors with just loading the genome itself during the STAR alignment step. I'm not exactly sure how to look into or debug this, since it seems like an internal nextflow error to me. Any thoughts would be appreciated!
N E X T F L O W ~ version 19.04.1
Launching `nf-core/rnaseq` [goofy_waddington] - revision: 37f260d360 [master]
----------------------------------------------------
,--./,-.
___ __ __ __ ___ /,-._.--~'
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'
nf-core/rnaseq v1.3
----------------------------------------------------
Pipeline Release : master
Run Name : goofy_waddington
Reads : s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/fastqs/*{R1,R2}*.fastq.gz
Data Type : Paired-End
Genome : GRCh38
Strandedness : None
Trimming : 5'R1: 0 / 5'R2: 0 / 3'R1: 0 / 3'R2: 0
Aligner : STAR
STAR Index : s3://czbiohub-reference/gencode/human/v30/STARIndex/
GTF Annotation : s3://czbiohub-reference/gencode/human/v30/gencode.v30.annotation.ERCC92.gtf
Save prefs : Ref Genome: Yes / Trimmed FastQ: Yes / Alignment intermediates: Yes
Max Resources : 1.9 TB memory, 96 cpus, 10d time per job
Container : docker - nfcore/rnaseq:1.3
Output dir : s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/results_nextflow
Launch dir : /home/shayanhoss/code/scSLAM_notebooks
Working dir : /darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates
Script dir : /home/shayanhoss/.nextflow/assets/nf-core/rnaseq
User : shayanhoss
Config Profile : czbiohub_aws
Config Description: Chan Zuckerberg Biohub AWS Batch profile provided by nf-core/configs.
Config Contact : Olga Botvinnik (@olgabot)
Config URL : https://www.czbiohub.org/
----------------------------------------------------
ERROR ~ Error executing process > 'star (A17_scSLAM_S17_R1_001)'
Caused by:
Process `star (A17_scSLAM_S17_R1_001)` terminated with an error exit status (1)
Command executed:
STAR --genomeDir STARIndex \
--sjdbGTFfile gencode.v30.annotation.ERCC92.gtf \
--readFilesIn A17_scSLAM_S17_R1_001_val_1.fq.gz A17_scSLAM_S17_R2_001_val_2.fq.gz \
--runThreadN 10 \
--twopassMode Basic \
--outWigType bedGraph \
--outSAMtype BAM SortedByCoordinate --limitBAMsortRAM 85799345920 \
--readFilesCommand zcat \
--runDirPerm All_RWX \
--outFileNamePrefix A17_scSLAM_S17_R1_001
samtools index A17_scSLAM_S17_R1_001Aligned.sortedByCoord.out.bam
Command exit status:
1
Command output:
(empty)
Command wrapper:
nxf-scratch-dir ip-10-0-0-252:/tmp/nxf.GVfYEvc2bE
Work dir:
s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74
Here is the nextflow log:
But we've also gotten weird errors with the genome not getting fully loaded for STAR? Here's an excerpt from that .nextflow.log file:
.nextflow.log.2:Jun-13 17:38:44.485 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'star (L1_scSLAM_S265_R1_001)'
.nextflow.log.2-
.nextflow.log.2-Caused by:
.nextflow.log.2- Process `star (L1_scSLAM_S265_R1_001)` terminated with an error exit status (105)
.nextflow.log.2-
.nextflow.log.2-Command executed:
.nextflow.log.2-
.nextflow.log.2- STAR --genomeDir star \
.nextflow.log.2- --sjdbGTFfile gencode.v30.annotation.ERCC92.gtf \
.nextflow.log.2- --readFilesIn L1_scSLAM_S265_R1_001_val_1.fq.gz L1_scSLAM_S265_R2_001_val_2.fq.gz \
.nextflow.log.2- --runThreadN 10 \
.nextflow.log.2- --twopassMode Basic \
.nextflow.log.2- --outWigType bedGraph \
.nextflow.log.2- --outSAMtype BAM SortedByCoordinate --limitBAMsortRAM 85799345920 \
.nextflow.log.2- --readFilesCommand zcat \
.nextflow.log.2- --runDirPerm All_RWX \
.nextflow.log.2- --outFileNamePrefix L1_scSLAM_S265_R1_001
.nextflow.log.2-
.nextflow.log.2- samtools index L1_scSLAM_S265_R1_001Aligned.sortedByCoord.out.bam
.nextflow.log.2-
.nextflow.log.2-Command exit status:
.nextflow.log.2- 105
.nextflow.log.2-
.nextflow.log.2-Command output:
.nextflow.log.2- Jun 13 20:13:23 ..... started STAR run
.nextflow.log.2- Jun 13 20:13:23 ..... loading genome
.nextflow.log.2-
.nextflow.log.2-Command error:
.nextflow.log.2-
.nextflow.log.2- EXITING because of FATAL ERROR: could not open genome file: star/SA
.nextflow.log.2- SOLUTION: check that the path to genome files, specified in --genomeDir is correct and the files are present, and have user read permissions
Do you know what may be happening?
olgabot
All 4 comments
Hmm, is this something about not being able to mount s3 directories, needing specific files?
ewels
on 18 Jun 2019
@ewels This error was happening when we were generating the genome on the fly, and somehow seems to go away when we copy that generated reference genome to our path. When we do that, we now get a new error! (my favorite!) Again, I'm not sure what is causing the problem. It seems like an internal nextflow error in creating temporary directory.
ERROR ~ Error executing process > 'star (A17_scSLAM_S17_R1_001)'
Caused by:
Process `star (A17_scSLAM_S17_R1_001)` terminated with an error exit status (1)
Command executed:
STAR --genomeDir STARIndex \
--sjdbGTFfile gencode.v30.annotation.ERCC92.gtf \
--readFilesIn A17_scSLAM_S17_R1_001_val_1.fq.gz A17_scSLAM_S17_R2_001_val_2.fq.gz \
--runThreadN 10 \
--twopassMode Basic \
--outWigType bedGraph \
--outSAMtype BAM SortedByCoordinate --limitBAMsortRAM 85799345920 \
--readFilesCommand zcat \
--runDirPerm All_RWX \
--outFileNamePrefix A17_scSLAM_S17_R1_001
samtools index A17_scSLAM_S17_R1_001Aligned.sortedByCoord.out.bam
Command exit status:
1
Command output:
(empty)
Command wrapper:
nxf-scratch-dir ip-10-0-0-252:/tmp/nxf.GVfYEvc2bE
Work dir:
s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
.command.run
Here's the .command.run:
#!/bin/bash
# NEXTFLOW TASK: star (A17_scSLAM_S17_R1_001)
set -e
set -u
NXF_DEBUG=${NXF_DEBUG:=0}; [[ $NXF_DEBUG > 1 ]] && set -x
NXF_ENTRY=${1:-nxf_main}
nxf_tree() {
local pid=$1
declare -a ALL_CHILDREN
while read P PP;do
ALL_CHILDREN[$PP]+=" $P"
done < <(ps -e -o pid= -o ppid=)
pstat() {
local x_pid=$1
local STATUS=$(2> /dev/null < /proc/$1/status egrep 'Vm|ctxt')
if [ $? = 0 ]; then
local x_vsz=$(echo "$STATUS" | grep VmSize | awk '{print $2}' || echo -n '0')
local x_rss=$(echo "$STATUS" | grep VmRSS | awk '{print $2}' || echo -n '0')
local x_peak=$(echo "$STATUS" | egrep 'VmPeak|VmHWM' | sed 's/^.*:\s*//' | sed 's/[\sa-zA-Z]*$//' | tr '\n' ' ' || echo -n '0 0')
local x_pmem=$(awk -v rss=$x_rss -v mem_tot=$mem_tot 'BEGIN {printf "%.0f", rss/mem_tot*100*10}' || echo -n '0')
local vol_ctxt=$(echo "$STATUS" | grep '\bvoluntary_ctxt_switches' | awk '{print $2}' || echo -n '0')
local inv_ctxt=$(echo "$STATUS" | grep '\bnonvoluntary_ctxt_switches' | awk '{print $2}' || echo -n '0')
cpu_stat[x_pid]="$x_pid $x_pmem $x_vsz $x_rss $x_peak $vol_ctxt $inv_ctxt"
fi
}
pwalk() {
pstat $1
for i in ${ALL_CHILDREN[$1]:=}; do pwalk $i; done
}
pwalk $1
}
nxf_stat() {
cpu_stat=()
nxf_tree $1
declare -a sum=(0 0 0 0 0 0 0 0)
local pid
local i
for pid in "${!cpu_stat[@]}"; do
local row=(${cpu_stat[pid]})
[ $NXF_DEBUG = 1 ] && echo "++ stat mem=${row[*]}"
for i in "${!row[@]}"; do
if [ $i != 0 ]; then
sum[i]=$((sum[i]+row[i]))
fi
done
done
[ $NXF_DEBUG = 1 ] && echo -e "++ stat SUM=${sum[*]}"
for i in {1..7}; do
if [ ${sum[i]} -lt ${cpu_peak[i]} ]; then
sum[i]=${cpu_peak[i]}
else
cpu_peak[i]=${sum[i]}
fi
done
[ $NXF_DEBUG = 1 ] && echo -e "++ stat PEAK=${sum[*]}\n"
nxf_stat_ret=(${sum[*]})
}
nxf_sleep() {
sleep $1 2>/dev/null || sleep 1;
}
nxf_mem_watch() {
set -o pipefail
local pid=$1
local trace_file=.command.trace
local count=0;
declare -a cpu_stat=(0 0 0 0 0 0 0 0)
declare -a cpu_peak=(0 0 0 0 0 0 0 0)
local mem_tot=$(< /proc/meminfo grep MemTotal | awk '{print $2}')
local timeout
local DONE
local STOP=''
[ $NXF_DEBUG = 1 ] && nxf_sleep 0.2 && ps fx
while true; do
nxf_stat $pid
if [ $count -lt 10 ]; then timeout=1;
elif [ $count -lt 120 ]; then timeout=5;
else timeout=30;
fi
read -t $timeout -r DONE || true
[[ $DONE ]] && break
if [ ! -e /proc/$pid ]; then
[ ! $STOP ] && STOP=$(nxf_date)
[ $(($(nxf_date)-STOP)) -gt 10000 ] && break
fi
count=$((count+1))
done
echo "%mem=${nxf_stat_ret[1]}" >> $trace_file
echo "vmem=${nxf_stat_ret[2]}" >> $trace_file
echo "rss=${nxf_stat_ret[3]}" >> $trace_file
echo "peak_vmem=${nxf_stat_ret[4]}" >> $trace_file
echo "peak_rss=${nxf_stat_ret[5]}" >> $trace_file
echo "vol_ctxt=${nxf_stat_ret[6]}" >> $trace_file
echo "inv_ctxt=${nxf_stat_ret[7]}" >> $trace_file
}
nxf_write_trace() {
echo "nextflow.trace/v2" > $trace_file
echo "realtime=$wall_time" >> $trace_file
echo "%cpu=$ucpu" >> $trace_file
echo "rchar=${io_stat1[0]}" >> $trace_file
echo "wchar=${io_stat1[1]}" >> $trace_file
echo "syscr=${io_stat1[2]}" >> $trace_file
echo "syscw=${io_stat1[3]}" >> $trace_file
echo "read_bytes=${io_stat1[4]}" >> $trace_file
echo "write_bytes=${io_stat1[5]}" >> $trace_file
}
nxf_trace_mac() {
local start_millis=$(nxf_date)
/bin/bash -euo pipefail .command.sh
local end_millis=$(nxf_date)
local wall_time=$((end_millis-start_millis))
local ucpu=''
local io_stat1=('' '' '' '' '' '')
nxf_write_trace
}
nxf_trace_linux() {
local pid=$$
local num_cpus=$(< /proc/cpuinfo grep '^processor' -c)
local tot_time0=$(grep '^cpu ' /proc/stat | awk '{sum=$2+$3+$4+$5+$6+$7+$8+$9; printf "%.0f",sum}')
local cpu_time0=$(2> /dev/null < /proc/$pid/stat awk '{printf "%.0f", ($16+$17)*10 }' || echo -n 'X')
local io_stat0=($(2> /dev/null < /proc/$pid/io sed 's/^.*:\s*//' | head -n 6 | tr '\n' ' ' || echo -n '0 0 0 0 0 0'))
local start_millis=$(nxf_date)
command -v ps &>/dev/null || { >&2 echo "Command 'ps' required by nextflow to collect task metrics cannot be found"; exit 1; }
/bin/bash -euo pipefail .command.sh &
local task=$!
exec 10> >(nxf_mem_watch $task)
local mem_proc=$!
wait $task
local end_millis=$(nxf_date)
local tot_time1=$(grep '^cpu ' /proc/stat | awk '{sum=$2+$3+$4+$5+$6+$7+$8+$9; printf "%.0f",sum}')
local cpu_time1=$(2> /dev/null < /proc/$pid/stat awk '{printf "%.0f", ($16+$17)*10 }' || echo -n 'X')
local ucpu=$(awk -v p1=$cpu_time1 -v p0=$cpu_time0 -v t1=$tot_time1 -v t0=$tot_time0 -v n=$num_cpus 'BEGIN { pct=(p1-p0)/(t1-t0)*100*n; printf("%.0f", pct>0 ? pct : 0) }' )
local io_stat1=($(2> /dev/null < /proc/$pid/io sed 's/^.*:\s*//' | head -n 6 | tr '\n' ' ' || echo -n '0 0 0 0 0 0'))
local i
for i in {0..5}; do
io_stat1[i]=$((io_stat1[i]-io_stat0[i]))
done
local wall_time=$((end_millis-start_millis))
[ $NXF_DEBUG = 1 ] && echo "+++ STATS %CPU=$ucpu TIME=$wall_time I/O=${io_stat1[*]}"
echo "nextflow.trace/v2" > $trace_file
echo "realtime=$wall_time" >> $trace_file
echo "%cpu=$ucpu" >> $trace_file
echo "rchar=${io_stat1[0]}" >> $trace_file
echo "wchar=${io_stat1[1]}" >> $trace_file
echo "syscr=${io_stat1[2]}" >> $trace_file
echo "syscw=${io_stat1[3]}" >> $trace_file
echo "read_bytes=${io_stat1[4]}" >> $trace_file
echo "write_bytes=${io_stat1[5]}" >> $trace_file
echo 'DONE' >&10
wait $mem_proc 2>/dev/null || true
while [ -e /proc/$mem_proc ]; do nxf_sleep 0.1; done
}
nxf_trace() {
local trace_file=.command.trace
touch $trace_file
if [[ $(uname) = Darwin ]]; then
nxf_trace_mac
else
nxf_trace_linux
fi
}
# aws helper
nxf_s3_upload() {
local pattern=$1
local s3path=$2
IFS=$'\n'
for name in $(eval "ls -1d $pattern");do
if [[ -d "$name" ]]; then
/home/ec2-user/miniconda/bin/aws --region us-west-2 s3 cp --only-show-errors --recursive --storage-class STANDARD "$name" "$s3path/$name"
else
/home/ec2-user/miniconda/bin/aws --region us-west-2 s3 cp --only-show-errors --storage-class STANDARD "$name" "$s3path/$name"
fi
done
unset IFS
}
nxf_s3_download() {
local source=$1
local target=$2
local file_name=$(basename $1)
local is_dir=$(/home/ec2-user/miniconda/bin/aws --region us-west-2 s3 ls $source | grep -F "PRE ${file_name}/" -c)
if [[ $is_dir == 1 ]]; then
/home/ec2-user/miniconda/bin/aws --region us-west-2 s3 cp --only-show-errors --recursive "$source" "$target"
else
/home/ec2-user/miniconda/bin/aws --region us-west-2 s3 cp --only-show-errors "$source" "$target"
fi
}
nxf_parallel() {
local cmd=("$@")
local cpus=$(nproc 2>/dev/null || < /proc/cpuinfo grep '^process' -c)
local max=$(if (( cpus>16 )); then echo 16; else echo $cpus; fi)
local i=0
local pid=()
(
set +u
while ((i<${#cmd[@]})); do
local copy=()
for x in "${pid[@]}"; do
[[ -e /proc/$x ]] && copy+=($x)
done
pid=("${copy[@]}")
if ((${#pid[@]}>=$max)); then
sleep 1
else
eval "${cmd[$i]}" &
pid+=($!)
((i+=1))
fi
done
((${#pid[@]}>0)) && wait ${pid[@]}
)
}
nxf_date() {
local ts=$(date +%s%3N); [[ $ts == *3N ]] && date +%s000 || echo $ts
}
nxf_env() {
echo '============= task environment ============='
env | sort | sed "s/\(.*\)AWS\(.*\)=\(.\{6\}\).*/\1AWS\2=\3xxxxxxxxxxxxx/"
echo '============= task output =================='
}
nxf_kill() {
declare -a children
while read P PP;do
children[$PP]+=" $P"
done < <(ps -e -o pid= -o ppid=)
kill_all() {
[[ $1 != $$ ]] && kill $1 2>/dev/null || true
for i in ${children[$1]:=}; do kill_all $i; done
}
kill_all $1
}
nxf_mktemp() {
local base=${1:-/tmp}
if [[ $(uname) = Darwin ]]; then mktemp -d $base/nxf.XXXXXXXXXX
else TMPDIR="$base" mktemp -d -t nxf.XXXXXXXXXX
fi
}
on_exit() {
exit_status=${nxf_main_ret:=$?}
printf $exit_status | /home/ec2-user/miniconda/bin/aws --region us-west-2 s3 cp --only-show-errors - s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74/.exitcode || true
set +u
[[ "$tee1" ]] && kill $tee1 2>/dev/null
[[ "$tee2" ]] && kill $tee2 2>/dev/null
[[ "$ctmp" ]] && rm -rf $ctmp || true
rm -rf $NXF_SCRATCH || true
exit $exit_status
}
on_term() {
set +e
[[ "$pid" ]] && nxf_kill $pid
}
nxf_launch() {
/bin/bash .command.run nxf_trace
}
nxf_stage() {
true
# stage input files
downloads=()
rm -f gencode.v30.annotation.ERCC92.gtf
rm -f .command.sh
rm -f where_are_my_files.txt
rm -f A17_scSLAM_S17_R1_001_val_1.fq.gz
rm -f A17_scSLAM_S17_R2_001_val_2.fq.gz
rm -f .command.run
rm -f STARIndex
downloads+=("nxf_s3_download s3://czbiohub-reference/gencode/human/v30/gencode.v30.annotation.ERCC92.gtf gencode.v30.annotation.ERCC92.gtf")
downloads+=("nxf_s3_download s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74/.command.sh .command.sh")
downloads+=("nxf_s3_download s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/stage/60/eda74e5ac98cb1b781c6557153ed78/where_are_my_files.txt where_are_my_files.txt")
downloads+=("nxf_s3_download s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/23/7f3f964ca52b97d0e60f6e37c63fcc/A17_scSLAM_S17_R1_001_val_1.fq.gz A17_scSLAM_S17_R1_001_val_1.fq.gz")
downloads+=("nxf_s3_download s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/23/7f3f964ca52b97d0e60f6e37c63fcc/A17_scSLAM_S17_R2_001_val_2.fq.gz A17_scSLAM_S17_R2_001_val_2.fq.gz")
downloads+=("nxf_s3_download s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74/.command.run .command.run")
downloads+=("nxf_s3_download s3://czbiohub-reference/gencode/human/v30/STARIndex STARIndex")
nxf_parallel "${downloads[@]}"
}
nxf_unstage() {
true
nxf_s3_upload .command.out s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74 || true
nxf_s3_upload .command.err s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74 || true
nxf_s3_upload .command.trace s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74 || true
[[ ${nxf_main_ret:=0} != 0 ]] && return
uploads=()
uploads+=("nxf_s3_upload '*Log.final.out' s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74")
uploads+=("nxf_s3_upload '*.bam' s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74")
uploads+=("nxf_s3_upload '*.out' s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74")
uploads+=("nxf_s3_upload '*SJ.out.tab' s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74")
uploads+=("nxf_s3_upload '*Log.out' s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74")
uploads+=("nxf_s3_upload 'where_are_my_files.txt' s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74")
uploads+=("nxf_s3_upload 'A17_scSLAM_S17_R1_001Aligned.sortedByCoord.out.bam.bai' s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74")
nxf_parallel "${uploads[@]}"
}
nxf_main() {
trap on_exit EXIT
trap on_term TERM INT USR1 USR2
NXF_SCRATCH="$(set +u; nxf_mktemp $TMPDIR)"
[[ $NXF_DEBUG > 0 ]] && nxf_env
echo start | /home/ec2-user/miniconda/bin/aws --region us-west-2 s3 cp --only-show-errors - s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/1d/f5d07b40ecaa54e68fe14308561d74/.command.begin
set +u
set -u
/home/ec2-user/miniconda/bin/aws --region us-west-2 s3 cp --recursive --only-show-errors s3://darmanis-group/shayanhoss/scSLAM/190610_M05295_0286_000000000-G43TH/nextflow-intermediates/tmp/5e/9b7a8f4e6ef82d3a5f53bc4f5f8e55/bin $PWD/nextflow-bin
chmod +x $PWD/nextflow-bin/*
export PATH=$PWD/nextflow-bin:$PATH
[[ $NXF_SCRATCH ]] && echo "nxf-scratch-dir $HOSTNAME:$NXF_SCRATCH" && cd $NXF_SCRATCH
nxf_stage
set +e
local ctmp=$(set +u; nxf_mktemp /dev/shm 2>/dev/null || nxf_mktemp $TMPDIR)
local cout=$ctmp/.command.out; mkfifo $cout
local cerr=$ctmp/.command.err; mkfifo $cerr
tee .command.out < $cout &
tee1=$!
tee .command.err < $cerr >&2 &
tee2=$!
( nxf_launch ) >$cout 2>$cerr &
pid=$!
wait $pid || nxf_main_ret=$?
wait $tee1 $tee2
nxf_unstage
}
$NXF_ENTRY
.command.sh
#!/bin/bash -euo pipefail
STAR --genomeDir STARIndex \
--sjdbGTFfile gencode.v30.annotation.ERCC92.gtf \
--readFilesIn A17_scSLAM_S17_R1_001_val_1.fq.gz A17_scSLAM_S17_R2_001_val_2.fq.gz \
--runThreadN 10 \
--twopassMode Basic \
--outWigType bedGraph \
--outSAMtype BAM SortedByCoordinate --limitBAMsortRAM 85799345920 \
--readFilesCommand zcat \
--runDirPerm All_RWX \
--outFileNamePrefix A17_scSLAM_S17_R1_001
samtools index A17_scSLAM_S17_R1_001Aligned.sortedByCoord.out.bam
This seems to be a heisenbug because then it ran fine now.. not sure what happened
olgabot
on 21 Jun 2019
👀2
😕1
Ah well, thanks for the detailed report. At least if anyone else finds something similar then a pattern may start to emerge..? (probably not though 馃槄)
ewels
on 25 Jun 2019
Was this page helpful?
0 / 5 - 0 ratings
Related issues
mjsteinbaugh
路
5Comments
santiagorevale
路
5Comments
carrere
路
11Comments
ewels
路
7Comments
arnoldliaoILMN
路
11Comments