您的位置:首页 > 其它

SNAP建立索引的时候出现问题

2016-01-08 21:49 393 查看
hadoop@Mcnode1:~/cloud/adam/xubo/snap$ ./snap-aligner index ../../down/xubo/GRCH38/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna xubo/GRAH38_hs38dl
Welcome to SNAP version 1.0beta.22.

Hash table slack 0.300000
Loading FASTA file '../../down/xubo/GRCH38/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna' into memory...
FASTA file contained a character that's not a valid base (or N): 'M', full line 'TCACCCCCCACACACACCAAACAMCCCACACAACACACACACACCACACCACACAAACACAAACACACCA';
converting to 'N'.  This may happen again, but there will be no more warnings.
76s
Saving genome...55s
Computing bias table.
Bias computation: 100000000 / 3216671450
Bias computation: 200000000 / 3216671450
Bias computation: 300000000 / 3216671450
Bias computation: 400000000 / 3216671450
Bias computation: 500000000 / 3216671450
Bias computation: 600000000 / 3216671450
Bias computation: 700000000 / 3216671450
Bias computation: 800000000 / 3216671450
Bias computation: 900000000 / 3216671450
Bias computation: 1000000000 / 3216671450
Bias computation: 1100000000 / 3216671450
Bias computation: 1200000000 / 3216671450
Bias computation: 1300000000 / 3216671450
Bias computation: 1400000000 / 3216671450
Bias computation: 1500000000 / 3216671450
Bias computation: 1600000000 / 3216671450
Bias computation: 1700000000 / 3216671450
Bias computation: 1800000000 / 3216671450
Bias computation: 1900000000 / 3216671450
Bias computation: 2000000000 / 3216671450
Bias computation: 2100000000 / 3216671450
Bias computation: 2200000000 / 3216671450
Bias computation: 2300000000 / 3216671450
Bias computation: 2400000000 / 3216671450
Bias computation: 2500000000 / 3216671450
Bias computation: 2600000000 / 3216671450
Bias computation: 2700000000 / 3216671450
Bias computation: 2800000000 / 3216671450
Bias computation: 2900000000 / 3216671450
Bias computation: 3000000000 / 3216671450
Bias computation: 3100000000 / 3216671450
Bias computation: 3200000000 / 3216671450
Computed bias table in 139s
Allocating memory for hash tables...mmap: Cannot allocate memory
SNAP exited with exit code 1 from line 394 of file SNAPLib/BigAlloc.cpp


问题:建立hash table的时候memory不够

第二次运行同样的问题:6G的内存不够来分配hashtable。用top查看大概在90%的使用率。

Hash table slack 0.300000
Loading FASTA file '../../down/xubo/GRCH38/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna' into memory...
FASTA file contained a character that's not a valid base (or N): 'M', full line 'TCACCCCCCACACACACCAAACAMCCCACACAACACACACACACCACACCACACAAACACAAACACACCA';
converting to 'N'. This may happen again, but there will be no more warnings.
74s
Saving genome...59s
Computing bias table.
Bias computation: 100000000 / 3216671450
......
Bias computation: 3200000000 / 3216671450
Computed bias table in 135s
Allocating memory for hash tables...mmap: Cannot allocate memory
SNAP exited with exit code 1 from line 394 of file SNAPLib/BigAlloc.cpp


数据源大小:

hadoop@Mcnode1:~/cloud/adam/xubo/snap$ ll -h ../../down/xubo/GRCH38/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna
-rw------- 1 hadoop hadoop 3.1G  1月  8 00:40 ../../down/xubo/GRCH38/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna


已经建立的部分:

hadoop@Mcnode1:~/cloud/adam/xubo/snap$ du -sh  xubo/GRAH38_hs38dl/*
3.0G	xubo/GRAH38_hs38dl/Genome
hadoop@Mcnode1:~/cloud/adam/xubo/snap$ cd xubo/
datatest/      GRAH38_hs38dl/ index1/        index2/        index3/        index4/        index5/
hadoop@Mcnode1:~/cloud/adam/xubo/snap$ cd xubo/GRAH38_hs38dl/
hadoop@Mcnode1:~/cloud/adam/xubo/snap/xubo/GRAH38_hs38dl$ ls
Genome


换了数据后:

hadoop@Mcnode1:~/cloud/adam/xubo/snap$ ./snap-aligner index ../../down/1000genomes/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa index11
Welcome to SNAP version 1.0beta.22.

Hash table slack 0.300000
Loading FASTA file '../../down/1000genomes/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa' into memory...
FASTA file contained a character that's not a valid base (or N): 'M', full line 'TCACCCCCCACACACACCAAACAMCCCACACAACACACACACACCACACCACACAAACACAAACACACCA';
converting to 'N'. This may happen again, but there will be no more warnings.
174s
Saving genome...58s
Computing bias table.
Bias computation: 100000000 / 3219030417
Bias computation: 200000000 / 3219030417
Bias computation: 300000000 / 3219030417
Bias computation: 400000000 / 3219030417
Bias computation: 500000000 / 3219030417
Bias computation: 600000000 / 3219030417
Bias computation: 700000000 / 3219030417
Bias computation: 800000000 / 3219030417
Bias computation: 900000000 / 3219030417
Bias computation: 1000000000 / 3219030417
Bias computation: 1100000000 / 3219030417
Bias computation: 1200000000 / 3219030417
Bias computation: 1300000000 / 3219030417
Bias computation: 1400000000 / 3219030417
Bias computation: 1500000000 / 3219030417
Bias computation: 1600000000 / 3219030417
Bias computation: 1700000000 / 3219030417
Bias computation: 1800000000 / 3219030417
Bias computation: 1900000000 / 3219030417
Bias computation: 2000000000 / 3219030417
Bias computation: 2100000000 / 3219030417
Bias computation: 2200000000 / 3219030417
Bias computation: 2300000000 / 3219030417
Bias computation: 2400000000 / 3219030417
Bias computation: 2500000000 / 3219030417
Bias computation: 2600000000 / 3219030417
Bias computation: 2700000000 / 3219030417
Bias computation: 2800000000 / 3219030417
Bias computation: 2900000000 / 3219030417
Bias computation: 3000000000 / 3219030417
Bias computation: 3100000000 / 3219030417
Bias computation: 3200000000 / 3219030417
Computed bias table in 137s
Allocating memory for hash tables...mmap: Cannot allocate memory
SNAP exited with exit code 1 from line 394 of file SNAPLib/BigAlloc.cpp
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签:  alignment SNAP