SNAP建立索引的时候出现问题
2016-01-08 21:49
393 查看
hadoop@Mcnode1:~/cloud/adam/xubo/snap$ ./snap-aligner index ../../down/xubo/GRCH38/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna xubo/GRAH38_hs38dl Welcome to SNAP version 1.0beta.22. Hash table slack 0.300000 Loading FASTA file '../../down/xubo/GRCH38/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna' into memory... FASTA file contained a character that's not a valid base (or N): 'M', full line 'TCACCCCCCACACACACCAAACAMCCCACACAACACACACACACCACACCACACAAACACAAACACACCA'; converting to 'N'. This may happen again, but there will be no more warnings. 76s Saving genome...55s Computing bias table. Bias computation: 100000000 / 3216671450 Bias computation: 200000000 / 3216671450 Bias computation: 300000000 / 3216671450 Bias computation: 400000000 / 3216671450 Bias computation: 500000000 / 3216671450 Bias computation: 600000000 / 3216671450 Bias computation: 700000000 / 3216671450 Bias computation: 800000000 / 3216671450 Bias computation: 900000000 / 3216671450 Bias computation: 1000000000 / 3216671450 Bias computation: 1100000000 / 3216671450 Bias computation: 1200000000 / 3216671450 Bias computation: 1300000000 / 3216671450 Bias computation: 1400000000 / 3216671450 Bias computation: 1500000000 / 3216671450 Bias computation: 1600000000 / 3216671450 Bias computation: 1700000000 / 3216671450 Bias computation: 1800000000 / 3216671450 Bias computation: 1900000000 / 3216671450 Bias computation: 2000000000 / 3216671450 Bias computation: 2100000000 / 3216671450 Bias computation: 2200000000 / 3216671450 Bias computation: 2300000000 / 3216671450 Bias computation: 2400000000 / 3216671450 Bias computation: 2500000000 / 3216671450 Bias computation: 2600000000 / 3216671450 Bias computation: 2700000000 / 3216671450 Bias computation: 2800000000 / 3216671450 Bias computation: 2900000000 / 3216671450 Bias computation: 3000000000 / 3216671450 Bias computation: 3100000000 / 3216671450 Bias computation: 3200000000 / 3216671450 Computed bias table in 139s Allocating memory for hash tables...mmap: Cannot allocate memory SNAP exited with exit code 1 from line 394 of file SNAPLib/BigAlloc.cpp
问题:建立hash table的时候memory不够
第二次运行同样的问题:6G的内存不够来分配hashtable。用top查看大概在90%的使用率。
Hash table slack 0.300000 Loading FASTA file '../../down/xubo/GRCH38/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna' into memory... FASTA file contained a character that's not a valid base (or N): 'M', full line 'TCACCCCCCACACACACCAAACAMCCCACACAACACACACACACCACACCACACAAACACAAACACACCA'; converting to 'N'. This may happen again, but there will be no more warnings. 74s Saving genome...59s Computing bias table. Bias computation: 100000000 / 3216671450 ...... Bias computation: 3200000000 / 3216671450 Computed bias table in 135s Allocating memory for hash tables...mmap: Cannot allocate memory SNAP exited with exit code 1 from line 394 of file SNAPLib/BigAlloc.cpp
数据源大小:
hadoop@Mcnode1:~/cloud/adam/xubo/snap$ ll -h ../../down/xubo/GRCH38/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna -rw------- 1 hadoop hadoop 3.1G 1月 8 00:40 ../../down/xubo/GRCH38/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna
已经建立的部分:
hadoop@Mcnode1:~/cloud/adam/xubo/snap$ du -sh xubo/GRAH38_hs38dl/* 3.0G xubo/GRAH38_hs38dl/Genome hadoop@Mcnode1:~/cloud/adam/xubo/snap$ cd xubo/ datatest/ GRAH38_hs38dl/ index1/ index2/ index3/ index4/ index5/ hadoop@Mcnode1:~/cloud/adam/xubo/snap$ cd xubo/GRAH38_hs38dl/ hadoop@Mcnode1:~/cloud/adam/xubo/snap/xubo/GRAH38_hs38dl$ ls Genome
换了数据后:
hadoop@Mcnode1:~/cloud/adam/xubo/snap$ ./snap-aligner index ../../down/1000genomes/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa index11
Welcome to SNAP version 1.0beta.22.
Hash table slack 0.300000
Loading FASTA file '../../down/1000genomes/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa' into memory...
FASTA file contained a character that's not a valid base (or N): 'M', full line 'TCACCCCCCACACACACCAAACAMCCCACACAACACACACACACCACACCACACAAACACAAACACACCA';
converting to 'N'. This may happen again, but there will be no more warnings.
174s
Saving genome...58s
Computing bias table.
Bias computation: 100000000 / 3219030417
Bias computation: 200000000 / 3219030417
Bias computation: 300000000 / 3219030417
Bias computation: 400000000 / 3219030417
Bias computation: 500000000 / 3219030417
Bias computation: 600000000 / 3219030417
Bias computation: 700000000 / 3219030417
Bias computation: 800000000 / 3219030417
Bias computation: 900000000 / 3219030417
Bias computation: 1000000000 / 3219030417
Bias computation: 1100000000 / 3219030417
Bias computation: 1200000000 / 3219030417
Bias computation: 1300000000 / 3219030417
Bias computation: 1400000000 / 3219030417
Bias computation: 1500000000 / 3219030417
Bias computation: 1600000000 / 3219030417
Bias computation: 1700000000 / 3219030417
Bias computation: 1800000000 / 3219030417
Bias computation: 1900000000 / 3219030417
Bias computation: 2000000000 / 3219030417
Bias computation: 2100000000 / 3219030417
Bias computation: 2200000000 / 3219030417
Bias computation: 2300000000 / 3219030417
Bias computation: 2400000000 / 3219030417
Bias computation: 2500000000 / 3219030417
Bias computation: 2600000000 / 3219030417
Bias computation: 2700000000 / 3219030417
Bias computation: 2800000000 / 3219030417
Bias computation: 2900000000 / 3219030417
Bias computation: 3000000000 / 3219030417
Bias computation: 3100000000 / 3219030417
Bias computation: 3200000000 / 3219030417
Computed bias table in 137s
Allocating memory for hash tables...mmap: Cannot allocate memory
SNAP exited with exit code 1 from line 394 of file SNAPLib/BigAlloc.cpp
相关文章推荐
- WAVEFORMATEX 格式说明
- 仲夏夜--js特效--左右内容的选择
- QGridLayout中multi cell处理
- Byte Alignment
- 微软类库 中文(汉语)转拼音
- JavaFX Script语言教程
- 组件制作之四(定制外观)
- 关于ACE_InputCDR和ACE_OutputCDR读写数据时的经验
- #progma 使用详解
- uboot
- 串行的blast+2.2.25本地化: blastdbcmd,makeblastdb,blastn.简介
- 解析#pragma指令(转)
- 如何修改ovp中button 的状态(ABAP Webdynpro)
- 前期总结之jacob说明
- 经验丰富的开发者总结的网站UI基本设计原则,写的挺好的!
- which would be better served with an electric machine. If your needs are small or moderate
- HOWTO:在 Visual Basic 中自动运行 Microsoft Excel
- The first paper reading——Guttman, R-TREES A DYNAMIC INDEX STRUCTURE FOR SPATIAL SEARCHING
- 2.CreateWindowEx
- 信息部门人员角色划分及任职资格