Native memory allocation (mmap) failed to map 142606336 bytes for committing reserved memory.

这里写链接内容

问题描述

Java程序运行过程中抛出java.lang.OutOfMemoryError: unable to create new native thread，如下所示：

[java] view plain copy

java.lang.OutOfMemoryError: unable to create new native thread

at java.lang.Thread.start0(Native Method)

at java.lang.Thread.start(Thread.java:691)

at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)

at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1017)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1163)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)

at java.lang.Thread.run(Thread.java:722)

[java] view plain copy

Caused by: java.lang.OutOfMemoryError

at java.util.zip.ZipFile.open(Native Method)

at java.util.zip.ZipFile.(ZipFile.java:214)

at java.util.zip.ZipFile.(ZipFile.java:144)

at java.util.jar.JarFile.(JarFile.java:153)

at java.util.jar.JarFile.(JarFile.java:117)

从JVM层面去解决

减小thread stack的大小

JVM默认thread stack的大小为1024，这样当线程多时导致Native virtual memory被耗尽，实际上当thread stack的大小为128K 或 256K时是足够的，所以我们如果明确指定thread stack为128K 或 256K即可，具体使用-Xss，例如在JVM启动的JVM_OPT中添加如下配置

[java] view plain copy

-Xss128k

减小heap或permgen初始分配的大小

如果JVM启动的JVM_OPT中有如下配置

[java] view plain copy

-Xms1303m -Xmx1303m -XX:PermSize=256m -XX:MaxPermSize=256m

我们可以删除或减小初始化最小值的配置，如下

[java] view plain copy

-Xms256m -Xmx1303m -XX:PermSize=64m -XX:MaxPermSize=256m

[java] view plain copy

-Xmx1303m -XX:MaxPermSize=256m

升级JVM到最新的版本

最新版本的JVM一般在内存优化方面做的更好，升级JVM到最新的版本可能会缓解测问题

从操作系统层面去解决

使用64位操作系统

如果使用32位操作系统遇到unable to create new native thread，建议使用64位操作系统

增大OS对线程的限制

在Linux操作系统设定nofile和nproc，具体编辑/etc/security/limits.conf添加如下：

[html] view plain copy

soft nofile 2048

hard nofile 8192

[html] view plain copy

soft nproc 2048

hard nproc 8192

如果使用Red Hat Enterprise Linux 6，编辑/etc/security/limits.d/90-nproc.conf，添加如下配置：

[html] view plain copy

cat /etc/security/limits.d/90-nproc.conf

soft nproc 1024

root soft nproc unlimited

user - nproc 2048

这里写链接内容

JVM Crash抛出如下信息：

[java] view plain copy

There is insufficient memory for the Java Runtime Environment to continue.

Native memory allocation (malloc) failed to allocate 813056 bytes for Chunk::new

An error report file with more information is saved as:

/home/kylin/work/brms/brms-standalone-5.3.1/jboss-as/bin/hs_err_pid26819.log

hs_err_pid26819.log文件内容如下：

[java] view plain copy

There is insufficient memory for the Java Runtime Environment to continue.

Native memory allocation (malloc) failed to allocate 813056 bytes for Chunk::new

Possible reasons:

The system is out of physical RAM or swap space

In 32 bit mode, the process size limit was hit

Possible solutions:

Reduce memory load on the system

Increase physical memory or swap space

Check if swap backing store is full

Use 64 bit Java on a 64 bit OS

Decrease Java heap size (-Xmx/-Xms)

Decrease number of Java threads

Decrease Java thread stack sizes (-Xss)

Set larger code cache with -XX:ReservedCodeCacheSize=

This output file may be truncated or incomplete.

Out of Memory Error (allocation.cpp:328), pid=26819, tid=1286601584

JRE version: 7.0_21-b11

Java VM: Java HotSpot(TM) Server VM (23.21-b01 mixed mode linux-x86 )

Failed to write core dump. Core dumps have been disabled. To enable core dumping, try “ulimit -c unlimited” before starting Java again

————— T H R E A D —————

Current thread (0x4cff4c00): JavaThread “C2 CompilerThread0” daemon [_thread_in_native, id=26829, stack(0x4ca7f000,0x4cb00000)]

Stack: [0x4ca7f000,0x4cb00000], sp=0x4cafd9a0, free space=506k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

V [libjvm.so+0x722359] VMError::report_and_die()+0x199

V [libjvm.so+0x2e8ef2] report_vm_out_of_memory(char const*, int, unsigned int, char const*)+0x72

V [libjvm.so+0x14ddad] Chunk::operator new(unsigned int, unsigned int)+0x10d

V [libjvm.so+0x14dde9] Arena::grow(unsigned int)+0x29

V [libjvm.so+0x3d5fe2] PhaseIFG::init(unsigned int)+0x1c2

V [libjvm.so+0x23835c] PhaseChaitin::Register_Allocate()+0x75c

V [libjvm.so+0x2a2991] Compile::Code_Gen()+0x3b1

V [libjvm.so+0x2a5250] Compile::Compile(ciEnv*, C2Compiler*, ciMethod*, int, bool, bool)+0xbd0

V [libjvm.so+0x220f16] C2Compiler::compile_method(ciEnv*, ciMethod*, int)+0x176

V [libjvm.so+0x2a9f6a] CompileBroker::invoke_compiler_on_method(CompileTask*)+0x33a

V [libjvm.so+0x2ab037] CompileBroker::compiler_thread_loop()+0x417

V [libjvm.so+0x6ddd98] compiler_thread_entry(JavaThread*, Thread*)+0x18

V [libjvm.so+0x6e4944] JavaThread::thread_main_inner()+0xf4

V [libjvm.so+0x6e4ad1] JavaThread::run()+0x161

V [libjvm.so+0x5e1091] java_start(Thread*)+0x111

C [libpthread.so.0+0x6a2e] abort@@GLIBC_2.0+0x6a2e

这里写链接内容

阿里云 Java环境 每小时在根目录下生成 hs_err_pid*.log ，求教！

错误日志

OS环境：阿里云 1核 1G内存；

安装软件：安装了 JDK 1.7 TOMCAT 7.X；

部署内容：部署了 java web 工程，工程正常运行；

问题：每小时在跟目录下生成一个 hs_err_pid.log

已经采取的措施：

1、配置服务器Swap

2、配置tomcat jvm 内存 （bin/catalina.sh）

未果，求牛人指点。

文件内容如下：

#

There is insufficient memory for the Java Runtime Environment to continue.

pthread_getattr_np

Possible reasons:

The system is out of physical RAM or swap space

In 32 bit mode, the process size limit was hit

Possible solutions:

Reduce memory load on the system

Increase physical memory or swap space

Check if swap backing store is full

Use 64 bit Java on a 64 bit OS

Decrease Java heap size (-Xmx/-Xms)

Decrease number of Java threads

Decrease Java thread stack sizes (-Xss)

Set larger code cache with -XX:ReservedCodeCacheSize=

This output file may be truncated or incomplete.

#

Out of Memory Error (os_linux_x86.cpp:718), pid=29229, tid=140505843455744

#

JRE version: Java(TM) SE Runtime Environment (7.0_67-b01) (build 1.7.0_67-b01)

Java VM: Java HotSpot(TM) 64-Bit Server VM (24.65-b04 mixed mode linux-amd64 compressed oops)

Failed to write core dump. Core dumps have been disabled. To enable core dumping, try “ulimit -c unlimited” before starting Java again

#

————— T H R E A D —————

Current thread (0x00007fca0c08a800): JavaThread “Service Thread” daemon [_thread_new, id=29237, stack(0x0000000000000000,0x0000000000000000)]

Stack: [0x0000000000000000,0x0000000000000000], sp=0x00007fca10e34880, free space=137212737746k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

V [libjvm.so+0x99eb8a]

V [libjvm.so+0x49721b]

V [libjvm.so+0x8237ca]

V [libjvm.so+0x823815]

V [libjvm.so+0x952614]

V [libjvm.so+0x958dd4]

V [libjvm.so+0x81f988]

————— P R O C E S S —————

Java Threads: ( => current thread )

=>0x00007fca0c08a800 JavaThread “Service Thread” daemon [_thread_new, id=29237, stack(0x0000000000000000,0x0000000000000000)]

0x00007fca0c088000 JavaThread “C2 CompilerThread1” daemon [_thread_blocked, id=29236, stack(0x00007fca10e36000,0x00007fca10f37000)]

0x00007fca0c085800 JavaThread “C2 CompilerThread0” daemon [_thread_blocked, id=29235, stack(0x00007fca10f37000,0x00007fca11038000)]

0x00007fca0c084000 JavaThread “Signal Dispatcher” daemon [_thread_blocked, id=29234, stack(0x00007fca11038000,0x00007fca11139000)]

0x00007fca0c064800 JavaThread “Finalizer” daemon [_thread_blocked, id=29233, stack(0x00007fca11139000,0x00007fca1123a000)]

0x00007fca0c062800 JavaThread “Reference Handler” daemon [_thread_blocked, id=29232, stack(0x00007fca1123a000,0x00007fca1133b000)]

0x00007fca0c008800 JavaThread “main” [_thread_in_vm, id=29230, stack(0x00007fca12195000,0x00007fca12296000)]

Other Threads:

0x00007fca0c05e000 VMThread [stack: 0x00007fca1133b000,0x00007fca1143c000] [id=29231]

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: ([mutex/lock_event])

[0x00007fca0c006bc0] PeriodicTask_lock - owner thread: 0x00007fca0c008800

Heap

def new generation total 4800K, used 343K [0x00000000eb400000, 0x00000000eb930000, 0x00000000f0750000)

eden space 4288K, 8% used [0x00000000eb400000, 0x00000000eb455e40, 0x00000000eb830000)

from space 512K, 0% used [0x00000000eb830000, 0x00000000eb830000, 0x00000000eb8b0000)

to space 512K, 0% used [0x00000000eb8b0000, 0x00000000eb8b0000, 0x00000000eb930000)

tenured generation total 10624K, used 0K [0x00000000f0750000, 0x00000000f11b0000, 0x00000000fae00000)

the space 10624K, 0% used [0x00000000f0750000, 0x00000000f0750000, 0x00000000f0750200, 0x00000000f11b0000)

compacting perm gen total 21248K, used 2132K [0x00000000fae00000, 0x00000000fc2c0000, 0x0000000100000000)

the space 21248K, 10% used [0x00000000fae00000, 0x00000000fb015148, 0x00000000fb015200, 0x00000000fc2c0000)

No shared spaces configured.

Card table byte_map: [0x00007fca117bd000,0x00007fca11864000] byte_map_base: 0x00007fca11063000

Polling page: 0x00007fca1332b000

Code Cache [0x00007fca09000000, 0x00007fca09270000, 0x00007fca0c000000)

total_blobs=156 nmethods=0 adapters=126 free_code_cache=48779Kb largest_free_block=49950080

Compilation events (0 events):

No events

GC Heap History (0 events):

No events

Deoptimization events (0 events):

No events

Internal exceptions (1 events):

Event: 0.027 Thread 0x00007fca0c008800 Threw 0x00000000eb410380 at /HUDSON/workspace/7u-2-build-linux-amd64/jdk7u67/1368/hotspot/src/share/vm/prims/jni.cpp:3991

Events (10 events):

Event: 0.058 loading class 0x00007fca11701880

Event: 0.058 loading class 0x00007fca11701880 done

Event: 0.058 loading class 0x00007fca1170df80

Event: 0.058 loading class 0x00007fca1170df80 done

Event: 0.058 loading class 0x00007fca116c8ee0

Event: 0.058 loading class 0x00007fca116c8ee0 done

Event: 0.058 Thread 0x00007fca0c084000 Thread added: 0x00007fca0c084000

Event: 0.058 Thread 0x00007fca0c085800 Thread added: 0x00007fca0c085800

Event: 0.059 Thread 0x00007fca0c088000 Thread added: 0x00007fca0c088000

Event: 0.059 Thread 0x00007fca0c08a800 Thread added: 0x00007fca0c08a800

…..

这里写链接内容

资源暂时不可用错误(Out of memery)完美解决方案

首先说一下环境：

centos6.3

业务环境：weiboyi

情况说明一下：

为了适应公司业务发展需要，公司新购买了一批新机器，作为weiboyi的新环境，安装了比较新的操作系统：通统：centos6.3

一开始硬件（Dell R410）主板坏了，更换后再进行各种测试。

测试前是用root用户启动业务，没有什么问题。但是后来在用非root用户启动时出现种问题。java报如下错误：

There is insufficient memory for the Java Runtime Environment to continue.

Cannot create GC thread. Out of system resources.

Possible reasons:

The system is out of physical RAM or swap space

In 32 bit mode, the process size limit was hit

Possible solutions:

Reduce memory load on the system

Increase physical memory or swap space

Check if swap backing store is full

Use 64 bit Java on a 64 bit OS

Decrease Java heap size (-Xmx/-Xms)

Decrease number of Java threads

Decrease Java thread stack sizes (-Xss)

Set larger code cache with -XX:ReservedCodeCacheSize=

This output file may be truncated or incomplete.

#

Out of Memory Error (gcTaskThread.cpp:46), pid=50128, tid=140649625663232

同时运行java -version也报错

执行一个ls也会报资源暂时不足

害得我又是查看系统的/etc/security/limit.conf 又是搞/etc/sysctl.conf 然后换JDK版本等等，搞得精疲力尽，到处找相关的资料！

搞了几天没有搞定，今天再接着进行这方面的研究，在以前的生产环境：centos5.5下可以正常使用的系统参数时突然发现：新系统的max user processes 只有1024 而旧系统默认为278528

你M呀，问题是出在这里了，再在/etc/security/下一看。centos6多出来一个limits.d目录，下面有个文件： 90-nproc.config

此文件内容：

Default limit for number of user’s processes to prevent

accidental fork bombs.

See rhbz #432903 for reasoning.

soft nproc 1024

这里限制了1024呀，果断注释。

再运行业务程序，问题解决！汗一下，centos6加这个搞什么呢，没有搞清楚！