JVM crash while memcpy during class load

java, jvm, jvm-crash

Solution

we've seen similar errors. our current suspect is jar files which are re-written (by an upgrade process) while the process is running.

Problem

My JVM crashed the and the hs_err file showed that it crashed while attempting to load a class. Specifically while trying to memcpy ([libc.so.6+0x6aa2c] memcpy+0x1c). I looked at the .class file and was able to determine what class was being loaded. But can any one tell me what could cause this or how i could determine more about the cause? If the JVM was out of memory wouldn't it throw an Error. Any insight is greatly appreciated. I've included an excerpt from my hs_err file. ``` # # An unexpected error has been detected by Java Runtime Environment: # # SIGBUS (0x7) at pc=0x005aba2c, pid=20841, tid=2427227056 # # Java VM: Java HotSpot(TM) Client VM (1.6.0_02-b05 mixed mode) # Problematic frame: # C [libc.so.6+0x6aa2c] memcpy+0x1c # # If you would like to submit a bug report, please visit: # http://java.sun.com/webapps/bugreport/crash.jsp # --------------- T H R E A D --------------- Current thread (0x90d0dc00): JavaThread "ORDERHANDLER" [_thread_in_native, id=20881] siginfo:si_signo=7, si_errno=0, si_code=2, si_addr=0x915e3000 Registers: EAX=0x91218298, EBX=0xb7f2e71c, ECX=0x0000079b, EDX=0x915dfef2 ESP=0x90ac6a34, EBP=0x90ac6a60, ESI=0x915e2ffd, EDI=0x914f0a0d EIP=0x005aba2c, CR2=0x915e3000, EFLAGS=0x00010206 Top of Stack: (sp=0x90ac6a34) 0x90ac6a34: b7f29d4b 914ed930 915dff20 00004f49 0x90ac6a44: 082e7bc4 00006f6f 00004243 00004f49 0x90ac6a54: b7f2e71c 080e3e54 00000000 90ac6a90 0x90ac6a64: b7f29fbb 080e3b00 080e3e54 00000000 0x90ac6a74: 00000000 90d0dc00 00000000 d68dd1b6 0x90ac6a84: b7f2e71c 90ac6ad8 90d0dcec 90ac6f00 0x90ac6a94: b7f21169 080e3b00 90ac6ad8 0000002b 0x90ac6aa4: 0000002b 90ac6ad8 00000008 00000000 Instructions: (pc=0x005aba2c) 0x005aba1c: 8b 74 24 08 fc d1 e9 73 01 a4 d1 e9 73 02 66 a5 0x005aba2c: f3 a5 89 c7 89 d6 8b 44 24 04 c3 90 90 90 90 90 Stack: [0x90a78000,0x90ac9000), sp=0x90ac6a34, free space=314k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [libc.so.6+0x6aa2c] memcpy+0x1c C [libzip.so+0xbfbb] ZIP_GetEntry+0x10b C [libzip.so+0x3169] Java_java_util_zip_ZipFile_getEntry+0xc9 J java.util.zip.ZipFile.getEntry(JLjava/lang/String;Z)J J java.util.zip.ZipFile.getEntry(Ljava/lang/String;)Ljava/util/zip/ZipEntry; J java.util.jar.JarFile.getEntry(Ljava/lang/String;)Ljava/util/zip/ZipEntry; J sun.misc.URLClassPath$JarLoader.getResource(Ljava/lang/String;Z)Lsun/misc/Resource; J sun.misc.URLClassPath.getResource(Ljava/lang/String;Z)Lsun/misc/Resource; J java.net.URLClassLoader$1.run()Ljava/lang/Object; v ~StubRoutines::call_stub V [libjvm.so+0x20bbbd] V [libjvm.so+0x30a6b8] V [libjvm.so+0x20ba50] V [libjvm.so+0x26190b] C [libjava.so+0xaa5c] Java_java_security_AccessController_doPrivileged__Ljava_security_PrivilegedExceptionAction_2Ljava_security_AccessControlContext_2+0x3 c J java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;Ljava/security/AccessControlContext;)Ljava/lang/Object; J java.net.URLClassLoader.findClass(Ljava/lang/String;)Ljava/lang/Class; J java.lang.ClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class; J sun.misc.Launcher$AppClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class; j java.lang.ClassLoader.loadClass(Ljava/lang/String;)Ljava/lang/Class;+3 j java.lang.ClassLoader.loadClassInternal(Ljava/lang/String;)Ljava/lang/Class;+2 v ~StubRoutines::call_stub V [libjvm.so+0x20bbbd] V [libjvm.so+0x30a6b8] V [libjvm.so+0x20b6e1] V [libjvm.so+0x20b7ca] V [libjvm.so+0x367621] V [libjvm.so+0x3662a5] V [libjvm.so+0x365357] V [libjvm.so+0x365112] V [libjvm.so+0x1adb03] V [libjvm.so+0x1aeb32] V [libjvm.so+0x2d75cb] V [libjvm.so+0x2d8a94] V [libjvm.so+0x2d8a17] V [libjvm.so+0x1fe7f8] j com.aqua.foms.book.OrderHandler.handleNewOrder(Lcom/aqua/NmsApi/OrderMap;Lcom/aqua/api/AtsMessage;)V+221 j com.aqua.foms.book.FMSNewOrderTask.execute()V+12 j com.aqua.api.EEDefaultWorkerThread.run()V+96 v ~StubRoutines::call_stub V [libjvm.so+0x20bbbd] V [libjvm.so+0x30a6b8] V [libjvm.so+0x20b4d0] V [libjvm.so+0x20b55d] V [libjvm.so+0x27b795] V [libjvm.so+0x383ef0] V [libjvm.so+0x30b5a9] C [libpthread.so.0+0x5371] Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) J java.util.zip.ZipFile.getEntry(JLjava/lang/String;Z)J J java.util.zip.ZipFile.getEntry(Ljava/lang/String;)Ljava/util/zip/ZipEntry; J java.util.jar.JarFile.getEntry(Ljava/lang/String;)Ljava/util/zip/ZipEntry; J sun.misc.URLClassPath$JarLoader.getResource(Ljava/lang/String;Z)Lsun/misc/Resource; J sun.misc.URLClassPath.getResource(Ljava/lang/String;Z)Lsun/misc/Resource; J java.net.URLClassLoader$1.run()Ljava/lang/Object; v ~StubRoutines::call_stub J java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;Ljava/security/AccessControlContext;)Ljava/lang/Object; J java.net.URLClassLoader.findClass(Ljava/lang/String;)Ljava/lang/Class; J java.lang.ClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class; J sun.misc.Launcher$AppClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class; j java.lang.ClassLoader.loadClass(Ljava/lang/String;)Ljava/lang/Class;+3 j java.lang.ClassLoader.loadClassInternal(Ljava/lang/String;)Ljava/lang/Class;+2 v ~StubRoutines::call_stub j com.aqua.foms.book.OrderHandler.handleNewOrder(Lcom/aqua/NmsApi/OrderMap;Lcom/aqua/api/AtsMessage;)V+221 j com.aqua.foms.book.FMSNewOrderTask.execute()V+12 j com.aqua.api.EEDefaultWorkerThread.run()V+96 v ~StubRoutines::call_stub --------------- P R O C E S S --------------- Java Threads: ( => current thread ) 0x080c9c00 JavaThread "pool-1-thread-3" [_thread_blocked, id=18725] 0x0824f800 JavaThread "pool-1-thread-2" [_thread_blocked, id=13693] 0x91217c00 JavaThread "AquaSchedulerService_7" daemon [_thread_blocked, id=23675] 0x91215c00 JavaThread "AquaSchedulerService_6" daemon [_thread_blocked, id=23001] 0x91215400 JavaThread "AquaSchedulerService_5" daemon [_thread_blocked, id=22759] 0x91213400 JavaThread "AquaSchedulerService_4" daemon [_thread_blocked, id=22410] 0x91212c00 JavaThread "AquaSchedulerService_3" daemon [_thread_blocked, id=22262] 0x08316400 JavaThread "pool-1-thread-1" [_thread_blocked, id=22260] 0x0827d000 JavaThread "JmsConn_1_sender_0" daemon [_thread_blocked, id=21196] 0x90d0cc00 JavaThread "Timer-0" [_thread_blocked, id=20882] =>0x90d0dc00 JavaThread "ORDERHANDLER" [_thread_in_native, id=20881] 0x90d0d400 JavaThread "TradeInviteMonitor" [_thread_blocked, id=20880] 0x90d09c00 JavaThread "ROUTERT" [_thread_blocked, id=20878] 0x90d09000 JavaThread "TIBCO EMS Session Dispatcher (33197)" [_thread_blocked, id=20877] 0x08310800 JavaThread "DORDERHANDLER" [_thread_blocked, id=20874] 0x90d01c00 JavaThread "Thread-12" daemon [_thread_blocked, id=20873] 0x90d03000 JavaThread "Thread-11" daemon [_thread_in_native, id=20872] 0x082e1c00 JavaThread "DELAYEDORDMON" [_thread_blocked, id=20871] 0x082e8000 JavaThread "DBUPD" [_thread_blocked, id=20870] 0x914e5000 JavaThread "pool-2-thread-1" [_thread_blocked, id=20869] 0x914e3c00 JavaThread "StatusStatsEventDispatcherThread" [_thread_blocked, id=20868] 0x082c8400 JavaThread "TimerQueue" daemon [_thread_blocked, id=20866] 0x082ca000 JavaThread "MDATATHREAD" [_thread_blocked, id=20865] 0x082c9400 JavaThread "AquaSchedulerService_2" daemon [_thread_blocked, id=20864] 0x9122b000 JavaThread "DestroyJavaVM" [_thread_blocked, id=20843] 0x91200800 JavaThread "FirmMatchingServer" [_thread_blocked, id=20863] 0x914de800 JavaThread "TIBCO EMS TCPLink Reader (32084)" daemon [_thread_in_native, id=20861] 0x9122a400 JavaThread "TIBCO EMS Connections Pinger" daemon [_thread_blocked, id=20859] 0x914d4000 JavaThread "WDISTQ" [_thread_blocked, id=20858] 0x9121f400 JavaThread "JmsConn_1_connector_0" daemon [_thread_blocked, id=20857] 0x914d8000 JavaThread "JmsConn_1_receiver_0" daemon [_thread_blocked, id=20856] 0x9149ac00 JavaThread "AquaSchedulerService_1" daemon [_thread_blocked, id=20855] 0x9149b400 JavaThread "AquaSchedulerService_0" daemon [_thread_blocked, id=20854] 0x9142a000 JavaThread "MySQL Statement Cancellation Timer" daemon [_thread_blocked, id=20852] 0x91425c00 JavaThread "Dispatcher-Thread-0" daemon [_thread_blocked, id=20851] 0x080bf800 JavaThread "Low Memory Detector" daemon [_thread_blocked, id=20849] 0x080bdc00 JavaThread "CompilerThread0" daemon [_thread_blocked, id=20848] 0x080bcc00 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=20847] 0x080a9800 JavaThread "Finalizer" daemon [_thread_blocked, id=20846] 0x080a8800 JavaThread "Reference Handler" daemon [_thread_blocked, id=20845] Other Threads: 0x080a5400 VMThread [id=20844] 0x080c1000 WatcherThread [id=20850] VM state:not at safepoint (normal execution) VM Mutex/Monitor currently owned by a thread: None ```

Original source