Describe the bug
I occasionally encounter JVM hangs when running benchmarks locally. I've identified some non-daemon Java threads, which might be tokio runtime worker threads attached to the JVM.
jstack.log:
"Thread-17" #205 prio=5 os_prio=31 cpu=61330.26ms elapsed=785.90s tid=0x00000007eeb59800 nid=0x14e03 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Thread-18" #206 prio=5 os_prio=31 cpu=60816.99ms elapsed=785.90s tid=0x00000007eeb58c00 nid=0x10b03 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Thread-19" #207 prio=5 os_prio=31 cpu=61118.53ms elapsed=785.90s tid=0x00000007eeb58600 nid=0x10a03 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
"Thread-20" #208 prio=5 os_prio=31 cpu=61453.10ms elapsed=785.90s tid=0x00000007eeb59200 nid=0x10c03 runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
sample.txt:
3600 Thread_4540969: tokio-rt-worker
+ 3600 thread_start (in libsystem_pthread.dylib) + 8 [0x184d84c1c]
+ 3600 _pthread_start (in libsystem_pthread.dylib) + 136 [0x184d89c58]
+ 3600 _RNvNvMs0_NtNtNtCsaLOjE9VYtxK_3std3sys6thread4unixNtB7_6Thread3new12thread_start (in libcomet-10312534928326113772.dylib) + 408 [0x13bbf2020]
+ 3600 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h6be6a551dd5ca5c3 (in libcomet-10312534928326113772.dylib) + 76 [0x13b10bdf4]
+ 3600 std::sys::backtrace::__rust_begin_short_backtrace::hcee216148fc0f6e3 (in libcomet-10312534928326113772.dylib) + 584 [0x13b10aee8]
+ 3600 tokio::runtime::task::raw::poll::he4ba3e1a70e3b5b6 (in libcomet-10312534928326113772.dylib) + 440 [0x13b11d080]
+ 3600 tokio::runtime::scheduler::multi_thread::worker::run::h0a3e57d507e45651 (in libcomet-10312534928326113772.dylib) + 4092 [0x13b1276a4]
+ 3600 tokio::runtime::scheduler::multi_thread::worker::Context::park_internal::hd55bed5b56ca7137 (in libcomet-10312534928326113772.dylib) + 236 [0x13b128800]
+ 3600 tokio::runtime::time::Driver::park_internal::h43326194bcf2d527 (in libcomet-10312534928326113772.dylib) + 676 [0x13b1208cc]
+ 3600 tokio::runtime::io::driver::Driver::turn::h57d24de64565a0a9 (in libcomet-10312534928326113772.dylib) + 392 [0x13b11a0ac]
+ 3600 mio::poll::Poll::poll::hf5c244c71fcac8f5 (in libcomet-10312534928326113772.dylib) + 116 [0x13b12fdb8]
+ 3600 kevent (in libsystem_kernel.dylib) + 8 [0x184d4bfc4]
3600 Thread_4540970: tokio-rt-worker
+ 3600 thread_start (in libsystem_pthread.dylib) + 8 [0x184d84c1c]
+ 3600 _pthread_start (in libsystem_pthread.dylib) + 136 [0x184d89c58]
+ 3600 _RNvNvMs0_NtNtNtCsaLOjE9VYtxK_3std3sys6thread4unixNtB7_6Thread3new12thread_start (in libcomet-10312534928326113772.dylib) + 408 [0x13bbf2020]
+ 3600 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h6be6a551dd5ca5c3 (in libcomet-10312534928326113772.dylib) + 76 [0x13b10bdf4]
+ 3600 std::sys::backtrace::__rust_begin_short_backtrace::hcee216148fc0f6e3 (in libcomet-10312534928326113772.dylib) + 584 [0x13b10aee8]
+ 3600 tokio::runtime::task::raw::poll::he4ba3e1a70e3b5b6 (in libcomet-10312534928326113772.dylib) + 440 [0x13b11d080]
+ 3600 tokio::runtime::scheduler::multi_thread::worker::run::h0a3e57d507e45651 (in libcomet-10312534928326113772.dylib) + 4092 [0x13b1276a4]
+ 3600 tokio::runtime::scheduler::multi_thread::worker::Context::park_internal::hd55bed5b56ca7137 (in libcomet-10312534928326113772.dylib) + 428 [0x13b1288c0]
+ 3600 tokio::runtime::scheduler::multi_thread::park::Inner::park_condvar::hd7068c61b246bc1c (in libcomet-10312534928326113772.dylib) + 320 [0x13b125e30]
+ 3600 _pthread_cond_wait (in libsystem_pthread.dylib) + 980 [0x184d8a128]
+ 3600 __psynch_cvwait (in libsystem_kernel.dylib) + 8 [0x184d4950c]
3600 Thread_4540971: tokio-rt-worker
+ 3600 thread_start (in libsystem_pthread.dylib) + 8 [0x184d84c1c]
+ 3600 _pthread_start (in libsystem_pthread.dylib) + 136 [0x184d89c58]
+ 3600 _RNvNvMs0_NtNtNtCsaLOjE9VYtxK_3std3sys6thread4unixNtB7_6Thread3new12thread_start (in libcomet-10312534928326113772.dylib) + 408 [0x13bbf2020]
+ 3600 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h6be6a551dd5ca5c3 (in libcomet-10312534928326113772.dylib) + 76 [0x13b10bdf4]
+ 3600 std::sys::backtrace::__rust_begin_short_backtrace::hcee216148fc0f6e3 (in libcomet-10312534928326113772.dylib) + 584 [0x13b10aee8]
+ 3600 tokio::runtime::task::raw::poll::he4ba3e1a70e3b5b6 (in libcomet-10312534928326113772.dylib) + 440 [0x13b11d080]
+ 3600 tokio::runtime::scheduler::multi_thread::worker::run::h0a3e57d507e45651 (in libcomet-10312534928326113772.dylib) + 4092 [0x13b1276a4]
+ 3600 tokio::runtime::scheduler::multi_thread::worker::Context::park_internal::hd55bed5b56ca7137 (in libcomet-10312534928326113772.dylib) + 428 [0x13b1288c0]
+ 3600 tokio::runtime::scheduler::multi_thread::park::Inner::park_condvar::hd7068c61b246bc1c (in libcomet-10312534928326113772.dylib) + 320 [0x13b125e30]
+ 3600 _pthread_cond_wait (in libsystem_pthread.dylib) + 980 [0x184d8a128]
+ 3600 __psynch_cvwait (in libsystem_kernel.dylib) + 8 [0x184d4950c]
3600 Thread_4540972: tokio-rt-worker
+ 3600 thread_start (in libsystem_pthread.dylib) + 8 [0x184d84c1c]
+ 3600 _pthread_start (in libsystem_pthread.dylib) + 136 [0x184d89c58]
+ 3600 _RNvNvMs0_NtNtNtCsaLOjE9VYtxK_3std3sys6thread4unixNtB7_6Thread3new12thread_start (in libcomet-10312534928326113772.dylib) + 408 [0x13bbf2020]
+ 3600 core::ops::function::FnOnce::call_once$u7b$$u7b$vtable.shim$u7d$$u7d$::h6be6a551dd5ca5c3 (in libcomet-10312534928326113772.dylib) + 76 [0x13b10bdf4]
+ 3600 std::sys::backtrace::__rust_begin_short_backtrace::hcee216148fc0f6e3 (in libcomet-10312534928326113772.dylib) + 584 [0x13b10aee8]
+ 3600 tokio::runtime::task::raw::poll::he4ba3e1a70e3b5b6 (in libcomet-10312534928326113772.dylib) + 440 [0x13b11d080]
+ 3600 tokio::runtime::scheduler::multi_thread::worker::run::h0a3e57d507e45651 (in libcomet-10312534928326113772.dylib) + 4092 [0x13b1276a4]
+ 3600 tokio::runtime::scheduler::multi_thread::worker::Context::park_internal::hd55bed5b56ca7137 (in libcomet-10312534928326113772.dylib) + 428 [0x13b1288c0]
+ 3600 tokio::runtime::scheduler::multi_thread::park::Inner::park_condvar::hd7068c61b246bc1c (in libcomet-10312534928326113772.dylib) + 320 [0x13b125e30]
+ 3600 _pthread_cond_wait (in libsystem_pthread.dylib) + 980 [0x184d8a128]
+ 3600 __psynch_cvwait (in libsystem_kernel.dylib) + 8 [0x184d4950c]
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response
Describe the bug
I occasionally encounter JVM hangs when running benchmarks locally. I've identified some non-daemon Java threads, which might be tokio runtime worker threads attached to the JVM.
jstack.log:
sample.txt:
Steps to reproduce
No response
Expected behavior
No response
Additional context
No response