So, I collected the following logs with the process:
- All packages up to date, except candlepin and candlepin-selinux on 4.4.10
- Foreman on 3.11.1 and Katello on 4.13.1
- Everything running without issues right now
- Doing the
dnf update
to update candlepin
The following messages might have occured while doing the update or right after.
==> /var/log/tomcat/catalina.2024-07-26.log <==
26-Jul-2024 13:37:31.799 INFO [Catalina-utility-1] org.apache.catalina.startup.HostConfig.reload Reloading context [/candlepin]
26-Jul-2024 13:37:31.799 INFO [Catalina-utility-1] org.apache.catalina.core.StandardContext.reload Reloading Context with name [/candlepin] has started
26-Jul-2024 13:37:32.341 SEVERE [Catalina-utility-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [candlepin] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@3d3835c2]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@4c9a69cc]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
26-Jul-2024 13:37:32.342 SEVERE [Catalina-utility-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [candlepin] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@409d58d3]) and a value of type [org.hibernate.internal.SessionImpl] (value [SessionImpl(1921267161<closed>)]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
26-Jul-2024 13:37:32.342 SEVERE [Catalina-utility-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [candlepin] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@e6e886e]) and a value of type [liquibase.SingletonScopeManager] (value [liquibase.SingletonScopeManager@11255466]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
26-Jul-2024 13:37:32.342 SEVERE [Catalina-utility-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [candlepin] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@3d3835c2]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@3bac8789]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
26-Jul-2024 13:37:32.350 SEVERE [Catalina-utility-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [candlepin] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@4147da62]) and a value of type [org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl] (value [OperationContextImpl [1592894089] [minimalStore=9223372036854775807, storeLineUp=1, stored=1, minimalReplicated=9223372036854775807, replicationLineUp=0, replicated=0, paged=0, minimalPage=9223372036854775807, pageLineUp=0, errorCode=-1, errorMessage=null, executorsPending=0]]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
26-Jul-2024 13:37:32.351 SEVERE [Catalina-utility-1] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [candlepin] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@3d3835c2]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.util.internal.InternalThreadLocalMap@5c032a6c]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
26-Jul-2024 13:37:32.932 INFO [Finalizer] org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading Illegal access: this web application instance has been stopped already. Could not load [io.netty.util.internal.shaded.org.jctools.queues.IndexedQueueSizeUtil]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
java.lang.IllegalStateException: Illegal access: this web application instance has been stopped already. Could not load [io.netty.util.internal.shaded.org.jctools.queues.IndexedQueueSizeUtil]. The following stack trace is thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access.
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForResourceLoading(WebappClassLoaderBase.java:1349)
at org.apache.catalina.loader.WebappClassLoaderBase.checkStateForClassLoading(WebappClassLoaderBase.java:1337)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1174)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1141)
at io.netty.util.internal.shaded.org.jctools.queues.ConcurrentCircularArrayQueue.isEmpty(ConcurrentCircularArrayQueue.java:71)
at io.netty.util.internal.shaded.org.jctools.queues.MpscArrayQueue.isEmpty(MpscArrayQueue.java:207)
at io.netty.buffer.PoolThreadCache.checkCacheMayLeak(PoolThreadCache.java:236)
at io.netty.buffer.PoolThreadCache.free(PoolThreadCache.java:227)
at io.netty.buffer.PoolThreadCache$FreeOnFinalize.finalize(PoolThreadCache.java:500)
at java.base/java.lang.System$2.invokeFinalize(System.java:2301)
at java.base/java.lang.ref.Finalizer.runFinalizer(Finalizer.java:88)
at java.base/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:173)
26-Jul-2024 13:37:35.531 INFO [Catalina-utility-1] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
==> /var/log/candlepin/candlepin.log <==
2024-07-26 13:36:58,995 [thread=https-jsse-nio-127.0.0.1-23443-exec-10] [req=df8eaf9a-ff02-4df9-b5ea-a838589e2797, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/8cc7a859-c991-4bcd-b350-808d703178f1/certificates/serials
2024-07-26 13:36:59,033 [thread=https-jsse-nio-127.0.0.1-23443-exec-10] [req=df8eaf9a-ff02-4df9-b5ea-a838589e2797, org=LUMA, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=38
2024-07-26 13:36:59,075 [thread=https-jsse-nio-127.0.0.1-23443-exec-7] [req=5cae2812-cd5c-4d25-b1c5-fc8fe8b203fd, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/status
2024-07-26 13:36:59,076 [thread=https-jsse-nio-127.0.0.1-23443-exec-7] [req=5cae2812-cd5c-4d25-b1c5-fc8fe8b203fd, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=1
2024-07-26 13:36:59,117 [thread=https-jsse-nio-127.0.0.1-23443-exec-3] [req=b63962f3-1b36-4cfd-bf0d-c051d7cfc7cc, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/8cc7a859-c991-4bcd-b350-808d703178f1/accessible_content
2024-07-26 13:36:59,127 [thread=https-jsse-nio-127.0.0.1-23443-exec-3] [req=b63962f3-1b36-4cfd-bf0d-c051d7cfc7cc, org=LUMA, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=304, content-type="application/json", time=10
2024-07-26 13:36:59,169 [thread=https-jsse-nio-127.0.0.1-23443-exec-10] [req=60613b6a-ac9c-46b7-ae31-90fe09f70d07, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/8cc7a859-c991-4bcd-b350-808d703178f1/content_overrides
2024-07-26 13:36:59,178 [thread=https-jsse-nio-127.0.0.1-23443-exec-10] [req=60613b6a-ac9c-46b7-ae31-90fe09f70d07, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=9
2024-07-26 13:37:11,420 [thread=https-jsse-nio-127.0.0.1-23443-exec-7] [req=ee7bb45c-40c5-46b8-b6a5-0d909d5d3cf3, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/status
2024-07-26 13:37:11,423 [thread=https-jsse-nio-127.0.0.1-23443-exec-7] [req=ee7bb45c-40c5-46b8-b6a5-0d909d5d3cf3, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=3
2024-07-26 13:37:11,459 [thread=https-jsse-nio-127.0.0.1-23443-exec-4] [req=6e396a94-e2b1-40d4-a750-9d44bd35f1a2, org=, csid=55a250e7-9845-4bc1-9131-cf0e65d9e6bf] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/4ae71432-e8c9-4554-b06a-9d41a417bd0c
2024-07-26 13:37:11,499 [thread=https-jsse-nio-127.0.0.1-23443-exec-4] [req=6e396a94-e2b1-40d4-a750-9d44bd35f1a2, org=LUMA, csid=55a250e7-9845-4bc1-9131-cf0e65d9e6bf] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=40
2024-07-26 13:37:11,560 [thread=https-jsse-nio-127.0.0.1-23443-exec-1] [req=c9e7072a-407c-4188-9f9d-349dc8fb758d, org=, csid=c39f57dc-9b1e-4f3d-87eb-4d90f3622dca] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/4ae71432-e8c9-4554-b06a-9d41a417bd0c
2024-07-26 13:37:11,590 [thread=https-jsse-nio-127.0.0.1-23443-exec-1] [req=c9e7072a-407c-4188-9f9d-349dc8fb758d, org=LUMA, csid=c39f57dc-9b1e-4f3d-87eb-4d90f3622dca] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=30
2024-07-26 13:37:11,631 [thread=https-jsse-nio-127.0.0.1-23443-exec-8] [req=35c2ef02-46aa-462d-accd-14f1c2d0539d, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/4ae71432-e8c9-4554-b06a-9d41a417bd0c/compliance
2024-07-26 13:37:11,671 [thread=https-jsse-nio-127.0.0.1-23443-exec-8] [req=35c2ef02-46aa-462d-accd-14f1c2d0539d, org=LUMA, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=40
2024-07-26 13:37:11,728 [thread=https-jsse-nio-127.0.0.1-23443-exec-6] [req=feb719de-809e-4edc-85f0-4adf16579dab, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/4ae71432-e8c9-4554-b06a-9d41a417bd0c/certificates/serials
2024-07-26 13:37:11,761 [thread=https-jsse-nio-127.0.0.1-23443-exec-6] [req=feb719de-809e-4edc-85f0-4adf16579dab, org=LUMA, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=33
2024-07-26 13:37:11,827 [thread=https-jsse-nio-127.0.0.1-23443-exec-2] [req=3f1f3f9f-ab2e-4b36-9946-e76cc623759e, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/4ae71432-e8c9-4554-b06a-9d41a417bd0c/accessible_content
2024-07-26 13:37:11,845 [thread=https-jsse-nio-127.0.0.1-23443-exec-2] [req=3f1f3f9f-ab2e-4b36-9946-e76cc623759e, org=LUMA, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=304, content-type="application/json", time=18
2024-07-26 13:37:21,057 [thread=https-jsse-nio-127.0.0.1-23443-exec-9] [req=f24a3b6d-3947-4f62-9af0-e99233451181, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/82022984-0258-4d12-abfc-ea01414de566/certificates/serials
2024-07-26 13:37:21,089 [thread=https-jsse-nio-127.0.0.1-23443-exec-9] [req=f24a3b6d-3947-4f62-9af0-e99233451181, org=LUMA, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=32
2024-07-26 13:37:21,134 [thread=https-jsse-nio-127.0.0.1-23443-exec-5] [req=a39de66d-e71c-46f0-bbda-9720f412decf, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/status
2024-07-26 13:37:21,137 [thread=https-jsse-nio-127.0.0.1-23443-exec-5] [req=a39de66d-e71c-46f0-bbda-9720f412decf, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=3
2024-07-26 13:37:21,184 [thread=https-jsse-nio-127.0.0.1-23443-exec-3] [req=a12c5405-5c01-4bc3-abcb-7893d5eb3f67, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/82022984-0258-4d12-abfc-ea01414de566/accessible_content
2024-07-26 13:37:21,197 [thread=https-jsse-nio-127.0.0.1-23443-exec-3] [req=a12c5405-5c01-4bc3-abcb-7893d5eb3f67, org=LUMA, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=304, content-type="application/json", time=13
2024-07-26 13:37:21,265 [thread=https-jsse-nio-127.0.0.1-23443-exec-10] [req=ac531529-d6d0-44e4-9a81-45762c800dc8, org=, csid=5a7bbbcd-28ad-451b-8dd0-34b2da25c627] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/82022984-0258-4d12-abfc-ea01414de566
2024-07-26 13:37:21,284 [thread=https-jsse-nio-127.0.0.1-23443-exec-10] [req=ac531529-d6d0-44e4-9a81-45762c800dc8, org=LUMA, csid=5a7bbbcd-28ad-451b-8dd0-34b2da25c627] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=19
2024-07-26 13:37:21,335 [thread=https-jsse-nio-127.0.0.1-23443-exec-7] [req=69fa9576-9598-4680-a547-769dc961b5e7, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/82022984-0258-4d12-abfc-ea01414de566/content_overrides
2024-07-26 13:37:21,346 [thread=https-jsse-nio-127.0.0.1-23443-exec-7] [req=69fa9576-9598-4680-a547-769dc961b5e7, org=, csid=] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=11
2024-07-26 13:37:23,377 [thread=https-jsse-nio-127.0.0.1-23443-exec-4] [req=4fb2650b-192a-43df-94a4-6b1f883d2674, org=, csid=5d328f9a-2266-4bbe-83f3-8a4d10dce8ac] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/82022984-0258-4d12-abfc-ea01414de566
2024-07-26 13:37:23,415 [thread=https-jsse-nio-127.0.0.1-23443-exec-4] [req=4fb2650b-192a-43df-94a4-6b1f883d2674, org=LUMA, csid=5d328f9a-2266-4bbe-83f3-8a4d10dce8ac] INFO org.candlepin.servlet.filter.logging.LoggingFilter - Response: status=200, content-type="application/json", time=38
2024-07-26 13:37:31,800 [thread=Catalina-utility-1] [=, org=, csid=] INFO org.candlepin.async.JobManager - Shutting down job manager
2024-07-26 13:37:32,160 [thread=Catalina-utility-1] [=, org=, csid=] INFO org.candlepin.async.JobManager - Job manager shut down
2024-07-26 13:37:32,198 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] ERROR org.apache.activemq.artemis.core.client - AMQ214013: Failed to decode packet
java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@241433a1 rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.onAddedTaskIfNotRunning(ProcessorBase.java:199)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.task(ProcessorBase.java:187)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.execute(OrderedExecutor.java:72)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$SessionFactoryTopologyHandler.nodeDisconnected(ClientSessionFactoryImpl.java:1572)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handleDisconnect(ActiveMQClientProtocolManager.java:505)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handlePacket(ActiveMQClientProtocolManager.java:484)
at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:835)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:419)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:392)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1365)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
==> /var/log/candlepin/error.log <==
2024-07-26 13:37:32,198 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] ERROR org.apache.activemq.artemis.core.client - AMQ214013: Failed to decode packet
java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@241433a1 rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.onAddedTaskIfNotRunning(ProcessorBase.java:199)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.task(ProcessorBase.java:187)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.execute(OrderedExecutor.java:72)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$SessionFactoryTopologyHandler.nodeDisconnected(ClientSessionFactoryImpl.java:1572)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handleDisconnect(ActiveMQClientProtocolManager.java:505)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handlePacket(ActiveMQClientProtocolManager.java:484)
at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:835)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:419)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:392)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1365)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
==> /var/log/candlepin/candlepin.log <==
2024-07-26 13:37:32,200 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] ERROR org.apache.activemq.artemis.core.client - AMQ214031: Failed to decode buffer, disconnect immediately.
java.lang.IllegalStateException: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@241433a1 rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:397)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1365)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@241433a1 rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.onAddedTaskIfNotRunning(ProcessorBase.java:199)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.task(ProcessorBase.java:187)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.execute(OrderedExecutor.java:72)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$SessionFactoryTopologyHandler.nodeDisconnected(ClientSessionFactoryImpl.java:1572)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handleDisconnect(ActiveMQClientProtocolManager.java:505)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handlePacket(ActiveMQClientProtocolManager.java:484)
at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:835)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:419)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:392)
... 8 common frames omitted
2024-07-26 13:37:32,201 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] ERROR org.apache.activemq.artemis.core.server - AMQ224027: Failed to write to handler on invm connector org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1@7fba1808
java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler$1@3a52b17b rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1368)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
2024-07-26 13:37:32,202 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] WARN org.apache.activemq.artemis.utils.actors.OrderedExecutor - Failed to write to handler on connector org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1@7fba1808
java.lang.IllegalStateException: Failed to write to handler on connector org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1@7fba1808
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:231)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler$1@3a52b17b rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1368)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
... 6 common frames omitted
==> /var/log/candlepin/error.log <==
2024-07-26 13:37:32,200 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] ERROR org.apache.activemq.artemis.core.client - AMQ214031: Failed to decode buffer, disconnect immediately.
java.lang.IllegalStateException: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@241433a1 rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:397)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1365)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@241433a1 rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.onAddedTaskIfNotRunning(ProcessorBase.java:199)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.task(ProcessorBase.java:187)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.execute(OrderedExecutor.java:72)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$SessionFactoryTopologyHandler.nodeDisconnected(ClientSessionFactoryImpl.java:1572)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handleDisconnect(ActiveMQClientProtocolManager.java:505)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handlePacket(ActiveMQClientProtocolManager.java:484)
at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:835)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:419)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:392)
... 8 common frames omitted
2024-07-26 13:37:32,201 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] ERROR org.apache.activemq.artemis.core.server - AMQ224027: Failed to write to handler on invm connector org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1@7fba1808
java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler$1@3a52b17b rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1368)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
2024-07-26 13:37:32,202 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] WARN org.apache.activemq.artemis.utils.actors.OrderedExecutor - Failed to write to handler on connector org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1@7fba1808
java.lang.IllegalStateException: Failed to write to handler on connector org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1@7fba1808
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:231)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler$1@3a52b17b rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1368)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
... 6 common frames omitted
==> /var/log/candlepin/candlepin.log <==
2024-07-26 13:37:32,208 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] ERROR org.apache.activemq.artemis.core.client - AMQ214013: Failed to decode packet
java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@74ce81dc rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.onAddedTaskIfNotRunning(ProcessorBase.java:199)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.task(ProcessorBase.java:187)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.execute(OrderedExecutor.java:72)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$SessionFactoryTopologyHandler.nodeDisconnected(ClientSessionFactoryImpl.java:1572)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handleDisconnect(ActiveMQClientProtocolManager.java:505)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handlePacket(ActiveMQClientProtocolManager.java:484)
at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:835)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:419)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:392)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1365)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
==> /var/log/candlepin/error.log <==
2024-07-26 13:37:32,208 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] ERROR org.apache.activemq.artemis.core.client - AMQ214013: Failed to decode packet
java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@74ce81dc rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.onAddedTaskIfNotRunning(ProcessorBase.java:199)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.task(ProcessorBase.java:187)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.execute(OrderedExecutor.java:72)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$SessionFactoryTopologyHandler.nodeDisconnected(ClientSessionFactoryImpl.java:1572)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handleDisconnect(ActiveMQClientProtocolManager.java:505)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handlePacket(ActiveMQClientProtocolManager.java:484)
at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:835)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:419)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:392)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1365)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
==> /var/log/candlepin/candlepin.log <==
2024-07-26 13:37:32,208 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] ERROR org.apache.activemq.artemis.core.client - AMQ214031: Failed to decode buffer, disconnect immediately.
java.lang.IllegalStateException: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@74ce81dc rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:397)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1365)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@74ce81dc rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.onAddedTaskIfNotRunning(ProcessorBase.java:199)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.task(ProcessorBase.java:187)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.execute(OrderedExecutor.java:72)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$SessionFactoryTopologyHandler.nodeDisconnected(ClientSessionFactoryImpl.java:1572)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handleDisconnect(ActiveMQClientProtocolManager.java:505)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handlePacket(ActiveMQClientProtocolManager.java:484)
at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:835)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:419)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:392)
... 8 common frames omitted
==> /var/log/candlepin/error.log <==
2024-07-26 13:37:32,208 [thread=Thread-6 (org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnector)] [=, org=, csid=] ERROR org.apache.activemq.artemis.core.client - AMQ214031: Failed to decode buffer, disconnect immediately.
java.lang.IllegalStateException: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@74ce81dc rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:397)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$DelegatingBufferHandler.bufferReceived(ClientSessionFactoryImpl.java:1365)
at org.apache.activemq.artemis.core.remoting.impl.invm.InVMConnection$1.run(InVMConnection.java:223)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$1124/0x00007fedb4aa6b08@74ce81dc rejected from org.apache.activemq.artemis.utils.ActiveMQThreadPoolExecutor@59898202[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 7]
at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2065)
at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:833)
at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1365)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.onAddedTaskIfNotRunning(ProcessorBase.java:199)
at org.apache.activemq.artemis.utils.actors.ProcessorBase.task(ProcessorBase.java:187)
at org.apache.activemq.artemis.utils.actors.OrderedExecutor.execute(OrderedExecutor.java:72)
at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$SessionFactoryTopologyHandler.nodeDisconnected(ClientSessionFactoryImpl.java:1572)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handleDisconnect(ActiveMQClientProtocolManager.java:505)
at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQClientProtocolManager$Channel0Handler.handlePacket(ActiveMQClientProtocolManager.java:484)
at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:835)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:419)
at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:392)
... 8 common frames omitted