Candlepin crash after upgrade to 1.22.1

Support
1

Problem:
On registering a new host (at least), candlepin tomcat crashes with a log in audit:
type=ANOM_ABEND msg=audit(1568229368.306:507): auid=4294967295 uid=53 gid=53 ses=4294967295 subj=system_u:system_r:tomcat_t:s0 pid=17703 comm=“http-bio-8443-e” reason=“memory violation” sig=6
type=SERVICE_STOP msg=audit(1568229368.348:508): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg=‘unit=tomcat comm=“systemd” exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed’

Expected outcome:
No crash, host successfully registers

Foreman and Proxy versions:
Foreman 1.22.1
Katello 1.12

Foreman and Proxy plugin versions:

Other relevant data:
[e.g. logs from Foreman and/or the Proxy, modified templates, commands issued, etc]
(for logs, surround with three back-ticks to get proper formatting, e.g.)

type=ANOM_ABEND msg=audit(1568229368.306:507): auid=4294967295 uid=53 gid=53 ses=4294967295 subj=system_u:system_r:tomcat_t:s0 pid=17703 comm="http-bio-8443-e" reason="memory violation" sig=6
type=SERVICE_STOP msg=audit(1568229368.348:508): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=tomcat comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'

Since i’m no selinux specialist, i haven’t the slightest idea how to fix this or even where to start looking.

2

Journal logs when tomcat crashes:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f4717dfda4f, pid=6509, tid=0x00007f469e1e5700
#
# JRE version: OpenJDK Runtime Environment (8.0_222-b10) (build 1.8.0_222-b10)
# Java VM: OpenJDK 64-Bit Server VM (25.222-b10 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libplds4.so+0x1a4f]  PL_HashTableLookupConst+0xf
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /usr/share/tomcat/hs_err_pid6509.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
3

Temporary fix:
yum downgrade java-1.8.0-openjdk-1:1.8.0.222.b10-0.el7_6.x86_64 java-1.8.0-openjdk-headless-1:1.8.0.222.b10-0.el7_6.x86_64

1 Like
4

Seems to then be a java bug and not a foreman 1.22.1 bug - just happened at the same time since i did a full system update at the same time :slight_smile:

1 Like
5

Hello, worked for me, thank you for investigating! :slight_smile:

6

Would you mind reporting this in Red Hat Bugzilla, project Candlepin, component SELinux? The Candlepin team carries its own policy.

7

We had the same problem with 1.21.3. Downgrading fixed it. Thanks to Birkirf.

8
9

Java bug reported.
Mentioned SELinux as well, but since I’m not sure thats the actual problem I opened the report as Java causing the crash. :slight_smile:

https://bugzilla.redhat.com/show_bug.cgi?id=1751967

10

There is a scratch build for 2.6.8 that we intend to include in the next 3.12 stream at http://koji.katello.org/koji/buildinfo?buildID=48386. Should be a drop in and we’d appreciate some testing before we release it.

3 Likes