Hello,
following packages are installed
rpm -qa --last |grep -E "(gutterb|celer|candlep)"
candlepin-selinux-0.9.50-1.el7.noarch Mo 03 Aug 2015 12:43:04 CEST
candlepin-tomcat-0.9.50-1.el7.noarch Mo 03 Aug 2015 12:43:03 CEST
candlepin-0.9.50-1.el7.noarch Mo 03 Aug 2015 12:43:01 CEST
gutterball-1.0.16-1.el7.noarch Mo 03 Aug 2015 11:40:47 CEST
gutterball-certs-1.0-1.noarch Do 30 Jul 2015 14:00:40 CEST
ruby193-rubygem-foreman_gutterball-0.0.1-1.201507061443gitb9974c7.git.0.64e9626.el7.noarch
Do 30 Jul 2015 11:43:04 CEST
candlepin-common-1.0.23-1.el7.noarch Do 30 Jul 2015 11:42:12 CEST
python-celery-3.1.11-1.el7.noarch Do 30 Jul 2015 11:40:10 CEST
rubygem-hammer_cli_gutterball-1.0.0-1.el7.noarch Do 30 Jul 2015 11:36:59
CEST
Problem:
hammer ping / tomcat candlepin app / tomcat gutterball tell 404 not found.
celery beat fails as well.
It seems the webapp is not deployed properly when starting tomcat.
systemctl status tomcat -l
tomcat.service - Apache Tomcat Web Application Container
Loaded: loaded (/usr/lib/systemd/system/tomcat.service; enabled)
Active: active (running) since Mo 2015-08-03 12:43:43 CEST; 20s ago
Process: 21764 ExecStop=/usr/libexec/tomcat/server stop (code=exited,
status=1/FAILURE)
Main PID: 23775 (java)
CGroup: /system.slice/tomcat.service
└─23775 java -classpath
/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar
-Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat
-Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp
-Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
org.apache.catalina.startup.Bootstrap start
Aug 03 12:43:59 rzsv0490.fzg.local server[23775]: Aug 03, 2015 12:43:59 PM
org.apache.catalina.loader.WebappClassLoader validateJarFile
Aug 03 12:43:59 rzsv0490.fzg.local server[23775]: INFORMATION:
validateJarFile(/usr/share/tomcat/webapps/gutterball/WEB-INF/lib/servlet-api-2.5.jar)
- jar not loaded. See Servlet Spec 3.0, section 10.7.2. Offending class:
javax/servlet/Servlet.class
Aug 03 12:44:03 rzsv0490.fzg.local server[23775]: Aug 03, 2015 12:44:03 PM
org.apache.catalina.startup.HostConfig deployDirectory
Aug 03 12:44:03 rzsv0490.fzg.local server[23775]: INFORMATION: Deployment
of web application directory /var/lib/tomcat/webapps/gutterball has
finished in 4,157 ms
Aug 03 12:44:03 rzsv0490.fzg.local server[23775]: Aug 03, 2015 12:44:03 PM
org.apache.coyote.AbstractProtocol start
Aug 03 12:44:03 rzsv0490.fzg.local server[23775]: INFORMATION: Starting
ProtocolHandler ["http-apr-8080"]
Aug 03 12:44:03 rzsv0490.fzg.local server[23775]: Aug 03, 2015 12:44:03 PM
org.apache.coyote.AbstractProtocol start
Aug 03 12:44:03 rzsv0490.fzg.local server[23775]: INFORMATION: Starting
ProtocolHandler ["ajp-apr-8009"]
Aug 03 12:44:03 rzsv0490.fzg.local server[23775]: Aug 03, 2015 12:44:03 PM
org.apache.catalina.startup.Catalina start
Aug 03 12:44:03 rzsv0490.fzg.local server[23775]: INFORMATION: Server
startup in 18088 ms
Same is with candlepin Web-App.
I tried as well the latest nightly build from
https://fedorapeople.org/groups/katello/releases/yum/nightly/candlepin/RHEL/7/x86_64/
But always the same.
katello-service status tells:
Redirecting to /bin/systemctl status qpidd.service
qpidd.service - An AMQP message broker daemon.
Loaded: loaded (/usr/lib/systemd/system/qpidd.service; enabled)
Active: active (running) since Mo 2015-08-03 10:13:18 CEST; 2h 38min ago
Docs: man:qpidd(1)
http://qpid.apache.org/
Main PID: 19712 (qpidd)
CGroup: /system.slice/qpidd.service
└─19712 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf
Aug 03 10:13:18 dep.loy.local systemd[1]: Starting An AMQP message broker
daemon…
Aug 03 10:13:18 dep.loy.local systemd[1]: Started An AMQP message broker
daemon…
Aug 03 11:48:07 dep.loy.local qpidd[19712]: 2015-08-03 11:48:07 [System]
error Connection qpid.10.110.1.130:5671-10.110.1.130:55442 No protocol
received closing
Aug 03 11:48:07 dep.loy.local qpidd[19712]: 2015-08-03 11:48:07 [System]
error Connection qpid.10.110.1.130:5671-10.110.1.130:55442 No protocol
received closing
Redirecting to /bin/systemctl status qdrouterd.service
qdrouterd.service - Qpid Dispatch router daemon
Loaded: loaded (/usr/lib/systemd/system/qdrouterd.service; enabled)
Active: active (running) since Mo 2015-08-03 10:13:19 CEST; 2h 38min ago
Main PID: 19738 (qdrouterd)
CGroup: /system.slice/qdrouterd.service
└─19738 /usr/sbin/qdrouterd -c /etc/qpid-dispatch/qdrouterd.conf
Aug 03 10:13:19 dep.loy.local qdrouterd[19738]: Mon Aug 3 10:13:19 2015
ROUTER (info) Configured Address: prefix=/ phase=0
fanout=QD_SCHEMA_FIXEDADDRESS_FANOUT_MULTIPLE
Aug 03 10:13:19 dep.loy.local qdrouterd[19738]: Mon Aug 3 10:13:19 2015
CONN_MGR (info) Configured Listener: 0.0.0.0:5647 role=normal
Aug 03 10:13:19 dep.loy.local qdrouterd[19738]: Mon Aug 3 10:13:19 2015
CONN_MGR (info) Configured Listener: 0.0.0.0:5646 role=inter-router
Aug 03 10:13:19 dep.loy.local qdrouterd[19738]: Mon Aug 3 10:13:19 2015
CONN_MGR (info) Configured on-demand connector: dep.loy.local:5671
name=broker
Aug 03 10:13:19 dep.loy.local qdrouterd[19738]: Mon Aug 3 10:13:19 2015
ROUTER (info) Configured Link-route-pattern: prefix=pulp. connector=broker
Aug 03 10:13:19 dep.loy.local qdrouterd[19738]: Mon Aug 3 10:13:19 2015
ROUTER (info) Configured Link-route-pattern: prefix=qmf. connector=broker
Aug 03 10:13:19 dep.loy.local qdrouterd[19738]: Mon Aug 3 10:13:19 2015
SERVER (info) Operational, 2 Threads Running
Aug 03 10:13:19 dep.loy.local qdrouterd[19738]: Mon Aug 3 10:13:19 2015
CONN_MGR (info) Starting on-demand connector: broker
Aug 03 10:13:19 dep.loy.local qdrouterd[19738]: Mon Aug 3 10:13:19 2015
ROUTER (info) Activating Prefix 'pulp.' for routed links to 'broker'
Aug 03 10:13:19 dep.loy.local qdrouterd[19738]: Mon Aug 3 10:13:19 2015
ROUTER (info) Activating Prefix 'qmf.' for routed links to 'broker'
elasticsearch.service - LSB: Starts elasticsearch
Loaded: loaded (/etc/rc.d/init.d/elasticsearch)
Active: active (running) since Mo 2015-08-03 10:13:20 CEST; 2h 38min ago
CGroup: /system.slice/elasticsearch.service
└─19893 java -Xms256m -Xmx256m -Xss256k -Djava.awt.headless=true
-XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly
-XX:+HeapDumpOnOutOfMemoryError -Delasticsearch
-Des.pidfile=/var/run/elasticsearch/elasticsearch.pid
-Des.path.home=/usr/share/java/elasticsearch -cp
:/usr/share/java/elasticsearch/lib/elasticsearch-0.90.10.jar:/usr/share/java/lucene4-core.jar:/usr/share/java/lucene4-contrib/*:/usr/share/java/sigar/sigar.jar
-Des.config=/etc/elasticsearch/elasticsearch.yml
-Des.path.conf=/etc/elasticsearch
-Des.path.home=/usr/share/java/elasticsearch
-Des.path.logs=/var/log/elasticsearch -Des.path.data=/var/lib/elasticsearch
-Des.path.work=/tmp/elasticsearch
-Des.path.plugins=/usr/share/java/elasticsearch/plugins
org.elasticsearch.bootstrap.ElasticSearch
Aug 03 10:13:20 dep.loy.local runuser[19884]: pam_unix(runuser:session):
session opened for user elasticsearch by (uid=0)
Aug 03 10:13:20 dep.loy.local elasticsearch[19880]: Starting elasticsearch:
[ OK ]
Aug 03 10:13:20 dep.loy.local systemd[1]: Started LSB: Starts elasticsearch.
Redirecting to /bin/systemctl status tomcat.service
tomcat.service - Apache Tomcat Web Application Container
Loaded: loaded (/usr/lib/systemd/system/tomcat.service; enabled)
Active: active (running) since Mo 2015-08-03 12:43:43 CEST; 8min ago
Process: 21764 ExecStop=/usr/libexec/tomcat/server stop (code=exited,
status=1/FAILURE)
Main PID: 23775 (java)
CGroup: /system.slice/tomcat.service
└─23775 java -classpath
/usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar
-Dcatalina.base=/usr/share/tomcat -Dcatalina.home=/usr/share/tomcat
-Djava.endorsed.dirs= -Djava.io.tmpdir=/var/cache/tomcat/temp
-Djava.util.logging.config.file=/usr/share/tomcat/conf/logging.properties
-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
org.apache.catalina.startup.Bootstrap start
Aug 03 12:48:56 dep.loy.local server[23775]: Exception in thread "Timer-0"
java.lang.NoClassDefFoundError:
com/mchange/v2/resourcepool/BasicResourcePool$AsyncTestIdleResourceTask
Aug 03 12:48:56 dep.loy.local server[23775]: at
com.mchange.v2.resourcepool.BasicResourcePool.checkIdleResources(BasicResourcePool.java:1481)
Aug 03 12:48:56 dep.loy.local server[23775]: at
com.mchange.v2.resourcepool.BasicResourcePool.access$2000(BasicResourcePool.java:32)
Aug 03 12:48:56 dep.loy.local server[23775]: at
com.mchange.v2.resourcepool.BasicResourcePool$CheckIdleResourcesTask.run(BasicResourcePool.java:1964)
Aug 03 12:48:56 dep.loy.local server[23775]: at
java.util.TimerThread.mainLoop(Timer.java:555)
Aug 03 12:48:56 dep.loy.local server[23775]: at
java.util.TimerThread.run(Timer.java:505)
Aug 03 12:48:56 dep.loy.local server[23775]: Caused by:
java.lang.ClassNotFoundException:
com.mchange.v2.resourcepool.BasicResourcePool$AsyncTestIdleResourceTask
Aug 03 12:48:56 dep.loy.local server[23775]: at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1720)
Aug 03 12:48:56 dep.loy.local server[23775]: at
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1571)
Aug 03 12:48:56 dep.loy.local server[23775]: … 5 more
Redirecting to /bin/systemctl status pulp_workers.service
pulp_workers.service - Pulp Celery Workers
Loaded: loaded (/usr/lib/systemd/system/pulp_workers.service; enabled)
Active: active (exited) since Mo 2015-08-03 10:13:33 CEST; 2h 38min ago
Main PID: 20796 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/pulp_workers.service
Aug 03 10:13:32 dep.loy.local systemd[1]: Starting Pulp Celery Workers…
Aug 03 10:13:33 dep.loy.local systemd[1]: Started Pulp Celery Workers.
Redirecting to /bin/systemctl status foreman-proxy.service
foreman-proxy.service - Foreman Proxy
Loaded: loaded (/usr/lib/systemd/system/foreman-proxy.service; enabled)
Active: active (running) since Mo 2015-08-03 10:13:35 CEST; 2h 38min ago
Main PID: 20880 (ruby)
CGroup: /system.slice/foreman-proxy.service
└─20880 ruby /usr/share/foreman-proxy/bin/smart-proxy
Aug 03 10:13:33 dep.loy.local systemd[1]: Starting Foreman Proxy…
Aug 03 10:13:35 dep.loy.local systemd[1]: PID file
/run/foreman-proxy/foreman-proxy.pid not readable (yet?) after start.
Aug 03 10:13:35 dep.loy.local systemd[1]: Started Foreman Proxy.
Redirecting to /bin/systemctl status pulp_resource_manager.service
pulp_resource_manager.service - Pulp Resource Manager
Loaded: loaded (/usr/lib/systemd/system/pulp_resource_manager.service;
enabled)
Active: active (running) since Mo 2015-08-03 10:13:36 CEST; 2h 38min ago
Main PID: 20911 (celery)
CGroup: /system.slice/pulp_resource_manager.service
├─20911 /usr/bin/python /usr/bin/celery worker -A
pulp.server.async.app -n resource_manager@%h -Q resource_manager -c 1
–events --umask 18
└─20996 /usr/bin/python /usr/bin/celery worker -A
pulp.server.async.app -n resource_manager@%h -Q resource_manager -c 1
–events --umask 18
Aug 03 10:13:37 dep.loy.local celery[20911]: - ** ---------- .>
transport: qpid://guest@dep.loy.local:5671//
Aug 03 10:13:37 dep.loy.local celery[20911]: - ** ---------- .>
results: mongodb
Aug 03 10:13:37 dep.loy.local celery[20911]: - *** — * — .>
concurrency: 1 (prefork)
Aug 03 10:13:37 dep.loy.local celery[20911]: – ******* ----
Aug 03 10:13:37 dep.loy.local celery[20911]: — ***** ----- [queues]
Aug 03 10:13:37 dep.loy.local celery[20911]: -------------- .>
resource_manager exchange=resource_manager(direct) key=resource_manager
Aug 03 10:13:37 dep.loy.local celery[20911]: .>
resource_manager@dep.loy.local.dq exchange=C.dq(direct)
key=resource_manager@dep.loy.local
Aug 03 10:13:37 dep.loy.local pulp[20911]: kombu.transport.qpid:INFO:
Connected to qpid with SASL mechanism PLAIN
Aug 03 10:13:37 dep.loy.local pulp[20911]: celery.worker.consumer:INFO:
Connected to qpid://guest@dep.loy.local:5671//
Aug 03 10:13:37 dep.loy.local pulp[20911]: kombu.transport.qpid:INFO:
Connected to qpid with SASL mechanism PLAIN
Redirecting to /bin/systemctl status mongod.service
mongod.service - High-performance, schema-free document-oriented database
Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled)
Active: active (running) since Mo 2015-08-03 10:13:36 CEST; 2h 38min ago
Main PID: 20943 (mongod)
CGroup: /system.slice/mongod.service
└─20943 /usr/bin/mongod --smallfiles --quiet -f /etc/mongod.conf
run
Aug 03 10:13:36 dep.loy.local systemd[1]: Starting High-performance,
schema-free document-oriented database…
Aug 03 10:13:36 dep.loy.local mongod[20941]: about to fork child process,
waiting until server is ready for connections.
Aug 03 10:13:36 dep.loy.local mongod[20941]: forked process: 20943
Aug 03 10:13:36 dep.loy.local mongod[20941]: child process started
successfully, parent exiting
Aug 03 10:13:36 dep.loy.local systemd[1]: Started High-performance,
schema-free document-oriented database.
Aug 03 11:31:02 dep.loy.local systemd[1]:
[/usr/lib/systemd/system/mongod.service:9] Not an absolute path, ignoring:
${PIDFILE-/var/run/mongodb/mongod.pid}
Aug 03 11:33:46 dep.loy.local systemd[1]:
[/usr/lib/systemd/system/mongod.service:9] Not an absolute path, ignoring:
${PIDFILE-/var/run/mongodb/mongod.pid}
Aug 03 11:34:17 dep.loy.local systemd[1]:
[/usr/lib/systemd/system/mongod.service:9] Not an absolute path, ignoring:
${PIDFILE-/var/run/mongodb/mongod.pid}
Redirecting to /bin/systemctl status pulp_celerybeat.service
pulp_celerybeat.service - Pulp's Celerybeat
Loaded: loaded (/usr/lib/systemd/system/pulp_celerybeat.service; enabled)
Active: failed (Result: exit-code) since Mo 2015-08-03 11:46:59 CEST; 1h
5min ago
Process: 1433 ExecStart=/usr/bin/celery beat
–scheduler=pulp.server.async.scheduler.Scheduler (code=exited,
status=1/FAILURE)
Main PID: 1433 (code=exited, status=1/FAILURE)
CGroup: /system.slice/pulp_celerybeat.service
Aug 03 11:46:59 dep.loy.local celery[1433]: pidlock.acquire()
Aug 03 11:46:59 dep.loy.local celery[1433]: File
"/usr/lib/python2.7/site-packages/celery/platforms.py", line 153, in acquire
Aug 03 11:46:59 dep.loy.local celery[1433]: reraise(LockFailed,
LockFailed(str(exc)), sys.exc_info()[2])
Aug 03 11:46:59 dep.loy.local celery[1433]: File
"/usr/lib/python2.7/site-packages/celery/platforms.py", line 151, in acquire
Aug 03 11:46:59 dep.loy.local celery[1433]: self.write_pid()
Aug 03 11:46:59 dep.loy.local celery[1433]: File
"/usr/lib/python2.7/site-packages/celery/platforms.py", line 212, in
write_pid
Aug 03 11:46:59 dep.loy.local celery[1433]: pidfile_fd = os.open(self.path,
PIDFILE_FLAGS, PIDFILE_MODE)
Aug 03 11:46:59 dep.loy.local celery[1433]: celery.platforms.LockFailed:
[Errno 13] Permission denied: '/var/lib/pulp/celery/celerybeat.pid'
Aug 03 11:46:59 dep.loy.local systemd[1]: pulp_celerybeat.service: main
process exited, code=exited, status=1/FAILURE
Aug 03 11:46:59 dep.loy.local systemd[1]: Unit pulp_celerybeat.service
entered failed state.
Redirecting to /bin/systemctl status httpd.service
httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled)
Active: active (running) since Mo 2015-08-03 10:13:44 CEST; 2h 38min ago
Main PID: 21090 (httpd)
Status: "Total requests: 0; Current requests/sec: 0; Current traffic:
0 B/sec"
CGroup: /system.slice/httpd.service
├─21090 /usr/sbin/httpd -DFOREGROUND
├─21115 (wsgi:pulp) -DFOREGROUND
├─21116 PassengerWatchdog
├─21119 PassengerHelperAgent
├─21125 PassengerLoggingAgent
├─21133 /usr/sbin/httpd -DFOREGROUND
├─21134 /usr/sbin/httpd -DFOREGROUND
├─21135 /usr/sbin/httpd -DFOREGROUND
├─21136 /usr/sbin/httpd -DFOREGROUND
├─21137 /usr/sbin/httpd -DFOREGROUND
├─21139 /usr/sbin/httpd -DFOREGROUND
├─21140 /usr/sbin/httpd -DFOREGROUND
├─21141 /usr/sbin/httpd -DFOREGROUND
├─21381 Passenger RackApp: /usr/share/foreman
├─29453 Passenger RackApp: /etc/puppet/rack
└─32756 Passenger RackApp: /usr/share/foreman
Aug 03 10:46:03 dep.loy.local pulp[21115]: qpid.messaging:WARNING:
recoverable error[attempt 1]: heartbeat timeout
Aug 03 10:46:03 dep.loy.local pulp[21115]: qpid.messaging:WARNING: sleeping
1 seconds
Aug 03 10:46:04 dep.loy.local pulp[21115]: qpid.messaging:WARNING: trying:
dep.loy.local:5671
Aug 03 10:46:04 dep.loy.local pulp[21115]: qpid.messaging:WARNING:
reconnect succeeded: dep.loy.local:5671
Aug 03 10:58:03 dep.loy.local puppet-master[29453]: Compiled catalog for
dep.loy.local in environment production in 0.02 seconds
Aug 03 11:28:02 dep.loy.local puppet-master[29453]: Compiled catalog for
dep.loy.local in environment production in 0.01 seconds
Aug 03 11:58:03 dep.loy.local puppet-master[29453]: Compiled catalog for
dep.loy.local in environment production in 0.04 seconds
Aug 03 12:28:00 dep.loy.local puppet-master[29453]: Compiled catalog for
dep.loy.local in environment production in 0.01 seconds
Aug 03 12:44:48 dep.loy.local pulp[21115]: kombu.transport.qpid:INFO:
Connected to qpid with SASL mechanism PLAIN
Aug 03 12:46:38 dep.loy.local pulp[21115]: kombu.transport.qpid:INFO:
Connected to qpid with SASL mechanism PLAIN
Redirecting to /bin/systemctl status foreman-tasks.service
foreman-tasks.service - Foreman jobs daemon
Loaded: loaded (/usr/lib/systemd/system/foreman-tasks.service; enabled)
Active: active (running) since Mo 2015-08-03 10:14:05 CEST; 2h 37min ago
Docs: GitHub - theforeman/foreman-tasks: Tasks management engine and plugin for Foreman
CGroup: /system.slice/foreman-tasks.service
├─21300 dynflow_executor_monitor
└─21302 dynflow_executor
Aug 03 10:13:55 dep.loy.local systemd[1]: Starting Foreman jobs daemon…
Aug 03 10:14:00 dep.loy.local foreman-tasks[21243]: Performance on MRI may
be improved with the concurrent-ruby-ext gem. Please see
Aug 03 10:14:04 dep.loy.local forema