Actions::Pulp::Consumer::ContentUpdate (waiting for Pulp to start the task)

Hi all,

I’m having an issue with Foreman …
3: Actions::Pulp::Consumer::ContentUpdate (waiting for Pulp to start the task) while trying to update a package on a client.
But the update doesn’t start and the system is steal on this activity.

I tried to search over the network but i couldn’t find any kind of solution.

Please, could you help me? Fell free to ask any kind of additional information.

Thanks!
Stefano

If you look at output of the Actions::Pulp::Consumer::ContentUpdate action, is there mentioned an id of the task in pulp? If so, you could inspect the task in pulp using pulp-admin. Something along the lines of

pulp-admin tasks details --task-id $TASK_ID

might work. It is possible that the task in pulp started (and finished) and the task in Foreman just missed it. In general if the task in pulp finished it should be safe to cancel the task in Foreman and trigger it again. If the task in pulp is stuck, then we’d have to ask Katello or Pulp folks for guidance.

@Justin_Sherrill1 any ideas here?

Hi aruzicka,
thanks for your response.

Following pulp-admin command result.

+----------------------------------------------------------------------+
                              Task Details
+----------------------------------------------------------------------+

Operations:       unit_update
Resources:        d2cec437-5643-4c57-8c2b-94655403962e (consumer)
State:            Waiting
Start Time:       Unstarted
Finish Time:      Incomplete
Result:           Incomplete
Task Id:          ebd51591-ce8f-4162-a6d7-11882024261c
Progress Report:

Can you verify that goferd is running on the client?

service goferd status

Hi Justin,

the goferd service seems active but during a restart of the service I’ve seen this in the log :slight_smile:

[root@srvlzkatello-host02 ~]# service goferd status -l
Redirecting to /bin/systemctl status  -l goferd.service
● goferd.service - Gofer Agent
   Loaded: loaded (/usr/lib/systemd/system/goferd.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2018-01-24 09:44:31 CET; 6s ago
 Main PID: 16996 (python)
   CGroup: /system.slice/goferd.service
           └─16996 python /usr/bin/goferd --foreground

[ERROR][worker-0] gofer.messaging.adapter.connect:33 - connect: proton+amqps://srvlzkatello.conte.it:5647, failed: Connection amqps://srvlzkatello.conte.it:5647 disconnected: Condition('proton:io', 'recv: Connection refused') 

Could you please give me any tips on this?
Thanks
Stefano

Hi Stefano,
Could it be that you’ve updated qpid on TFM server? EPEL pushed qpid 1.0 to its repository and this version doesn’t start the the actual TFM config.
Check if qdrouterd service is running:
systemctl status qdrouterd.service

I did have a similar problem.
rgds,
Arsène

HI Arsene

this is the output of the command suggested us:

systemctl status -l qdrouterd.service
● qdrouterd.service - Qpid Dispatch router daemon
   Loaded: loaded (/usr/lib/systemd/system/qdrouterd.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Thu 2018-01-25 12:17:39 CET; 16s ago
  Process: 12819 ExecStart=/usr/sbin/qdrouterd -c /etc/qpid-dispatch/qdrouterd.conf (code=exited, status=1/FAILURE)
Main PID: 12819 (code=exited, status=1/FAILURE)
 
Jan 25 12:17:37 srvlzkatello.conte.it systemd[1]: Started Qpid Dispatch router daemon.
Jan 25 12:17:37 srvlzkatello.conte.it systemd[1]: Starting Qpid Dispatch router daemon...
Jan 25 12:17:39 srvlzkatello.conte.it qdrouterd[12819]: 2018-01-25 12:17:39.362738 +0100 ERROR (error) Python: Exception: Cannot load configuration file /etc/qpid-dispatch/qdrouterd.conf: No such entity type 'org.apache.qpid.dispatch.container'
Jan 25 12:17:39 srvlzkatello.conte.it qdrouterd[12819]: 2018-01-25 12:17:39.373828 +0100 ERROR (error) Traceback (most recent call last):
Jan 25 12:17:39 srvlzkatello.conte.it qdrouterd[12819]: File "/usr/lib/qpid-dispatch/python/qpid_dispatch_internal/management/config.py", line 142, in configure_dispatch
Jan 25 12:17:39 srvlzkatello.conte.it systemd[1]: qdrouterd.service: main process exited, code=exited, status=1/FAILURE
Jan 25 12:17:39 srvlzkatello.conte.it systemd[1]: Unit qdrouterd.service entered failed state.
Jan 25 12:17:39 srvlzkatello.conte.it systemd[1]: qdrouterd.service failed.

and these are our qpid libraries:

tfm-rubygem-qpid_messaging-0.34.1-1.el7.x86_64
srvlzkatello.conte.it-qpid-router-server-1.0-1.noarch
qpid-proton-c-0.18.1-1.el7.x86_64
python-qpid-proton-0.18.1-1.el7.x86_64
qpid-cpp-server-1.37.0-1.el7.x86_64
qpid-tools-1.37.0-1.el7.noarch
qpid-cpp-client-1.37.0-1.el7.x86_64
qpid-dispatch-router-1.0.0-1.el7.x86_64
qpid-cpp-server-linearstore-1.37.0-1.el7.x86_64
qpid-cpp-client-devel-1.37.0-1.el7.x86_64
srvlzkatello.conte.it-qpid-broker-1.0-1.noarch
python-qpid-qmf-1.37.0-1.el7.x86_64
python-qpid-1.37.0-1.el7.noarch
python-gofer-qpid-2.7.6-1.el7.noarch
qpid-qmf-1.37.0-1.el7.x86_64
srvlzkatello.conte.it-qpid-router-client-1.0-1.noarch
srvlzkatello.conte.it-qpid-client-cert-1.0-1.noarch

Could you please take a look?
Thanks
Stefano

Hi Stefano,

I did downgrade qpid to latest 0.x version.
You may find additional information in following thread : QPID Broken Default Configuration

rgds,
Arsène

1 Like

Hi Arsene,
solution you told me works fine.

Downgrading libraries solved issue.

Many thanks
Stefano