Pulp redis connection issue at https://myhostname.honeywell.com/pulp/api/v3

Support
,
#1

Problem:
Hello,

I update from :

Foreman 3.1/Katello4.3 to Foreman 3.2/Katello4.4

then

Foreman 3.2/Katello4.4 to Foreman 3.3/Katello4.5

Since the last update I have an issue with pulp3 , I’m not able to update products for example.
All services seems to be up and running.

Some output :

[root@az66u1971 ~]# hammer ping
pulp3:
Status: FAIL
Server Response: Message: Pulp redis connection issue at https://az66u1971.honeywell.com/pulp/api/v3.

For status :
https://az66u1971.honeywell.com/pulp/api/v3/status/

},
“redis_connection”: {
“connected”: false
},

cat /var/log/foreman/production.log | grep redis

2022-06-29T13:51:30 [E|bac|83dc7a74] There was an issue with the backend service pulp3: Pulp redis connection issue at https://az66u1971.honeywell.com/pulp/api/v3. (RuntimeError)

2022-06-29T14:35:23 [W|app|438eb570] Pulp redis connection issue at https://az66u1971.honeywell.com/pulp/api/v3.

I follow the documentation : Upgrading and Updating Foreman

I am not able to find the issue, Could you somebody give a direction ?

Thank you very much,

#3

Hi!
Katello 4.5 is still an RC, and yeah that’s a known bug right now, at least reported twice here :slight_smile:

So if this is a prod system, I hope you made a backup before, not sure if there is a way to roll back without a backup / snapshot :thinking:

#4

Hello,

Thank you for your feedback.

It’s in Prod, I didn’t do a snapshot this time :confused:

It’s not possible to downgrade Katello ?

#5

So there is only RC for Katello 4.5 and Foreman 3.3 is not compatible with Katello 4.4

I will try to Downgrade Foreman to 3.2 and Katello 4.4…

If not I will try a foreman-maintain backup offline, fresh install of a Foreman 3.2 and restore

#6

I’m not sure if this is even possible, because there are made database upgrades during the upgrade :confused:

Someone from the Katello dev team would have to comment on that, I guess :thinking:

#7

I will wait if I received a feedback from the Katello Dev Team.
if not I will reinstall everything from scratch and re-register all nodes during the Weekend :sweat:

1 Like
#8

@iballou Sorry for bothering you here as well, but do you know a way back, if no Snapshot/Backup is present?

#9

A downgrade would be pretty difficult, especially since a new version of Pulp was introduced. I’m trying to get a 4.4 env up now to upgrade and check for issues.

In the meantime, what is the status of your pulpcore services?

#10

Also, are there any errors on your smart proxy? And this is EL8 ?

#11

Hello,

Thank you for your help @iballou and @lumarel

It’s running on Centos 7.9

All services are up.

I found a backup of the VM , I am currently upgrading it to 3.2.

But I kept the broken one, If you want me to try something I will do it.

#12

Happy to hear you found a backup! I can’t think of anything to try right now, but let’s see if anyone else has any ideas.

1 Like
#13

Hi @FabR,

Just wanted to make sure you saw this: Upgrade issue to 4.4.1 - Internal Server Error (and a notice about 4.5 RC2) - #11 by jamesn

Seems to be the solution. We’ll take a look at why that got flipped.

2 Likes