236 … doh … 
Katello updates a remote with the latest remote configuration options during every sync. That’s done in the RefreshRemote action. Are these bad remote options not being updated when Katello runs RefreshRemote? If not then that may be part of the issue as well.
What we’re looking for is the first step on a repo sync dynflow task:
Clicking on it, you should see a pulp task:
I don‘t quite understand: should Katello update the username, password, client key during each sync? I don’t think it does. If it would, the sync should work just fine and double encryption or missing encryption in the pulp database shouldn’t matter, because it would be overwritten immediately… Instead, each sync fails.
We call the Pulp remote update API through the RefreshRemote task that @Justin_Sherrill pointed out above. It should be updating all of those fields every time. We don’t update the fields for remotes immediately when remote configuration is changed in Katello, rather we defer it until sync time.
So, if these fields are still bad, then either the update isn’t working in Pulp, or there’s a Katello bug that’s skipping the update.
Ah okay, the username and password aren’t updated if there aren’t any set for a repository. That is likely related. katello/repository.rb at master · Katello/katello · GitHub
The keys should be updated, however: katello/repository.rb at master · Katello/katello · GitHub
This PR may help if anyone is able to test it: Fixes #34619 - Remotes should have username and password cleared out if set blank by ianballou · Pull Request #10018 · Katello/katello · GitHub
Ignore the test files when patching.
I have checked the repositories I have problems with (mentioned above):
https://download.postgresql.org/pub/repos/yum/common/redhat/rhel-7-x86_64/
https://cdn.redhat.com/content/dist/rhel/server/7/7Server/x86_64/os
http://mirror.centos.org/centos/7/storage/x86_64/samba-411/
http://mirror.centos.org/centos/7/cloud/x86_64/openstack-queens/
http://mirror.centos.org/centos/7/cloud/x86_64/openstack-rocky/
http://mirror.centos.org/centos/7/cloud/x86_64/openstack-stein/
https://download.fedoraproject.org/pub/epel/8/Everything/x86_64/
All those except the redhat repo have the problem that they an empty string username and password in the pulpcore core_remote table, i.e. they did not get encrypted. They fail each time.
I have also checked the redhat repository and it does have a client_key which is also not encrypted. It’s actually the first RefreshRemote that is failing:
So it looks to me as if the update of the remote is already failing…
Ah that’s interesting, what is the error on the Pulp side? I wonder if that’s related to what @dralley is working on.
Yes. I think it’s related. I guess, before you can change the username, etc. in the pulpcore remote, it tries to load it first and thus automatically tries to decrypt the encrypted fields, which fails.
The error message is the exact message as mentioned above, e.g. in the first post…
Looks like the Pulp side of this should be taken care of soon. We’ll make sure the changes get into the Katello repo as soon as possible. I’ll add an entry to the docs to run the repair command for users who migrated to Pulpcore 3.16 without the fix.
1 Like
Don’t forget that this also seems to affect content proxies.
2 Likes
Thanks for the reminder. That’ll be added to the docs as well.