Katello repository sync warning

Support
1

Problem:
Oracle Linux 8 “Codeready builder” repository sync completes with a warning state
Expected outcome:
The sync completes with a successful state
Foreman and Proxy versions:
foreman-3.2.0-1.el8.noarch
Foreman and Proxy plugin versions:
katello-4.4.0-1.el8.noarch
katello-repos-4.4.0-1.el8.noarch
Distribution and version:
Oracle Linux 8.5
Other relevant data:
Syncing the repository Oracle Linux 8 (x86_64) CodeReady Builder - Unsupported | Oracle, Software. Hardware. Complete. completes with the warning:

PG::CardinalityViolation: ERROR:  ON CONFLICT DO UPDATE command cannot affect row a second time
HINT:  Ensure that no rows proposed for insertion within the same command have duplicate constrained values.

Backtrace:

2022-04-27T08:23:34 [E|bac|327d8a7f] PG::CardinalityViolation: ERROR:  ON CONFLICT DO UPDATE command cannot affect row a second time
 327d8a7f | HINT:  Ensure that no rows proposed for insertion within the same command have duplicate constrained values.
 327d8a7f |  (ActiveRecord::StatementInvalid)
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:675:in `exec_params'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:675:in `block (2 levels) in exec_no_cache'
 327d8a7f | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
 327d8a7f | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares'
 327d8a7f | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:674:in `block in exec_no_cache'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract_adapter.rb:722:in `block (2 levels) in log'
 327d8a7f | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
 327d8a7f | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
 327d8a7f | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
 327d8a7f | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
 327d8a7f | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract_adapter.rb:721:in `block in log'
 327d8a7f | /usr/share/gems/gems/activesupport-6.0.3.7/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract_adapter.rb:712:in `log'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:673:in `exec_no_cache'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql_adapter.rb:654:in `execute_and_clear'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/postgresql/database_statements.rb:98:in `exec_query'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/database_statements.rb:153:in `exec_insert_all'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/connection_adapters/abstract/query_cache.rb:22:in `exec_insert_all'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/insert_all.rb:27:in `execute'
 327d8a7f | /usr/share/gems/gems/activerecord-6.0.3.7/lib/active_record/persistence.rb:243:in `upsert_all'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/services/katello/content_unit_indexer.rb:45:in `block in import_all'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/services/katello/pulp3/pulp_content_unit.rb:97:in `yield'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/services/katello/pulp3/pulp_content_unit.rb:97:in `block (2 levels) in pulp_units_batch_for_repo'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/services/katello/pulp3/pulp_content_unit.rb:85:in `loop'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/services/katello/pulp3/pulp_content_unit.rb:85:in `block in pulp_units_batch_for_repo'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/services/katello/content_unit_indexer.rb:27:in `each'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/services/katello/content_unit_indexer.rb:27:in `import_all'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/models/katello/repository.rb:918:in `block (2 levels) in index_content'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/lib/katello/logging.rb:8:in `time'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/models/katello/repository.rb:917:in `block in index_content'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/models/katello/repository.rb:916:in `each'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/models/katello/repository.rb:916:in `index_content'
 327d8a7f | /usr/share/gems/gems/katello-4.4.0/app/lib/actions/katello/repository/index_content.rb:22:in `run'
2

Hi @John_Beranek ,

Thanks for the issue report. Looking at the error and comparing to the source on the 4.4 branch, my best guess is that the repository contains some errata with duplicate IDs, and that this caused the duplicate constrained values error during Actions::Katello::Repository::IndexContent.

With that said, I tried syncing this repository on a Katello 4.4 test instance and it worked OK for me.

Browsing the repository metadata at Index of /repo/OracleLinux/OL8/codeready/builder/x86_64/repodata/ I can see that the $UUID-updateinfo.xml.gz files are very recently (past ~3 days of this writing) updated.

Have you tried syncing the repository again more recently since the issue first occurred, and if so what was the result?

Looking forward to hearing from you,

3

Thanks for taking a look.

This morning’s sync has also failed, with the same error message. Do I need to do some sort of forced sync maybe?

4

Hi @John_Beranek ,

In /var/lib/pgsql/data/log/*.log can you locate ERROR: ON CONFLICT DO UPDATE command cannot affect row a second time so that we can see what exactly it tried to do when the issue occurred?

Kind regards,

5

The following, I’ve vastly truncated the full SQL statement as it’s huge

2022-05-06 02:06:52 UTC ERROR:  ON CONFLICT DO UPDATE command cannot affect row a second time
2022-05-06 02:06:52 UTC HINT:  Ensure that no rows proposed for insertion within the same command have duplicate constrained values.
2022-05-06 02:06:52 UTC STATEMENT:  INSERT INTO "katello_errata" ("title","severity","description","reboot_suggested","solution","summary","pulp_id","issued","updated","errata_id","errata_type","created_at","updated_at") VALUES (' mariadb:10.3 security and bug fix update', 'Moderate', 'galera
        [25.3.34-4]
        - Explicitly require the ''procps-ng'' package
        - Otherwise it will not require it in the lightweight systems (e.g. containers)
        - and Galera won''t work properly

        [25.3.34-3]
        - Use downstream garbd-wrapper and garbd.service to ensure compatibility
        - Add upstream versions of garbd-wrapper (called garbd-systemd) and garbd.service
          in case user want''s to use them

        [ 25.3.34-2]
        - Switch from SCONS build tooling to CMAKE build tooling

        [25.3.34-1]
        - Rebase to 25.3.34
          Resolves: #2042298

        mariadb
        [3:10.3.32-2.0.1]
        - Fix changelog

        [3:10.3.32-2]
        - Add delaycompress option to the logrotate script

        [3:10.3.32-1]
        - Rebase to 10.3.32

        [3:10.3.31-1]
        - Rebase to 10.3.31

        [3:10.3.30-1]
        - Rebase to 10.3.30

        [3:10.3.29-2]
        - Set user_map.conf file to be noreplace config file

        [3:10.3.29-1]
        - Rebase to 10.3.29
        ', FALSE, 'This update is available via the Unbreakable Linux Network (ULN) and the Oracle Public Yum Server. Details on how to use ULN or http://public-yum.oracle.com to apply this update are available at http://linux.oracle.com/applying_updates.html.', '', 'ELSA-2022-1556', '2022-04-28', '2022-04-28', 'ELSA-2022-1556', 'security', '2022-05-06 02:06:52.414931', '2022-05-06 02:06:52.414967'), (' xmlrpc-c security update', 'Important',
6

These last few parameters may be pertinent too

 'ELBA-2021-1603', '2021-05-25', '2021-05-25', 'ELBA-2021-1603', 'bugfix', '2022-05-06 02:06:52.422006', '2022-05-06 02:06:52.422013')

Oh, I only just realised it’s so big as it’s many errata…so it could be any of them I guess…?