Katello 4.3.0: Bullseye sync takes more than 5 hours

Support
1

I have hopefully upgraded for the last time from 4.2 to 4.3.0 yesterday. Set all debian repositories (buster, bullseye and the puppet deb repo) to download policy immediate.

I have noticed that the bullseye main repo sync now takes substantially longer: more than 5 hours compared to approx. 20 minutes before the upgrade.

Buster main went from approx. 13 minutes to more than 40 minutes.

The security and updates repos also take much longer, but as it is only from less than 5 seconds to still under a minute it’s not so bad.

I have synced all repositories twice already and both times it took so long.

During the 5 hours bullseye main sync I have noticed that one postgres process was close to 100% all the time. During that time, task status slowing incremented the associated content.

---
pulp_tasks:
- pulp_href: "/pulp/api/v3/tasks/0b173f48-45ab-44f1-854e-b7281c19fb7a/"
  pulp_created: '2022-03-29T23:34:49.840+00:00'
  state: completed
  name: pulp_deb.app.tasks.synchronizing.synchronize
  logging_cid: 44c8674f-a256-4294-9071-50e1a1915636
  started_at: '2022-03-29T23:34:50.028+00:00'
  finished_at: '2022-03-30T04:42:51.392+00:00'
  worker: "/pulp/api/v3/workers/17c2d73f-34b6-4930-9cd1-265d5b7e1b92/"
  child_tasks: []
  progress_reports:
  - message: Update ReleaseFile units
    code: update.release_file
    state: completed
    done: 1
  - message: Update PackageIndex units
    code: update.packageindex
    state: completed
    done: 2
  - message: Downloading Artifacts
    code: sync.downloading.artifacts
    state: completed
    done: 3
  - message: Associating Content
    code: associating.content
    state: completed
    done: 57352
  - message: Un-Associating Content
    code: unassociating.content
    state: completed
    done: 0
  created_resources: []
  reserved_resources_record:
  - "/pulp/api/v3/repositories/deb/apt/5cf43358-9694-4dc5-a602-be660c479856/"
  - shared:/pulp/api/v3/remotes/deb/apt/f541a1f2-6a32-418f-835e-165596a4f150/
create_version: true
task_groups: []
poll_attempts:
  total: 1160
  failed: 0

Foreman 3.1.2, Katello 4.3.0 (upgraded yesterday from 4.2.0) running on CentOS 7.9.

tfm-pulpcore-python3-pulp-deb-2.16.1-1.el7.noarch
tfm-pulpcore-python3-pulpcore-3.16.6-1.el7.noarch

2

Hi,

Could you file an issue here? Issues · pulp/pulp_deb · GitHub

3

O.K. https://github.com/pulp/pulp_deb/issues/466

4

I’m also experiencing this issue with bullseye incremental sync on Katello 4.3.1 / Foreman 3.1.2.
I’m syncing bullseye and bullseye-updates in the same product and it takes about 11 hours.

---
pulp_tasks:
- pulp_href: "/pulp/api/v3/tasks/b0525daa-67b9-49a8-a969-74e767251007/"
  pulp_created: '2022-04-14T02:30:07.870+00:00'
  state: completed
  name: pulp_deb.app.tasks.synchronizing.synchronize
  logging_cid: 309f50e6-9629-41e3-b040-986f0a7dd88f
  started_at: '2022-04-14T02:30:07.944+00:00'
  finished_at: '2022-04-14T13:25:37.096+00:00'
  worker: "/pulp/api/v3/workers/12f63e93-fdc2-48d4-8e55-2a1b931cdc8e/"
  child_tasks: []
  progress_reports:
  - message: Un-Associating Content
    code: unassociating.content
    state: completed
    done: 1
  - message: Downloading Artifacts
    code: sync.downloading.artifacts
    state: completed
    done: 6
  - message: Update ReleaseFile units
    code: update.release_file
    state: completed
    done: 2
  - message: Update PackageIndex units
    code: update.packageindex
    state: completed
    done: 8
  - message: Associating Content
    code: associating.content
    state: completed
    done: 57657
  created_resources:
  - "/pulp/api/v3/repositories/deb/apt/8a8d65ad-9c12-4e05-bc40-812dec42485b/versions/186/"
  reserved_resources_record:
  - "/pulp/api/v3/repositories/deb/apt/8a8d65ad-9c12-4e05-bc40-812dec42485b/"
  - shared:/pulp/api/v3/remotes/deb/apt/3e174370-93a2-47a2-a6f3-735e16852099/
create_version: true
task_groups: []
poll_attempts:
  total: 2468
  failed: 0