On Foreman 3.8/Katello 4.10 and noticed that when a user deploys a new docker image and triggers a sync of the docker registry from Foreman, the image is added to foreman and a sync task is created called “Sync Repository on Smart Proxy(ies)” but when I look in the task it does not contain any subtasks for each proxy. It is more or less empty. When looking on each proxy, they never tried to sync the image. These syncs worked fine on previous version.
The role created for allowing that sync contains these filters:
Resource: Product and Repositories, Permissions: sync_products, view_products, create_products, edit_products
Resource: Smart proxy, Permissions: view_capsule_content
Looking in the production log I do not really see anything strange:
2023-11-21T08:32:31 [I|app|6a90e2f3] Started POST "/katello/api/repositories/278/sync" for 192.168.120.209 at 2023-11-21 08:32:31 +0000
2023-11-21T08:32:31 [I|app|6a90e2f3] Processing by Katello::Api::V2::RepositoriesController#sync as */*
2023-11-21T08:32:31 [I|app|6a90e2f3] Parameters: {"api_version"=>"v2", "id"=>"278", "repository"=>{}}
2023-11-21T08:32:31 [I|app|6a90e2f3] Authorized user xxxxxx(xxxxxx)
2023-11-21T08:32:31 [I|bac|6a90e2f3] Task {label: , execution_plan_id: acc2a2f3-1d63-441a-8b37-9992b1ce6286} state changed: pending
2023-11-21T08:32:31 [I|bac|6a90e2f3] Task {label: Actions::Katello::Repository::Sync, id: 8342cac2-dc71-4d57-9920-43907dec71ec, execution_plan_id: acc2a2f3-1d63-441a-8b37-9992b1ce62
86} state changed: planning
2023-11-21T08:32:31 [I|bac|6a90e2f3] Task {label: Actions::Katello::Repository::Sync, id: 8342cac2-dc71-4d57-9920-43907dec71ec, execution_plan_id: acc2a2f3-1d63-441a-8b37-9992b1ce62
86} state changed: planned
2023-11-21T08:32:31 [I|app|6a90e2f3] Rendered /usr/share/gems/gems/katello-4.10.0/app/views/katello/api/v2/repositories/sync.json.rabl within katello/api/v2/layouts/resource (Dura
tion: 22.9ms | Allocations: 19867)
2023-11-21T08:32:31 [I|app|6a90e2f3] Rendered layout /usr/share/gems/gems/katello-4.10.0/app/views/katello/api/v2/layouts/resource.json.erb (Duration: 25.0ms | Allocations: 22489)
2023-11-21T08:32:31 [I|app|6a90e2f3] Completed 202 Accepted in 568ms (Views: 23.0ms | ActiveRecord: 44.2ms | Allocations: 128950)
2023-11-21T08:32:31 [I|bac|6a90e2f3] Task {label: Actions::Katello::Repository::Sync, id: 8342cac2-dc71-4d57-9920-43907dec71ec, execution_plan_id: acc2a2f3-1d63-441a-8b37-9992b1ce62
86} state changed: running
2023-11-21T08:32:31 [I|app|d41d43ac] Started POST "/katello/api/repositories/279/sync" for 192.168.120.209 at 2023-11-21 08:32:31 +0000
2023-11-21T08:32:31 [I|app|d41d43ac] Processing by Katello::Api::V2::RepositoriesController#sync as */*
2023-11-21T08:32:31 [I|app|d41d43ac] Parameters: {"api_version"=>"v2", "id"=>"279", "repository"=>{}}
2023-11-21T08:32:32 [I|app|d41d43ac] Authorized user xxxxxx(xxxxxx)
2023-11-21T08:32:32 [I|bac|d41d43ac] Task {label: , execution_plan_id: 8df8c980-8343-426b-a2a5-1fde56f74aff} state changed: pending
2023-11-21T08:32:32 [I|bac|d41d43ac] Task {label: Actions::Katello::Repository::Sync, id: bcd3b832-70fc-4b7d-a030-eb7633c89aeb, execution_plan_id: 8df8c980-8343-426b-a2a5-1fde56f74a
ff} state changed: planning
2023-11-21T08:32:32 [I|bac|d41d43ac] Task {label: Actions::Katello::Repository::Sync, id: bcd3b832-70fc-4b7d-a030-eb7633c89aeb, execution_plan_id: 8df8c980-8343-426b-a2a5-1fde56f74a
ff} state changed: planned
2023-11-21T08:32:32 [I|app|d41d43ac] Rendered /usr/share/gems/gems/katello-4.10.0/app/views/katello/api/v2/repositories/sync.json.rabl within katello/api/v2/layouts/resource (Dura
tion: 23.1ms | Allocations: 19860)
2023-11-21T08:32:32 [I|app|d41d43ac] Rendered layout /usr/share/gems/gems/katello-4.10.0/app/views/katello/api/v2/layouts/resource.json.erb (Duration: 25.1ms | Allocations: 22478)
2023-11-21T08:32:32 [I|app|d41d43ac] Completed 202 Accepted in 545ms (Views: 24.3ms | ActiveRecord: 41.2ms | Allocations: 120736)
2023-11-21T08:32:32 [I|bac|d41d43ac] Task {label: Actions::Katello::Repository::Sync, id: bcd3b832-70fc-4b7d-a030-eb7633c89aeb, execution_plan_id: 8df8c980-8343-426b-a2a5-1fde56f74a
ff} state changed: running
2023-11-21T08:32:33 [I|app|9fa98ae7] Started GET "/rhsm/" for 10.69.0.67 at 2023-11-21 08:32:33 +0000
2023-11-21T08:32:33 [I|app|9fa98ae7] Processing by Katello::Api::V2::RootController#rhsm_resource_list as JSON
2023-11-21T08:32:33 [I|app|9fa98ae7] Parameters: {"root"=>{}}
2023-11-21T08:32:33 [I|app|9fa98ae7] Completed 200 OK in 3ms (Views: 0.8ms | ActiveRecord: 0.8ms | Allocations: 1500)
2023-11-21T08:32:33 [I|app|b3bdd2eb] Started GET "/rhsm/consumers/594941dc-b28c-4041-bc85-92d39a92a3d4/certificates/serials" for 10.69.0.67 at 2023-11-21 08:32:33 +0000
2023-11-21T08:32:33 [I|app|b3bdd2eb] Processing by Katello::Api::Rhsm::CandlepinProxiesController#serials as JSON
2023-11-21T08:32:33 [I|app|b3bdd2eb] Parameters: {"id"=>"594941dc-b28c-4041-bc85-92d39a92a3d4"}
2023-11-21T08:32:33 [I|app|b3bdd2eb] Completed 200 OK in 65ms (Views: 0.3ms | ActiveRecord: 18.8ms | Allocations: 5904)
2023-11-21T08:32:33 [I|app|506080c6] Started GET "/rhsm/status" for 10.69.0.67 at 2023-11-21 08:32:33 +0000
2023-11-21T08:32:33 [I|app|506080c6] Processing by Katello::Api::Rhsm::CandlepinProxiesController#server_status as JSON
2023-11-21T08:32:33 [I|app|506080c6] Completed 200 OK in 21ms (Views: 0.3ms | ActiveRecord: 1.1ms | Allocations: 4557)
2023-11-21T08:32:33 [I|app|785f33b0] Started GET "/rhsm/consumers/594941dc-b28c-4041-bc85-92d39a92a3d4/accessible_content" for 10.69.0.67 at 2023-11-21 08:32:33 +0000
2023-11-21T08:32:33 [I|app|785f33b0] Processing by Katello::Api::Rhsm::CandlepinProxiesController#get as JSON
2023-11-21T08:32:33 [I|app|785f33b0] Parameters: {"id"=>"594941dc-b28c-4041-bc85-92d39a92a3d4"}
2023-11-21T08:32:33 [I|app|785f33b0] Completed 304 Not Modified in 25ms (Views: 0.2ms | ActiveRecord: 1.1ms | Allocations: 4585)
2023-11-21T08:32:33 [I|app|adcda231] Started GET "/notification_recipients" for 10.107.17.18 at 2023-11-21 08:32:33 +0000
2023-11-21T08:32:33 [I|app|adcda231] Processing by NotificationRecipientsController#index as JSON
2023-11-21T08:32:33 [I|app|adcda231] Completed 200 OK in 5ms (Views: 0.1ms | ActiveRecord: 0.6ms | Allocations: 1638)
2023-11-21T08:32:33 [I|app|c7443d67] Started GET "/rhsm/consumers/594941dc-b28c-4041-bc85-92d39a92a3d4" for 10.69.0.67 at 2023-11-21 08:32:33 +0000
2023-11-21T08:32:33 [I|app|c7443d67] Processing by Katello::Api::Rhsm::CandlepinProxiesController#consumer_show as JSON
2023-11-21T08:32:33 [I|app|c7443d67] Parameters: {"id"=>"594941dc-b28c-4041-bc85-92d39a92a3d4"}
2023-11-21T08:32:33 [I|app|c7443d67] Completed 200 OK in 39ms (Views: 3.0ms | ActiveRecord: 2.0ms | Allocations: 12105)
2023-11-21T08:32:33 [I|app|9f455928] Started GET "/rhsm/consumers/594941dc-b28c-4041-bc85-92d39a92a3d4/content_overrides" for 10.69.0.67 at 2023-11-21 08:32:33 +0000
2023-11-21T08:32:33 [I|app|9f455928] Processing by Katello::Api::Rhsm::CandlepinProxiesController#get as JSON
2023-11-21T08:32:33 [I|app|9f455928] Parameters: {"id"=>"594941dc-b28c-4041-bc85-92d39a92a3d4"}
2023-11-21T08:32:33 [I|app|9f455928] Completed 200 OK in 40ms (Views: 0.1ms | ActiveRecord: 1.0ms | Allocations: 7589)
2023-11-21T08:32:35 [I|app|4e2ed41b] Started GET "/rhsm/consumers/594941dc-b28c-4041-bc85-92d39a92a3d4" for 10.69.0.67 at 2023-11-21 08:32:35 +0000
2023-11-21T08:32:35 [I|app|4e2ed41b] Processing by Katello::Api::Rhsm::CandlepinProxiesController#consumer_show as JSON
2023-11-21T08:32:35 [I|app|4e2ed41b] Parameters: {"id"=>"594941dc-b28c-4041-bc85-92d39a92a3d4"}
2023-11-21T08:32:35 [I|app|4e2ed41b] Completed 200 OK in 75ms (Views: 3.2ms | ActiveRecord: 35.3ms | Allocations: 11964)
2023-11-21T08:32:37 [I|aud|d41d43ac] Katello::Repository (279) update event on last_contents_changed 2023-11-16 13:49:06 UTC, 2023-11-21 08:32:37 UTC
2023-11-21T08:32:37 [I|aud|d41d43ac] Katello::Repository (279) update event on version_href /pulp/api/v3/repositories/container/container/018bc8fe-20ec-7e75-b44f-dad1737fd8a6/versio
ns/9/, /pulp/api/v3/repositories/container/container/018bc8fe-20ec-7e75-b44f-dad1737fd8a6/versions/10/
2023-11-21T08:32:37 [I|aud|6a90e2f3] Katello::Repository (278) update event on last_contents_changed 2023-11-16 13:48:50 UTC, 2023-11-21 08:32:37 UTC
2023-11-21T08:32:37 [I|aud|6a90e2f3] Katello::Repository (278) update event on version_href /pulp/api/v3/repositories/container/container/018bc8fc-53de-7ba7-a542-02475efe4654/versio
ns/10/, /pulp/api/v3/repositories/container/container/018bc8fc-53de-7ba7-a542-02475efe4654/versions/11/
2023-11-21T08:32:38 [I|app|d41d43ac] CONTENT_INDEX type=Katello::DockerManifest success=true duration=227.49
2023-11-21T08:32:38 [I|app|d41d43ac] CONTENT_INDEX type=Katello::DockerManifestList success=true duration=194.99
2023-11-21T08:32:38 [I|app|d41d43ac] CONTENT_INDEX type=Katello::DockerTag success=true duration=160.91
2023-11-21T08:32:38 [I|aud|d41d43ac] Katello::Repository (279) update event on last_indexed 2023-11-16 13:49:08 UTC, 2023-11-21 08:32:38 UTC
2023-11-21T08:32:38 [I|bac|d41d43ac] Task {label: , execution_plan_id: 13460027-f297-417f-b6d8-dc8db880eb1a} state changed: pending
2023-11-21T08:32:38 [I|bac|d41d43ac] Task {label: Actions::Katello::Repository::CapsuleSync, id: 0dcacc11-fdca-41da-8946-2a6b10737742, execution_plan_id: 13460027-f297-417f-b6d8-dc8
db880eb1a} state changed: planning
2023-11-21T08:32:39 [I|bac|d41d43ac] Task {label: Actions::Katello::Repository::CapsuleSync, id: 0dcacc11-fdca-41da-8946-2a6b10737742, execution_plan_id: 13460027-f297-417f-b6d8-dc8
db880eb1a} state changed: planned
2023-11-21T08:32:39 [I|bac|d41d43ac] Task {label: Actions::Katello::Repository::CapsuleSync, id: 0dcacc11-fdca-41da-8946-2a6b10737742, execution_plan_id: 13460027-f297-417f-b6d8-dc8
db880eb1a} state changed: running
2023-11-21T08:32:39 [I|bac|d41d43ac] Task {label: Actions::Katello::Repository::CapsuleSync, id: 0dcacc11-fdca-41da-8946-2a6b10737742, execution_plan_id: 13460027-f297-417f-b6d8-dc8
db880eb1a} state changed: stopped result: success
2023-11-21T08:32:39 [I|app|6a90e2f3] CONTENT_INDEX type=Katello::DockerManifest success=true duration=202.89
2023-11-21T08:32:39 [I|bac|d41d43ac] Task {label: Actions::Katello::Repository::Sync, id: bcd3b832-70fc-4b7d-a030-eb7633c89aeb, execution_plan_id: 8df8c980-8343-426b-a2a5-1fde56f74a
ff} state changed: stopped result: success
2023-11-21T08:32:39 [I|app|6a90e2f3] CONTENT_INDEX type=Katello::DockerManifestList success=true duration=117.54
2023-11-21T08:32:39 [I|app|6a90e2f3] CONTENT_INDEX type=Katello::DockerTag success=true duration=204.31
2023-11-21T08:32:39 [I|aud|6a90e2f3] Katello::Repository (278) update event on last_indexed 2023-11-16 13:48:51 UTC, 2023-11-21 08:32:39 UTC
2023-11-21T08:32:39 [I|bac|6a90e2f3] Task {label: , execution_plan_id: 4c17512d-849f-4ac5-a4f5-b095e05372cc} state changed: pending
2023-11-21T08:32:39 [I|bac|6a90e2f3] Task {label: Actions::Katello::Repository::CapsuleSync, id: ff8db92f-8f2b-4db2-8a52-1c55e49a1937, execution_plan_id: 4c17512d-849f-4ac5-a4f5-b09
5e05372cc} state changed: planning
2023-11-21T08:32:39 [I|bac|6a90e2f3] Task {label: Actions::Katello::Repository::CapsuleSync, id: ff8db92f-8f2b-4db2-8a52-1c55e49a1937, execution_plan_id: 4c17512d-849f-4ac5-a4f5-b09
5e05372cc} state changed: planned
2023-11-21T08:32:39 [I|bac|6a90e2f3] Task {label: Actions::Katello::Repository::CapsuleSync, id: ff8db92f-8f2b-4db2-8a52-1c55e49a1937, execution_plan_id: 4c17512d-849f-4ac5-a4f5-b09
5e05372cc} state changed: running
2023-11-21T08:32:39 [I|bac|6a90e2f3] Task {label: Actions::Katello::Repository::CapsuleSync, id: ff8db92f-8f2b-4db2-8a52-1c55e49a1937, execution_plan_id: 4c17512d-849f-4ac5-a4f5-b09
5e05372cc} state changed: stopped result: success
2023-11-21T08:32:39 [I|bac|6a90e2f3] Task {label: Actions::Katello::Repository::Sync, id: 8342cac2-dc71-4d57-9920-43907dec71ec, execution_plan_id: acc2a2f3-1d63-441a-8b37-9992b1ce62
86} state changed: stopped result: success