yum fails on all rhel7 clients with “Error: requested datatype primary not available”. Widely accepted fix is to republish metadata for affected repo. When I attempt this via “Products → ‘Redhat Enterprise Linux Server’ → ‘Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server’ → Republish Metadata” however, the process only ever gets to 50% and enters State:Suspended and never finishes
Expected outcome: Republish Metadata completes
Foreman and Proxy versions: 1.24.3-1.el7
Foreman and Proxy plugin versions: Unsure
Distribution and version:
Other relevant data:
Action:
Actions::Pulp::Repository::DistributorPublish
State:suspended
Input:
{"smart_proxy_id"=>1,
"options"=>
{"force"=>true,
"source_repository"=>nil,
"matching_content"=>false,
"dependency"=>nil,
"repository_creation"=>false},
"dependency"=>nil,
"repository_id"=>2,
"remote_user"=>"admin",
"remote_cp_user"=>"admin",
"current_request_id"=>nil,
"current_timezone"=>"Australia/Darwin",
"current_user_id"=>4,
"current_organization_id"=>1,
"current_location_id"=>2}
Output:
{"pulp_tasks"=>
[{"exception"=>nil,
"task_type"=>"pulp.server.managers.repo.publish.publish",
"_href"=>"/pulp/api/v2/tasks/5116b48b-88b1-45f5-97f1-83eb03250e37/",
"task_id"=>"5116b48b-88b1-45f5-97f1-83eb03250e37",
"tags"=>
["pulp:repository:88550c21-537a-477c-a9a7-eb1d3f93c662",
"pulp:action:publish"],
"finish_time"=>nil,
"_ns"=>"task_status",
"start_time"=>"2021-10-29T00:23:09Z",
"traceback"=>nil,
"spawned_tasks"=>[],
"progress_report"=>
{"88550c21-537a-477c-a9a7-eb1d3f93c662"=>
[{"num_success"=>1,
"description"=>"Initializing repo metadata",
"step_type"=>"initialize_repo_metadata",
"items_total"=>1,
"state"=>"FINISHED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"8e8c4445-92b4-4e08-a2f0-08e4e47a3d8f",
"num_processed"=>1},
{"num_success"=>0,
"description"=>"Publishing Distribution files",
"step_type"=>"distribution",
"items_total"=>0,
"state"=>"FINISHED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"b258c520-b89a-42f0-8d47-6a19ee217a76",
"num_processed"=>0},
{"num_success"=>28057,
"description"=>"Publishing RPMs",
"step_type"=>"rpms",
"items_total"=>32332,
"state"=>"IN_PROGRESS",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"bbbaff33-008c-4450-ad6b-dccc9743e0db",
"num_processed"=>28057},
{"num_success"=>0,
"description"=>"Publishing Delta RPMs",
"step_type"=>"drpms",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"3c4b6898-850c-46c3-b343-6c1232b524e6",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Publishing Errata",
"step_type"=>"errata",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"35217820-1ffe-4252-a2e4-aab66d35eb1d",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Publishing Modules",
"step_type"=>"modules",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"13c7a8f6-220a-470d-a3d2-a32b214c1309",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Publishing Comps file",
"step_type"=>"comps",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"35739569-0473-4932-b864-501d9cbe6b37",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Publishing Metadata.",
"step_type"=>"metadata",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"d1026467-117d-43c9-93c5-42afd85b8259",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Closing repo metadata",
"step_type"=>"close_repo_metadata",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"22c3398a-a944-453e-8acb-f5911ef79faa",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Generating sqlite files",
"step_type"=>"generate sqlite",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"836582c0-08b5-4874-8e0f-130b98906452",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Removing old repodata",
"step_type"=>"remove_old_repodata",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"ac89df36-7275-4f62-9b68-42f3fc08f631",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Generating HTML files",
"step_type"=>"repoview",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"04b93232-74a1-4d0c-b878-6e8744a50ba6",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Publishing files to web",
"step_type"=>"publish_directory",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"668ac84b-1c9e-459d-a621-a938c48969f8",
"num_processed"=>0},
{"num_success"=>0,
"description"=>"Writing Listings File",
"step_type"=>"initialize_repo_metadata",
"items_total"=>1,
"state"=>"NOT_STARTED",
"error_details"=>[],
"details"=>"",
"num_failures"=>0,
"step_id"=>"da2782f6-658d-40ad-bbfe-b63778d7485e",
"num_processed"=>0}]},
"queue"=>"reserved_resource_worker-3@platnt-adm-13p.platnt.internal.dq2",
"state"=>"running",
"worker_name"=>"reserved_resource_worker-3@platnt-adm-13p.platnt.internal",
"result"=>nil,
"error"=>nil,
"_id"=>{"$oid"=>"617b3eed14ac25aa9ab67b3e"},
"id"=>"617b3eed14ac25aa9ab67b3e"}],
"poll_attempts"=>{"total"=>148, "failed"=>0}}
Further info: I can successfully republish metadata for all other repos in our list. It is only this particular repo (Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server) that has the issue.
The republish process has previously worked for this repo and/but has needed to be used many times for reasons I cannot explain despite many hours googling it. It usually fails at the same point but after running “katello-service restart” and then resuming the task, it will often complete. I have tried that numerous times on this particular occasion but this time it will not complete no matter what.
I am only proficient with the web UI controls of katello (I inherited the system from its original installer who no longer works for the company) so I apologise if there is more I should have provided by way of logs etc (I’m not sure which logs will produce the output required to troubleshoot this issue).
Any help would be most appreciated.