jcaniza
February 14, 2025, 11:50am
1
Problem:
Foreman and Proxy versions:
Name: foreman-tasks
Name: foreman_ansible
Name: foreman_puppet
Name: foreman_remote_execution
Name: foreman_webhooks
Name: katello
Foreman and Proxy plugin versions:
Name: dynflow
Name: script
Name: pulpcore
Name: ansible
Name: shellhooks
Name: tftp
Name: dhcp
Name: puppetca
Name: puppet
Distribution and version:
Other relevant data: Repository Layout ’ is ‘ansible-default ’.
I have published my Ansible collection using the command ansible-galaxy collection publish test/collection/test-collection-1.0.0.tar.gz -s https://artifactory.example.com/artifactory/api/ansible/ansible-repo-example --token=....
The structure created in the Artifactory repository is:
In the repository configuration of type ‘ansible_collection’ I have indicated the following fields:
Upstream URL : https://artifactory.example.com/artifactory/api/ansible/ansible-repo-example/api/v3/collections/test/collection/Upstream Authorization : user/token
Thanks in advance
To add more information, I add the response from the Artifactory API and the Ansible Galaxy API.
Artifactory:
{
"href": "/artifactory/api/ansible/core-ansible-collections/api/v3/collections/ansible/posix",
"namespace": "ansible",
"name": "posix",
"deprecated": false,
"versions_url": "/artifactory/api/ansible/core-ansible-collections/api/v3/collections/ansible/posix/versions/",
"highest_version": {
"href": "/artifactory/api/ansible/core-ansible-collections/api/v3/collections/ansible/posix/versions/2.0.0",
"version": "2.0.0"
},
"created_at": "2025-04-02T06:48:36.000265Z",
"updated_at": "2024-12-05T01:08:39.000000Z",
"download_count": 0
}
Galaxy:
{
"href": "/api/v3/plugin/ansible/content/published/collections/index/ansible/posix/",
"namespace": "ansible",
"name": "posix",
"deprecated": false,
"versions_url": "/api/v3/plugin/ansible/content/published/collections/index/ansible/posix/versions/",
"highest_version": {
"href": "/api/v3/plugin/ansible/content/published/collections/index/ansible/posix/versions/2.0.0/",
"version": "2.0.0"
},
"created_at": "2023-05-08T20:27:28.415377Z",
"updated_at": "2024-12-05T01:08:46.203837Z",
"download_count": 70024107
}
What does Pulp3 need to synchronise an Ansible Collection repository?
For example, for a repository of type File, it needs the existence of a PULP_MANIFEST file with certain content.
Thanks again.
sajha
April 9, 2025, 1:46pm
3
I am not a hundred percent sure of the structure of the upstream ansible repo expected by pulp. If you have the exact error message on the sync that may be helpful.@dralley ^
Thanks for your reply.
The task log is as follows:
And the complete log in /var/log/foreman/production.log:
[E|bac|c08b4ed8] Could not find 'available_versions' at https://artifactory.example.com/artifactory/api/ansible/core-ansible-collections/api/v3/collections/ansible/posix/ (Katello::Errors::Pulp3Error)
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/lib/actions/pulp3/abstract_async_task.rb:107:in `block in check_for_errors'
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/lib/actions/pulp3/abstract_async_task.rb:105:in `each'
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/lib/actions/pulp3/abstract_async_task.rb:105:in `check_for_errors'
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/lib/actions/pulp3/abstract_async_task.rb:161:in `poll_external_task'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action/polling.rb:100:in `poll_external_task_with_rescue'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action/polling.rb:22:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action/cancellable.rb:15:in `run'
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/lib/actions/pulp3/abstract_async_task.rb:10:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action.rb:590:in `block (3 levels) in execute_run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:28:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:20:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:33:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:24:in `call'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:28:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:20:in `pass'
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
c08b4ed8 | /usr/share/gems/gems/katello-4.14.1/app/lib/actions/middleware/remote_action.rb:16:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:24:in `call'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:28:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:20:in `pass'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
c08b4ed8 | /usr/share/gems/gems/activesupport-6.1.7.9/lib/active_support/execution_wrapper.rb:91:in `wrap'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:24:in `call'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:28:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:20:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action/progress.rb:29:in `with_progress_calculation'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action/progress.rb:15:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:24:in `call'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:28:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:20:in `pass'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/load_setting_values.rb:20:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:24:in `call'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:28:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:20:in `pass'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_request_id.rb:52:in `restore_current_request_id'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:24:in `call'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:28:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:20:in `pass'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:24:in `call'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:28:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:20:in `pass'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:24:in `call'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:28:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:20:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:33:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:24:in `call'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:28:in `pass'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware.rb:20:in `pass'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_user.rb:54:in `restore_curent_user'
c08b4ed8 | /usr/share/gems/gems/foreman-tasks-9.2.3/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/stack.rb:24:in `call'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/middleware/world.rb:31:in `execute'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action.rb:589:in `block (2 levels) in execute_run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action.rb:588:in `catch'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action.rb:588:in `block in execute_run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action.rb:491:in `block in with_error_handling'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action.rb:491:in `catch'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action.rb:491:in `with_error_handling'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action.rb:583:in `execute_run'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/action.rb:304:in `execute'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/execution_plan/steps/abstract.rb:168:in `with_meta_calculation'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/director.rb:95:in `execute'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/executors/sidekiq/worker_jobs.rb:12:in `block (2 levels) in perform'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/executors.rb:18:in `run_user_code'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/executors/sidekiq/worker_jobs.rb:10:in `block in perform'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/executors/sidekiq/worker_jobs.rb:26:in `with_telemetry'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `perform'
c08b4ed8 | /usr/share/gems/gems/dynflow-1.9.0/lib/dynflow/executors/sidekiq/serialization.rb:28:in `perform'
c08b4ed8 | [ sidekiq ]
c08b4ed8 | [ concurrent-ruby ]
Do you need more information?
sajha
April 10, 2025, 4:26pm
5
I was able to find the code path in pulp that goes through the available versions and throws this error here.
try:
api_data = parse_metadata(await downloader.run())
except (json.decoder.JSONDecodeError, ClientResponseError):
if root.endswith("/api/"):
raise
root = urljoin(root, "api/")
downloader = self.remote.get_downloader(url=root)
api_data = parse_metadata(await downloader.run())
if "available_versions" not in api_data:
raise RuntimeError(_("Could not find 'available_versions' at {}").format(root))
if "v3" in api_data.get("available_versions", {}):
api_version = 3
elif "v2" in api_data.get("available_versions", {}):
api_version = 2
else:
raise RuntimeError(_("Unsupported API versions at {}").format(root))
endpoint = f"{root}v{api_version}"
Checking the galaxy endpoint on my local pulp galaxy endpoint, I see this:
Without knowing all the details, I am assuming the artifactory is not listing available versions in the api view.
If that does not help, I would also check in https://discourse.pulpproject.org/ .
.
