Pulpcore 3.18 RPM pipeline failed:
https://ci.theforeman.org/job/pulpcore-3.18-rpm-pipeline/225/
foreman-pipeline-pulpcore-3.18-centos8-stream-pulpcore (failed) (remote job)
foreman-pipeline-pulpcore-3.18-centos9-stream-pulpcore (failed) (remote job)
foreman-pipeline-pulpcore-3.18-centos7-pulpcore (failed) (remote job)
I think something is wrong with the forklift config since it doesn’t recognize the box.
[2022-09-29T14:38:57.451Z] + ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -F ./jenkins-jobs/centos.org/ansible/ssh_config duffy_box 'source otel_env && cd forklift && ansible-playbook playbooks/collect_debug.yml --limit pipe-pulpcore-*3.18-centos9-stream -e pipeline_version=3.18 -e pipeline_os=centos9-stream -e pipeline_type=pulpcore -e foreman_expected_version='
[2022-09-29T14:38:57.953Z] e[0;35m[DEPRECATION WARNING]: ANSIBLE_CALLBACK_WHITELIST option, normalizing names to e[0m
[2022-09-29T14:38:57.953Z] e[0;35mnew standard, use ANSIBLE_CALLBACKS_ENABLED instead. This feature will be e[0m
[2022-09-29T14:38:57.953Z] e[0;35mremoved from ansible-core in version 2.15. Deprecation warnings can be disablede[0m
[2022-09-29T14:38:57.953Z] e[0;35m by setting deprecation_warnings=False in ansible.cfg.e[0m
[2022-09-29T14:38:59.157Z] e[1;35m[WARNING]: Could not match supplied host pattern, ignoring: pipe-e[0m
[2022-09-29T14:38:59.157Z] e[1;35mpulpcore-*3.18-centos9-streame[0m
[2022-09-29T14:38:59.157Z] e[0;31mERROR! Specified hosts and/or --limit does not match any hostse[0m
Edit: oh, it fails here:
2022-09-29T14:38:56.723Z] e[0;31m /usr/share/gems/gems/fog-libvirt-0.7.0/lib/fog/libvirt/requests/compute/vm_action.rb:7:in `create': Call to virDomainCreateWithFlags failed: Cannot access storage file '/root/.local/share/libvirt/images/forklift_pipe-pulpcore-server-3.18-centos9-stream.img' (as uid:107, gid:107): Permission denied (Libvirt::Error)e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/gems/gems/fog-libvirt-0.7.0/lib/fog/libvirt/requests/compute/vm_action.rb:7:in `vm_action'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/gems/gems/fog-libvirt-0.7.0/lib/fog/libvirt/models/compute/server.rb:75:in `start'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/start_domain.rb:399:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/set_boot_order.rb:80:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/create_network_interfaces.rb:190:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/create_networks.rb:33:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/share_folders.rb:22:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/prepare_nfs_settings.rb:21:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/builtin/synced_folders.rb:87:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/builtin/delayed.rb:19:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/builtin/synced_folder_cleanup.rb:28:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/plugins/synced_folders/nfs/action_cleanup.rb:25:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/prepare_nfs_valid_ids.rb:14:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/builtin/provision.rb:80:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/create_domain.rb:443:in `call'e[0m
[2022-09-29T14:38:56.723Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/create_domain_volume.rb:94:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/handle_box_image.rb:123:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/builtin/handle_box.rb:56:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/handle_storage_pool.rb:63:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-libvirt-0.7.0/lib/vagrant-libvirt/action/set_name_of_domain.rb:34:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:127:in `block in finalize_action'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/builder.rb:149:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/runner.rb:89:in `block in run'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/util/busy.rb:19:in `busy'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/runner.rb:89:in `run'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/builtin/call.rb:53:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/builtin/box_check_outdated.rb:31:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/builtin/config_validate.rb:25:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/warden.rb:48:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/builder.rb:149:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/runner.rb:89:in `block in run'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/util/busy.rb:19:in `busy'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/action/runner.rb:89:in `run'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/machine.rb:246:in `action_raw'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/machine.rb:215:in `block in action'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/environment.rb:614:in `lock'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/machine.rb:201:in `call'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/machine.rb:201:in `action'e[0m
[2022-09-29T14:38:56.724Z] e[0;31m from /usr/share/vagrant/gems/gems/vagrant-2.2.19/lib/vagrant/batch_action.rb:86:in `block (2 levels) in run'e[0m
@evgeni any idea? This could be related to your multi-user work.
Yeah… Probably. It tries to access to root user session as the libvirt user, which, uh, fails…
I’ll look in detail tomorrow morning
Yeah, so I can repro the issue locally. It uploads the data to the local storage (/root/.local/share/libvirt) but then tries to access it via the system libvirtd which runs qemu as user qemu/107, not as root.
Given we’re switching away from this execution method, I am inclined to ignore this issue for now and finalize the EC2 migration.
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.