Error during upload Product (Repository type file)

Problem:
Can not upload files in Product (Repository type file).
Yum repository is working.

Error output:

Any ideas?

Expected outcome:

Foreman and Proxy versions:

  • foreman-2.3.2-1.el7.noarch
  • katello-3.18.1-1.el7.noarch

Pulp
Version2.1.0
Pulp server version2.21.5

Pulpcore
Version2.1.0
Supported Content Types
docker
file
deb (supported, but overridden to use Pulp 2)
yum (supported, but overridden to use Pulp 2)

Foreman and Proxy plugin versions:

Distribution and version:

Other relevant data:

Hi @senetm ! Is there a chance that you upgraded from Foreman 2.3.x to 2.3.2 without running the foreman-installer? I wonder if there are any other stack traces in journalctl related to this error (not including what you’ve shared from /var/log/foreman/production.log)

Hi…i have upgraded from Foreman 2.2.x direct to 2.3.2 with foreman-installer. I have try to migrate from pulp2 to pulp3 and unfortionaly not successful.

Thank you. This is the most interesting part of your pastebin:

Response body: {"detail":"You do not have permission to perform this action."}undefined method each’ for nil:NilClass`

When you perform the file upload do you see a stack trace in journalctl related to this error? The one you shared is from the Forman app in /var/log/foreman/production.log which relates to the Ruby stack. I’m looking for a Python stack trace in particular.

Also, what error did you encounter with the pulp 2->3 migration? Could be related.

Hi…on journalctl there is nothing related to this.
I have lots of this errors:

Feb 24 10:55:09 foreman.com qdrouterd[66860]: 2021-02-24 10:55:09.879056 +0100 SERVER (info) [C231479] Accepted connection to :5647 from xx.xx.xx.172:59278
Feb 24 10:55:09 foreman.com qdrouterd[66860]: 2021-02-24 10:55:09.894206 +0100 ROUTER (info) [C231479] Connection Opened: dir=in host=xx.xx.xx.172:59278 vhost= encrypted=TLSv1/SSLv3 auth=ANONYMOUS user=anonymous container_id=5d8653b7-9792-4e1e-bed3-e13ec0860419 props=

By migrating i get this:

Another thing that i have with pulp is that he stops working time to time with error:

Component Status Message
candlepin OK
candlepin_auth OK
foreman_tasks OK
katello_events OK 106 Processed, 1 Failed
candlepin_events OK 1788 Processed, 2 Failed
pulp3 OK
pulp FAIL Not all necessary pulp workers running at https://foreman.lan/pulp/api/v2.
pulp_auth FAIL Skipped pulp_auth check after failed pulp check

Dropping a link to another thread that has a similar issue (although no resolution). Likely the resolution we find here will address it as well.

Now i am getting ( katello-3.18.2-1.el7.noarch and foreman-2.3.3-1.el7.noarch):

foreman-maintain content prepare
Running Prepare content for Pulp 3

Prepare content for Pulp 3:
Starting task.
2021-03-26 09:33:09 +0100: Content migration starting. Migration failed, You will want to investigate: https://foreman.com/foreman_tasks/tasks/fae253a9-0f64-48da-97f0-1326b8e6a467
rake aborted!
ForemanTasks::TaskError: Task fae253a9-0f64-48da-97f0-1326b8e6a467: Pulp2to3MigrationClient::ApiError: Error message: the server returns an error
HTTP status code: 400
Response headers: {“date”=>“Fri, 26 Mar 2021 08:33:10 GMT”, “server”=>“gunicorn/20.0.4”, “content-type”=>“application/json”, “vary”=>“Accept,Cookie”, “allow”=>“POST, OPTIONS”, “x-frame-options”=>“SAMEORIGIN”, “content-length”=>“57”, “via”=>“1.1 foreman.com”, “connection”=>“close”}
Response body: [“Only one migration plan can run or be reset at a time”]
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.2.1/lib/katello/tasks/pulp3_migration.rake:33:in block (2 levels) in <top (required)>' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in <top (required)>’
Tasks: TOP => katello:pulp3_migration
(See full trace by running task with --trace)
[FAIL]
Failed executing foreman-rake katello:pulp3_migration, exit status 1

Scenario [Prepare content for Pulp 3] failed.

The following steps ended up in failing state:

[content-prepare]

Resolve the failed steps and rerun
the command. In case the failures are false positives,
use --whitelist=“content-prepare”

foreman-rake katello:pulp3_migration --trace

** Invoke katello:pulp3_migration (first_time)
** Invoke dynflow:client (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute dynflow:client
** Execute katello:pulp3_migration
Starting task.
2021-03-26 09:34:30 +0100: Content migration starting. Migration failed, You will want to investigate: https://foreman.com/foreman_tasks/tasks/17132279-07da-457a-b488-d26f57fcef92
rake aborted!
ForemanTasks::TaskError: Task 17132279-07da-457a-b488-d26f57fcef92: Pulp2to3MigrationClient::ApiError: Error message: the server returns an error
HTTP status code: 400
Response headers: {“date”=>“Fri, 26 Mar 2021 08:34:31 GMT”, “server”=>“gunicorn/20.0.4”, “content-type”=>“application/json”, “vary”=>“Accept,Cookie”, “allow”=>“POST, OPTIONS”, “x-frame-options”=>“SAMEORIGIN”, “content-length”=>“57”, “via”=>“1.1 foreman.com”, “connection”=>“close”}
Response body: [“Only one migration plan can run or be reset at a time”]
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.2.1/lib/katello/tasks/pulp3_migration.rake:33:in block (2 levels) in <top (required)>' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in block in execute’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in each' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:251:in execute’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:195:in block in invoke_with_call_chain' /opt/rh/rh-ruby25/root/usr/share/ruby/monitor.rb:226:in mon_synchronize’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:188:in invoke_with_call_chain' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/task.rb:181:in invoke’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:160:in invoke_task' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in block (2 levels) in top_level’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in each' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:116:in block in top_level’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:125:in run_with_threads' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:110:in top_level’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:83:in block in run' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:186:in standard_exception_handling’
/opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/lib/rake/application.rb:80:in run' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in <top (required)>’
/opt/rh/rh-ruby25/root/usr/bin/rake:23:in load' /opt/rh/rh-ruby25/root/usr/bin/rake:23:in
Tasks: TOP => katello:pulp3_migration