Subscription Manifest Upload Fails (v1.17.1)

Problem:

When attempting to upload a subscription manifest either by web GUI or hammer CLI we get an ‘API 500 Internal Server Error’ with an Exception message stating 'wrong number of arguments (given 4, expected 2…3)

[ERROR 2018-07-03T13:46:01 API] 500 Internal Server Error
[ERROR 2018-07-03T13:46:01 Exception] wrong number of arguments (given 4, expected 2..3)
Manifest upload failed:
  wrong number of arguments (given 4, expected 2..3)
[ERROR 2018-07-03T13:46:01 Exception]

RestClient::InternalServerError (500 Internal Server Error):
    /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.1/lib/restclient/abstract_response.rb:223:in `exception_with_response'
    /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.1/lib/restclient/abstract_response.rb:103:in `return!'
    /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.2.2/lib/apipie_bindings/api.rb:353:in `block in rest_client_call_block'
    /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.1/lib/restclient/request.rb:807:in `process_result'
    /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.1/lib/restclient/request.rb:725:in `block in transmit'
    /opt/rh/rh-ruby24/root/usr/share/ruby/net/http.rb:877:in `start'
    /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.1/lib/restclient/request.rb:715:in `transmit'
    /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.1/lib/restclient/request.rb:145:in `execute'
    /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.1/lib/restclient/request.rb:52:in `execute'
    /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-2.0.1/lib/restclient/resource.rb:67:in `post'
    /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.2.2/lib/apipie_bindings/api.rb:327:in `call_client'
    /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.2.2/lib/apipie_bindings/api.rb:240:in `http_call'
    /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.2.2/lib/apipie_bindings/api.rb:190:in `call_action'
    /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.2.2/lib/apipie_bindings/api.rb:185:in `call'
    /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.2.2/lib/apipie_bindings/resource.rb:21:in `call'
    /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.12.0/lib/hammer_cli/apipie/command.rb:43:in `send_request'
    /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.12.0/lib/hammer_cli_foreman/commands.rb:168:in `send_request'
    /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman_tasks-0.0.12/lib/hammer_cli_foreman_tasks/async_command.rb:14:in `execute'
    /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.1.2/lib/clamp/command.rb:63:in `run'
    /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.12.0/lib/hammer_cli/abstract.rb:29:in `run'
    /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.1.2/lib/clamp/subcommand/execution.rb:11:in `execute'
    /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.1.2/lib/clamp/command.rb:63:in `run'
    /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.12.0/lib/hammer_cli/abstract.rb:29:in `run'
    /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.1.2/lib/clamp/subcommand/execution.rb:11:in `execute'
    /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.1.2/lib/clamp/command.rb:63:in `run'
    /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.12.0/lib/hammer_cli/abstract.rb:29:in `run'
    /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.1.2/lib/clamp/command.rb:132:in `run'
    /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.12.0/bin/hammer:142:in `<top (required)>'
    /usr/bin/hammer:22:in `load'
    /usr/bin/hammer:22:in `<main>'

Expected outcome:

New manifest file is uploaded

Foreman and Proxy versions:

v 1.17.1

Foreman and Proxy plugin versions:

Other relevant data:
[e.g. logs from Foreman and/or the Proxy, modified templates, commands issued, etc]

Do you happen to see any errors on the server /var/log/foreman/production.log?

Command I ran:

hammer -v subscription upload --file <manifest.zip> --organization-label

Error messages in production.log:

2018-07-04T10:56:34 50f54a25 [background] [E] wrong number of arguments (given 4, expected 2..3) (ArgumentError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `block in plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in `propagate_candlepin_errors'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:16:in `block in rollback_on_error'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:15:in `rollback_on_error'
 | /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:8:in `call'
2018-07-04T10:56:34 50f54a25 [dynflow] [D]          Step 962db5a9-d853-420d-b9da-1c0331b98974:237   running >>     error in phase     Plan Actions::Pulp::Repository::Refresh
2018-07-04T10:56:34 50f54a25 [background] [E] wrong number of arguments (given 4, expected 2..3) (ArgumentError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `block in plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in `propagate_candlepin_errors'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:16:in `block in rollback_on_error'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:15:in `rollback_on_error'
 | /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:8:in `call'
2018-07-04T10:56:34 50f54a25 [dynflow] [D]          Step 962db5a9-d853-420d-b9da-1c0331b98974:236   running >>     error in phase     Plan Actions::Katello::Repository::RefreshRepository
2018-07-04T10:56:34 50f54a25 [background] [E] wrong number of arguments (given 4, expected 2..3) (ArgumentError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `block in plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in `propagate_candlepin_errors'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:16:in `block in rollback_on_error'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:15:in `rollback_on_error'
 | /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:8:in `call'
2018-07-04T10:56:34 50f54a25 [dynflow] [D]          Step 962db5a9-d853-420d-b9da-1c0331b98974: 1   running >>     error in phase     Plan Actions::Katello::Organization::ManifestImport
2018-07-04T10:56:35 50f54a25 [app] [E] ArgumentError: wrong number of arguments (given 4, expected 2..3)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `block in plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in `propagate_candlepin_errors'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:16:in `block in rollback_on_error'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:15:in `rollback_on_error'
 | /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:8:in `call'
2018-07-04T10:56:35 50f54a25 [app] [I] Completed 500 Internal Server Error in 6742ms (Views: 0.3ms | ActiveRecord: 56.7ms)
2018-07-04T10:56:35 50f54a25 [app] [D] With body: {"displayMessage":"wrong number of arguments (given 4, expected 2..3)","errors":["wrong number of arguments (given 4, expected 2..3)"]}

Error messages in production.log when uploading a manifest from the web application:

2018-07-04T11:01:58 f73a16e7 [background] [E] wrong number of arguments (given 4, expected 2..3) (ArgumentError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `block in plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in `propagate_candlepin_errors'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:16:in `block in rollback_on_error'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:15:in `rollback_on_error'
 | /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:8:in `call'
2018-07-04T11:01:58 f73a16e7 [dynflow] [D]          Step 733f4532-9bc1-49e8-ad59-e4c74e20a304:237   running >>     error in phase     Plan Actions::Pulp::Repository::Refresh
2018-07-04T11:01:58 f73a16e7 [background] [E] wrong number of arguments (given 4, expected 2..3) (ArgumentError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `block in plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in `propagate_candlepin_errors'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:16:in `block in rollback_on_error'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:15:in `rollback_on_error'
 | /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:8:in `call'
2018-07-04T11:01:58 f73a16e7 [dynflow] [D]          Step 733f4532-9bc1-49e8-ad59-e4c74e20a304:236   running >>     error in phase     Plan Actions::Katello::Repository::RefreshRepository
2018-07-04T11:01:58 f73a16e7 [background] [E] wrong number of arguments (given 4, expected 2..3) (ArgumentError)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `block in plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in `propagate_candlepin_errors'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:16:in `block in rollback_on_error'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:15:in `rollback_on_error'
 | /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:8:in `call'
2018-07-04T11:01:58 f73a16e7 [dynflow] [D]          Step 733f4532-9bc1-49e8-ad59-e4c74e20a304: 1   running >>     error in phase     Plan Actions::Katello::Organization::ManifestImport
2018-07-04T11:01:58 f73a16e7 [app] [E] ArgumentError: wrong number of arguments (given 4, expected 2..3)
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `block in plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:19:in `propagate_candlepin_errors'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.6.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:5:in `plan'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `block in with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/action.rb:444:in `with_error_handling'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:16:in `block in rollback_on_error'
 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-0.8.37/lib/dynflow/middleware/common/transaction.rb:15:in `rollback_on_error'
 | /usr/share/foreman/lib/middleware/catch_json_parse_errors.rb:8:in `call'
2018-07-04T11:01:58 f73a16e7 [app] [I] Completed 500 Internal Server Error in 7503ms (Views: 0.3ms | ActiveRecord: 103.0ms)
2018-07-04T11:01:58 f73a16e7 [app] [D] With body: {"displayMessage":"wrong number of arguments (given 4, expected 2..3)","errors":["wrong number of arguments (given 4, expected 2..3)"]}

The API POST request and parameters are also captured in the log output below. There looks to be a parameters passed which is not the in the API documentation - “api_version”. I’m not sure on the significance of the content string. Could there be a parsing error also? Just guessing really but hopefully someone knows a little more than me :slight_smile:

2018-07-04T11:23:10 0401c1b4 [app] [I] Started POST “/katello/api/organizations/6/subscriptions/upload” for 22.188.84.8 at 2018-07-04 11:23:10 +0100
2018-07-04T11:23:10 0401c1b4 [app] [I] Processing by Katello::Api::V2::SubscriptionsController#upload as JSON
2018-07-04T11:23:10 0401c1b4 [app] [I] Parameters: {“content”=>#<ActionDispatch::Http::UploadedFile:0x00007f285b8ecf30 @tempfile=#Tempfile:/tmp/RackMultipart20180704-2813-h4c08o.zip, @original_filename=“manifest.zip”, @content_type=“application/zip”, @headers=“Content-Disposition: form-data; name=“content”; filename=“manifest.zip”\r\nContent-Type: application/zip\r\n”>, “api_version”=>“v2”, “organization_id”=>“6”}

It appears that you may be getting an error from candlepin and then we are not handling the error properly. Could you check /var/log/candlepin/errors.log or /var/log/candlepin/candlepin.log to see if there are any errors there?

Hi, I work with Chris and replying on his behalf.

We have tried to re upload the manifest again and monitored both log files that you have specified and cannot see anything of interest there. I have attached the log output below in case you can see anything wrong.

$ tail -f candlepin.log
2018-07-05 15:31:20,011 [thread=QuartzScheduler_Worker-7] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:31:20,013 [thread=QuartzScheduler_Worker-7] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:31:25,009 [thread=QuartzScheduler_Worker-13] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:31:25,011 [thread=QuartzScheduler_Worker-13] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:31:30,011 [thread=QuartzScheduler_Worker-5] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:31:30,011 [thread=QuartzScheduler_Worker-5] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:31:35,008 [thread=QuartzScheduler_Worker-1] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:31:35,009 [thread=QuartzScheduler_Worker-1] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:31:40,010 [thread=QuartzScheduler_Worker-4] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:31:40,010 [thread=QuartzScheduler_Worker-4] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:31:45,008 [thread=QuartzScheduler_Worker-15] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:31:45,008 [thread=QuartzScheduler_Worker-15] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:31:50,012 [thread=QuartzScheduler_Worker-12] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:31:50,016 [thread=QuartzScheduler_Worker-12] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:31:55,013 [thread=QuartzScheduler_Worker-10] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:31:55,014 [thread=QuartzScheduler_Worker-10] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:31:57,047 [thread=http-bio-8443-exec-8] [req=e6379492-4ae1-427a-9399-f457077b1446, org=, csid=] INFO  org.candlepin.common.filter.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/69139313-ea41-4a91-85ce-fbd223be5413/compliance
2018-07-05 15:31:57,103 [thread=http-bio-8443-exec-8] [req=e6379492-4ae1-427a-9399-f457077b1446, org=gisdev, csid=] INFO  org.candlepin.common.filter.LoggingFilter - Response: status=200, content-type="application/json", time=56
2018-07-05 15:31:57,173 [thread=http-bio-8443-exec-2] [req=8e323d94-7472-4b6d-bd79-1a369ffe7248, org=, csid=] INFO  org.candlepin.common.filter.LoggingFilter - Request: verb=GET, uri=/candlepin/consumers/69139313-ea41-4a91-85ce-fbd223be5413/entitlements?exclude=certificates.key&exclude=certificates.cert
2018-07-05 15:31:57,258 [thread=http-bio-8443-exec-2] [req=8e323d94-7472-4b6d-bd79-1a369ffe7248, org=gisdev, csid=] INFO  org.candlepin.common.filter.LoggingFilter - Response: status=200, content-type="application/json", time=86
2018-07-05 15:32:00,010 [thread=QuartzScheduler_Worker-3] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:32:00,012 [thread=QuartzScheduler_Worker-3] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:32:05,010 [thread=QuartzScheduler_Worker-2] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:32:05,012 [thread=QuartzScheduler_Worker-2] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:32:10,009 [thread=QuartzScheduler_Worker-11] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:32:10,013 [thread=QuartzScheduler_Worker-11] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler
2018-07-05 15:32:15,015 [thread=QuartzScheduler_Worker-14] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Deleting 0 cancelled jobs from scheduler.
2018-07-05 15:32:15,017 [thread=QuartzScheduler_Worker-14] [job=CancelJobJob-33c080ec-a96e-4a2b-aa43-4c5c12f40432, org=, csid=] INFO  org.candlepin.pinsetter.core.PinsetterKernel - Finished deleting jobs from scheduler

There were no logs in the error.log file that were generated at the time of upload.

Thanks
George

Hi All,

Is there any update on the above? We are really struggling to find a solution to this problem.

Regards,
George