Katello 2.2 - repo sync pauses at 53% with error

Support
1

I have just upgraded to the latest Katello RC (2.2).

This was supposed to solve the repo sync freezing issues by replacing qpid
0.28 with qpid 0.3.

When I try to sync a repo, it downloads all the packages, but then pauses
and goes into error state.

Here is the log.

The last line in the log, stating that 'block in
create_with_logging_context' is the same message I got with Katello 2.1 and
qpid .28.

Wasn't this supposed to be fixed in the new version? I have rebooted the
computer after the upgrade, so it should not be due to anything old hanging
around.

Here are the logs from the dynflow console :

Started at: 2015-04-16 19:32:32 UTC

Ended at: 2015-04-16 19:38:30 UTC

Real time: 357.88s

Execution time (excluding suspended state): 357.88s

Input:

··· --- dependency: - exception: task_type: pulp.server.managers.repo.sync.sync _href: /pulp/api/v2/tasks/0be07ad7-3451-4f4b-a23a-94d6a284dfed/ task_id: 0be07ad7-3451-4f4b-a23a-94d6a284dfed tags: - pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base - pulp:action:sync finish_time: '2015-04-16T19:09:05Z' _ns: task_status start_time: '2015-04-16T18:46:10Z' traceback: spawned_tasks: - _href: /pulp/api/v2/tasks/a0a945df-39ba-487e-9ca2-b9a08bfe2bbe/ task_id: a0a945df-39ba-487e-9ca2-b9a08bfe2bbe - _href: /pulp/api/v2/tasks/8ca9362c-fe5b-4034-8296-cada29f5744b/ task_id: 8ca9362c-fe5b-4034-8296-cada29f5744b progress_report: yum_importer: content: size_total: 1822408364 items_left: 0 items_total: 1739 state: FINISHED size_left: 0 details: rpm_total: 1739 rpm_done: 1739 drpm_total: 0 drpm_done: 0 error_details: [] comps: state: FINISHED distribution: items_total: 4 state: FINISHED error_details: [] items_left: 0 errata: state: FINISHED metadata: state: FINISHED queue: reserved_resource_worker-0@katello.dq state: finished worker_name: reserved_resource_worker-0@katello result: importer_type_id: yum_importer importer_id: yum_importer exception: repo_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base removed_count: 0 started: '2015-04-16T18:46:10Z' _ns: repo_sync_results completed: '2015-04-16T19:09:04Z' traceback: summary: content: state: FINISHED comps: state: FINISHED distribution: state: FINISHED errata: state: FINISHED metadata: state: FINISHED added_count: 1966 error_message: updated_count: 4779 details: content: size_total: 1822408364 items_left: 0 items_total: 1739 state: FINISHED size_left: 0 details: rpm_total: 1739 rpm_done: 1739 drpm_total: 0 drpm_done: 0 error_details: [] comps: state: FINISHED distribution: items_total: 4 state: FINISHED error_details: [] items_left: 0 errata: state: FINISHED metadata: state: FINISHED id: 553008d0229dcf0abe8162df result: success error: _id: $oid: 553003710fa7ca8c110a4c6e id: 55300371229dcf0ac430c3e0 - exception: task_type: pulp.server.managers.repo.publish.publish _href: /pulp/api/v2/tasks/a0a945df-39ba-487e-9ca2-b9a08bfe2bbe/ task_id: a0a945df-39ba-487e-9ca2-b9a08bfe2bbe tags: - pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base - pulp:action:publish finish_time: '2015-04-16T19:17:56Z' _ns: task_status start_time: '2015-04-16T19:09:05Z' traceback: spawned_tasks: [] progress_report: {} queue: reserved_resource_worker-0@katello.dq state: finished worker_name: reserved_resource_worker-0@katello result: exception: repo_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base started: '2015-04-16T19:09:05Z' _ns: repo_publish_results completed: '2015-04-16T19:17:55Z' traceback: distributor_type_id: nodes_http_distributor summary: succeeded error_message: details: unit_count: 6745 distributor_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base_nodes id: 55300ae4229dcf0abe8162e2 result: success error: _id: $oid: 553008d00fa7ca8c110a4c70 id: 553008d1229dcf0ac430c3e4 - exception: task_type: pulp.server.managers.repo.publish.publish _href: /pulp/api/v2/tasks/8ca9362c-fe5b-4034-8296-cada29f5744b/ task_id: 8ca9362c-fe5b-4034-8296-cada29f5744b tags: - pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base - pulp:action:publish finish_time: '2015-04-16T19:32:31Z' _ns: task_status start_time: '2015-04-16T19:17:56Z' traceback: spawned_tasks: [] progress_report: My_Company-CentOS_6-GR_Centos_6_6_-_Base: - num_success: 1 description: Copying files step_type: save_tar items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 594fd78f-174b-4d27-a9d9-6887f283377c num_processed: 1 - num_success: 1 description: Initializing repo metadata step_type: initialize_repo_metadata items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 11341e5f-06a4-4f32-b9f8-e15be60ab383 num_processed: 1 - num_success: 1 description: Publishing Distribution files step_type: distribution items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 99019b48-9f5b-45ac-ae1f-19ec0fb35c68 num_processed: 1 - num_success: 6518 description: Publishing RPMs step_type: rpms items_total: 12885 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 5c100b15-a5a0-4932-b77a-58b4d9895683 num_processed: 6518 - num_success: 0 description: Publishing Delta RPMs step_type: drpms items_total: 1 state: SKIPPED error_details: [] details: '' num_failures: 0 step_id: 2d3b6464-2a32-4fde-9bc5-65b7a64b4150 num_processed: 0 - num_success: 0 description: Publishing Errata step_type: errata items_total: 0 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 59e0a1ff-4d70-47f5-a034-e0eef925b598 num_processed: 0 - num_success: 226 description: Publishing Comps file step_type: comps items_total: 226 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 1877dac7-615c-4c60-b141-66b57a0bf3e4 num_processed: 226 - num_success: 0 description: Publishing Metadata. step_type: metadata items_total: 0 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 65f78ddf-d1a1-4ea0-bd65-c5c47d7fe531 num_processed: 0 - num_success: 1 description: Closing repo metadata step_type: close_repo_metadata items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 57f8223c-c944-41c6-b3f0-2464449e96da num_processed: 1 - num_success: 0 description: Generating sqlite files step_type: generate sqlite items_total: 1 state: SKIPPED error_details: [] details: '' num_failures: 0 step_id: 939b092e-03e8-41c5-ba3d-974dad161ea7 num_processed: 0 - num_success: 1 description: Publishing files to web step_type: publish_directory items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: a6306970-941a-4498-8e1d-30d2dc85ceb7 num_processed: 1 - num_success: 1 description: Writing Listings File step_type: initialize_repo_metadata items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 35b0951b-cb20-428d-a88a-ded2d1ed1375 num_processed: 1 - num_success: 1 description: Writing Listings File step_type: initialize_repo_metadata items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: e79b5f82-0da5-4dd4-b83f-b4c6e98e12b4 num_processed: 1 queue: reserved_resource_worker-0@katello.dq state: finished worker_name: reserved_resource_worker-0@katello result: exception: repo_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base started: '2015-04-16T19:17:56Z' _ns: repo_publish_results completed: '2015-04-16T19:32:29Z' traceback: distributor_type_id: yum_distributor summary: generate sqlite: SKIPPED initialize_repo_metadata: FINISHED rpms: FINISHED close_repo_metadata: FINISHED drpms: SKIPPED comps: FINISHED distribution: FINISHED metadata: FINISHED publish_directory: FINISHED errata: FINISHED save_tar: FINISHED error_message: details: - num_success: 1 description: Copying files step_type: save_tar items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 594fd78f-174b-4d27-a9d9-6887f283377c num_processed: 1 - num_success: 1 description: Initializing repo metadata step_type: initialize_repo_metadata items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 11341e5f-06a4-4f32-b9f8-e15be60ab383 num_processed: 1 - num_success: 1 description: Publishing Distribution files step_type: distribution items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 99019b48-9f5b-45ac-ae1f-19ec0fb35c68 num_processed: 1 - num_success: 6518 description: Publishing RPMs step_type: rpms items_total: 12885 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 5c100b15-a5a0-4932-b77a-58b4d9895683 num_processed: 6518 - num_success: 0 description: Publishing Delta RPMs step_type: drpms items_total: 1 state: SKIPPED error_details: [] details: '' num_failures: 0 step_id: 2d3b6464-2a32-4fde-9bc5-65b7a64b4150 num_processed: 0 - num_success: 0 description: Publishing Errata step_type: errata items_total: 0 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 59e0a1ff-4d70-47f5-a034-e0eef925b598 num_processed: 0 - num_success: 226 description: Publishing Comps file step_type: comps items_total: 226 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 1877dac7-615c-4c60-b141-66b57a0bf3e4 num_processed: 226 - num_success: 0 description: Publishing Metadata. step_type: metadata items_total: 0 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 65f78ddf-d1a1-4ea0-bd65-c5c47d7fe531 num_processed: 0 - num_success: 1 description: Closing repo metadata step_type: close_repo_metadata items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 57f8223c-c944-41c6-b3f0-2464449e96da num_processed: 1 - num_success: 0 description: Generating sqlite files step_type: generate sqlite items_total: 1 state: SKIPPED error_details: [] details: '' num_failures: 0 step_id: 939b092e-03e8-41c5-ba3d-974dad161ea7 num_processed: 0 - num_success: 1 description: Publishing files to web step_type: publish_directory items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: a6306970-941a-4498-8e1d-30d2dc85ceb7 num_processed: 1 - num_success: 1 description: Writing Listings File step_type: initialize_repo_metadata items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: 35b0951b-cb20-428d-a88a-ded2d1ed1375 num_processed: 1 - num_success: 1 description: Writing Listings File step_type: initialize_repo_metadata items_total: 1 state: FINISHED error_details: [] details: '' num_failures: 0 step_id: e79b5f82-0da5-4dd4-b83f-b4c6e98e12b4 num_processed: 1 distributor_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base id: 55300e4d229dcf0abe8162eb result: success error: _id: $oid: 553008d10fa7ca8c110a4c71 id: 553008d1229dcf0ac430c3e8 id: 12 remote_user: admin-375a9869 remote_cp_user: admin locale: en

Output:

— {}

Error:

RestClient::RequestTimeout

Request Timeout

/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.35/lib/restclient_ext/request.rb:56:in
`rescue in transmit’

/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.35/lib/restclient_ext/request.rb:11:in
`transmit’

/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in
`execute’

/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in
`execute’

/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in
`post’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/base.rb:91:in
`get_response’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/base.rb:82:in
`call’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/resources/repository.rb:161:in
`unit_search’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:151:in
`block in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:139:in
`<<’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:139:in
`block (2 levels) in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:138:in
`loop’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:138:in
`block in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`each’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`each’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`reduce’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`rescue in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:130:in
`rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/pulp/repo.rb:244:in
`package_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/pulp/repo.rb:309:in
`packages’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/elastic_search/repository.rb:61:in
`index_packages’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/elastic_search/repository.rb:123:in
`index_content’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/elastic_search/repository/index_content.rb:24:in
`run’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/action.rb:456:in
`block (3 levels) in execute_run’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware/stack.rb:26:in
`call’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware/stack.rb:26:in
`pass’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware.rb:16:in
`pass’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:28:in
`block in run’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:58:in
`block (2 levels) in as_remote_user’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/katello/util/thread_session.rb:84:in
`pulp_config’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:44:in
`as_pulp_user’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:57:in
`block in as_remote_user’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/katello/util/thread_session.rb:91:in
`cp_config’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:39:in
`as_cp_user’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:56:in
`as_remote_user’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:28:in
`run’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware/stack.rb:22:in
`call’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware/stack.rb:26:in
`pass’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware.rb:16:in
`pass’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/action/progress.rb:30:in
`with_progress_calculation’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/action/progress.rb:16:in
`run’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware/stack.rb:22:in
`call’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware/stack.rb:26:in
`pass’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware.rb:16:in
`pass’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/keep_locale.rb:23:in
`block in run’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/keep_locale.rb:34:in
`with_locale’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/keep_locale.rb:23:in
`run’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware/stack.rb:22:in
`call’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware/world.rb:30:in
`execute’

/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/action.rb:455:in
`block

2

Some questions to help me diagnose or re-create:

Do you have capsules other than the default?
Is the Library environment that the repository is being synced to attached
to any external capsules?
Are qpidd, pulp, and mongo all running?
Do you see anything specific in /var/log/messages ?

Eric

··· On Thu, Apr 16, 2015 at 4:16 PM, Nathan Peters wrote:

I have just upgraded to the latest Katello RC (2.2).

This was supposed to solve the repo sync freezing issues by replacing qpid
0.28 with qpid 0.3.

When I try to sync a repo, it downloads all the packages, but then pauses
and goes into error state.

Here is the log.

The last line in the log, stating that ‘block in
create_with_logging_context’ is the same message I got with Katello 2.1 and
qpid .28.

Wasn’t this supposed to be fixed in the new version? I have rebooted the
computer after the upgrade, so it should not be due to anything old hanging
around.

Here are the logs from the dynflow console :

Started at: 2015-04-16 19:32:32 UTC

Ended at: 2015-04-16 19:38:30 UTC

Real time: 357.88s

Execution time (excluding suspended state): 357.88s

Input:

dependency:

  • exception:
    task_type: pulp.server.managers.repo.sync.sync
    _href: /pulp/api/v2/tasks/0be07ad7-3451-4f4b-a23a-94d6a284dfed/
    task_id: 0be07ad7-3451-4f4b-a23a-94d6a284dfed
    tags:
    • pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base
    • pulp:action:sync
      finish_time: ‘2015-04-16T19:09:05Z’
      _ns: task_status
      start_time: '2015-04-16T18:46:10Z’
      traceback:
      spawned_tasks:
    • _href: /pulp/api/v2/tasks/a0a945df-39ba-487e-9ca2-b9a08bfe2bbe/
      task_id: a0a945df-39ba-487e-9ca2-b9a08bfe2bbe
    • href: /pulp/api/v2/tasks/8ca9362c-fe5b-4034-8296-cada29f5744b/
      task_id: 8ca9362c-fe5b-4034-8296-cada29f5744b
      progress_report:
      yum_importer:
      content:
      size_total: 1822408364
      items_left: 0
      items_total: 1739
      state: FINISHED
      size_left: 0
      details:
      rpm_total: 1739
      rpm_done: 1739
      drpm_total: 0
      drpm_done: 0
      error_details: []
      comps:
      state: FINISHED
      distribution:
      items_total: 4
      state: FINISHED
      error_details: []
      items_left: 0
      errata:
      state: FINISHED
      metadata:
      state: FINISHED
      queue: reserved_resource_worker-0@katello.dq
      state: finished
      worker_name: reserved_resource_worker-0@katello
      result:
      importer_type_id: yum_importer
      importer_id: yum_importer
      exception:
      repo_id: My_Company-CentOS_6-GR_Centos_6_6      -_Base
      removed_count: 0
      started: ‘2015-04-16T18:46:10Z’
      _ns: repo_sync_results
      completed: '2015-04-16T19:09:04Z’
      traceback:
      summary:
      content:
      state: FINISHED
      comps:
      state: FINISHED
      distribution:
      state: FINISHED
      errata:
      state: FINISHED
      metadata:
      state: FINISHED
      added_count: 1966
      error_message:
      updated_count: 4779
      details:
      content:
      size_total: 1822408364
      items_left: 0
      items_total: 1739
      state: FINISHED
      size_left: 0
      details:
      rpm_total: 1739
      rpm_done: 1739
      drpm_total: 0
      drpm_done: 0
      error_details: []
      comps:
      state: FINISHED
      distribution:
      items_total: 4
      state: FINISHED
      error_details: []
      items_left: 0
      errata:
      state: FINISHED
      metadata:
      state: FINISHED
      id: 553008d0229dcf0abe8162df
      result: success
      error:
      _id:
      $oid: 553003710fa7ca8c110a4c6e
      id: 55300371229dcf0ac430c3e0
  • exception:
    task_type: pulp.server.managers.repo.publish.publish
    _href: /pulp/api/v2/tasks/a0a945df-39ba-487e-9ca2-b9a08bfe2bbe/
    task_id: a0a945df-39ba-487e-9ca2-b9a08bfe2bbe
    tags:
    • pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base
    • pulp:action:publish
      finish_time: '2015-04-16T19:17:56Z’
      ns: task_status
      start_time: '2015-04-16T19:09:05Z’
      traceback:
      spawned_tasks: []
      progress_report: {}
      queue: reserved_resource_worker-0@katello.dq
      state: finished
      worker_name: reserved_resource_worker-0@katello
      result:
      exception:
      repo_id: My_Company-CentOS_6-GR_Centos_6_6      -_Base
      started: '2015-04-16T19:09:05Z’
      ns: repo_publish_results
      completed: '2015-04-16T19:17:55Z’
      traceback:
      distributor_type_id: nodes_http_distributor
      summary: succeeded
      error_message:
      details:
      unit_count: 6745
      distributor_id: My_Company-CentOS_6-GR_Centos_6_6      -_Base_nodes
      id: 55300ae4229dcf0abe8162e2
      result: success
      error:
      _id:
      $oid: 553008d00fa7ca8c110a4c70
      id: 553008d1229dcf0ac430c3e4
  • exception:
    task_type: pulp.server.managers.repo.publish.publish
    _href: /pulp/api/v2/tasks/8ca9362c-fe5b-4034-8296-cada29f5744b/
    task_id: 8ca9362c-fe5b-4034-8296-cada29f5744b
    tags:
    • pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base
    • pulp:action:publish
      finish_time: '2015-04-16T19:32:31Z’
      ns: task_status
      start_time: '2015-04-16T19:17:56Z’
      traceback:
      spawned_tasks: []
      progress_report:
      My_Company-CentOS_6-GR_Centos_6_6      -_Base:
      • num_success: 1
        description: Copying files
        step_type: save_tar
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 594fd78f-174b-4d27-a9d9-6887f283377c
        num_processed: 1
      • num_success: 1
        description: Initializing repo metadata
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 11341e5f-06a4-4f32-b9f8-e15be60ab383
        num_processed: 1
      • num_success: 1
        description: Publishing Distribution files
        step_type: distribution
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 99019b48-9f5b-45ac-ae1f-19ec0fb35c68
        num_processed: 1
      • num_success: 6518
        description: Publishing RPMs
        step_type: rpms
        items_total: 12885
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 5c100b15-a5a0-4932-b77a-58b4d9895683
        num_processed: 6518
      • num_success: 0
        description: Publishing Delta RPMs
        step_type: drpms
        items_total: 1
        state: SKIPPED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 2d3b6464-2a32-4fde-9bc5-65b7a64b4150
        num_processed: 0
      • num_success: 0
        description: Publishing Errata
        step_type: errata
        items_total: 0
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 59e0a1ff-4d70-47f5-a034-e0eef925b598
        num_processed: 0
      • num_success: 226
        description: Publishing Comps file
        step_type: comps
        items_total: 226
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 1877dac7-615c-4c60-b141-66b57a0bf3e4
        num_processed: 226
      • num_success: 0
        description: Publishing Metadata.
        step_type: metadata
        items_total: 0
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 65f78ddf-d1a1-4ea0-bd65-c5c47d7fe531
        num_processed: 0
      • num_success: 1
        description: Closing repo metadata
        step_type: close_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 57f8223c-c944-41c6-b3f0-2464449e96da
        num_processed: 1
      • num_success: 0
        description: Generating sqlite files
        step_type: generate sqlite
        items_total: 1
        state: SKIPPED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 939b092e-03e8-41c5-ba3d-974dad161ea7
        num_processed: 0
      • num_success: 1
        description: Publishing files to web
        step_type: publish_directory
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: a6306970-941a-4498-8e1d-30d2dc85ceb7
        num_processed: 1
      • num_success: 1
        description: Writing Listings File
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 35b0951b-cb20-428d-a88a-ded2d1ed1375
        num_processed: 1
      • num_success: 1
        description: Writing Listings File
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: e79b5f82-0da5-4dd4-b83f-b4c6e98e12b4
        num_processed: 1
        queue: reserved_resource_worker-0@katello.dq
        state: finished
        worker_name: reserved_resource_worker-0@katello
        result:
        exception:
        repo_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base
        started: ‘2015-04-16T19:17:56Z’
        _ns: repo_publish_results
        completed: '2015-04-16T19:32:29Z’
        traceback:
        distributor_type_id: yum_distributor
        summary:
        generate sqlite: SKIPPED
        initialize_repo_metadata: FINISHED
        rpms: FINISHED
        close_repo_metadata: FINISHED
        drpms: SKIPPED
        comps: FINISHED
        distribution: FINISHED
        metadata: FINISHED
        publish_directory: FINISHED
        errata: FINISHED
        save_tar: FINISHED
        error_message:
        details:
      • num_success: 1
        description: Copying files
        step_type: save_tar
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 594fd78f-174b-4d27-a9d9-6887f283377c
        num_processed: 1
      • num_success: 1
        description: Initializing repo metadata
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 11341e5f-06a4-4f32-b9f8-e15be60ab383
        num_processed: 1
      • num_success: 1
        description: Publishing Distribution files
        step_type: distribution
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 99019b48-9f5b-45ac-ae1f-19ec0fb35c68
        num_processed: 1
      • num_success: 6518
        description: Publishing RPMs
        step_type: rpms
        items_total: 12885
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 5c100b15-a5a0-4932-b77a-58b4d9895683
        num_processed: 6518
      • num_success: 0
        description: Publishing Delta RPMs
        step_type: drpms
        items_total: 1
        state: SKIPPED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 2d3b6464-2a32-4fde-9bc5-65b7a64b4150
        num_processed: 0
      • num_success: 0
        description: Publishing Errata
        step_type: errata
        items_total: 0
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 59e0a1ff-4d70-47f5-a034-e0eef925b598
        num_processed: 0
      • num_success: 226
        description: Publishing Comps file
        step_type: comps
        items_total: 226
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 1877dac7-615c-4c60-b141-66b57a0bf3e4
        num_processed: 226
      • num_success: 0
        description: Publishing Metadata.
        step_type: metadata
        items_total: 0
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 65f78ddf-d1a1-4ea0-bd65-c5c47d7fe531
        num_processed: 0
      • num_success: 1
        description: Closing repo metadata
        step_type: close_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 57f8223c-c944-41c6-b3f0-2464449e96da
        num_processed: 1
      • num_success: 0
        description: Generating sqlite files
        step_type: generate sqlite
        items_total: 1
        state: SKIPPED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 939b092e-03e8-41c5-ba3d-974dad161ea7
        num_processed: 0
      • num_success: 1
        description: Publishing files to web
        step_type: publish_directory
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: a6306970-941a-4498-8e1d-30d2dc85ceb7
        num_processed: 1
      • num_success: 1
        description: Writing Listings File
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 35b0951b-cb20-428d-a88a-ded2d1ed1375
        num_processed: 1
      • num_success: 1
        description: Writing Listings File
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: e79b5f82-0da5-4dd4-b83f-b4c6e98e12b4
        num_processed: 1
        distributor_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base
        id: 55300e4d229dcf0abe8162eb
        result: success
        error:
        _id:
        $oid: 553008d10fa7ca8c110a4c71
        id: 553008d1229dcf0ac430c3e8
        id: 12
        remote_user: admin-375a9869
        remote_cp_user: admin
        locale: en

Output:

— {}

Error:

RestClient::RequestTimeout

Request Timeout

/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.35/lib/restclient_ext/request.rb:56:in
`rescue in transmit’

/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.35/lib/restclient_ext/request.rb:11:in
`transmit’

/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in
`execute’

/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in
`execute’

/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in
`post’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/base.rb:91:in
`get_response’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/base.rb:82:in
`call’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/resources/repository.rb:161:in
`unit_search’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:151:in
`block in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:139:in
`<<’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:139:in
`block (2 levels) in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:138:in
`loop’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:138:in
`block in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`each’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`each’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`reduce’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`rescue in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:130:in
`rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/pulp/repo.rb:244:in
`package_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/pulp/repo.rb:309:in
`packages’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/elastic_search/repository.rb:61:in
`index_packages’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/elastic_search/repository.rb:123:in
`index_cont

3

What version of qpid-dispatch does the box have?

··· On Apr 16, 2015 4:16 PM, "Nathan Peters" wrote:

I have just upgraded to the latest Katello RC (2.2).

This was supposed to solve the repo sync freezing issues by replacing qpid
0.28 with qpid 0.3.

When I try to sync a repo, it downloads all the packages, but then pauses
and goes into error state.

Here is the log.

The last line in the log, stating that ‘block in
create_with_logging_context’ is the same message I got with Katello 2.1 and
qpid .28.

Wasn’t this supposed to be fixed in the new version? I have rebooted the
computer after the upgrade, so it should not be due to anything old hanging
around.

Here are the logs from the dynflow console :

Started at: 2015-04-16 19:32:32 UTC

Ended at: 2015-04-16 19:38:30 UTC

Real time: 357.88s

Execution time (excluding suspended state): 357.88s

Input:

dependency:

  • exception:
    task_type: pulp.server.managers.repo.sync.sync
    _href: /pulp/api/v2/tasks/0be07ad7-3451-4f4b-a23a-94d6a284dfed/
    task_id: 0be07ad7-3451-4f4b-a23a-94d6a284dfed
    tags:
    • pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base
    • pulp:action:sync
      finish_time: ‘2015-04-16T19:09:05Z’
      _ns: task_status
      start_time: '2015-04-16T18:46:10Z’
      traceback:
      spawned_tasks:
    • _href: /pulp/api/v2/tasks/a0a945df-39ba-487e-9ca2-b9a08bfe2bbe/
      task_id: a0a945df-39ba-487e-9ca2-b9a08bfe2bbe
    • href: /pulp/api/v2/tasks/8ca9362c-fe5b-4034-8296-cada29f5744b/
      task_id: 8ca9362c-fe5b-4034-8296-cada29f5744b
      progress_report:
      yum_importer:
      content:
      size_total: 1822408364
      items_left: 0
      items_total: 1739
      state: FINISHED
      size_left: 0
      details:
      rpm_total: 1739
      rpm_done: 1739
      drpm_total: 0
      drpm_done: 0
      error_details: []
      comps:
      state: FINISHED
      distribution:
      items_total: 4
      state: FINISHED
      error_details: []
      items_left: 0
      errata:
      state: FINISHED
      metadata:
      state: FINISHED
      queue: reserved_resource_worker-0@katello.dq
      state: finished
      worker_name: reserved_resource_worker-0@katello
      result:
      importer_type_id: yum_importer
      importer_id: yum_importer
      exception:
      repo_id: My_Company-CentOS_6-GR_Centos_6_6      -_Base
      removed_count: 0
      started: ‘2015-04-16T18:46:10Z’
      _ns: repo_sync_results
      completed: '2015-04-16T19:09:04Z’
      traceback:
      summary:
      content:
      state: FINISHED
      comps:
      state: FINISHED
      distribution:
      state: FINISHED
      errata:
      state: FINISHED
      metadata:
      state: FINISHED
      added_count: 1966
      error_message:
      updated_count: 4779
      details:
      content:
      size_total: 1822408364
      items_left: 0
      items_total: 1739
      state: FINISHED
      size_left: 0
      details:
      rpm_total: 1739
      rpm_done: 1739
      drpm_total: 0
      drpm_done: 0
      error_details: []
      comps:
      state: FINISHED
      distribution:
      items_total: 4
      state: FINISHED
      error_details: []
      items_left: 0
      errata:
      state: FINISHED
      metadata:
      state: FINISHED
      id: 553008d0229dcf0abe8162df
      result: success
      error:
      _id:
      $oid: 553003710fa7ca8c110a4c6e
      id: 55300371229dcf0ac430c3e0
  • exception:
    task_type: pulp.server.managers.repo.publish.publish
    _href: /pulp/api/v2/tasks/a0a945df-39ba-487e-9ca2-b9a08bfe2bbe/
    task_id: a0a945df-39ba-487e-9ca2-b9a08bfe2bbe
    tags:
    • pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base
    • pulp:action:publish
      finish_time: '2015-04-16T19:17:56Z’
      ns: task_status
      start_time: '2015-04-16T19:09:05Z’
      traceback:
      spawned_tasks: []
      progress_report: {}
      queue: reserved_resource_worker-0@katello.dq
      state: finished
      worker_name: reserved_resource_worker-0@katello
      result:
      exception:
      repo_id: My_Company-CentOS_6-GR_Centos_6_6      -_Base
      started: '2015-04-16T19:09:05Z’
      ns: repo_publish_results
      completed: '2015-04-16T19:17:55Z’
      traceback:
      distributor_type_id: nodes_http_distributor
      summary: succeeded
      error_message:
      details:
      unit_count: 6745
      distributor_id: My_Company-CentOS_6-GR_Centos_6_6      -_Base_nodes
      id: 55300ae4229dcf0abe8162e2
      result: success
      error:
      _id:
      $oid: 553008d00fa7ca8c110a4c70
      id: 553008d1229dcf0ac430c3e4
  • exception:
    task_type: pulp.server.managers.repo.publish.publish
    _href: /pulp/api/v2/tasks/8ca9362c-fe5b-4034-8296-cada29f5744b/
    task_id: 8ca9362c-fe5b-4034-8296-cada29f5744b
    tags:
    • pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base
    • pulp:action:publish
      finish_time: '2015-04-16T19:32:31Z’
      ns: task_status
      start_time: '2015-04-16T19:17:56Z’
      traceback:
      spawned_tasks: []
      progress_report:
      My_Company-CentOS_6-GR_Centos_6_6      -_Base:
      • num_success: 1
        description: Copying files
        step_type: save_tar
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 594fd78f-174b-4d27-a9d9-6887f283377c
        num_processed: 1
      • num_success: 1
        description: Initializing repo metadata
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 11341e5f-06a4-4f32-b9f8-e15be60ab383
        num_processed: 1
      • num_success: 1
        description: Publishing Distribution files
        step_type: distribution
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 99019b48-9f5b-45ac-ae1f-19ec0fb35c68
        num_processed: 1
      • num_success: 6518
        description: Publishing RPMs
        step_type: rpms
        items_total: 12885
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 5c100b15-a5a0-4932-b77a-58b4d9895683
        num_processed: 6518
      • num_success: 0
        description: Publishing Delta RPMs
        step_type: drpms
        items_total: 1
        state: SKIPPED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 2d3b6464-2a32-4fde-9bc5-65b7a64b4150
        num_processed: 0
      • num_success: 0
        description: Publishing Errata
        step_type: errata
        items_total: 0
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 59e0a1ff-4d70-47f5-a034-e0eef925b598
        num_processed: 0
      • num_success: 226
        description: Publishing Comps file
        step_type: comps
        items_total: 226
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 1877dac7-615c-4c60-b141-66b57a0bf3e4
        num_processed: 226
      • num_success: 0
        description: Publishing Metadata.
        step_type: metadata
        items_total: 0
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 65f78ddf-d1a1-4ea0-bd65-c5c47d7fe531
        num_processed: 0
      • num_success: 1
        description: Closing repo metadata
        step_type: close_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 57f8223c-c944-41c6-b3f0-2464449e96da
        num_processed: 1
      • num_success: 0
        description: Generating sqlite files
        step_type: generate sqlite
        items_total: 1
        state: SKIPPED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 939b092e-03e8-41c5-ba3d-974dad161ea7
        num_processed: 0
      • num_success: 1
        description: Publishing files to web
        step_type: publish_directory
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: a6306970-941a-4498-8e1d-30d2dc85ceb7
        num_processed: 1
      • num_success: 1
        description: Writing Listings File
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 35b0951b-cb20-428d-a88a-ded2d1ed1375
        num_processed: 1
      • num_success: 1
        description: Writing Listings File
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: e79b5f82-0da5-4dd4-b83f-b4c6e98e12b4
        num_processed: 1
        queue: reserved_resource_worker-0@katello.dq
        state: finished
        worker_name: reserved_resource_worker-0@katello
        result:
        exception:
        repo_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base
        started: ‘2015-04-16T19:17:56Z’
        _ns: repo_publish_results
        completed: '2015-04-16T19:32:29Z’
        traceback:
        distributor_type_id: yum_distributor
        summary:
        generate sqlite: SKIPPED
        initialize_repo_metadata: FINISHED
        rpms: FINISHED
        close_repo_metadata: FINISHED
        drpms: SKIPPED
        comps: FINISHED
        distribution: FINISHED
        metadata: FINISHED
        publish_directory: FINISHED
        errata: FINISHED
        save_tar: FINISHED
        error_message:
        details:
      • num_success: 1
        description: Copying files
        step_type: save_tar
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 594fd78f-174b-4d27-a9d9-6887f283377c
        num_processed: 1
      • num_success: 1
        description: Initializing repo metadata
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 11341e5f-06a4-4f32-b9f8-e15be60ab383
        num_processed: 1
      • num_success: 1
        description: Publishing Distribution files
        step_type: distribution
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 99019b48-9f5b-45ac-ae1f-19ec0fb35c68
        num_processed: 1
      • num_success: 6518
        description: Publishing RPMs
        step_type: rpms
        items_total: 12885
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 5c100b15-a5a0-4932-b77a-58b4d9895683
        num_processed: 6518
      • num_success: 0
        description: Publishing Delta RPMs
        step_type: drpms
        items_total: 1
        state: SKIPPED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 2d3b6464-2a32-4fde-9bc5-65b7a64b4150
        num_processed: 0
      • num_success: 0
        description: Publishing Errata
        step_type: errata
        items_total: 0
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 59e0a1ff-4d70-47f5-a034-e0eef925b598
        num_processed: 0
      • num_success: 226
        description: Publishing Comps file
        step_type: comps
        items_total: 226
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 1877dac7-615c-4c60-b141-66b57a0bf3e4
        num_processed: 226
      • num_success: 0
        description: Publishing Metadata.
        step_type: metadata
        items_total: 0
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 65f78ddf-d1a1-4ea0-bd65-c5c47d7fe531
        num_processed: 0
      • num_success: 1
        description: Closing repo metadata
        step_type: close_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 57f8223c-c944-41c6-b3f0-2464449e96da
        num_processed: 1
      • num_success: 0
        description: Generating sqlite files
        step_type: generate sqlite
        items_total: 1
        state: SKIPPED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 939b092e-03e8-41c5-ba3d-974dad161ea7
        num_processed: 0
      • num_success: 1
        description: Publishing files to web
        step_type: publish_directory
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: a6306970-941a-4498-8e1d-30d2dc85ceb7
        num_processed: 1
      • num_success: 1
        description: Writing Listings File
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: 35b0951b-cb20-428d-a88a-ded2d1ed1375
        num_processed: 1
      • num_success: 1
        description: Writing Listings File
        step_type: initialize_repo_metadata
        items_total: 1
        state: FINISHED
        error_details: []
        details: ''
        num_failures: 0
        step_id: e79b5f82-0da5-4dd4-b83f-b4c6e98e12b4
        num_processed: 1
        distributor_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base
        id: 55300e4d229dcf0abe8162eb
        result: success
        error:
        _id:
        $oid: 553008d10fa7ca8c110a4c71
        id: 553008d1229dcf0ac430c3e8
        id: 12
        remote_user: admin-375a9869
        remote_cp_user: admin
        locale: en

Output:

— {}

Error:

RestClient::RequestTimeout

Request Timeout

/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.35/lib/restclient_ext/request.rb:56:in
`rescue in transmit’

/opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.35/lib/restclient_ext/request.rb:11:in
`transmit’

/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in
`execute’

/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in
`execute’

/opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in
`post’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/base.rb:91:in
`get_response’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/base.rb:82:in
`call’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/resources/repository.rb:161:in
`unit_search’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:151:in
`block in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:139:in
`<<’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:139:in
`block (2 levels) in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:138:in
`loop’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:138:in
`block in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`each’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`each’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`reduce’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`rescue in rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:130:in
`rpm_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/pulp/repo.rb:244:in
`package_ids’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/pulp/repo.rb:309:in
`packages’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/elastic_search/repository.rb:61:in
`index_packages’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/elastic_search/repository.rb:123:in
`index_content’

/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/elastic_search/repository/index_content.rb:24:in
`run’

/opt/rh/ruby193/root/usr/share/gems/gems/d

4

This server has 2 nics and is acting as a router/masquerade server for my
katello test lab to keep it separated from our general environment. It is
running CentOS 7.

It is otherwise working fine and I can provision machines with it using
multiple operating systems (fedora/centos)

Katello was installed with this command line:

katello-installer
–enable-foreman-plugin-setup
–capsule-tftp=true
–capsule-tftp-servername=192.168.1.1
–capsule-dhcp=true
–capsule-dhcp-interface=ens33
–capsule-dhcp-gateway=192.168.1.1
–capsule-dhcp-range="192.168.1.2 192.168.1.100"
–capsule-dhcp-nameservers="192.168.1.1"
–capsule-dns=true
–capsule-dns-interface=ens33
–capsule-dns-zone=dev-globalrelay.net
–capsule-dns-reverse=1.168.192.in-addr.arpa
–capsule-dns-forwarders=10.21.0.14

There are no external capsules. Everything is running on the Katello
server so the Library environment is the default and not attached to any
external capsules.

When I run katello-service status everything comes back ok except this next
bit :

[root@katello ~]# katello-service status
Redirecting to /bin/systemctl status qpidd.service
qpidd.service - An AMQP message broker daemon.
Loaded: loaded (/usr/lib/systemd/system/qpidd.service; enabled)
Active: active (running) since Thu 2015-04-16 17:31:56 UTC; 3h 14min ago
Docs: man:qpidd(1)
http://qpid.apache.org/
Main PID: 1228 (qpidd)
CGroup: /system.slice/qpidd.service
└─1228 /usr/sbin/qpidd --config /etc/qpid/qpidd.conf

Apr 16 17:31:56 katello systemd[1]: Starting An AMQP message broker
daemon…
Apr 16 17:31:56 katello systemd[1]: Started An AMQP message broker daemon…
Redirecting to /bin/systemctl status qdrouterd.service
qdrouterd.service - Qpid Dispatch router daemon
Loaded: loaded (/usr/lib/systemd/system/qdrouterd.service; enabled)
Active: failed (Result: core-dump) since Thu 2015-04-16 17:31:48 UTC; 3h
14min ago
Process: 506 ExecStart=/usr/sbin/qdrouterd -c
/etc/qpid-dispatch/qdrouterd.conf (code=dumped, signal=SEGV)
Main PID: 506 (code=dumped, signal=SEGV)
CGroup: /system.slice/qdrouterd.service

Apr 16 17:31:47 katello qdrouterd[506]: Thu Apr 16 17:31:47 2015 CONN_MGR
(info) Configured Listener: 0.0.0.0:5647 role=normal
Apr 16 17:31:47 katello qdrouterd[506]: Thu Apr 16 17:31:47 2015 CONN_MGR
(info) Configured Listener: 0.0.0.0:5646 role=inter-router
Apr 16 17:31:47 katello qdrouterd[506]: Thu Apr 16 17:31:47 2015 CONN_MGR
(info) Configured on-demand connector: katello.mydomain.net:5671 name=broker
Apr 16 17:31:47 katello qdrouterd[506]: Thu Apr 16 17:31:47 2015 ROUTER
(info) Configured Link-route-pattern: prefix=pulp. connector=broker
Apr 16 17:31:47 katello qdrouterd[506]: Thu Apr 16 17:31:47 2015 ROUTER
(info) Configured Link-route-pattern: prefix=qmf. connector=broker
Apr 16 17:31:47 katello qdrouterd[506]: Thu Apr 16 17:31:47 2015 SERVER
(info) Operational, 2 Threads Running
Apr 16 17:31:47 katello qdrouterd[506]: Thu Apr 16 17:31:47 2015 CONN_MGR
(info) Starting on-demand connector: broker
Apr 16 17:31:47 katello qdrouterd[506]: Thu Apr 16 17:31:47 2015 DRIVER
(error) connect: Network is unreachable
Apr 16 17:31:48 katello systemd[1]: qdrouterd.service: main process exited,
code=dumped, status=11/SEGV
Apr 16 17:31:48 katello systemd[1]: Unit qdrouterd.service entered failed
state.

I could not find anything useful in /var/log/messages but it's full of
stuff so I'm not sure what I'd be looking for.

··· On Thursday, April 16, 2015 at 1:25:46 PM UTC-7, Eric Helms wrote: > > Some questions to help me diagnose or re-create: > > Do you have capsules other than the default? > Is the Library environment that the repository is being synced to attached > to any external capsules? > Are qpidd, pulp, and mongo all running? > Do you see anything specific in /var/log/messages ? > > > Eric > > On Thu, Apr 16, 2015 at 4:16 PM, Nathan Peters > wrote: > >> I have just upgraded to the latest Katello RC (2.2). >> >> This was supposed to solve the repo sync freezing issues by replacing >> qpid 0.28 with qpid 0.3. >> >> When I try to sync a repo, it downloads all the packages, but then pauses >> and goes into error state. >> >> Here is the log. >> >> The last line in the log, stating that 'block in >> create_with_logging_context' is the same message I got with Katello 2.1 and >> qpid .28. >> >> Wasn't this supposed to be fixed in the new version? I have rebooted the >> computer after the upgrade, so it should not be due to anything old hanging >> around. >> >> Here are the logs from the dynflow console : >> >> Started at: 2015-04-16 19:32:32 UTC >> >> Ended at: 2015-04-16 19:38:30 UTC >> >> Real time: 357.88s >> >> Execution time (excluding suspended state): 357.88s >> >> Input: >> >> --- >> dependency: >> - exception: >> task_type: pulp.server.managers.repo.sync.sync >> _href: /pulp/api/v2/tasks/0be07ad7-3451-4f4b-a23a-94d6a284dfed/ >> task_id: 0be07ad7-3451-4f4b-a23a-94d6a284dfed >> tags: >> - pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base >> - pulp:action:sync >> finish_time: '2015-04-16T19:09:05Z' >> _ns: task_status >> start_time: '2015-04-16T18:46:10Z' >> traceback: >> spawned_tasks: >> - _href: /pulp/api/v2/tasks/a0a945df-39ba-487e-9ca2-b9a08bfe2bbe/ >> task_id: a0a945df-39ba-487e-9ca2-b9a08bfe2bbe >> - _href: /pulp/api/v2/tasks/8ca9362c-fe5b-4034-8296-cada29f5744b/ >> task_id: 8ca9362c-fe5b-4034-8296-cada29f5744b >> progress_report: >> yum_importer: >> content: >> size_total: 1822408364 >> items_left: 0 >> items_total: 1739 >> state: FINISHED >> size_left: 0 >> details: >> rpm_total: 1739 >> rpm_done: 1739 >> drpm_total: 0 >> drpm_done: 0 >> error_details: [] >> comps: >> state: FINISHED >> distribution: >> items_total: 4 >> state: FINISHED >> error_details: [] >> items_left: 0 >> errata: >> state: FINISHED >> metadata: >> state: FINISHED >> queue: reserved_resource_worker-0@katello.dq >> state: finished >> worker_name: reserved_resource_worker-0@katello >> result: >> importer_type_id: yum_importer >> importer_id: yum_importer >> exception: >> repo_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base >> removed_count: 0 >> started: '2015-04-16T18:46:10Z' >> _ns: repo_sync_results >> completed: '2015-04-16T19:09:04Z' >> traceback: >> summary: >> content: >> state: FINISHED >> comps: >> state: FINISHED >> distribution: >> state: FINISHED >> errata: >> state: FINISHED >> metadata: >> state: FINISHED >> added_count: 1966 >> error_message: >> updated_count: 4779 >> details: >> content: >> size_total: 1822408364 >> items_left: 0 >> items_total: 1739 >> state: FINISHED >> size_left: 0 >> details: >> rpm_total: 1739 >> rpm_done: 1739 >> drpm_total: 0 >> drpm_done: 0 >> error_details: [] >> comps: >> state: FINISHED >> distribution: >> items_total: 4 >> state: FINISHED >> error_details: [] >> items_left: 0 >> errata: >> state: FINISHED >> metadata: >> state: FINISHED >> id: 553008d0229dcf0abe8162df >> result: success >> error: >> _id: >> $oid: 553003710fa7ca8c110a4c6e >> id: 55300371229dcf0ac430c3e0 >> - exception: >> task_type: pulp.server.managers.repo.publish.publish >> _href: /pulp/api/v2/tasks/a0a945df-39ba-487e-9ca2-b9a08bfe2bbe/ >> task_id: a0a945df-39ba-487e-9ca2-b9a08bfe2bbe >> tags: >> - pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base >> - pulp:action:publish >> finish_time: '2015-04-16T19:17:56Z' >> _ns: task_status >> start_time: '2015-04-16T19:09:05Z' >> traceback: >> spawned_tasks: [] >> progress_report: {} >> queue: reserved_resource_worker-0@katello.dq >> state: finished >> worker_name: reserved_resource_worker-0@katello >> result: >> exception: >> repo_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base >> started: '2015-04-16T19:09:05Z' >> _ns: repo_publish_results >> completed: '2015-04-16T19:17:55Z' >> traceback: >> distributor_type_id: nodes_http_distributor >> summary: succeeded >> error_message: >> details: >> unit_count: 6745 >> distributor_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base_nodes >> id: 55300ae4229dcf0abe8162e2 >> result: success >> error: >> _id: >> $oid: 553008d00fa7ca8c110a4c70 >> id: 553008d1229dcf0ac430c3e4 >> - exception: >> task_type: pulp.server.managers.repo.publish.publish >> _href: /pulp/api/v2/tasks/8ca9362c-fe5b-4034-8296-cada29f5744b/ >> task_id: 8ca9362c-fe5b-4034-8296-cada29f5744b >> tags: >> - pulp:repository:My_Company-CentOS_6-GR_Centos_6_6_-_Base >> - pulp:action:publish >> finish_time: '2015-04-16T19:32:31Z' >> _ns: task_status >> start_time: '2015-04-16T19:17:56Z' >> traceback: >> spawned_tasks: [] >> progress_report: >> My_Company-CentOS_6-GR_Centos_6_6_-_Base: >> - num_success: 1 >> description: Copying files >> step_type: save_tar >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 594fd78f-174b-4d27-a9d9-6887f283377c >> num_processed: 1 >> - num_success: 1 >> description: Initializing repo metadata >> step_type: initialize_repo_metadata >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 11341e5f-06a4-4f32-b9f8-e15be60ab383 >> num_processed: 1 >> - num_success: 1 >> description: Publishing Distribution files >> step_type: distribution >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 99019b48-9f5b-45ac-ae1f-19ec0fb35c68 >> num_processed: 1 >> - num_success: 6518 >> description: Publishing RPMs >> step_type: rpms >> items_total: 12885 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 5c100b15-a5a0-4932-b77a-58b4d9895683 >> num_processed: 6518 >> - num_success: 0 >> description: Publishing Delta RPMs >> step_type: drpms >> items_total: 1 >> state: SKIPPED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 2d3b6464-2a32-4fde-9bc5-65b7a64b4150 >> num_processed: 0 >> - num_success: 0 >> description: Publishing Errata >> step_type: errata >> items_total: 0 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 59e0a1ff-4d70-47f5-a034-e0eef925b598 >> num_processed: 0 >> - num_success: 226 >> description: Publishing Comps file >> step_type: comps >> items_total: 226 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 1877dac7-615c-4c60-b141-66b57a0bf3e4 >> num_processed: 226 >> - num_success: 0 >> description: Publishing Metadata. >> step_type: metadata >> items_total: 0 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 65f78ddf-d1a1-4ea0-bd65-c5c47d7fe531 >> num_processed: 0 >> - num_success: 1 >> description: Closing repo metadata >> step_type: close_repo_metadata >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 57f8223c-c944-41c6-b3f0-2464449e96da >> num_processed: 1 >> - num_success: 0 >> description: Generating sqlite files >> step_type: generate sqlite >> items_total: 1 >> state: SKIPPED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 939b092e-03e8-41c5-ba3d-974dad161ea7 >> num_processed: 0 >> - num_success: 1 >> description: Publishing files to web >> step_type: publish_directory >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: a6306970-941a-4498-8e1d-30d2dc85ceb7 >> num_processed: 1 >> - num_success: 1 >> description: Writing Listings File >> step_type: initialize_repo_metadata >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 35b0951b-cb20-428d-a88a-ded2d1ed1375 >> num_processed: 1 >> - num_success: 1 >> description: Writing Listings File >> step_type: initialize_repo_metadata >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: e79b5f82-0da5-4dd4-b83f-b4c6e98e12b4 >> num_processed: 1 >> queue: reserved_resource_worker-0@katello.dq >> state: finished >> worker_name: reserved_resource_worker-0@katello >> result: >> exception: >> repo_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base >> started: '2015-04-16T19:17:56Z' >> _ns: repo_publish_results >> completed: '2015-04-16T19:32:29Z' >> traceback: >> distributor_type_id: yum_distributor >> summary: >> generate sqlite: SKIPPED >> initialize_repo_metadata: FINISHED >> rpms: FINISHED >> close_repo_metadata: FINISHED >> drpms: SKIPPED >> comps: FINISHED >> distribution: FINISHED >> metadata: FINISHED >> publish_directory: FINISHED >> errata: FINISHED >> save_tar: FINISHED >> error_message: >> details: >> - num_success: 1 >> description: Copying files >> step_type: save_tar >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 594fd78f-174b-4d27-a9d9-6887f283377c >> num_processed: 1 >> - num_success: 1 >> description: Initializing repo metadata >> step_type: initialize_repo_metadata >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 11341e5f-06a4-4f32-b9f8-e15be60ab383 >> num_processed: 1 >> - num_success: 1 >> description: Publishing Distribution files >> step_type: distribution >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 99019b48-9f5b-45ac-ae1f-19ec0fb35c68 >> num_processed: 1 >> - num_success: 6518 >> description: Publishing RPMs >> step_type: rpms >> items_total: 12885 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 5c100b15-a5a0-4932-b77a-58b4d9895683 >> num_processed: 6518 >> - num_success: 0 >> description: Publishing Delta RPMs >> step_type: drpms >> items_total: 1 >> state: SKIPPED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 2d3b6464-2a32-4fde-9bc5-65b7a64b4150 >> num_processed: 0 >> - num_success: 0 >> description: Publishing Errata >> step_type: errata >> items_total: 0 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 59e0a1ff-4d70-47f5-a034-e0eef925b598 >> num_processed: 0 >> - num_success: 226 >> description: Publishing Comps file >> step_type: comps >> items_total: 226 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 1877dac7-615c-4c60-b141-66b57a0bf3e4 >> num_processed: 226 >> - num_success: 0 >> description: Publishing Metadata. >> step_type: metadata >> items_total: 0 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 65f78ddf-d1a1-4ea0-bd65-c5c47d7fe531 >> num_processed: 0 >> - num_success: 1 >> description: Closing repo metadata >> step_type: close_repo_metadata >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 57f8223c-c944-41c6-b3f0-2464449e96da >> num_processed: 1 >> - num_success: 0 >> description: Generating sqlite files >> step_type: generate sqlite >> items_total: 1 >> state: SKIPPED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 939b092e-03e8-41c5-ba3d-974dad161ea7 >> num_processed: 0 >> - num_success: 1 >> description: Publishing files to web >> step_type: publish_directory >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: a6306970-941a-4498-8e1d-30d2dc85ceb7 >> num_processed: 1 >> - num_success: 1 >> description: Writing Listings File >> step_type: initialize_repo_metadata >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: 35b0951b-cb20-428d-a88a-ded2d1ed1375 >> num_processed: 1 >> - num_success: 1 >> description: Writing Listings File >> step_type: initialize_repo_metadata >> items_total: 1 >> state: FINISHED >> error_details: [] >> details: '' >> num_failures: 0 >> step_id: e79b5f82-0da5-4dd4-b83f-b4c6e98e12b4 >> num_processed: 1 >> distributor_id: My_Company-CentOS_6-GR_Centos_6_6_-_Base >> id: 55300e4d229dcf0abe8162eb >> result: success >> error: >> _id: >> $oid: 553008d10fa7ca8c110a4c71 >> id: 553008d1229dcf0ac430c3e8 >> id: 12 >> remote_user: admin-375a9869 >> remote_cp_user: admin >> locale: en >> >> Output: >> >> --- {}
5

libqpid-dispatch.x86_64 0.4-2.20150402.el7
installed
qpid-dispatch-router.x86_64 0.4-2.20150402.el7
installed

··· On Thursday, April 16, 2015 at 2:15:25 PM UTC-7, Eric Helms wrote: > > What version of qpid-dispatch does the box have? >