RPM1008: Checksum type "sha256" is not available for all units in the repository

kraba · October 15, 2018, 3:10pm

Problem:

After new errata are present, an incremental update of the CV version (for example 1.1) fail for checksum type error:

RPM1008: Checksum type “sha256” is not available for all units in the repository. Make sure those units have been downloaded.

If I’ll create a new CV version (for example 2) it’ll work.

Expected outcome:

The incremental update terminate w/out errors

Foreman and Proxy versions:

Foreman 1.18.2

foreman-1.18.2-1.el7.noarch
foreman-release-1.18.2-1.el7.noarch

Foreman and Proxy plugin versions:

katello 3.7.0
katello-common-3.7.0-4.el7.noarch

Other relevant data:

2018-10-15T14:32:45 [I|kat|] GET: https://mysatelliteserver.local.net/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 200: {"exception": null, "task_type": "pulp.server.managers.repo.unit_association.associate_from_repo", "_href": "/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/", "task_id": "ec15962c-034e-4486-8e75-e7ef1e045934", "tags": ["pulp:repository:1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:repository:2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:action:associate"], "finish_time": null, "_ns": "task_status", "start_time": "2018-10-15T13:21:44Z", "traceback": null, "spawned_tasks": [], "progress_report": {}, "queue": "reserved_resource_worker-1@mysatelliteserver.dq2", "state": "running", "worker_name": "reserved_resource_worker-1@mysatelliteserver", "result": null, "error": null, "_id": {"$oid": "5bc4946099569390e449afd7"}, "id": "5bc4946099569390e449afd7"}
2018-10-15T14:33:01 [I|kat|] GET: https://mysatelliteserver.local.net/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 200: {"exception": null, "task_type": "pulp.server.managers.repo.unit_association.associate_from_repo", "_href": "/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/", "task_id": "ec15962c-034e-4486-8e75-e7ef1e045934", "tags": ["pulp:repository:1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:repository:2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:action:associate"], "finish_time": null, "_ns": "task_status", "start_time": "2018-10-15T13:21:44Z", "traceback": null, "spawned_tasks": [], "progress_report": {}, "queue": "reserved_resource_worker-1@mysatelliteserver.dq2", "state": "running", "worker_name": "reserved_resource_worker-1@mysatelliteserver", "result": null, "error": null, "_id": {"$oid": "5bc4946099569390e449afd7"}, "id": "5bc4946099569390e449afd7"}
2018-10-15T14:33:17 [I|kat|] GET: https://mysatelliteserver.local.net/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 200: {"exception": null, "task_type": "pulp.server.managers.repo.unit_association.associate_from_repo", "_href": "/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/", "task_id": "ec15962c-034e-4486-8e75-e7ef1e045934", "tags": ["pulp:repository:1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:repository:2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:action:associate"], "finish_time": null, "_ns": "task_status", "start_time": "2018-10-15T13:21:44Z", "traceback": null, "spawned_tasks": [], "progress_report": {}, "queue": "reserved_resource_worker-1@mysatelliteserver.dq2", "state": "running", "worker_name": "reserved_resource_worker-1@mysatelliteserver", "result": null, "error": null, "_id": {"$oid": "5bc4946099569390e449afd7"}, "id": "5bc4946099569390e449afd7"}
2018-10-15T14:33:35 [I|kat|] GET: https://mysatelliteserver.local.net/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 200: {"exception": null, "task_type": "pulp.server.managers.repo.unit_association.associate_from_repo", "_href": "/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/", "task_id": "ec15962c-034e-4486-8e75-e7ef1e045934", "tags": ["pulp:repository:1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:repository:2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:action:associate"], "finish_time": null, "_ns": "task_status", "start_time": "2018-10-15T13:21:44Z", "traceback": null, "spawned_tasks": [], "progress_report": {}, "queue": "reserved_resource_worker-1@mysatelliteserver.dq2", "state": "running", "worker_name": "reserved_resource_worker-1@mysatelliteserver", "result": null, "error": null, "_id": {"$oid": "5bc4946099569390e449afd7"}, "id": "5bc4946099569390e449afd7"}
2018-10-15T14:33:51 [I|kat|] GET: https://mysatelliteserver.local.net/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 200: {"exception": null, "task_type": "pulp.server.managers.repo.unit_association.associate_from_repo", "_href": "/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/", "task_id": "ec15962c-034e-4486-8e75-e7ef1e045934", "tags": ["pulp:repository:1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:repository:2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:action:associate"], "finish_time": null, "_ns": "task_status", "start_time": "2018-10-15T13:21:44Z", "traceback": null, "spawned_tasks": [], "progress_report": {}, "queue": "reserved_resource_worker-1@mysatelliteserver.dq2", "state": "running", "worker_name": "reserved_resource_worker-1@mysatelliteserver", "result": null, "error": null, "_id": {"$oid": "5bc4946099569390e449afd7"}, "id": "5bc4946099569390e449afd7"}
2018-10-15T14:34:07 [I|kat|] GET: https://mysatelliteserver.local.net/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 200: {"exception": null, "task_type": "pulp.server.managers.repo.unit_association.associate_from_repo", "_href": "/pulp/api/v2/tasks/ec15962c-034e-4486-8e75-e7ef1e045934/", "task_id": "ec15962c-034e-4486-8e75-e7ef1e045934", "tags": ["pulp:repository:1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:repository:2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:action:associate"], "finish_time": "2018-10-15T13:34:02Z", "_ns": "task_status", "start_time": "2018-10-15T13:21:44Z", "traceback": null, "spawned_tasks": [], "progress_report": {}, "queue": "reserved_resource_worker-1@mysatelliteserver.dq2", "state": "finished", "worker_name": "reserved_resource_worker-1@mysatelliteserver", "result": {"units_successful": [{"unit_key": {"id": "RHSA-2018:2884"}, "type_id": "erratum"}, {"signing_key": null, "unit_key": {"name": "firefox", "checksum": "84cf742f0e694dd0350b19b7f21a953a1c328a86", "epoch": "0", "version": "60.2.2", "release": "1.el7_5", "arch": "x86_64", "checksumtype": "sha1"}, "type_id": "rpm"}], "units_failed_signature_filter": []}, "error": null, "_id": {"$oid": "5bc4946099569390e449afd7"}, "id": "5bc4946099569390e449afd7"}
2018-10-15T14:34:10 [I|kat|] GET: https://mysatelliteserver.local.net/pulp/api/v2/repositories/1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686/?details=true: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 200: {"scratchpad": {}, "display_name": "Red Hat Enterprise Linux 7 Server RPMs x86_64 7Server", "description": null, "distributors": [{"repo_id": "1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "last_updated": "2018-10-15T13:18:55Z", "_href": "/pulp/api/v2/repositories/1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686/distributors/export_distributor/", "last_override_config": {}, "last_publish": null, "distributor_type_id": "export_distributor", "auto_publish": false, "scratchpad": {}, "_ns": "repo_distributors", "_id": {"$oid": "5bc493bf10f8e019046433bb"}, "config": {"http": false, "relative_url": "mycompany-vCentre/content_views/RHEL_Views/9.3/content/dist/rhel/server/7/7Server/x86_64/os", "https": false}, "id": "export_distributor"}, {"repo_id": "1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "last_updated": "2018-10-15T13:18:55Z", "_href": "/pulp/api/v2/repositories/1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686/distributors/1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686_clone/", "last_override_config": {}, "last_publish": null, "distributor_type_id": "yum_clone_distributor", "auto_publish": false, "scratchpad": {}, "_ns": "repo_distributors", "_id": {"$oid": "5bc493bf10f8e019046433bd"}, "config": {"destination_distributor_id": "1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686"}, "id": "1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686_clone"}, {"repo_id": "1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "last_updated": "2018-10-15T13:18:55Z", "_href": "/pulp/api/v2/repositories/1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686/distributors/1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686/", "last_override_config": {}, "last_publish": null, "distributor_type_id": "yum_distributor", "auto_publish": true, "scratchpad": {}, "_ns": "repo_distributors", "_id": {"$oid": "5bc493bf10f8e019046433b8"}, "config": {"checksum_type": "sha256", "protected": true, "http": false, "https": true, "relative_url": "mycompany-vCentre/content_views/RHEL_Views/9.3/content/dist/rhel/server/7/7Server/x86_64/os"}, "id": "1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686"}], "last_unit_added": "2018-10-15T13:34:02Z", "notes": {"_repo-type": "rpm-repo"}, "last_unit_removed": null, "content_unit_counts": {"package_group": 76, "rpm": 21065, "yum_repo_metadata_file": 1, "erratum": 3102}, "_ns": "repos", "importers": [{"repo_id": "1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "last_updated": "2018-10-15T13:18:55Z", "_href": "/pulp/api/v2/repositories/1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686/importers/yum_importer/", "_ns": "repo_importers", "importer_type_id": "yum_importer", "last_override_config": {}, "last_sync": null, "scratchpad": null, "_id": {"$oid": "5bc493bf10f8e019046433b4"}, "config": {"ssl_validation": true, "remove_missing": true, "download_policy": "on_demand"}, "id": "yum_importer"}], "locally_stored_units": 4491, "_id": {"$oid": "5bc493bf10f8e019046433af"}, "total_repository_units": 24244, "id": "1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "_href": "/pulp/api/v2/repositories/1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686/"}
2018-10-15T14:34:10 [I|kat|] POST: https://mysatelliteserver.local.net/pulp/api/v2/repositories/1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686/actions/publish/: {"id":"1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686","override_config":{"force_full":false}}: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 202: {"spawned_tasks": [{"_href": "/pulp/api/v2/tasks/756d46bb-b79f-4b46-9b93-20632ca2aa59/", "task_id": "756d46bb-b79f-4b46-9b93-20632ca2aa59"}], "result": null, "error": null}
2018-10-15T14:34:10 [I|kat|] GET: https://mysatelliteserver.local.net/pulp/api/v2/tasks/756d46bb-b79f-4b46-9b93-20632ca2aa59/: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 200: {"exception": null, "task_type": "pulp.server.managers.repo.publish.publish", "_href": "/pulp/api/v2/tasks/756d46bb-b79f-4b46-9b93-20632ca2aa59/", "task_id": "756d46bb-b79f-4b46-9b93-20632ca2aa59", "tags": ["pulp:repository:1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:action:publish"], "finish_time": null, "_ns": "task_status", "start_time": null, "traceback": null, "spawned_tasks": [], "progress_report": {}, "queue": "reserved_resource_worker-3@mysatelliteserver.dq2", "state": "waiting", "worker_name": "reserved_resource_worker-3@mysatelliteserver", "result": null, "error": null, "_id": {"$oid": "5bc4975299569390e449df13"}, "id": "5bc4975299569390e449df13"}
2018-10-15T14:34:11 [I|kat|] GET: https://mysatelliteserver.local.net/pulp/api/v2/tasks/756d46bb-b79f-4b46-9b93-20632ca2aa59/: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 200: {"exception": null, "task_type": "pulp.server.managers.repo.publish.publish", "_href": "/pulp/api/v2/tasks/756d46bb-b79f-4b46-9b93-20632ca2aa59/", "task_id": "756d46bb-b79f-4b46-9b93-20632ca2aa59", "tags": ["pulp:repository:1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:action:publish"], "finish_time": null, "_ns": "task_status", "start_time": "2018-10-15T13:34:10Z", "traceback": null, "spawned_tasks": [], "progress_report": {"1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686": [{"num_success": 1, "description": "Initializing repo metadata", "step_type": "initialize_repo_metadata", "items_total": 1, "state": "FINISHED", "error_details": [], "details": "", "num_failures": 0, "step_id": "831c34a4-7067-4c40-b85f-9bcf3ff84687", "num_processed": 1}, {"num_success": 0, "description": "Publishing Distribution files", "step_type": "distribution", "items_total": 0, "state": "FINISHED", "error_details": [], "details": "", "num_failures": 0, "step_id": "ad10d658-3d25-46d9-8e53-bb043d1a0b95", "num_processed": 0}, {"num_success": 0, "description": "Publishing RPMs", "step_type": "rpms", "items_total": 21065, "state": "IN_PROGRESS", "error_details": [], "details": "", "num_failures": 0, "step_id": "5fa3613e-2910-4059-9a23-6393ebd9490d", "num_processed": 0}, {"num_success": 0, "description": "Publishing Delta RPMs", "step_type": "drpms", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "a3024f1a-adc0-4d9f-8278-98bf8210dd21", "num_processed": 0}, {"num_success": 0, "description": "Publishing Errata", "step_type": "errata", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "8a8c42a6-b2fb-4efd-86e6-e44c451f8a4f", "num_processed": 0}, {"num_success": 0, "description": "Publishing Comps file", "step_type": "comps", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "53d13c84-8330-4088-b459-5cd074a2e0f1", "num_processed": 0}, {"num_success": 0, "description": "Publishing Metadata.", "step_type": "metadata", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "02b850c5-2da5-42e6-b439-28cf18eee949", "num_processed": 0}, {"num_success": 0, "description": "Closing repo metadata", "step_type": "close_repo_metadata", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "11d95bfa-d7b6-43b2-9f6e-b5df10159c9f", "num_processed": 0}, {"num_success": 0, "description": "Generating sqlite files", "step_type": "generate sqlite", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "e43d0be3-88ec-4be8-9adc-426774a2e735", "num_processed": 0}, {"num_success": 0, "description": "Removing old repodata", "step_type": "remove_old_repodata", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "07568048-daed-474c-833d-d56d755a3d8a", "num_processed": 0}, {"num_success": 0, "description": "Generating HTML files", "step_type": "repoview", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "d4b6cf24-f12a-4c83-b913-24bf268c8db4", "num_processed": 0}, {"num_success": 0, "description": "Publishing files to web", "step_type": "publish_directory", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "561e3532-af9d-4f68-a329-cefadc4f4aef", "num_processed": 0}, {"num_success": 0, "description": "Writing Listings File", "step_type": "initialize_repo_metadata", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "185aeba5-7938-46ef-8943-21dc68a68471", "num_processed": 0}]}, "queue": "reserved_resource_worker-3@mysatelliteserver.dq2", "state": "running", "worker_name": "reserved_resource_worker-3@mysatelliteserver", "result": null, "error": null, "_id": {"$oid": "5bc4975299569390e449df13"}, "id": "5bc4975299569390e449df13"}
2018-10-15T14:34:12 [I|kat|] GET: https://mysatelliteserver.local.net/pulp/api/v2/tasks/756d46bb-b79f-4b46-9b93-20632ca2aa59/: {"content_type"=>"application/json", "accept"=>"application/json"}
Response: 200: {"exception": null, "task_type": "pulp.server.managers.repo.publish.publish", "_href": "/pulp/api/v2/tasks/756d46bb-b79f-4b46-9b93-20632ca2aa59/", "task_id": "756d46bb-b79f-4b46-9b93-20632ca2aa59", "tags": ["pulp:repository:1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686", "pulp:action:publish"], "finish_time": "2018-10-15T13:34:11Z", "_ns": "task_status", "start_time": "2018-10-15T13:34:10Z", "traceback": "Traceback (most recent call last):\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 367, in trace_task\n    R = retval = fun(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 529, in __call__\n    return super(Task, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 107, in __call__\n    return super(PulpTask, self).__call__(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 622, in __protected_call__\n    return self.run(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 1109, in publish\n    result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 1206, in check_publish\n    result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 1258, in _do_publish\n    publish_report = publish_repo(transfer_repo, conduit, call_config)\n  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 737, in wrap_f\n    return f(*args, **kwargs)\n  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py\", line 174, in publish_repo\n    return self._publisher.process_lifecycle()\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 572, in process_lifecycle\n    super(PluginStep, self).process_lifecycle()\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 163, in process_lifecycle\n    step.process()\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 239, in process\n    self._process_block(item=item)\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 301, in _process_block\n    self.process_main(item=item)\n  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py\", line 496, in process_main\n    context.add_unit_metadata(unit)\n  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/filelists.py\", line 42, in add_unit_metadata\n    self.metadata_file_handle.write(unit.render_filelists(self.checksum_type))\n  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py\", line 863, in render_filelists\n    context = Context({'pkgid': self.get_or_calculate_and_save_checksum(checksumtype)})\n  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py\", line 258, in get_or_calculate_and_save_checksum\n    checksumtype=checksumtype)\nPulpCodedException: Checksum type \"sha256\" is not available for all units in the repository. Make sure those units have been downloaded.\n", "spawned_tasks": [], "progress_report": {"1-RHEL_Views-v9_3-2732f7f1-620c-48ad-9f31-fecdcef60686": [{"num_success": 1, "description": "Initializing repo metadata", "step_type": "initialize_repo_metadata", "items_total": 1, "state": "FINISHED", "error_details": [], "details": "", "num_failures": 0, "step_id": "831c34a4-7067-4c40-b85f-9bcf3ff84687", "num_processed": 1}, {"num_success": 0, "description": "Publishing Distribution files", "step_type": "distribution", "items_total": 0, "state": "FINISHED", "error_details": [], "details": "", "num_failures": 0, "step_id": "ad10d658-3d25-46d9-8e53-bb043d1a0b95", "num_processed": 0}, {"num_success": 0, "description": "Publishing RPMs", "step_type": "rpms", "items_total": 21065, "state": "FAILED", "error_details": [{"traceback": "  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 239, in process\n    self._process_block(item=item)\n\n  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 301, in _process_block\n    self.process_main(item=item)\n\n  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py\", line 496, in process_main\n    context.add_unit_metadata(unit)\n\n  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/filelists.py\", line 42, in add_unit_metadata\n    self.metadata_file_handle.write(unit.render_filelists(self.checksum_type))\n\n  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py\", line 863, in render_filelists\n    context = Context({'pkgid': self.get_or_calculate_and_save_checksum(checksumtype)})\n\n  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py\", line 258, in get_or_calculate_and_save_checksum\n    checksumtype=checksumtype)\n", "error": "Checksum type \"sha256\" is not available for all units in the repository. Make sure those units have been downloaded."}], "details": "", "num_failures": 1, "step_id": "5fa3613e-2910-4059-9a23-6393ebd9490d", "num_processed": 1}, {"num_success": 0, "description": "Publishing Delta RPMs", "step_type": "drpms", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "a3024f1a-adc0-4d9f-8278-98bf8210dd21", "num_processed": 0}, {"num_success": 0, "description": "Publishing Errata", "step_type": "errata", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "8a8c42a6-b2fb-4efd-86e6-e44c451f8a4f", "num_processed": 0}, {"num_success": 0, "description": "Publishing Comps file", "step_type": "comps", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "53d13c84-8330-4088-b459-5cd074a2e0f1", "num_processed": 0}, {"num_success": 0, "description": "Publishing Metadata.", "step_type": "metadata", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "02b850c5-2da5-42e6-b439-28cf18eee949", "num_processed": 0}, {"num_success": 0, "description": "Closing repo metadata", "step_type": "close_repo_metadata", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "11d95bfa-d7b6-43b2-9f6e-b5df10159c9f", "num_processed": 0}, {"num_success": 0, "description": "Generating sqlite files", "step_type": "generate sqlite", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "e43d0be3-88ec-4be8-9adc-426774a2e735", "num_processed": 0}, {"num_success": 0, "description": "Removing old repodata", "step_type": "remove_old_repodata", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "07568048-daed-474c-833d-d56d755a3d8a", "num_processed": 0}, {"num_success": 0, "description": "Generating HTML files", "step_type": "repoview", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "d4b6cf24-f12a-4c83-b913-24bf268c8db4", "num_processed": 0}, {"num_success": 0, "description": "Publishing files to web", "step_type": "publish_directory", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "561e3532-af9d-4f68-a329-cefadc4f4aef", "num_processed": 0}, {"num_success": 0, "description": "Writing Listings File", "step_type": "initialize_repo_metadata", "items_total": 1, "state": "NOT_STARTED", "error_details": [], "details": "", "num_failures": 0, "step_id": "185aeba5-7938-46ef-8943-21dc68a68471", "num_processed": 0}]}, "queue": "reserved_resource_worker-3@mysatelliteserver.dq2", "state": "error", "worker_name": "reserved_resource_worker-3@mysatelliteserver", "result": null, "error": {"code": "RPM1008", "data": {"checksumtype": "sha256"}, "description": "Checksum type \"sha256\" is not available for all units in the repository. Make sure those units have been downloaded.", "sub_errors": []}, "_id": {"$oid": "5bc4975299569390e449df13"}, "id": "5bc4975299569390e449df13"}
2018-10-15T14:34:12 [E|bac|] RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded. (Katello::Errors::PulpError)
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/pulp/abstract_async_task.rb:121:in `block in external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/pulp/abstract_async_task.rb:119:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/pulp/abstract_async_task.rb:119:in `external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action/polling.rb:98:in `poll_external_task_with_rescue'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action/polling.rb:21:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action/cancellable.rb:13:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/pulp/abstract_async_task.rb:45:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:538:in `block (3 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/middleware/skip_if_matching_content.rb:5:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:31:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/middleware/remote_action.rb:16:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/execution_wrapper.rb:85:in `wrap'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action/progress.rb:16:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/middleware/keep_locale.rb:11:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.7.0/app/lib/actions/middleware/keep_locale.rb:11:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/app/lib/actions/middleware/keep_current_user.rb:43:in `restore_curent_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:31:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/world.rb:30:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:537:in `block (2 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:536:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:536:in `block in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:451:in `block in with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:451:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:451:in `with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:531:in `execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:278:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block (2 levels) in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract.rb:162:in `with_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `block in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:30:in `open_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:15:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/director.rb:56:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/executors/parallel/worker.rb:12:in `block in on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/executors.rb:12:in `run_user_code'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/executors/parallel/worker.rb:11:in `on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/context.rb:46:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/actor.rb:26:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:161:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:95:in `block in on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `block in synchronize'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:115:in `block in schedule_execution'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:18:in `call'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:96:in `work'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

After sync & errata calculation I tried to apply the errata, I tried to generate an incremental update of CV but foreman fail at 66% with this error:

RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.
RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.
RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.
RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.
RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.
RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.
RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.
RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.
RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.
RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.

I tried to set up my repo as written on rh documentation:

   repo = Katello::Repository.find_by_pulp_id("<IDENTIFIER>")
   repo.checksum_type = "sha256"
   repo.save!

but nothing changed.
Following Bug #24730: unable to publish a content view including RH repo with a filter - Katello - Foreman it’s a bug, there’s a way/fix to solve the issue before the katello package upgrade (and w/out patching the code)?
Thanks !

John_Mitsch · October 16, 2018, 5:08pm

Looking at the fix, I don’t see an easy way to solve the issue except for upgrading to 3.8 so the proper upgrade task runs.

kraba · October 16, 2018, 7:34pm

I upgraded the server to katello 3.8 but the problem remains…

I checked the pulp version and is the same of the previous version (3.7/1.18.2):

pulp-docker-plugins-3.1.4-2.el7.noarch
pulp-selinux-2.16.4-1.el7.noarch
pulp-server-2.16.4-1.el7.noarch
pulp-katello-1.0.2-1.el7.noarch
pulp-admin-client-2.16.4-1.el7.noarch
pulp-client-1.0-1.noarch
pulp-puppet-tools-2.16.4-1.el7.noarch
pulp-puppet-plugins-2.16.4-1.el7.noarch
pulp-deb-plugins-1.7.1-1.el7.noarch
pulp-rpm-plugins-2.16.4-1.el7.noarch

There is a way to upgrade only pulp now?
on my yum file I have

[katello]
name=Katello 3.8
baseurl=https://fedorapeople.org/groups/katello/releases/yum/3.8/katello/el7/$basearch/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-katello
enabled=1
gpgcheck=1

[pulp]
name=Pulp Community Release
baseurl=https://repos.fedorapeople.org/repos/pulp/pulp/stable/2.16/$releasever/$basearch
gpgkey=https://repos.fedorapeople.org/repos/pulp/pulp/GPG-RPM-KEY-pulp-2
enabled=1
gpgcheck=1
```

but no package was upgraded....

I'm the only one with this problem?

kraba · November 17, 2018, 3:35pm

I’m come back to this issue, I upgraded katello to 3.8.1 who solved this problem ( Bug #24730: unable to publish a content view including RH repo with a filter - Katello - Foreman ), I tried to publish a new incremental update of the CV but the issue remain.
Actual version is foreman 1.19.1 and katello 3.8.1.
We’ve a mixed rhel6 and 7 repos to sync in the same CV, a new CV publishing is working properly.
Do someone know where I can check/fix?

If you need other info, test, log…I’ll provide all. I’m very frustrated about that, I want to solve the issue and maybe a is a miss configured issue (or my fault…)

The error log is :

Action:
Actions::Pulp::Repository::DistributorPublish

Input:
{"pulp_id"=>"1-RHEL_Views-v18_3-92304746-5061-436a-abb6-67ae17fda5d3",
 "distributor_type_id"=>"yum_distributor",
 "source_pulp_id"=>nil,
 "override_config"=>{"force_full"=>false},
 "dependency"=>nil,
 "matching_content"=>nil,
 "remote_user"=>"admin",
 "remote_cp_user"=>"admin"}

Output:
{"pulp_tasks"=>
  [{"exception"=>nil,
    "task_type"=>"pulp.server.managers.repo.publish.publish",
    "_href"=>"/pulp/api/v2/tasks/96234b82-e95a-45da-8589-a6fcad0dcae3/",
    "task_id"=>"96234b82-e95a-45da-8589-a6fcad0dcae3",
    "tags"=>
     ["pulp:repository:1-RHEL_Views-v18_3-92304746-5061-436a-abb6-67ae17fda5d3",
      "pulp:action:publish"],
    "finish_time"=>"2018-11-17T14:56:42Z",
    "_ns"=>"task_status",
    "start_time"=>"2018-11-17T14:56:42Z",
    "traceback"=>
     "Traceback (most recent call last):\n" +
     "  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 367, in trace_task\n" +
     "    R = retval = fun(*args, **kwargs)\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 529, in __call__\n" +
     "    return super(Task, self).__call__(*args, **kwargs)\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 107, in __call__\n" +
     "    return super(PulpTask, self).__call__(*args, **kwargs)\n" +
     "  File \"/usr/lib/python2.7/site-packages/celery/app/trace.py\", line 622, in __protected_call__\n" +
     "    return self.run(*args, **kwargs)\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 1109, in publish\n" +
     "    result = check_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 1206, in check_publish\n" +
     "    result = _do_publish(repo_obj, dist_id, dist_inst, transfer_repo, conduit, call_config)\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py\", line 1258, in _do_publish\n" +
     "    publish_report = publish_repo(transfer_repo, conduit, call_config)\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py\", line 737, in wrap_f\n" +
     "    return f(*args, **kwargs)\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py\", line 174, in publish_repo\n" +
     "    return self._publisher.process_lifecycle()\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 572, in process_lifecycle\n" +
     "    super(PluginStep, self).process_lifecycle()\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 163, in process_lifecycle\n" +
     "    step.process()\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 239, in process\n" +
     "    self._process_block(item=item)\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 301, in _process_block\n" +
     "    self.process_main(item=item)\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py\", line 496, in process_main\n" +
     "    context.add_unit_metadata(unit)\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/filelists.py\", line 42, in add_unit_metadata\n" +
     "    self.metadata_file_handle.write(unit.render_filelists(self.checksum_type))\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py\", line 863, in render_filelists\n" +
     "    context = Context({'pkgid': self.get_or_calculate_and_save_checksum(checksumtype)})\n" +
     "  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py\", line 258, in get_or_calculate_and_save_checksum\n" +
     "    checksumtype=checksumtype)\n" +
     "PulpCodedException: Checksum type \"sha256\" is not available for all units in the repository. Make sure those units have been downloaded.\n",
    "spawned_tasks"=>[],
    "progress_report"=>
     {"1-RHEL_Views-v18_3-92304746-5061-436a-abb6-67ae17fda5d3"=>
       [{"num_success"=>1,
         "description"=>"Initializing repo metadata",
         "step_type"=>"initialize_repo_metadata",
         "items_total"=>1,
         "state"=>"FINISHED",
         "error_details"=>[],
          "details"=>"",
          "num_failures"=>0,
         "step_id"=>"4b6da448-d4cb-4ef5-9fb3-7e70b0bba0c9",
         "num_processed"=>1},
         {"num_success"=>0,
          "description"=>"Publishing Distribution files",
          "step_type"=>"distribution",
          "items_total"=>0,
          "state"=>"FINISHED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
         "step_id"=>"c83c1330-cdb1-4fc2-9958-71568f913cf6",
         "num_processed"=>0},> >         {"num_success"=>0,
         "description"=>"Publishing RPMs",
         "step_type"=>"rpms",
         "items_total"=>9129,
         "state"=>"FAILED",
         "error_details"=>
          [{"traceback"=>
             "  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 239, in process\n" +
             "    self._process_block(item=item)\n" +
             "\n" +
             "  File \"/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py\", line 301, in _process_block\n" +
             "    self.process_main(item=item)\n" +
             "\n" +
             "  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/publish.py\", line 496, in process_main\n" +
             "    context.add_unit_metadata(unit)\n" +
             "\n" +
             "  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/yum/metadata/filelists.py\", line 42, in add_unit_metadata\n" +
             "    self.metadata_file_handle.write(unit.render_filelists(self.checksum_type))\n" +
             "\n" +
             "  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py\", line 863, in render_filelists\n" +
             "    context = Context({'pkgid': self.get_or_calculate_and_save_checksum(checksumtype)})\n" +
             "\n" +
             "  File \"/usr/lib/python2.7/site-packages/pulp_rpm/plugins/db/models.py\", line 258, in get_or_calculate_and_save_checksum\n" +
             "    checksumtype=checksumtype)\n",
            "error"=>
             "Checksum type \"sha256\" is not available for all units in the repository. Make sure those units have been downloaded."}],
         "details"=>"",
         "num_failures"=>1,
         "step_id"=>"06435eff-2e59-48c0-b55d-268e6f37a4b4",
         "num_processed"=>1},
        {"num_success"=>0,
         "description"=>"Publishing Delta RPMs",
         "step_type"=>"drpms",
         "items_total"=>1,
         "state"=>"NOT_STARTED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
         "step_id"=>"8618f646-95fe-4bc3-8972-bfe1d96d41f4",
        "num_processed"=>0},
        {"num_success"=>0,
         "description"=>"Publishing Errata",
         "step_type"=>"errata",
         "items_total"=>1,
         "state"=>"NOT_STARTED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
         "step_id"=>"0cbbfec8-badd-4cc0-817c-dcd8475ca0aa",
         "num_processed"=>0},
        {"num_success"=>0,
         "description"=>"Publishing Comps file",
         "step_type"=>"comps",
         "items_total"=>1,
         "state"=>"NOT_STARTED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
         "step_id"=>"2bcac1fe-719c-4d81-9ed5-c7a729089e58",
         "num_processed"=>0},
        {"num_success"=>0,
         "description"=>"Publishing Metadata.",
         "step_type"=>"metadata",
         "items_total"=>1,
         "state"=>"NOT_STARTED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
         "step_id"=>"a609364b-f519-4f70-99ed-6ec3e660dedb",
         "num_processed"=>0},
        {"num_success"=>0,
         "description"=>"Closing repo metadata",
         "step_type"=>"close_repo_metadata",
         "items_total"=>1,
         "state"=>"NOT_STARTED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
         "step_id"=>"57f81562-2191-4676-b2fd-18d9b39f3090",
         "num_processed"=>0},
        {"num_success"=>0,
         "description"=>"Generating sqlite files",
         "step_type"=>"generate sqlite",
         "items_total"=>1,
         "state"=>"NOT_STARTED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
        "step_id"=>"855a92e1-76fd-42a0-b295-4ea162977821",
         "num_processed"=>0},
        {"num_success"=>0,
         "description"=>"Removing old repodata",
         "step_type"=>"remove_old_repodata",
         "items_total"=>1,
         "state"=>"NOT_STARTED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
         "step_id"=>"fbcb387a-277e-4abc-bd52-1f9b6f533e0c",
         "num_processed"=>0},
        {"num_success"=>0,
         "description"=>"Generating HTML files",
         "step_type"=>"repoview",
         "items_total"=>1,
         "state"=>"NOT_STARTED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
         "step_id"=>"240c5e46-d4d3-47c4-9c72-238dd8c4a710",
         "num_processed"=>0},
        {"num_success"=>0,
         "description"=>"Publishing files to web",
         "step_type"=>"publish_directory",
         "items_total"=>1,
         "state"=>"NOT_STARTED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
         "step_id"=>"73dedd26-e72e-4c0c-95f4-5d80c21b4e6e",
         "num_processed"=>0},
        {"num_success"=>0,
         "description"=>"Writing Listings File",
         "step_type"=>"initialize_repo_metadata",
         "items_total"=>1,
         "state"=>"NOT_STARTED",
         "error_details"=>[],
         "details"=>"",
         "num_failures"=>0,
         "step_id"=>"090491ef-231a-4c83-909a-6f630c2d8423",
         "num_processed"=>0}]},
    "queue"=>"reserved_resource_worker-0@atsatellite.dq2",
    "state"=>"error",
    "worker_name"=>"reserved_resource_worker-0@
atsatellite",
    "result"=>nil,
    "error"=>
     {"code"=>"RPM1008",
      "data"=>{"checksumtype"=>"sha256"},
      "description"=>
       "Checksum type \"sha256\" is not available for all units in the repository. Make sure those units have been downloaded.",
      "sub_errors"=>[]},
    "_id"=>{"$oid"=>"5bf02c218255fd6b07e442f4"},
    "id"=>"5bf02c218255fd6b07e442f4"}],
 "poll_attempts"=>{"total"=>8, "failed"=>1}}
Exception:
Katello::Errors::PulpError: RPM1008: Checksum type "sha256" is not available for all units in the repository. Make sure those units have been downloaded.

Backtrace:

/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/pulp/abstract_async_task.rb:121:in `block in external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/pulp/abstract_async_task.rb:119:in `each'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/pulp/abstract_async_task.rb:119:in `external_task='
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action/polling.rb:98:in `poll_external_task_with_rescue'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action/polling.rb:21:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action/cancellable.rb:13:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/pulp/abstract_async_task.rb:45:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:538:in `block (3 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/middleware/skip_if_matching_content.rb:5:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:31:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/middleware/remote_action.rb:16:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/execution_wrapper.rb:85:in `wrap'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action/progress.rb:30:in `with_progress_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action/progress.rb:16:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/middleware/keep_locale.rb:11:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/middleware/keep_locale.rb:22:in `with_locale'
/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/app/lib/actions/middleware/keep_locale.rb:11:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/app/lib/actions/middleware/keep_current_user.rb:43:in `restore_curent_user'
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:26:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:18:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware.rb:31:in `run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/stack.rb:22:in `call'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/middleware/world.rb:30:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:537:in `block (2 levels) in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:536:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:536:in `block in execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:451:in `block in with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:451:in `catch'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:451:in `with_error_handling'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:531:in `execute_run'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/action.rb:278:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block (2 levels) in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract.rb:162:in `with_meta_calculation'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `block in execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:30:in `open_action'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:15:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/director.rb:56:in `execute'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/executors/parallel/worker.rb:12:in `block in on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/executors.rb:12:in `run_user_code'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/executors/parallel/worker.rb:11:in `on_message'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/context.rb:46:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/executes_context.rb:7:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.0/lib/dynflow/actor.rb:26:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/awaits.rb:15:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:38:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:31:in `process_envelopes?'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:20:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/termination.rb:55:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/removes_child.rb:10:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in `pass'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in `on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:161:in `process_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:95:in `block in on_envelope'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:118:in `block (2 levels) in schedule_execution'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in `synchronize'
/opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:115:in `block in schedule_execution'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:18:in `call'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:96:in `work'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:77:in `block in call_job'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in `run_task'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in `block (3 levels) in create_worker'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `loop'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in `block (2 levels) in create_worker'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `catch'
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/concurrent-ruby1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in `block in create_worker'
/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

kraba · November 23, 2018, 4:20pm

Sorry to keep calling you, but no one replied
Can someone from @katello help me to solve this issue? I can provide all necessary logs and tests.

John_Mitsch · November 27, 2018, 1:12pm

Hey @kraba apologies on the delay getting back to you. Sorry you are having such a frustrating time with the upgrade. My first question is, how are you upgrading? Did you run foreman-installer --upgrade each time after upgrading the katello rpm?

The upgrade task that was added in 3.8.1 should have taken care of this issue. The pulp versions look fine to me. Lets check a few things:

Can you paste the output of Katello::Repository.column_names.sort? (run in foreman-rake console)
Here is what I see on a 3.8.1 box for comparison:

=> ["arch", "checksum_type", "container_repository_name", "content_id", "content_type", "content_view_version_id", "cp_label", "created_at", "deb_architectures", "deb_components", "deb_releases", "description", "distribution_arch", "distribution_bootable", "distribution_family", "distribution_uuid", "distribution_variant", "distribution_version", "docker_upstream_name", "download_policy", "environment_id", "gpg_key_id", "id", "ignorable_content", "ignore_global_proxy", "label", "library_instance_id", "major", "minor", "mirror_on_sync", "name", "ostree_upstream_sync_depth", "ostree_upstream_sync_policy", "product_id", "pulp_id", "relative_path", "ssl_ca_cert_id", "ssl_client_cert_id", "ssl_client_key_id", "unprotected", "updated_at", "upstream_password", "upstream_username", "url", "verify_ssl_on_sync"]

Run this mongo query from the command line

mongo pulp_database --eval 'db.units_rpm.find({checksumtype: { $ne: "sha256" }}).count()'

kraba · November 27, 2018, 1:56pm

Hello John,
thanks for reply!
Yes, I ran foreman-installer --upgrade before installing katello rpm.
These are the two answers for your request:

=> ["arch", "checksum_type", "container_repository_name", "content_id", "content_type", "content_view_version_id", "cp_label", "created_at", "deb_architectures", "deb_components", "deb_releases", "description", "distribution_arch", "distribution_bootable", "distribution_family", "distribution_uuid", "distribution_variant", "distribution_version", "docker_upstream_name", "download_policy", "environment_id", "gpg_key_id", "id", "ignorable_content", "ignore_global_proxy", "label", "library_instance_id", "major", "minor", "mirror_on_sync", "name", "ostree_upstream_sync_depth", "ostree_upstream_sync_policy", "product_id", "pulp_id", "relative_path", "source_repo_checksum_type", "ssl_ca_cert_id", "ssl_client_cert_id", "ssl_client_key_id", "unprotected", "updated_at", "upstream_password", "upstream_username", "url", "verify_ssl_on_sync"]

and

MongoDB shell version v3.4.9
connecting to: mongodb://127.0.0.1:27017/pulp_database
MongoDB server version: 3.4.9
65330

The first one differ from your after “relative path” :

(mine)
"source_repo_checksum_type", "ssl_ca_cert_id", "ssl_client_cert_id", "ssl_client_key_id", "unprotected", "updated_at", "upstream_password", "upstream_username", "url", "verify_ssl_on_sync"]

(your)
"ssl_ca_cert_id", "ssl_client_cert_id", "ssl_client_key_id", "unprotected", "updated_at", "upstream_password", "upstream_username", "url", "verify_ssl_on_sync"]

It seems "source_repo_checksum_type" remained!

John_Mitsch · November 27, 2018, 2:20pm

From those two outputs, it looks like the upgrade tasks didn’t run

I think we found the issue, you have to run the upgrade command after installing the katello rpm. This ensures that the new database migration, upgrade tasks, etc… that the new rpm provide are correctly applied.

Try running ‘foreman-installer --upgrade’ now and see where that gets you

kraba · November 27, 2018, 3:42pm

Thanks John, I ran the upgrade the output is the same of before upgrade (source_repo_checksum_type is present):

=> ["arch", "checksum_type", "container_repository_name", "content_id", "content_type", "content_view_version_id", "cp_label", "created_at", "deb_architectures", "deb_components", "deb_releases", "description", "distribution_arch", "distribution_bootable", "distribution_family", "distribution_uuid", "distribution_variant", "distribution_version", "docker_upstream_name", "download_policy", "environment_id", "gpg_key_id", "id", "ignorable_content", "ignore_global_proxy", "label", "library_instance_id", "major", "minor", "mirror_on_sync", "name", "ostree_upstream_sync_depth", "ostree_upstream_sync_policy", "product_id", "pulp_id", "relative_path", "source_repo_checksum_type", "ssl_ca_cert_id", "ssl_client_cert_id", "ssl_client_key_id", "unprotected", "updated_at", "upstream_password", "upstream_username", "url", "verify_ssl_on_sync"]
irb(main):002:0> quit


MongoDB shell version v3.4.9
connecting to: mongodb://127.0.0.1:27017/pulp_database
MongoDB server version: 3.4.9
65330

This is the --upgrade log (i attach also katello.log)

Upgrading, to monitor the progress on all related services, please do:
  foreman-tail | tee upgrade-$(date +%Y-%m-%d-%H%M).log
Upgrade Step: stop_services...
Redirecting to 'foreman-maintain service'
Running Stop Services
================================================================================
Check if command is run as root user:                                 [OK]
--------------------------------------------------------------------------------
Stop applicable services: Stopping the following service(s):

postgresql, rh-mongodb34-mongod, qdrouterd, qpidd, squid, pulp_resource_manager, pulp_streamer, smart_proxy_dynflow_core, foreman-proxy, pulp_celerybeat, pulp_workers, tomcat, goferd, dynflowd, httpd, puppetserver

/ All services stopped                                                [OK]
--------------------------------------------------------------------------------

katello-service stop finished successfully!
Upgrade Step: start_postgresql...
Redirecting to 'foreman-maintain service'
Running Start Services
================================================================================
Check if command is run as root user:                                 [OK]
--------------------------------------------------------------------------------
Start applicable services: Starting the following service(s):

postgresql

\ All services started                                                [OK]
--------------------------------------------------------------------------------

katello-service start --only postgresql finished successfully!
Upgrade Step: migrate_pulp...
systemctl start rh-mongodb34-mongod finished successfully!
Attempting to connect to localhost:27017
Attempting to connect to localhost:27017
Write concern for Mongo connection: {}
Loading content types.
Loading type descriptors [deb.json]
Parsing type descriptors
Validating type descriptor syntactic integrity
Validating type descriptor semantic integrity
Loading unit model: puppet_module = pulp_puppet.plugins.db.models:Module
Loading unit model: docker_manifest = pulp_docker.plugins.models:Manifest
Loading unit model: docker_blob = pulp_docker.plugins.models:Blob
Loading unit model: docker_manifest_list = pulp_docker.plugins.models:ManifestList
Loading unit model: docker_image = pulp_docker.plugins.models:Image
Loading unit model: docker_tag = pulp_docker.plugins.models:Tag
Loading unit model: erratum = pulp_rpm.plugins.db.models:Errata
Loading unit model: distribution = pulp_rpm.plugins.db.models:Distribution
Loading unit model: srpm = pulp_rpm.plugins.db.models:SRPM
Loading unit model: package_group = pulp_rpm.plugins.db.models:PackageGroup
Loading unit model: package_category = pulp_rpm.plugins.db.models:PackageCategory
Loading unit model: iso = pulp_rpm.plugins.db.models:ISO
Loading unit model: package_environment = pulp_rpm.plugins.db.models:PackageEnvironment
Loading unit model: drpm = pulp_rpm.plugins.db.models:DRPM
Loading unit model: package_langpacks = pulp_rpm.plugins.db.models:PackageLangpacks
Loading unit model: rpm = pulp_rpm.plugins.db.models:RPM
Loading unit model: yum_repo_metadata_file = pulp_rpm.plugins.db.models:YumMetadataFile
Loading unit model: deb_release = pulp_deb.plugins.db.models:DebRelease
Loading unit model: deb = pulp_deb.plugins.db.models:DebPackage
Loading unit model: deb_component = pulp_deb.plugins.db.models:DebComponent
Loading auxiliary model: erratum_pkglists = pulp_rpm.plugins.db.models:ErratumPkglist
Updating the database with types [deb, deb_component, deb_release]
Found the following type definitions that were not present in the update collection [puppet_module, drpm, package_langpacks, docker_manifest, docker_blob, erratum, yum_repo_metadata_file, package_group, package_category, iso, package_e  nvironment, docker_tag, srpm, rpm, distribution, docker_image, docker_manifest_list]
Updating the database with types [puppet_module, drpm, deb_component, package_langpacks, docker_manifest, docker_blob, erratum, yum_repo_metadata_file, deb_release, package_group, package_category, iso, package_environment, docker_tag,   deb, distribution, rpm, srpm, docker_image, docker_manifest_list]
Content types loaded.
Ensuring the admin role and user are in place.
Admin role and user are in place.
Beginning database migrations.
Migration package pulp.server.db.migrations is up to date at version 28
Migration package pulp_deb.plugins.migrations is up to date at version 1
Migration package pulp_docker.plugins.migrations is up to date at version 5
Migration package pulp_puppet.plugins.migrations is up to date at version 5
Migration package pulp_rpm.plugins.migrations is up to date at version 42
Loading unit model: puppet_module = pulp_puppet.plugins.db.models:Module
Loading unit model: docker_manifest = pulp_docker.plugins.models:Manifest
Loading unit model: docker_blob = pulp_docker.plugins.models:Blob
Loading unit model: docker_manifest_list = pulp_docker.plugins.models:ManifestList
Loading unit model: docker_image = pulp_docker.plugins.models:Image
Loading unit model: docker_tag = pulp_docker.plugins.models:Tag
Loading unit model: erratum = pulp_rpm.plugins.db.models:Errata
Loading unit model: distribution = pulp_rpm.plugins.db.models:Distribution
Loading unit model: srpm = pulp_rpm.plugins.db.models:SRPM
Loading unit model: package_group = pulp_rpm.plugins.db.models:PackageGroup
Loading unit model: package_category = pulp_rpm.plugins.db.models:PackageCategory
Loading unit model: iso = pulp_rpm.plugins.db.models:ISO
Loading unit model: package_environment = pulp_rpm.plugins.db.models:PackageEnvironment
Loading unit model: drpm = pulp_rpm.plugins.db.models:DRPM
Loading unit model: package_langpacks = pulp_rpm.plugins.db.models:PackageLangpacks
Loading unit model: rpm = pulp_rpm.plugins.db.models:RPM
Loading unit model: yum_repo_metadata_file = pulp_rpm.plugins.db.models:YumMetadataFile
Loading unit model: deb_release = pulp_deb.plugins.db.models:DebRelease
Loading unit model: deb = pulp_deb.plugins.db.models:DebPackage
Loading unit model: deb_component = pulp_deb.plugins.db.models:DebComponent
Loading auxiliary model: erratum_pkglists = pulp_rpm.plugins.db.models:ErratumPkglist
Database migrations complete.
su - apache -s /bin/bash -c pulp-manage-db finished successfully!
Upgrade Step: migrate_candlepin...
Configuring PostgreSQL with JDBC URL: jdbc:postgresql://localhost/candlepin
Migrating Candlepin database
--driver=org.postgresql.Driver --classpath=/usr/share/java/postgresql-jdbc.jar:/var/lib/tomcat/webapps/candlepin/WEB-INF/classes/ --changeLogFile=db/changelog/changelog-update.xml --url=jdbc:postgresql://localhost/candlepin --username=  candlepin --password=UuzUMnK3XgDb9Heu2FXa4jZHwYUky7mR --logLevel=severe
Liquibase Update Successful
/usr/share/candlepin/cpdb --update --database '//localhost/caandlepin' --user 'candlepin' --password 'UuzUMnK3XgDb9Heu2FXa4jZHwYUky7mR' finished successfully!
Upgrade Step: migrate_foreman...
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
/usr/share/foreman/lib/core_extensions.rb:182: warning: already initialized constant ActiveSupport::MessageEncryptor::DEFAULT_CIPHER
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/message_encryptor.rb:22: warning: previous definition of DEFAULT_CIPHER was here
foreman-rake db:migrate finished successfully!
Upgrade Step: Running installer...
Installing             Done                                               [100%] [........................................................................................................................................................]
  The full log is at /var/log/foreman-installer/katello.log
Upgrade Step: restart_services...
Redirecting to 'foreman-maintain service'
Running Restart Services
================================================================================
Check if command is run as root user:                                 [OK]
--------------------------------------------------------------------------------
Restart applicable services: Stopping the following service(s):

postgresql, rh-mongodb34-mongod, qdrouterd, qpidd, squid, pulp_resource_manager, pulp_streamer, smart_proxy_dynflow_core, foreman-proxy, pulp_celerybeat, pulp_workers, tomcat, goferd, dynflowd, httpd, puppetserver

| All services stopped
Starting the following service(s):

postgresql, rh-mongodb34-mongod, qdrouterd, qpidd, squid, pulp_resource_manager, pulp_streamer, smart_proxy_dynflow_core, foreman-proxy, pulp_celerybeat, pulp_workers, tomcat, goferd, dynflowd, httpd, puppetserver

\ All services started                                                [OK]
--------------------------------------------------------------------------------

katello-service restart finished successfully!
Upgrade Step: db_seed...
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
/usr/share/foreman/lib/core_extensions.rb:182: warning: already initialized constant ActiveSupport::MessageEncryptor::DEFAULT_CIPHER
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/message_encryptor.rb:22: warning: previous definition of DEFAULT_CIPHER was here
Seeding /usr/share/foreman/db/seeds.d/020-partition_tables_list.rb
Seeding /usr/share/foreman/db/seeds.d/020-permissions_list.rb
Seeding /usr/share/foreman/db/seeds.d/020-provisioning_templates_list.rb
Seeding /usr/share/foreman/db/seeds.d/020-roles_list.rb
Seeding /usr/share/foreman/db/seeds.d/030-auth_sources.rb
Seeding /usr/share/foreman/db/seeds.d/030-permissions.rb
Seeding /usr/share/foreman/db/seeds.d/035-admin.rb
Seeding /usr/share/foreman/db/seeds.d/040-roles.rb
Seeding /usr/share/foreman/db/seeds.d/050-taxonomies.rb
Seeding /usr/share/foreman/db/seeds.d/060-architectures.rb
Seeding /usr/share/foreman/db/seeds.d/070-provisioning_templates.rb
Seeding /usr/share/foreman/db/seeds.d/080-partition_tables.rb
Seeding /usr/share/foreman/db/seeds.d/100-installation_media.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/db/seeds.d/101-locations.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/db/seeds.d/102-organizations.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/db/seeds.d/104-proxy.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/db/seeds.d/106-mail_notifications.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/db/seeds.d/107-enable_dynflow.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/db/seeds.d/108-subcription-bookmarks.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/db/seeds.d/109-katello-notification-blueprints.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/db/seeds.d/110-content-view-autopublish.rb
Seeding /usr/share/foreman/db/seeds.d/110-smart_proxy_features.rb
Seeding /usr/share/foreman/db/seeds.d/130-compute_profiles.rb
Seeding /usr/share/foreman/db/seeds.d/150-bookmarks.rb
Seeding /usr/share/foreman/db/seeds.d/160-mail_notifications.rb
Seeding /usr/share/foreman/db/seeds.d/170-notification_blueprints.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/db/seeds.d/20-foreman_tasks_permissions.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_bootdisk-12.0.0/db/seeds.d/50-bootdisk_templates.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.5.6/db/seeds.d/50-notification_blueprints.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/db/seeds.d/60-dynflow_proxy_feature.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.5.6/db/seeds.d/60-ssh_proxy_feature.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.13.4/db/seeds.d/61-foreman_tasks_bookmarks.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-2.2.9/db/seeds.d/62_ansible_proxy_feature.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.5.6/db/seeds.d/70-job_templates.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.8.1/db/seeds.d/75-job_templates.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-2.2.9/db/seeds.d/75_job_templates.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_remote_execution-1.5.6/db/seeds.d/90-bookmarks.rb
Seeding /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_ansible-2.2.9/db/seeds.d/90_notification_blueprints.rb
All seed files executed
foreman-rake db:seed finished successfully!
Upgrade Step: correct_repositories (this may take a while) ...
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
/usr/share/foreman/lib/core_extensions.rb:182: warning: already initialized constant ActiveSupport::MessageEncryptor::DEFAULT_CIPHER
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/message_encryptor.rb:22: warning: previous definition of DEFAULT_CIPHER was here
Processing Repository 1/51: 
.......<a class="attachment" href="/uploads/default/original/2X/6/6b09ba73aa957e4fbf53be6806a068b946b1cdb3.log">katello.log</a> (2.3 MB)
 
[cut]
.......
foreman-rake katello:correct_repositories COMMIT=true finished successfully!
Upgrade Step: correct_puppet_environments (this may take a while) ...
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
/usr/share/foreman/lib/core_extensions.rb:182: warning: already initialized constant ActiveSupport::MessageEncryptor::DEFAULT_CIPHER
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/message_encryptor.rb:22: warning: previous definition of DEFAULT_CIPHER was here
Processing Puppet Environment 1/1: 1-RHEL_Views-v18_0-puppet-c8f546ea-4d31-48e4-8371-76010ff1d687 (34)
foreman-rake katello:correct_puppet_environments COMMIT=true finished successfully!
Upgrade Step: clean_backend_objects (this may take a while) ...
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
/usr/share/foreman/lib/core_extensions.rb:182: warning: already initialized constant ActiveSupport::MessageEncryptor::DEFAULT_CIPHER
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/message_encryptor.rb:22: warning: previous definition of DEFAULT_CIPHER was here
0 orphaned consumer id(s) found in candlepin.
Candlepin orphaned consumers: []
0 orphaned consumer id(s) found in pulp.
Pulp orphaned consumers: []
foreman-rake katello:clean_backend_objects COMMIT=true finished successfully!
Upgrade completed!

John_Mitsch · November 27, 2018, 4:15pm

Just to confirm you are on the katello-3.8.1 rpm?

If so, I think the upgrade is skipping the migration as it thinks it already has ran and then that is affecting the upgrade task.

Try foreman-rake db:migrate

Check the Repository column names again to see if they have updated properly

then run the foreman-installer --upgrade again

John_Mitsch · November 27, 2018, 4:16pm

actually a better way would be to run foreman-installer --upgrade --force-upgrade-steps, so the steps are forcibly re-run

John_Mitsch · November 28, 2018, 1:11pm

Try running things manually
foreman-rake db:migrate
then source_repo_checksum_type should be gone from the column names

and then
foreman-rake katello:upgrades:3.8:clear_checksum_type
after this task runs, the mongo query should return 0

then for good measure foreman-installer --upgrade

kraba · November 28, 2018, 1:34pm

it return the same values:

foreman-rake db:migrate
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
/usr/share/foreman/lib/core_extensions.rb:182: warning: already initialized constant ActiveSupport::MessageEncryptor::DEFAULT_CIPHER
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/message_encryptor.rb:22: warning: previous definition of DEFAULT_CIPHER was here

foreman-rake console
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
/usr/share/foreman/lib/core_extensions.rb:182: warning: already initialized constant ActiveSupport::MessageEncryptor::DEFAULT_CIPHER
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/message_encryptor.rb:22: warning: previous definition of DEFAULT_CIPHER was here
Loading production environment (Rails 5.1.6)
Failed to load console gems, starting anyway
irb(main):001:0> Katello::Repository.column_names.sort
=> ["arch", "checksum_type", "container_repository_name", "content_id", "content_type", "content_view_version_id", "cp_label", "created_at", "deb_architectures", "deb_components", "deb_releases", "description", "distribution_arch", "distribution_bootable", "distribution_family", "distribution_uuid", "distribution_variant", "distribution_version", "docker_upstream_name", "download_policy", "environment_id", "gpg_key_id", "id", "ignorable_content", "ignore_global_proxy", "label", "library_instance_id", "major", "minor", "mirror_on_sync", "name", "ostree_upstream_sync_depth", "ostree_upstream_sync_policy", "product_id", "pulp_id", "relative_path", "source_repo_checksum_type", "ssl_ca_cert_id", "ssl_client_cert_id", "ssl_client_key_id", "unprotected", "updated_at", "upstream_password", "upstream_username", "url", "verify_ssl_on_sync"]
irb(main):002:0> quit

foreman-rake katello:upgrades:3.8:clear_checksum_type
/usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
/usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
/usr/share/foreman/lib/core_extensions.rb:182: warning: already initialized constant ActiveSupport::MessageEncryptor::DEFAULT_CIPHER
/opt/theforeman/tfm-ror51/root/usr/share/gems/gems/activesupport-5.1.6/lib/active_support/message_encryptor.rb:22: warning: previous definition of DEFAULT_CIPHER was here

mongo pulp_database --eval 'db.units_rpm.find({checksumtype: { $ne: "sha256" }}).count()'
MongoDB shell version v3.4.9
connecting to: mongodb://127.0.0.1:27017/pulp_database
MongoDB server version: 3.4.9
65330

I ran also the upgrade but nothing changed (same results from query after that).
If it’s useful, my mongo db packages are:

python-mongoengine.noarch              0.10.5-1.el7          @pulp
python-pymongo.x86_64                  3.2-2.el7             @pulp
python-pymongo-gridfs.x86_64           3.2-2.el7             @pulp
rh-mongodb34.x86_64                    3.0-14.el7            @centos-sclo-rh
rh-mongodb34-boost-chrono.x86_64       1.60.0-2.el7          @centos-sclo-rh
rh-mongodb34-boost-filesystem.x86_64   1.60.0-2.el7          @centos-sclo-rh
rh-mongodb34-boost-iostreams.x86_64    1.60.0-2.el7          @centos-sclo-rh
rh-mongodb34-boost-program-options.x86_64
rh-mongodb34-boost-regex.x86_64        1.60.0-2.el7          @centos-sclo-rh
rh-mongodb34-boost-system.x86_64       1.60.0-2.el7          @centos-sclo-rh
rh-mongodb34-boost-thread.x86_64       1.60.0-2.el7          @centos-sclo-rh
rh-mongodb34-gperftools-libs.x86_64    2.5-4.el7             @centos-sclo-rh
rh-mongodb34-libstemmer.x86_64         0-5.585svn.el7        @centos-sclo-rh
rh-mongodb34-libunwind.x86_64          1.2-3.el7             @centos-sclo-rh
rh-mongodb34-mongo-tools.x86_64        3.4.7-1.el7           @centos-sclo-rh
rh-mongodb34-mongo-tools-syspaths.x86_64
rh-mongodb34-mongodb.x86_64            3.4.9-1.el7           @centos-sclo-rh
rh-mongodb34-mongodb-server.x86_64     3.4.9-1.el7           @centos-sclo-rh
rh-mongodb34-mongodb-server-syspaths.x86_64
rh-mongodb34-mongodb-syspaths.x86_64   3.4.9-1.el7           @centos-sclo-rh
rh-mongodb34-runtime.x86_64            3.0-14.el7            @centos-sclo-rh
rh-mongodb34-syspaths.x86_64           3.0-14.el7            @centos-sclo-rh
rh-mongodb34-yaml-cpp.x86_64           0.5.3-1.el7           @centos-sclo-rh

John_Mitsch · November 28, 2018, 1:38pm

Just to check, you are definitely on Katello 3.8.1? There were no errors during the db:migrate command?

kraba · November 28, 2018, 1:43pm

Yes, I’m on Katello 3.8.1 and no errors about db:migrate, my production.log during db:migrate wrote only:

2018-11-28T13:42:33 [W|app|] Creating scope :path_of. Overwriting existing method Taxonomy.path_of.
2018-11-28T13:42:35 [W|app|] Creating scope :completer_scope. Overwriting existing method Location.completer_scope.
2018-11-28T13:42:35 [W|app|] Creating scope :completer_scope. Overwriting existing method Organization.completer_scope.
2018-11-28T13:42:38 [W|app|] Creating scope :path_of. Overwriting existing method Hostgroup.path_of.
2018-11-28T13:42:48 [W|app|] ignoring associations organization_ids, location_ids audit definition for Container, the resource is not audited
2018-11-28T13:42:48 [W|app|] Creating scope :path_of. Overwriting existing method FactName.path_of.
2018-11-28T13:42:49 [I|dyn|] start terminating throttle_limiter...
2018-11-28T13:42:49 [I|dyn|] start terminating executor...
2018-11-28T13:42:49 [I|dyn|] shutting down Core ...
2018-11-28T13:42:49 [E|dyn|] ... core terminated.
2018-11-28T13:42:49 [I|dyn|] start terminating executor dispatcher...
2018-11-28T13:42:49 [I|dyn|] start terminating client dispatcher...
2018-11-28T13:42:49 [I|dyn|] stop listening for new events...
2018-11-28T13:42:49 [I|dyn|] start terminating clock...

John_Mitsch · November 28, 2018, 1:45pm

Can you share the exact katello rpm version? I just want to compare to my instance to rule any difference out

kraba · November 28, 2018, 1:48pm

Sure:

foreman-installer-katello.noarch       3.8.1-1.el7  
katello.noarch                         3.8.1-1.el7         
katello-certs-tools.noarch             2.4.0-1.el7       
katello-client-bootstrap.noarch        1.4.2-1.el7           
katello-common.noarch                  3.8.1-1.el7           
katello-debug.noarch                   3.8.1-1.el7                       
katello-host-tools.noarch              3.3.5-3.el7           
katello-host-tools-fact-plugin.noarch  3.3.5-3.el7           
katello-installer-base.noarch          3.8.1-1.el7           
katello-repos.noarch                   3.8.1-1.el7          
katello-selinux.noarch                 3.0.3-1.el7           
katello-service.noarch                 3.8.1-1.el7

John_Mitsch · November 28, 2018, 7:50pm

What version of tfm-rubygem-katello do you have?

John_Mitsch · November 28, 2018, 8:48pm

Also, can you check if the original issue is still present? It actually looks like your database is fully migrated now, I compared to an updated katello 3.8.1, the instance I was on had an older tfm-rubygem-katello package, which is what ultimately contains the migrations. Comparing now, the column names look correct.

Looking more closely at the task to change the checksum_types, it only updates on-demand repositories, so the count doesn’t actually have to be zero for that mongo query.

Please double check the original issue is present, its possible that some of the above things we tried actually worked. If not, we can proceed with more debugging

kraba · November 29, 2018, 10:20am

I tried to publish an incremental update but the error is still present (it fail at 69%)

The installed tfm is:

tfm-rubygem-katello.noarch 3.8.1-1.el7

and the others from katello repo are:

tfm-rubygem-anemone.noarch             0.7.2-15.el7          
tfm-rubygem-hammer_cli_csv.noarch      2.3.1-1.el7          
tfm-rubygem-hammer_cli_katello.noarch  0.14.1-1.el7         
tfm-rubygem-qpid_messaging.x86_64      1.36.0-2.el7          
tfm-rubygem-robotex.noarch             1.0.0-20.el7