Remove orphan tasks failing on Katello 4.9

Support
,
1

Problem: Tasks regarding orphan removal are failing. The oldest instance I see of this is from back when I was using Foreman 3.5 and Katello 4.7. Currently this is the error message I get:

("Cannot delete some instances of model 'Content' because they are referenced through protected foreign keys: 'Package.content_ptr'.", {<RepositoryContent: pk=ff32c510-ad1c-4288-918a-442fba668125>})

As mentioned in other posts, I do already have python39-pulp-deb-2.20.2-1.el8.noarch and rubygem-pulp_deb_client-2.20.2-1.el8.noarch installed

Expected outcome: successful removal of orphans

Foreman and Proxy versions: 3.7.0

Foreman and Proxy plugin versions:

Name Description Author Version
foreman-tasks The goal of this plugin is to unify the way of showing task statuses across the Foreman instance. It defines Task model for keeping the information about the tasks and Lock for assigning the tasks to resources. The locking allows dealing with preventing multiple colliding tasks to be run on the same resource. It also optionally provides Dynflow infrastructure for using it for managing the tasks. Ivan Nečas 8.1.1
foreman_ansible Ansible integration with Foreman Daniel Lobato Garcia 12.0.4
foreman_remote_execution A plugin bringing remote execution to the Foreman, completing the config management functionality with remote management functionality. Foreman Remote Execution team 10.0.1
katello Katello adds Content and Subscription Management to Foreman. For this it relies on Candlepin and Pulp. N/A 4.9.0

Distribution and version: Rocky Linux 8.8

Other relevant data:

Installed Packages

    ansible-collection-theforeman-foreman-3.12.0-1.el8.noarch
    candlepin-4.3.1-1.el8.noarch
    candlepin-selinux-4.3.1-1.el8.noarch
    foreman-3.7.0-1.el8.noarch
    foreman-cli-3.7.0-1.el8.noarch
    foreman-debug-3.7.0-1.el8.noarch
    foreman-dynflow-sidekiq-3.7.0-1.el8.noarch
    foreman-installer-3.7.0-1.el8.noarch
    foreman-installer-katello-3.7.0-1.el8.noarch
    foreman-obsolete-packages-1.5-1.el8.noarch
    foreman-postgresql-3.7.0-1.el8.noarch
    foreman-proxy-3.7.0-1.el8.noarch
    foreman-release-3.7.0-1.el8.noarch
    foreman-selinux-3.7.0-1.el8.noarch
    foreman-service-3.7.0-1.el8.noarch
    foreman-vmware-3.7.0-1.el8.noarch
    katello-4.9.0-1.el8.noarch
    katello-certs-tools-2.9.0-2.el8.noarch
    katello-client-bootstrap-1.7.9-2.el8.noarch
    katello-common-4.9.0-1.el8.noarch
    katello-debug-4.9.0-1.el8.noarch
    katello-repos-4.9.0-1.el8.noarch
    katello-selinux-5.0.2-1.el8.noarch
    pulpcore-selinux-1.3.2-1.el8.x86_64
    python39-pulp-ansible-0.16.0-1.el8.noarch
    python39-pulp-certguard-1.5.6-1.el8.noarch
    python39-pulp-cli-0.14.0-4.el8.noarch
    python39-pulp-cli-deb-0.0.2-4.el8.noarch
    python39-pulp-container-2.14.6-1.el8.noarch
    python39-pulp-deb-2.20.2-1.el8.noarch
    python39-pulp-file-1.12.0-1.el8.noarch
    python39-pulp-python-3.8.0-1.el8.noarch
    python39-pulp-rpm-3.19.7-1.el8.noarch
    python39-pulpcore-3.22.7-1.el8.noarch
    qpid-proton-c-0.37.0-1.el8.x86_64
    rubygem-foreman-tasks-8.1.1-1.fm3_7.el8.noarch
    rubygem-foreman_ansible-12.0.4-1.fm3_7.el8.noarch
    rubygem-foreman_maintain-1.3.2-1.el8.noarch
    rubygem-foreman_remote_execution-10.0.1-1.fm3_7.el8.noarch
    rubygem-hammer_cli-3.7.0-1.el8.noarch
    rubygem-hammer_cli_foreman-3.7.0-1.el8.noarch
    rubygem-hammer_cli_foreman_remote_execution-0.2.3-1.fm3_7.el8.noarch
    rubygem-hammer_cli_foreman_tasks-0.0.19-1.fm3_7.el8.noarch
    rubygem-hammer_cli_katello-1.9.0-1.el8.noarch
    rubygem-katello-4.9.0-1.el8.noarch
    rubygem-pulp_ansible_client-0.16.1-1.el8.noarch
    rubygem-pulp_certguard_client-1.6.5-1.el8.noarch
    rubygem-pulp_container_client-2.14.5-1.el8.noarch
    rubygem-pulp_deb_client-2.20.2-1.el8.noarch
    rubygem-pulp_file_client-1.12.0-1.el8.noarch
    rubygem-pulp_ostree_client-2.0.0-1.el8.noarch
    rubygem-pulp_python_client-3.8.0-1.el8.noarch
    rubygem-pulp_rpm_client-3.19.6-1.el8.noarch
    rubygem-pulpcore_client-3.22.4-1.el8.noarch
    rubygem-qpid_proton-0.37.0-1.el8.x86_64
    rubygem-smart_proxy_pulp-3.2.0-3.fm3_3.el8.noarch

journalctl -u pulpcore-worker@*

Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]: pulp [b3a7bad0-751c-4973-8432-9a7eb92d6452]: pulpcore.tasking.pulpcore_worker:INFO:   File "/usr/lib/python3.9/site-packages/pulpcore/tasking/pulpcore_worker.py", line 44>
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     result = func(*args, **kwargs)
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:   File "/usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 486, in synchronize
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     remote_url = fetch_remote_url(remote, url)
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:   File "/usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 305, in fetch_remote_url
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     remote_url = fetch_mirror(remote)
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:   File "/usr/lib/python3.9/site-packages/pulp_rpm/app/tasks/synchronizing.py", line 254, in fetch_mirror
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     result = downloader.fetch()
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:   File "/usr/lib/python3.9/site-packages/pulpcore/download/base.py", line 186, in fetch
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     return done.pop().result()
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:   File "/usr/lib/python3.9/site-packages/pulpcore/download/http.py", line 273, in run
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     return await download_wrapper()
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:   File "/usr/lib/python3.9/site-packages/backoff/_async.py", line 151, in retry
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     ret = await target(*args, **kwargs)
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:   File "/usr/lib/python3.9/site-packages/pulpcore/download/http.py", line 258, in download_wrapper
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     return await self._run(extra_data=extra_data)
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:   File "/usr/lib/python3.9/site-packages/pulp_rpm/app/downloaders.py", line 117, in _run
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     self.raise_for_status(response)
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:   File "/usr/lib/python3.9/site-packages/pulp_rpm/app/downloaders.py", line 102, in raise_for_status
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     response.raise_for_status()
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:   File "/usr/lib64/python3.9/site-packages/aiohttp/client_reqrep.py", line 1005, in raise_for_status
Jul 19 00:00:24 foreman.example.com pulpcore-worker-2[152361]:     raise ClientResponseError(

PULP_SETTINGS=/etc/pulp/settings.py pulpcore-manager showmigrations rpm

rpm
 [X] 0001_initial
 [X] 0002_updaterecord_reboot_suggested
 [X] 0003_DATA_incorrect_json
 [X] 0004_add_metadata_signing_service_fk
 [X] 0005_optimize_sync
 [X] 0006_opensuse_support
 [X] 0007_checksum_types
 [X] 0008_advisory_pkg_sumtype_as_int
 [X] 0009_revision_null
 [X] 0010_revision_null_redo
 [X] 0011_rpmremote_sles_auth_token
 [X] 0012_remove_pkg_group_env_cat_related_pkgs
 [X] 0013_RAW_rpm_evr_extension
 [X] 0014_rpmrepository_package_retention_policy
 [X] 0015_repo_metadata
 [X] 0016_dist_tree_nofk
 [X] 0017_merge_advisory_collections
 [X] 0018_updatecollection__update_record
 [X] 0019_migrate_updatecollection_data
 [X] 0020_remove_updatecollection_m2m
 [X] 0021_rename_updatecollection_update_record
 [X] 0022_add_collections_related_name
 [X] 0023_increase_distribution_release_short
 [X] 0024_change_subrepo_relation_properties
 [X] 0025_remove_orphaned_subrepos
 [X] 0026_add_gpgcheck_options
 [X] 0027_checksum_null
 [X] 0028_rpmrepository_last_sync_repomd_cheksum
 [X] 0029_rpmpublication_sqlite_metadata
 [X] 0030_DATA_fix_updaterecord
 [X] 0031_modulemd_static_context
 [X] 0032_ulnremote
 [X] 0033_new_distribution_model
 [X] 0034_auto_publish
 [X] 0035_fix_auto_publish
 [X] 0036_checksum_type
 [X] 0037_DATA_remove_rpmrepository_sub_repo
 [X] 0037_update_json_field
 [X] 0038_fix_sync_optimization
 [X] 0039_disttree_digest
 [X] 0040_rpmalternatecontentsource
 [X] 0041_modulemdobsolete
 [X] 0042_alter_repometadatafile_data_type
 [X] 0043_textfield_conversion
 [X] 0044_noartifact_modules
 [X] 0045_modulemd_fields
 [X] 0046_rbac_perms
 [X] 0047_modulemd_datefield
 [X] 0048_artifacts_dependencies_fix
 [X] 0049_profiles_fix

Task Output:

{"pulp_tasks"=>
  [{"pulp_href"=>"/pulp/api/v3/tasks/2996831a-d553-4195-982a-12938f7ee779/",
    "pulp_created"=>"2023-07-31T13:10:02.449+00:00",
    "state"=>"failed",
    "name"=>"pulpcore.app.tasks.orphan.orphan_cleanup",
    "logging_cid"=>"5b70908fc43f4d14942561f8319dcc49",
    "started_at"=>"2023-07-31T13:10:02.642+00:00",
    "finished_at"=>"2023-07-31T13:10:07.675+00:00",
    "error"=>
     {"traceback"=>
       "  File \"/usr/lib/python3.9/site-packages/pulpcore/tasking/pulpcore_worker.py\", line 444, in _perform_task\n" +
       "    result = func(*args, **kwargs)\n" +
       "  File \"/usr/lib/python3.9/site-packages/pulpcore/app/tasks/orphan.py\", line 66, in orphan_cleanup\n" +
       "    c.delete()\n" +
       "  File \"/usr/lib/python3.9/site-packages/django/db/models/query.py\", line 745, in delete\n" +
       "    collector.collect(del_query)\n" +
       "  File \"/usr/lib/python3.9/site-packages/django/db/models/deletion.py\", line 302, in collect\n" +
       "    raise ProtectedError(\n",
      "description"=>
       "(\"Cannot delete some instances of model 'Content' because they are referenced through protected foreign keys: 'Package.content_ptr'.\", {<RepositoryContent: pk=ff32c510-ad1c-4288-918a-442fba668125>})"},
    "worker"=>"/pulp/api/v3/workers/92aeba58-871a-40e8-aac1-042e4a0887c6/",
    "child_tasks"=>[],
    "progress_reports"=>
     [{"message"=>"Clean up orphan Content",
       "code"=>"clean-up.content",
       "state"=>"running",
       "total"=>19089,
       "done"=>2000}],
    "created_resources"=>[],
    "reserved_resources_record"=>["/pulp/api/v3/orphans/cleanup/"]}],
 "task_groups"=>[],
 "poll_attempts"=>{"total"=>21, "failed"=>1}}
2

Hi @lravelo ,

Would you mind using the info provided here to create a Pulpcore issue? Issues · pulp/pulpcore · GitHub

Looks like you hit a Pulp bug.

CC @dralley

3

Just submitted the issue
https://github.com/pulp/pulpcore/issues/4209

1 Like
4

Hi…i have the same problem on ( * foreman-3.6.1-1.el8.noarch, * katello-4.8.4-1.el8.noarch, RHEL 8.6).

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=4ad788c7-9e48-49f5-8160-7a480662dee5>})

python39-pulp-deb.noarch 2.20.2-1.el8
rubygem-pulp_deb_client.noarch 2.20.2-1.el8

Task Output:

{“pulp_tasks”=>
[{“pulp_href”=>“/pulp/api/v3/tasks/b6779640-721e-4343-b309-ec36a174ab29/”,
“pulp_created”=>“2023-08-17T11:09:45.443+00:00”,
“state”=>“failed”,
“name”=>“pulpcore.app.tasks.orphan.orphan_cleanup”,
“logging_cid”=>“ed652e2287cc465597d9ac5691c2f131”,
“started_at”=>“2023-08-17T11:09:45.570+00:00”,
“finished_at”=>“2023-08-17T11:10:00.011+00:00”,
“error”=>
{“traceback”=>
#<Sequel::SQL::Blob:0xe9fb448 bytes=106 start=" File "/u" end=“form_task\n”> +
#<Sequel::SQL::Blob:0xe9fb45c bytes=35 start=" result" end=“**kwargs)\n”> +
#<Sequel::SQL::Blob:0xe9fb470 bytes=99 start=" File "/u" end=“n_cleanup\n”> +
#<Sequel::SQL::Blob:0xe9fb484 bytes=15 content=" c.delete()\n"> +
#<Sequel::SQL::Blob:0xe9fb498 bytes=89 start=" File "/u" end=“in delete\n”> +
#<Sequel::SQL::Blob:0xe9fb4ac bytes=33 start=" collec" end=“el_query)\n”> +
#<Sequel::SQL::Blob:0xe9fb4c0 bytes=93 start=" File "/u" end=“n collect\n”> +
#<Sequel::SQL::Blob:0xe9fb4d4 bytes=26 start=" raise " end=“tedError(\n”>,
“description”=>
“("Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.", {<RepositoryContent: pk=4ad788c7-9e48-49f5-8160-7a480662dee5>})”},
“worker”=>“/pulp/api/v3/workers/e029c1d6-5621-4b57-919b-9702582cea04/”,
“child_tasks”=>,
“progress_reports”=>
[{“message”=>“Clean up orphan Content”,
“code”=>“clean-up.content”,
“state”=>“running”,
“total”=>79587,
“done”=>2000}],
“created_resources”=>,
“reserved_resources_record”=>[“/pulp/api/v3/orphans/cleanup/”]}],
“task_groups”=>,
“poll_attempts”=>{“total”=>7, “failed”=>1}}

5

This is really weird. If some Content is referenced by a RepositoryContent it should never be identified as an orphan in the first place. That is the definition of not orphaned. That reference just means “This Content is present in some repository”. It might be worth going into the DB and retrieving the RepositoryContent with pk=4ad788c7-9e48-49f5-8160-7a480662dee5, and then retrieving what content and what repository this actually refers to. Perhaps this could give some clues as to why Pulp would consider the content an orphan in the first place.

6

@senetm If you simply re-run the orphan cleanup, do you get the exact same error (down to the pk=4ad788c7-9e48-49f5-8160-7a480662dee5) again?

2 Likes
8

Hi, yes i get always the exact (same pk=) same errors (14 tasks 13 errors and 1 success):

image
image1289×440 77 KB

All error tasks:

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=cf90b446-dbf1-463f-9c46-c76fe8ae9f01>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=b9181c1c-d9b0-413f-ae54-26ac03ebbf60>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=ad80be2b-3d3a-47c0-9846-0f2b1bab058c>, <RepositoryContent: pk=73adc39a-aa18-4baf-aa65-23cb61a740aa>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=1a3292ce-81e9-4eb6-bf18-ac3093c17303>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=f1d64450-be95-41b5-a3cd-d199c5c6f7bb>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=e3bd7b95-6e24-4f42-b9b6-15620ecfabc2>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=25eed4ee-cbef-4bca-94bd-9e877d18c93e>, <RepositoryContent: pk=ff32c588-2701-422a-b7ec-34a2176b6a19>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=8c6da9f9-7748-4c1c-a325-518c2cb6942a>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=09527aae-405f-4430-9e7c-ca729f1cd269>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=0a3160bb-cbec-49e1-ad08-53c513b7cc82>, <RepositoryContent: pk=cf14e4f7-d218-42fc-82d9-175f606e29f2>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=014501b2-c500-4a88-a790-45a479d671a4>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=0cad53d0-5698-45c6-8256-d648815532b0>})

(“Cannot delete some instances of model ‘Content’ because they are referenced through protected foreign keys: ‘Package.content_ptr’.”, {<RepositoryContent: pk=4ad788c7-9e48-49f5-8160-7a480662dee5>})

It is always exact the same?

9

Are those multiple “Remove orphans” tasks running in parallel?

If so, can you try running just one “Remove orphans” task (ideally without any other type of task running in parallel)? Does that result in “success”?

10

There are no other multiple tasks running parallel. First they are paused in with error and i stopped them before i run foreman-rake katello:delete_orphaned_content RAILS_ENV=production >/dev/null .

11

@iballou can you help us understand this screenshot? I see multiple orphan cleanup tasks that were started at the same time. Does katello trigger somehow multiple tasks in parallel?

12

I’m guessing there are multiple smart proxies in the environment? Each “Remove orphans” task represents orphan cleanup on a single smart proxy.

Katello, by default, runs a single orphan cleanup run on the Foreman plus every remote smart proxy weekly. This is done via a cron job.

13

I noticed this a while back which caused me to implement Serialize orphan cleanup tasks to prevent them from getting in each other's way by dralley · Pull Request #3031 · pulp/pulpcore · GitHub

However, because of that very PR I wouldn’t expect to see problems due to concurrent orphan cleanups.

1 Like
14

I left a comment here with some of m findings but I need @quba42’s eyes too Remove orphan tasks failing on Katello 4.9 · Issue #4209 · pulp/pulpcore · GitHub

1 Like
15

Hi, yes there are 14 smart proxies in the environment and exact the same number of tasks are listed with 13 errors?

16

Should i manually change the pulpcore/app/tasks/orphan.py file?
The difference is:

import gc							import gc

							      >	from logging import getLogger
							      >
from django.conf import settings				from django.conf import settings
							      >	from django.db.models.deletion import ProtectedError
							      >	from django.utils import timezone

from pulpcore.app.models import (				from pulpcore.app.models import (
    Artifact,							    Artifact,
    Content,							    Content,
    ProgressReport,						    ProgressReport,
    PublishedMetadata,						    PublishedMetadata,
							      >	    PulpTemporaryFile,
							      >	    Upload,
)								)

							      >	log = getLogger(__name__)
							      >

def queryset_iterator(qs, batchsize=2000, gc_collect=True):	def queryset_iterator(qs, batchsize=2000, gc_collect=True):
    """								    """
    Provide a batching functionality that returns individual 	    Provide a batching functionality that returns individual 

    Copied from here with minor changes:			    Copied from here with minor changes:
    https://www.guguweb.com/2020/03/27/optimize-django-memory	    https://www.guguweb.com/2020/03/27/optimize-django-memory
    """								    """
    iterator = qs.values_list("pk", flat=True).order_by("pk")	    iterator = qs.values_list("pk", flat=True).order_by("pk")
    eof = False							    eof = False
    while not eof:						    while not eof:
        primary_key_buffer = []					        primary_key_buffer = []
        try:							        try:
            while len(primary_key_buffer) < batchsize:		            while len(primary_key_buffer) < batchsize:
                primary_key_buffer.append(next(iterator))	                primary_key_buffer.append(next(iterator))
        except StopIteration:					        except StopIteration:
            eof = True						            eof = True
        yield qs.filter(pk__in=primary_key_buffer).order_by("	        yield qs.filter(pk__in=primary_key_buffer).order_by("
        if gc_collect:						        if gc_collect:
            gc.collect()					            gc.collect()


def orphan_cleanup(content_pks=None, orphan_protection_time=s	def orphan_cleanup(content_pks=None, orphan_protection_time=s
    """								    """
    Delete all orphan Content and Artifact records.		    Delete all orphan Content and Artifact records.
    Go through orphan Content multiple times to remove conten	    Go through orphan Content multiple times to remove conten
    This task removes Artifact files from the filesystem as w	    This task removes Artifact files from the filesystem as w

    Kwargs:							    Kwargs:
        content_pks (list): A list of content pks. If specifi	        content_pks (list): A list of content pks. If specifi

    """								    """
    progress_bar = ProgressReport(			      |	    with ProgressReport(
        message="Clean up orphan Content",			        message="Clean up orphan Content",
        total=0,					      |	        total=None,
        code="clean-up.content",				        code="clean-up.content",
        done=0,						      |	    ) as progress_bar:
        state="running",				      |	        while True:
    )							      |	            content = Content.objects.orphaned(orphan_protect
							      |	                pulp_type=PublishedMetadata.get_pulp_type()
    while True:						      |	            )
        content = Content.objects.orphaned(orphan_protection_ |	            if not content.exists():
            pulp_type=PublishedMetadata.get_pulp_type()	      |	                break
        )						      |
        content_count = content.count()			      |	            # delete the content
        if not content_count:				      |	            for c in queryset_iterator(content):
            break					      |	                count = c.count()
							      |	                try:
        progress_bar.total += content_count		      |	                    c.delete()
        progress_bar.save()				      |	                except ProtectedError as e:
							      |	                    # some orphan content might have been pic
        # delete the content				      |	                    # i.e. sync
        for c in queryset_iterator(content):		      |	                    log.info(e)
            progress_bar.increase_by(c.count())		      |	                else:
            c.delete()					      |	                    progress_bar.increase_by(count)
							      <
    progress_bar.state = "completed"			      <
    progress_bar.save()					      <

    # delete the artifacts that don't belong to any content	    # delete the artifacts that don't belong to any content
    artifacts = Artifact.objects.orphaned(orphan_protection_t	    artifacts = Artifact.objects.orphaned(orphan_protection_t

    progress_bar = ProgressReport(			      |	    with ProgressReport(
        message="Clean up orphan Artifacts",			        message="Clean up orphan Artifacts",
        total=artifacts.count(),				        total=artifacts.count(),
        code="clean-up.content",			      |	        code="clean-up.artifacts",
        done=0,						      |	    ) as progress_bar:
        state="running",				      |	        for artifact in progress_bar.iter(artifacts.iterator(
    )							      |	            try:
    progress_bar.save()					      |	                # we need to manually call delete() because i
							      |	                artifact.delete()
    counter = 0						      |	            except ProtectedError as e:
    interval = 100					      |	                # Rarely artifact could be shared between to 
    for artifact in artifacts.iterator():		      |	                # Just log and skip the artifact deletion in 
        # we need to manually call delete() because it cleans |	                log.info(e)
        artifact.delete()				      |
        progress_bar.done += 1				      |
        counter += 1					      |	def upload_cleanup():
							      |	    assert settings.UPLOAD_PROTECTION_TIME > 0
        if counter >= interval:				      |	    expiration = timezone.now() - timezone.timedelta(minutes=
            progress_bar.save()				      |	    qs = Upload.objects.filter(pulp_created__lt=expiration)
            counter = 0					      |	    with ProgressReport(
							      |	        message="Clean up uploads",
    progress_bar.state = "completed"			      |	        total=qs.count(),
    progress_bar.save()					      |	        code="clean-up.uploads",
							      >	    ) as pr:
							      >	        for upload in pr.iter(qs):
							      >	            upload.delete()
							      >
							      >
							      >	def tmpfile_cleanup():
							      >	    assert settings.TMPFILE_PROTECTION_TIME > 0
							      >	    expiration = timezone.now() - timezone.timedelta(minutes=
							      >	    qs = PulpTemporaryFile.objects.filter(pulp_created__lt=ex
							      >	    with ProgressReport(
							      >	        message="Clean up shared temporary files",
							      >	        total=qs.count(),
							      >	        code="clean-up.tmpfiles",
							      >	    ) as pr:
							      >	        for tmpfile in pr.iter(qs):
							      >	            tmpfile.delete()
17

Please don’t until that PR is at least approved.

2 Likes