Foreman 3.11/Katello4.13 - Unable to delete product/repository

Hi @sajha ,
This is consider the ID from my URL: https://forman.local/products/26/repositories/1620

The ID in this case will be 1620

Please review if the following command would be apropiate to execute:

repository = Katello::Repository.find(1620)
ForemanTasks.sync_task(::Actions::Pulp3::Orchestration::Repository::Delete,
repository,
SmartProxy.pulp_primary)
repository.destroy!

1 Like

Hi @sajha, I forgot to ask…do i need to do a restart after running the command or it should simple work via GUi refresh?

Those commands look right…You won’t need to restart your server. You should see the Pulp3::Orchestration::Repository::Delete task in your task tab and make sure there are no errors in the task.

Hi @sajha , how long should this normally take to execute and complete?

Hopefully not too long…This is the same pulp task that the UI starts so you should keep and eye on the task in the UI to see it’s not stuck…

If it gets into the stuck state again, we’l run a few more commands to delete the repo from pulp directly using pulp-cli.

Looks like its stuck for last 6 minutes at 17%.

Dont see any fails under the runnings steps yet:

The CLI appears to have timeout with the following output, the GUI still running with stuck at 17%

irb(main):009:0>   SmartProxy.pulp_primary)

Traceback (most recent call last):
        8: from lib/tasks/console.rake:5:in `block in <top (required)>'
        7: from (irb):7
        6: from foreman-tasks (9.1.1) lib/foreman_tasks.rb:55:in `sync_task'
        5: from foreman-tasks (9.1.1) lib/foreman_tasks.rb:24:in `trigger_task'
        4: from foreman-tasks (9.1.1) lib/foreman_tasks.rb:45:in `rails_safe_trigger_task'
        3: from foreman-tasks (9.1.1) lib/foreman_tasks.rb:46:in `block in rails_safe_trigger_task'
        2: from foreman-tasks (9.1.1) lib/foreman_tasks.rb:26:in `block in trigger_task'
        1: from foreman-tasks (9.1.1) lib/foreman_tasks.rb:36:in `block (2 levels) in trigger_task'
Timeout::Error (The time waiting for task a593cf4a-132e-4dcc-b26c-1e3f7fc2cf6a to finish exceeded the 'foreman_tasks_sync_task_timeout' (600s))
irb(main):010:0> repository.destroy!
=> nil

From the running tasks, I can see the poll_attemps is increasing, not sure if that means anything, but the status bar remains the same 17% stuck:

 "poll_attempts"=>{"total"=>63, "failed"=>0}}

Something off with the remote. This was the step pulp got stuck at in the earlier run.

Can you try pulp rpm remote destroy --href="/pulp/api/v3/remotes/rpm/rpm/018f9c12-2abd-7c92-937f-2b336a7331c3/"

Replace the href with the pulp remote href from the task output’s reserved resources in the image above.

Screenshot from 2024-06-27 12-23-05

I guess I have to cancel the running task before doing this?

I tried running it, but got an error back regarding safety:

 pulp rpm remote destroy --href="/pulp/api/v3/remotes/rpm/rpm/01905067-3b9a-7567-a5a6-5cef25d1221f/"
Error: Call aborted due to safe mode

It’ll be good to cancel the task before we proceed cause I don’t think that task is going to get unstuck.

Also, you might need to use the --force flag to do any writes with pulp-cli.

pulp --force rpm remote destroy --href=""

Hi @sajha ,
it looks like is doing something, will let you know the output shortly:

pulp --force rpm remote destroy --href="/pulp/api/v3/remotes/rpm/rpm/01905067-3b9a-7567-a5a6-5cef25d1221f/"
Started background task /pulp/api/v3/tasks/01905a88-3ff9-7cb7-ba8b-2fa17eeb8fa9/
...............................................................
1 Like

Hi @sajha ,
The task has been running for sometime now, do we know how long is this suppose to take:

pulp --force rpm remote destroy --href=“/pulp/api/v3/remotes/rpm/rpm/01905067-3b9a-7567-a5a6-5cef25d1221f/”
Started background task /pulp/api/v3/tasks/01905a88-3ff9-7cb7-ba8b-2fa17eeb8fa9/
…

You can use pulp show --href="/pulp/api/v3/tasks/...../" with the task href fom the output above.

Weird that remote delete is hangng in pup. AFAIU it’s a small record.

We can leave it for now and proceed to the repository delete. Let me write up some commands to do that.

This is what pulp show is showing me:

pulp show --href="/pulp/api/v3/remotes/rpm/rpm/01905067-3b9a-7567-a5a6-5cef25d1221f/"
{
  "pulp_href": "/pulp/api/v3/remotes/rpm/rpm/01905067-3b9a-7567-a5a6-5cef25d1221f/",
  "pulp_created": "2024-06-25T17:17:41.147532Z",
  "pulp_last_updated": "2024-06-25T17:45:02.865410Z",
  "name": "epel-rpm-internal-repo-16208765",
  "url": "http://foreman.local/internal-product/epel-rpm-internal-repo,
  "ca_cert": null,
  "client_cert": null,
  "tls_validation": false,
  "proxy_url": null,
  "pulp_labels": {},
  "download_concurrency": null,
  "max_retries": null,
  "policy": "immediate",
  "total_timeout": 3600.0,
  "connect_timeout": 60.0,
  "sock_connect_timeout": 60.0,
  "sock_read_timeout": 3600.0,
  "headers": null,
  "rate_limit": 0,
  "hidden_fields": [
    {
      "name": "client_key",
      "is_set": false
    },
    {
      "name": "proxy_username",
      "is_set": false
    },
    {
      "name": "proxy_password",
      "is_set": false
    },
    {
      "name": "username",
      "is_set": false
    },
    {
      "name": "password",
      "is_set": false
    }
  ],
  "sles_auth_token": null
}

Let’s try this in foreman-rake console :

ForemanTasks.sync_task(::Actions::Pulp3::Repository::Delete,
1620,
SmartProxy.pulp_primary)

Hi @sajha ,
Let me know if I can CTRL+C the running task via CLI since it apepars it has gone into a zombie/stuck loop. See below reference: