[Katello] Can't Delete Package "NoMethodError: undefined method `repository_href' for nil:NilClass"

@fresh-pie

Can you post the output of the following commands for me:

# foreman-rake console
Katello::Repository.find(106).version_href
exit

Of course:

# foreman-rake console
API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
Loading production environment (Rails 6.0.3.1)
irb(main):001:0> Katello::Repository.find(106).version_href
=> "/pulp/api/v3/repositories/rpm/rpm/3f33e4ad-c860-4535-bdf4-459f9b860669/versions/17/"

@jamie

Can I have you post the output of the same command too, replace x with the repo ID you are trying to delete from:

# foreman-rake console
Katello::Repository.find(x).version_href
exit

Oh, it is actually repo ID “106”. Unless I am misunderstanding your ask.

@fresh-pie

Yep I got yours, I am working with another developer to get your reference back. I was seeing if the same output would return for @jamie or if we would get something different other than a uuid.

2 Likes

Hi @cintrix84,

I can execute against one of my broken repos but my issue is slightly different and I have opened another support issue ([Katello] 3.16.1.2 “NoMethodError: undefined method `repository_href’ for nil:NilClass” when syncing yum repos) so as not to hijack @fresh-pie’s thread. However, I’d be glad to assist with troubleshooting on this issue.

foreman-rake console

Loading production environment (Rails 6.0.3.1)
irb(main):001:0> Katello::Repository.find(28).version_href
=> “/pulp/api/v3/repositories/rpm/rpm/0676d8d3-6782-4bfe-9a07-865a4048e13e/versions/1/”

Hi! First lets check something, run foreman-rake console and run:

REPO_ID = 106
Katello::Pulp3::RepositoryReference.where(:root_repository_id => Katello::Repository.find(REPO_ID).root_id, :content_view_id => Katello::Repository.find(REPO_ID).organization.library )

make sure to set the REPO_ID to whatever your repository _id actually is. I’m expecting that will return nothing (or an empty array). If it returns something like:

[#<Katello::Pulp3::RepositoryReference:0x000000000c225c48
  id: 1,
  repository_href: "/pulp/api/v3/repositories/container/container/a403416e-b6e2-4670-aaca-e5d5f9afe842/",
  content_view_id: 1,
  root_repository_id: 1>]

Stop, and let us know. If it does return an empty array, you can run:

href = "/pulp/api/v3/repositories/rpm/rpm/3f33e4ad-c860-4535-bdf4-459f9b860669/"
Katello::Pulp3::RepositoryReference.create!(:root_repository_id => Katello::Repository.find(REPO_ID).root_id, :content_view_id => Katello::Repository.find(REPO_ID).organization.library.id, repository_href: href)

set ‘href’ equal to the first part of the string you got from the previous ‘version_href’ command chris asked you to run, so if you got:
/pulp/api/v3/repositories/rpm/rpm/0676d8d3-6782-4bfe-9a07-865a4048e13e/versions/1/”

remove the versions/1/ and the href would be:

href = "/pulp/api/v3/repositories/rpm/rpm/0676d8d3-6782-4bfe-9a07-865a4048e13e/"

I’d also be interested in getting a sosreport to look for any errors that may have caused this!

3 Likes

Hello! Sure, here is my foreman-rake console session:

# foreman-rake console
API controllers newer than Apipie cache! Run apipie:cache rake task to regenerate cache.
Loading production environment (Rails 6.0.3.1)

irb(main):001:0> REPO_ID = 106
=> 106

irb(main):002:0> Katello::Pulp3::RepositoryReference.where(:root_repository_id => Katello::Repository.find(REPO_ID).root_id, :content_view_id => Katello::Repository.find(REPO_ID).organization.library )
=> #<ActiveRecord::Relation []>

irb(main):003:0> href = "/pulp/api/v3/repositories/rpm/rpm/3f33e4ad-c860-4535-bdf4-459f9b860669/"
=> "/pulp/api/v3/repositories/rpm/rpm/3f33e4ad-c860-4535-bdf4-459f9b860669/"

irb(main):004:0> Katello::Pulp3::RepositoryReference.create!(:root_repository_id => Katello::Repository.find(REPO_ID).root_id, :content_view_id => Katello::Repository.find(REPO_ID).organization.library.id, repository_href: href)
=> #<Katello::Pulp3::RepositoryReference id: 129, repository_href: "/pulp/api/v3/repositories/rpm/rpm/3f33e4ad-c860-45...", content_view_id: 2, root_repository_id: 14>

Working on getting you that sosreport.

@fresh-pie are you able to remove rpms from that repo now?

Ah, so sorry for making you wait! My multi-tasking skills are not the sharpest :wink:

YES, it does work now! Thank you very much. I’m trying to understand exactly what I just did. Is there some kind of relation/reference between the actual repository and its “object” (lack of better word at the moment) that broke and I “re-connected”?

I’ve uploaded my sosreport here. Was a little too big to attach to this thread.

@fresh-pie

Thanks for letting us know, I am updating our repo rake task that can correct broken repos to accommodate this situation going forward.

1 Like

@jamie can you try the steps @Justin_Sherrill left and see if it helps getting the href back with your issue?

@fresh-pie yes there’s some association that was missing, and i’m not sure how it happened. Would it be possible to tar up /var/log/foreman/* and send it too? The sosreport didn’t provide enough information. Thanks!

Sure, no problem! I’ve emailed them to you.

Hello! I executed the procedure, after the first command, it came back blank. I set href variable and executed the second and unfortunately it did not resolve the error.

@jamie

Can you email me a sosreport so I can look further into what is happening?

chris.roberts@redhat.com

1 Like

Hi @cintrix84,

After that didn’t fix the issue, I had to pull the products from the cv/ccv and destroy/rebuild them in order to get past the issue. I can still send an sos if you think it will help.

Jamie

I looked through the logs and sadly there wasn’t any more information. it looks like all the logs before october had been rotated and deleted. If you notice it happen again, please let us know!

Well thank you so much for all your help guys. Ya’ll are awesome.

Will be sure to let you know if it happens again.

I seem to have a reproducer machine I can easily share if you’re still interested.