October 18, 2021, 10:47am
In short I can’t create and sync Debian 11 repository.
How can I:
Import/Create the GPG correctly?
Setup the repository correctly?
There’s a few articles in google about setting it up but none are complete and none work with the latest version of Foreman/Katello/Pulp.
Is there an article that walks through the process correctly?
Working debian and fully sync’d repository.
Foreman and Proxy versions:
Other relevant data:
Have a look at the orcharhino documentation to
add GPG keys for Debian & managing Debian systems guide. This guide is for Debian 10, but I am confident you can make it work for Debian 11, too.
Let me know if you run into any trouble.
October 18, 2021, 12:23pm
@maximilian - thanks for getting back to me and providing the links, very much appreciated
I’m looking at the guides and I’m following the steps for Debian 10 so I can make sure this works before I go onto Debian 11.
I’ve setup the repository like so:
Content > Products, select the
Debian 10 product, and create three repositories of type
deb as follows:
Debian 10 main
and I’m using this key as the GPG key:
But when I run it, I’m getting this error:
No valid Release file found for 'buster'.
Is the GPG file OK? Or do I have to run:
wget http://ftp.debian.org/debian/dists/buster/Release && wget http://ftp.debian.org/debian/dists/buster/Release.gpg gpg --verify Release.gpg Release gpg --keyserver keys.gnupg.net --recv-key 16E90B3FDF65EDE3AA7F323C04EE7237B7D453EC gpg --keyserver keys.gnupg.net --recv-key 0146DC6D4A0B2914BDED34DB648ACFD622F3D138 gpg --keyserver keys.gnupg.net --recv-key 6D33866EDD8FFA41C0143AEDDCC9EFBF77E11517 gpg --armor --export E0B11894F66AEC98 DC30D7C23CBBABEE DCC9EFBF77E11517 > debian_buster_main.txt
Because if I do that - I get:
[root@foreman btop]# gpg --keyserver keys.gnupg.net --recv-key 16E90B3FDF65EDE3AA7F323C04EE7237B7D453EC gpg: requesting key B7D453EC from hkp server keys.gnupg.net gpgkeys: HTTP fetch error 6: Could not resolve host: keys.gnupg.net; Unknown error gpg: no valid OpenPGP data found. gpg: Total number processed: 0
October 18, 2021, 12:31pm
I don’t know where to get the correct GPG key for this repo, but I can confirm that the error you are getting most likely means that verification of the downloaded Release file failed with the provided GPG key.
I’ve ran the commands from orcharhino docs and the ones you’ve posted and I cannot reproduce the error.
If you download
Release and Release.gpg, you should be able to see which GPG keys are used to sign the Release file when running
gpg --verify Release.gpg Release. Where you get the keys from is not important as long as you cross-check the fingerprint.
Finally, you want to export enarmored gpg keys into one file, which you can then upload to Foreman, for example with
gpg --armor --export KEY_1_FINGERPRINT KEY_2_FINGERPRINT > /path/to/a/file.txt
October 18, 2021, 1:15pm
I managed to get slightly further by using:
gpg --keyserver pgpkeys.eu --recv-key 16E90B3FDF65EDE3AA7F323C04EE7237B7D453EC gpg --keyserver pgpkeys.eu --recv-key 0146DC6D4A0B2914BDED34DB648ACFD622F3D138 gpg --keyserver pgpkeys.eu --recv-key 6D33866EDD8FFA41C0143AEDDCC9EFBF77E11517 gpg --armor --export E0B11894F66AEC98 DC30D7C23CBBABEE DCC9EFBF77E11517 > debian_10_main.txt
To generate the key as
keys.gnupg.net does not exist in DNS anymore.
This time, it runs for a few minutes - I see that the disk usage goes up a few GB so it must be pulling down content, but then it stops with a warning as the details say:
Katello::Errors::Pulp3Error: Pulp task error
full error info here:
October 18, 2021, 1:22pm
This new error looks like what was previously discussed here:
Add Debian 10 repository - #12 by smurdza
I will need to have another look at this soon…
As you are running Pulp3, you’re in the best hands possible with
@quba42 being a maintainer of pulp_deb for Pulp3. We will report back to you!
October 18, 2021, 1:27pm
I am afraid, the last time this issue was reported, I was told by pulpcore developers, that this has been addressed for pulpcore 3.14, but
@Binky is using Katello 4.2, so it should already be solved…
October 18, 2021, 1:33pm
@Binky Can you try increasing the timeout at “settings > administer > content”? (suggested by @Justin_Sherrill on IRC)
October 18, 2021, 1:41pm
@quba42 , is it this setting?
if so, I’ve just upped it to 1000 and re-run, will let you know if this helps, thanks!
October 18, 2021, 1:44pm
I think that is the one. 1000s is more than 16 minutes, so if it still fails, then there is probably something going on that is not just slow networking…
October 18, 2021, 1:49pm
thanks for confirming
It is running on my test proxmox setup with 8GB of RAM allocated, so maybe that is what’s also contributing to the timeouts! Will let you know if I’m successful
October 18, 2021, 2:18pm
@Justin_Sherrill Do you have any ideas how to keep debugging this? It is a pity the error does not tell us what it was trying to download when it ran into the timeout!
I don’t know to what extent it makes sense to keep increasing the timeout. Downloads should not take more than 16 minutes…
October 18, 2021, 2:26pm
I’ve increased it anyway to 10000 to see if it helps
I’ve a 80MB fibre connection. Not sure how big an initial Debian 10 repo would be, but I’ve got 500GB allocated which should be plenty enough! Thanks for your help so far - I’m definitely at least a ton further than I was a few hours ago!
October 18, 2021, 2:29pm
Bare in mind that it will take 2h 46min to run into that timeout. If you do run into it, then this is NOT a timeout issue, but a connection times out no matter what issue!
One possible reason would be something blocking connections such as a firewall or http proxy (which it doesn’t look like you’ve configured).
It might helpful to try to curl some file from the debian repo on the foreman/katello server itself? And check if you’ve got an http proxy configured via an env variable, as curl might use that (but pulp/katello would not).
October 18, 2021, 3:17pm
@Justin_Sherrill thanks for the response.
I can curl to the repo:
[root@foreman btop]# curl http://deb.debian.org/debian/ -vvv
* About to connect() to deb.debian.org port 80 (#0)
* Trying 18.104.22.168...
* Connected to deb.debian.org (22.214.171.124) port 80 (#0)
> GET /debian/ HTTP/1.1
> User-Agent: curl/7.29.0
> Host: deb.debian.org
> Accept: */*
< HTTP/1.1 200 OK
< Server: Apache
and the subscription manager to
https://apt.atix.de/Debian10/ sync works:
Seems like the sync is progressing:
so it’s taken about 1hr to get to 41% - and the disk space usage has definitely gone up by over 20GBs so I think it’s getting there, let you know if it is successful!
October 18, 2021, 4:01pm
still running and disk usage went from 20gb to 82gb.
I guess the lack of RAM does not help:
Look at the swap usage!
8 GB is well below the recommended minimum amount of ram. I would highly recommend you increase it closer to the 20 we recommend.