Recently, the signatures for foreman-installer tarballs were accidentally deleted. This is a result of me doing manual steps in effort to fix the corrupted tarball uploaded by Jenkins and get 1.17.1 out of the door. The signatures for 1.17 and 1.16 are available and the tarballs themselves are present, however it seems that the signatures for older foreman-installer releases cannot be recovered at this point. I regret and apologize for this costly mistake.
3 Likes
Thanks for the transparency. Accidents happen, and those versions are now out of support, so don’t beat yourself up about it.
Whilst I wouldn’t want to try and backup the entire of our repos, if there are smaller, important files that need to be backed up, let us know. We already backup Jenkins, Redmine, Discourse, and the Puppetmaster itself, so adding another host to that isn’t difficult.
When we found about this, we looked in Rackspace, there seems to be an option to make Snapshots of the disk.
Would it be possible to schedule that (I don’t have myself access to it)? It would have fixed this problem quite easily.
I wouldn’t worry too much @Ondrej_Prazak - it’s very unlikely we would have to sign new packages for anything before 1.16. IIRC I sent the 1.15 key before going on vacation to @Justin_Sherrill , might be 1.14 though.
I would also like to propose saving the release keys somewhere in theforeman.org. Currently I do have the 1.16 key, Ondrej has 1.17 and 1.18. This is quite risky and any of us could either lose it, or a disk failure might take it away if you’re not careful enough (I have mine on a external HD as well as on my laptop).
What would you think of storing the public and private keys under ~root/release_keys in theforeman.org server as a global backup for trusted people? If the box has been compromised as to have root, they can just replace all packages at that point.
I think it’s a bad practice to have release keys on a publicly accessible machine. They should be spread, but our primary webserver is the wrong place.
That is why you have signed packages: to indicate you control them. If they replace tarballs/RPMs which are unsigned or signed with a different key then users can notice it. Storing the private keys next to the result would be equivalent to unsigned packages in terms of security while giving the impression of security.
Agreed. A backup on the Puppetmaster, which is far more restricted, might make sense?
Do we run rsync service on downloads and/or yum theforeman.org? How about adding those if we don’t and then promoting that we are seeking for mirrors. Running rsync from a cronjob is a simple task for everyone, I’d start mirroring to my workstation right away.
Webcrawling our site is not a nice way of doing this and it’s slow. I’d love to do this everyday.
Edit: We do have rsync set up, need to figure out client configuration.
Here is how you can easily mirror all content that matters: EDIT - moved into separate thread: