Satellite cannot provisioning CentOS 8 Server

hiero-nymus · April 3, 2020, 3:37pm

Problem:
Cannot provisioning CentOS 8 host on Satellite 6

Steps to Reproduce:

Provisioning configuration use a default kickstart, default templates and CentOS 8 kickstart as media.
Following repositories were defined in CentOS 8 Product: AppStream, AppStream kickstart, BaseOS, Extras, PowerTools, Centoplus and kickstart.
The deployment of the server failed at installation. It will be install from Katello (synced) repository.

Actual results:

Error in anaconda menu during installation
[…]

[!] Installation source: (Error setting up software source)
[!] Software selection: (Error checking software selection)

Error in /tmp/anaconda file
ERR payload: PaylodError: Failed to download metadata for repo ‘AppStream’

Expected outcome:
N/A

Foreman and Proxy versions:
Foreman + Proxy 1.22

Distribution and version:
Satellite 6.6.2-1

mcorr · April 3, 2020, 4:14pm

Hi,

This looks similar to Can't Kickstart CentOS 8

Would you mind taking a look and see if you can find a solution there?

Mkbrown · April 7, 2020, 8:42pm

I’m currently PXE booting CentOS 8 off of local Katello repos on a closed network. You do need to clone and modify the Kickstart default template as detailed here:
https://theforeman.org/plugins/katello/3.14/user_guide/provisioning/index.html

CentOS 8 Anaconda requires a repo called “AppStream”, which is what is failing for you right now. I created the following repos using hammer scripting (I’ve removed the CentOS 7 repos for clarity). So, AppStream shows up twice, just so I can remove the workaround later. Note, you have to sync the kickstart and os repos separately.

[root@foreman ~]# hammer repository list --search "CentOS"
----|-----------------------------|---------|--------------|--------------------------------------------------------------
ID  | NAME                        | PRODUCT | CONTENT TYPE | URL                                                        
----|-----------------------------|---------|--------------|--------------------------------------------------------------
114 | AppStream                   | CentOS  | yum          | http://mirror.centos.org/centos/8/AppStream/x86_64/kickstart/    
84  | CentOS8-AppStream_ks-x86_64 | CentOS  | yum          | http://mirror.centos.org/centos/8/AppStream/x86_64/kickstart/
82  | CentOS8-AppStream-x86_64    | CentOS  | yum          | http://mirror.centos.org/centos/8/AppStream/x86_64/os/     
60  | CentOS8-BaseOS_ks-x86_64    | CentOS  | yum          | http://mirror.centos.org/centos/8/BaseOS/x86_64/kickstart/ 
81  | CentOS8-BaseOS-x86_64       | CentOS  | yum          | http://mirror.centos.org/centos/8/BaseOS/x86_64/os/        
196 | CentOS8-PowerTools-x86_64   | CentOS  | yum          | http://mirror.centos.org/centos/8/PowerTools/x86_64/os/    
44  | ISOs                        | CentOS  | file         |                                                            
----|-----------------------------|---------|--------------|--------------------------------------------------------------

Create a content view presenting all of those to the client systems, and you should be good to go. It’s best if you have activation keys created as well, with all the necessary repositories and Content view associated.

If you plan on using Internet repos, you will need to create a custom Installation Media source with the URL of http://mirror.centos.org/centos/$major/BaseOS/$arch/kickstart

I would also suggest perusing the documentation sites for additional information, as they are getting more detailed in current releases.

http://docs.theforeman.org/guides/build/doc-Provisioning_Guide/index-foreman.html
https://theforeman.org/plugins/katello/3.14/user_guide/provisioning/index.html

Note: the subscription_manager snippet referenced in the last link is called redhat_register in my version (Foreman 1.24.2, Katello 3.14.1).

Hope that helps!

/Mike

lzap · April 8, 2020, 4:00pm

I believe that this should work out of box, @katello team can you give this a priority please?

iballou · April 8, 2020, 6:47pm

We are still advising people to follow the CentOS 8 instructions that @Mkbrown noted in the Provisioning Guide. The specific instructions are due to a bug in Anaconda for CentOS 8 that looks like is still around: https://bugs.centos.org/view.php?id=16470

Otherwise it should be working, please let us know if not.

lzap · April 9, 2020, 8:20am

I am afraid that the comment “it’s probably Foreman issue” indicates this might be a long one. Is there a workaround we could implement in Katello or somewhere else for the time being?

lzap · April 9, 2020, 8:27am

So an ungly workaround exists already:

https://projects.theforeman.org/issues/27948#note-4

But I don’t expect CentOS project to redo their new directory structure. I need to dig more into this to understand what’s wrong.

lzap · April 9, 2020, 8:52am

I believe the cleanest way to fix this is to implement a custom code in Pulp for CentOS8 - it needs to detect CentOS8 and set up repositories in a way that it is compatible with Anaconda.

iballou · April 9, 2020, 1:28pm

@lzap previously I was able to provision a CentOS 8 VM using the custom kickstart provisioning template that I noted in the Katello provisioning docs there. There is also code in Katello to detect the AppStream repo specifically, I mentioned it here: Bug #27948: CentOS 8 install fails missing "../../../AppStream" repo when Installation Media is foreman repo. - Katello - Foreman

Is the issue that this workaround with the custom kickstart template is too hacky? I haven’t heard yet that these instructions are out of date, unless there was a regression that we’ve missed.

I do agree that this should work out of the box though.

lzap · April 9, 2020, 2:22pm

Hmmm I haven’t realized we have that. While this is not ideal, that would work I guess. The OP @hiero-nymus is having issues with Satellite 6 tho, it probably missed 6.6. So can I assume that 6.7 will be fixed?

iballou · April 9, 2020, 2:42pm

The AppStream code is older than Katello 3.12 so it might be in there. @hiero-nymus can you try specifying a custom kickstart provisioning template like the instructions say here? Foreman :: Plugin Manuals

Mkbrown · April 9, 2020, 3:17pm

iballou,

As an FYI, the Foreman 1.24.2/Katello 3.14.1 releases do not include the necessary workaround in the ‘Kickstart default’ template, nor in the Installation Media configurations. Those might need to be added to the Community templates, or some special notes created around CentOS 8 provisioning, including all the necessary changes (OS configuration template assignments, Installation Media URL, Repos and Content Views).

For your consideration,

/Mike

iballou · April 9, 2020, 4:21pm

I’ll take a look at where we can add more support for this workaround. It would be nice if this can get fixed in Anaconda or Pulp so extra steps aren’t required by Katello users.

iballou · April 9, 2020, 8:08pm

@hiero-nymus a couple more questions that might help:

Are your AppStream and BaseOS repositories published in a content view that you’ve set up? If so, they should be same environment and content view. If you’ve only synced them but haven’t published them in a content view you should be all set there.

Is your AppStream repo literally named “AppStream”?

hiero-nymus · April 17, 2020, 3:37pm

Hi @iballou and sorry for the delay.
Yes AppStream and BaseOS repositories are on the same content view and were published too.

Thanks @Mkbrown, I’ve created a new AppStream repository as you described and follow the workaround’s installation from your link. Actually I’ve started the installation and until now, all is fine.

I just have a comment about the filesystems type. The installation seems to work only with ext4 because I’ve received this error with xfs : something like raise FSError : failed to mount.… As someone an Idea about that?

oarie · April 18, 2020, 6:45pm

Not sure if this will help anyone else but I am too impatient to wait for a real fix so here’s what I did:

I was getting the “…/…/…/AppStream/” error just like everyone else.
I read the googles.
Came up with bunk.
Impatience commenced.

Created /etc/httpd/conf.d/pulp_aliases.conf on my foreman server with the following contents:

Alias /pulp/repos/home/Library/BaseOS/x86_64/os /var/www/pub/yum/http/repos/home/Library/CentOS_8/custom/CentOS_8/CentOS_8_-_Base-x86_64
Alias /pulp/repos/home/Library/AppStream/x86_64/os /var/www/pub/yum/http/repos/home/Library/CentOS_8/custom/CentOS_8/CentOS_8-AppStream-x86_64
Alias /pulp/repos/home/Library/centosplus/x86_64/os /var/www/pub/yum/http/repos/home/Library/CentOS_8/custom/CentOS_8/CentOS_8-_centosplus-x86_64
Alias /pulp/repos/home/Library/Extras/x86_64/os /var/www/pub/yum/http/repos/home/Library/CentOS_8/custom/CentOS_8/CentOS_8-_Extras-x86_64
Alias /pulp/repos/home/Library/PowerTools/x86_64/os /var/www/pub/yum/http/repos/home/Library/CentOS_8/custom/CentOS_8/CentOS_8-_PowerTools-_x86_64

restorecon, chmod and chown the new file:

chown root: /etc/httpd/conf.d/pulp_aliases.conf
chmod 640 /etc/httpd/conf.d/pulp_aliases.conf
restorecon -vF /etc/httpd/conf.d/pulp_aliases.conf

restarted foreman stuffs:

foreman-maintain service restart

Partied on.

Note: you may have to tweak the aliases to match the names of your repositories. Once this is all fixed by whomever, you can delete the pulp_aliases.conf file but it shouldn’t hurt anything even after it is fixed.

lzap · April 23, 2020, 2:18pm

Thank you, I am marking your answer as a “temporary solution” so it pops up for googlers. This is an important thing to solve.

mcorr · April 23, 2020, 4:43pm

We’ve had several reports of difficulties provisioning with CentOS 8 in the last number of weeks. @lzap would it be an idea to look for a ‘volunteer’ to do a walk through of troubleshooting this?

lzap · April 24, 2020, 11:00am

Like creating a tutorial/blogpost/docs about this? Let me try all of this on my instance to see.

mcorr · April 24, 2020, 11:41am

Like creating a tutorial/blogpost/docs about this?

Exactly! Thank you. Let me know if you need a proofreader