Instructions to setup and integrate Foreman and Chef - help me fill the blanks

#1

Hey guys, since Foreman has so many was to be used and so many scenarios and plugins I get lost in all this documentation. I’m in the process of setup Foreman for my scenario below. When done, I plan to document in detail setup and process for my use case and post it to tutorials section here in the community.
Please help me out to complete my setup.

Problem/scenario:
I have servers in several locations which are manged through Chef.
Each location is a separate Chef environment. What I’m am missing is central patch management system and for that purpose I wish to use foreman in scenario Katello.

Architecture design:
As per documentation and official recommendations each location/Chef_environment will have one smart proxy that will report to Central Foreman server, and servers/nodes in specific location/Chef_environment will report to that foreman proxy server. Image on https://www.theforeman.org/plugins/foreman_chef/0.1/
Deploying central Foreman server:
In order to install central server with this sccenario to use Katello follow the instructions here Foreman :: Plugin Manuals but once you come to the foreman-installer line use the following:
foreman-installer --scenario katello --enable-foreman-plugin-chef --enable-foreman-plugin-tasks --enable-foreman-proxy-plugin-chef --no-enable-puppet --foreman-proxy-puppet false --foreman-proxy-puppetca false --foreman-proxy-content-puppet false

Once the installer is complete it will output instructions how to setup smart proxy:

foreman-proxy-certs-generate --foreman-proxy-fqdn "proxy1.domain.com" --certs-tar "/etc/ssl/proxy1.domain.com-certs.tar"

Setting up smart proxy:
On smart proxy server to install add the repo’s as before, copy certs.tar from previous command onto proxy server and run:
foreman-installer \ --scenario foreman-proxy-content \ --certs-tar-file "/root/proxy1.domain.com-certs.tar"\ --foreman-proxy-content-parent-fqdn "proxy1.domain.com"\ --foreman-proxy-register-in-foreman "true"\ --foreman-proxy-foreman-base-url "https://proxy1.domain.com"\ --foreman-proxy-trusted-hosts "foreman.domain.com"\ --foreman-proxy-trusted-hosts "proxy1.domain.com"\ --foreman-proxy-oauth-consumer-key "someKeyHere"\ --foreman-proxy-oauth-consumer-secret "someKeyHereToo"\ --puppet-server-foreman-url "https://foreman.domain.com"
This is what I got from instructions but I assume this needs to be modified in order to enable chef plugins and connect smart proxy to chef server. Please let me know hot to correct this.

Configuring chef-client:
Install gem foreman_chef
Add to client.rb
chef_server_url "https://chef.domain.com/organizations/myorg" validation_client_name "validator" validation_key '/etc/chef/validator.pem' client_key '/etc/chef/client.pem' log_location STDOUT node_name "node1.domain.com" trusted_certs_dir "/etc/chef/trusted_certs"

Also,do I need to install katello-agent on nodes?

Foreman and Proxy versions:

Foreman 1.23 and Katello 3.13

Distribution and version:

CentOS 7.6

#2

It’s recommended to have 1 proxy per Chef server. So there are more options. You may have one proxy that only deals with Chef, while for content, you have multiple of them split per location. Or you can have proxy with all feature (content + chef). Also make sure to look at most recent documentation - https://www.theforeman.org/plugins/foreman_chef/0.8/index.html

You’re correct, just adding --enable-foreman-proxy-plugin-chef with additional parameters listed at https://www.theforeman.org/plugins/foreman_chef/0.8/index.html#2.Installation is necessary.

You can also use chef cookbook for this - see https://github.com/theforeman/chef-handler-foreman#using-chef-client-cookbook

Not necessarily, AFAIK it’s mostly used for deploying patches for which you can use remote execution plugin as a replacement. Chef plugin also works nicely with REX plugin if you need to trigger chef run on demand. As long as you hosts are reachable over SSH from proxy, it’s easier I think.

#3

You’re correct, just adding --enable-foreman-proxy-plugin-chef with additional parameters listed at https://www.theforeman.org/plugins/foreman_chef/0.8/index.html#2.Installation is necessary.

Thank you for your reply and insights. I’m having issues with this above.If I simply add that flag in scenario foreman-proxy-content I get
ERROR: Unrecognised option '--enable-foreman-proxy-plugin-chef'
When I did the installation of the main Foreman server I used all these flags in scenario katello and it worked.

--enable-foreman-plugin-chef --enable-foreman-plugin-tasks --enable-foreman-proxy-plugin-chef --no-enable-puppet --foreman-proxy-puppet false --foreman-proxy-puppetca false --foreman-proxy-content-puppet false

Using same these in scenario foreman-proxy-content it fails as unrecognized option before it even starts.
I will change my plans accordingly then to only have one smart proxy communicating with Chef. I’ve created dedicated system user in Chef, and private key for auth, gave it appropriate rights needed, but not sure how to configure it since from the documentation all I found is to use these parameters during installation.
–foreman-proxy-plugin-chef-server-url=“https://chef-server.domain.com/organizations/my-org/environments/TEST”
–foreman-proxy-plugin-chef-client-name=“foreman-chef”
–foreman-proxy-plugin-chef-private-key="/root/admin.pem"
–foreman-proxy-plugin-chef-ssl-verify=“false”
But same as the flag you mentioned these fail as well as unrecognized in scenario foreman-proxy-content. I’ve hit a wall.

Tnx for the katello-agent advice. All my servers are running CentOS 7 on x86_64 architecture which is see in docs is most supported.

#4

I’m in bit of a conflict in docs. I’m setting up smart proxy which will communicate with Chef as per docs which you pointed out @Marek_Hulan https://www.theforeman.org/plugins/foreman_chef/0.8/index.html My main foreman server is installed in scenario katello. Which scenario should I use for this dedicated smart proxy for chef communication? In scenario foreman-proxy-content option --enable-foreman-proxy-plugin-chef is unrecognized. And when I generate cert on main server it tells me to use proxy scenario, but I cannot from that scenario use chef plugin. do I use katello or default foreman scenario? I’m lost :frowning:

#5

Sorry for that. We don’t have a scenario for this use case. You should be good only allowing foreman-proxy module (disable foreman) but enable chef options. Don’t use katello scenario in this case, you’ll probably only want to have Chef feature enabled on this proxy.

Also I’m not sure if that’s obvious or not, but the installer shouldn’t be run on the same machine you have your Foreman. You’re installing it on separate machine right?

#6

If you think this should be supported, adding it to the scenario is possible. Here’s how the foreman scenario exposes the Foreman Proxy Chef Plugin:

You’d do the same thing here:

Currently the parameter cache would be an issue so if you do this on a live system, you need to install puppet-agent-puppet-strings so it can rebuild that. Obviously a PR to the installer to add the support is the right way to support this long term.

#7

@Marek_Hulan Yes, the idea is to have one smart proxy dedicated for Chef communication.
Not to have it of main Foreman/Katello server. As is recommended in docs.

@ekohl Yes I was comparing the answers files and noticed this. Since docs state the other thing I needed someone from you guys to confirm this, that I’m not looking or doing something wrong. The docs on this need to be updated until Chef proxy plugin is supported in scenario foreman-proxy-content. Currently it is misleading that this is supported.

I was thinking not to connect Chef since it is not supported in proxy scenario but to write Chef cookbook to install katello-agent and connect it directly via Smart proxy to Foreman. And handle via Chef environment variables which physical location(=Chef env) uses which smart proxy, and separate smart proxies per location in Foreman. Do you think that would be a good idea?