RFC: Client workflows and tooling

Development RFCs
1

This RFC aims to target building out all the current ways hosts are ingested into Foreman and begin discussions on simplifications that can be made to existing tools and workflows. The goal is to bring both awareness and cohesion.

I may get a few or many things wrong as I have attempted to cull this information from as many sources as I could muster. Please feel free to edit or post changes and I will see that they get amended.

Motivating Goals

  • Simple client workflow for users
  • Simplify delivery mechanism for client tooling
  • Least amount of client tools as possible
  • Less duplication of workflows, more power into templates and REX
  • Enable more client operating systems with less work

Background

Host creation

Hosts can be created in Foreman through multiple methods:

  • Directly via API
  • Secondarily via a plugins API
  • Uploading facts
  • Directly via the UI

This allows a number of supported integrations to create hosts via their own workflows:

  • Puppetmaster uploading facts about managed puppet hosts
  • subscription-manager registering to a Katello endpoint
  • Ansible callback uploading facts about hosts
  • Import hosts from compute resource
  • Chef client fact upload
  • Salt client fact upload

Advanced Client features and tools

In addition to adding a host to Foreman’s inventory, there are some advanced features and tools delivered to the community to support some workflows. They are:

Proposals

Introduce foreman-bootstrap

Add a new client script named foreman-bootstrap to be as thin a layer as possible to connect a new or existing system to Foreman and prepare it to have actions taken against it from REX or configuration management system. The connection to Foreman would depend on the users desired method, e.g:

  • Registration API using JWT token
  • API using username/password
  • subscription-manager with username/password
  • subscription-manager with activation key
  • Config management fact upload

The bootstrap tool would focus on using templates available from the Foreman server to render functional scripts that can be ran on the client to enable functionality. This would centralize and re-use the hardened logic already present in the templates. Further, this would help keep the bootstrap tool thin do only the necessary auth and handshakes to connect a client back to Foreman.

This script could be delivered directly from Foreman opening it up to use across a broad array of client OSes. This would follow the model katello-client-bootstrap has today but extend out to include more operating systems with less packaging work.

Introduce foreman-host-tools for advanced client tooling

All other tooling needs for clients would be contained in a single foreman-host-tools package. The foreman-host-tools package would handle advanced functionality that is not needed to bootstrap a host to Foreman and cannot be maintained in a template. For example, yum plugins or scripts that need to exist on the client that are more easily maintained in the tools repository rather than from a generated template.

All other needed functionality for a client would be provided through the client repository. For example, tracer support.

This would be a single source code repository but allow for multiple sub-packages to allow users to install and using only advanced functionality they desire. For example, keeeping tracer as it’s own subpackage like it is today with katello-host-tools to allow users to choose if they want tracer support.

Fate of existing projects

  • katello-consumer RPM would be deprecated in favor of foreman-bootstrap configuring RHSM
  • katello-client-bootstrap would be turned into foreman-bootstrap and fold any additional functionality it has into templates or foreman-host-tools
  • katello-host-tools would be renamed to foreman-host-tools
  • foreman_scap_client would be built into foreman-host-tools
4 Likes
2

Yep, I believe this could streamline some workflows. REX could even be used to initially run the bootstrap process on hosts to add them to the inventory and setup a permanent connection. Leveraging the template system is key to a flexible solution.

Right now, we mostly render templates for hosts we kind of know. These templates would need to have no host scope (@host is not defined) but instead access to a list of all subnets defined in Foreman. How would we handle permissions here? What about taxonomies? Would this be a bash script? How about Windows (that would probably need powershell)? Would we allow to specify some metadata (e.g. the Foreman user selects the desired Org, Loc, HG and OS-Kind and gets a link with all that information encoded)?

1 Like
3

@host scope

We’d still have a step where something (API call, subscription-manager, you name it) creates a Host object, so the templates will have scope. Just maybe not enough information for a real provisioning workflow (missing network/subnet definition is something that comes in mind). So I’d expect (or hope) the finish template to render OK, and that’s what has all the “make this host a manageable entity” magic.

script

Yes, this would be a bash-y script, so no Windows, for now, sorry. Not that we can’t add something similar later on, when we’re confident the workflow as such works.

metadata

katello-client-bootstrap has today two modes (well, it has more, but two that are relevant here):

  1. create a Host using the API and populate this with information the user provided on the CLI (this includes Org, Loc, HG, etc)
  2. create a Host using subscription-manager, this does include Org and Loc as that’s something we can set via the RHSM API, but everything else will be left empty.

I think I’d retain those two ways of creating hosts, so there is at least one that you can pass metadata in.

Maybe even add a way to add “custom” JSON to the API call for whatever needs you have?

4

REX to run the bootstrap

This is kind of a bootstrap problem (lol). We can only run REX jobs against systems Foreman knows about, so we’d have to pre-create them in Foreman in this case. Personally I’ve been using Ansible (or other available task executors) to run such jobs in the past, thus avoiding the chicken-n-egg problem. And if the system is known to Foreman, and can receive REX jobs, there is really little left to bootstrap. Maybe Katello stuff, we can have a REX action for that. But otherwise?

5

Two things I dislike at the current situation and hope that can be improved:

Hosts that get created via facts upload are unmanaged. To get them in a managed state I recommend and use quite often default hostgroup plugin which feels always a bit old with a yaml hash in a file based configuration. So if changes are done to bootstraping I would hope to get all hosts as managed and allow to assign sane defaults or specified values. Perhaps we could get rid of unmanaged host at all, but there are at least hypervisors as content hosts which make sense to be unmanaged.

The other was already mentioned:

We should concentrate on Linux as it is the normal “client”, but try to support at least the big three osfamilies Red Hat, Debian and SUSE in an equal fashion. I totally understand the developer’s focus and knowledge is Red Hat centric and for Red Hat as company providing support for Red Hat is key, resulting also in a good support for all derivatives, but for community and other companies (NETWAYS and ATIX) it is just one of many.

6

Yes.

7

Currently yes, but I can totally see some host enrollment workflow using REX, where you wouldn’t have to create the hosts in advance. Just put in hostnames and usernames/passwords and off you go.

Even though there are relatively straightforward ways of achieving this outside of Foreman, wouldn’t it be nice to not have to leave Foreman to do it?

1 Like
8

katello-client-bootstrap creates hosts as managed by default (when using the API method), but has a switch to create them “unmanaged” as it’s not always easy to provide all the data that Foreman requires to accept a host as “managed”. I always disliked that name anyways, as it’s more “provisionable”, as Foreman can manage such hosts just fine with Puppet/Ansible/etc.

9

If that’s easily doable, sure, I’ll take it! :slight_smile:

10

I think virt-who may also belong to this list or the list above (also creates hosts)

To avoid all of these hard questions, I was rather thinking about improving REX target input so that user could enter arbitrary IP/hostname. REX would first create a host with such information. Unmanaged hosts only requires the name. The rest would be populated from the REX job results running the register template.

The register job could have an input “OS type”, based on that it can include a snippet for Linux or for Windows. I’d only start with that when we see a demand though.

11

Could we somehow have some unification with discovered hosts? Perhaps this could also allow running REX jobs on discovered hosts.

1 Like
12

For discovered hosts, there were several throughts in the past. I think technically REX should work on them with just one-line change, but user needs to feed in the public key during the boot somehow. Or root password at least. I think @lzap was investigating this recently.

13

That’s the plan - I would like to see discovered node completely without HTTP endpoints, just with one initial bootstrap one and SSH for the rest.

14

Can we leverage the authorized command of sshd technique somehow?
sshd can call a shell script to get a list of all authorized keys for a user. Discovery image would just need to know the URL of a Foreman API endpoint (it already knows what Foreman to talk to, so we’re good). The script could then ask foreman for the authorized key. So Foreman can control who is allowed to log in to the discovered node and who isn’t. To make it secure, a user would need to feed the root certificate of the foreman instance to discovery image. Or we could just transfer it via the first PXE template (as a kernel parameter or so).

1 Like
15

We already have the endpoint at a smart-proxy with ssh feature to get the authorized key from at https://foreman.localdomain:8443/ssh/pubkey. Should be possible to pull this one into the discovered host to allow REX to work.
As we have no owner at this stage it would be more difficult to get the keys of specific users, but these are also as endpoints on Foreman if we want individual logins.

2 Likes
16

Sure, I guess for the discovery image it would be enough to use this key and a local user account (for rey) that can sudo to root without a password.
Bonus points if the authorised keys from all Foreman admins would be automatically allowed to login to the discovered host as well.

17

That’s the plan, apologies if my words “completely without HTTP endpoints” confused you.

1 Like