Foreman Manual Reboot

Community
1

@ehelms @lzap and @ekohl have done a lot of work lately to lay the foundations for docs.theforeman.org becoming the place for Foreman and Katello documentation.

As well as the infrastructure, I want to ensure that this does not become a garbage in/garbage out style migration of legacy content. I’d like if everything we publish at docs.theforeman.org is a marked improvement on what we already have. This would make the effort worthwhile.

The original motivation for new documentation/open sourcing of Red Hat docs was that we would have enterprise-quality docs for our open source project, and this is something that I still think is important to maintain.

As part of the effort to move our official documentation to docs.theforeman.org, I’ve been looking at the Foreman Manual.

In my opinion we shouldn’t migrate the Foreman Manual upstream, but stop, look at what is in the manual, and check if the content is already better described in the upstreamised Satellite guides. Where we identify holes, I suggest we enhance what exists in the open-sourced guides. Where we find content completely missing, we add to the new location.

This is my attempt at identifying the above in the current Foreman Manual and I am very much depending on your feedback to steer me. I’m hoping that the various component owners will be able to lend me a hand.

Scope

For now, I am excluding the QSG and the Installation parts of the manual. I think the QSG is pretty good. I think the Installation section requires a plan on its own.

1.2 Foreman Architecture

I would suggest that Foreman Architecture requires more explanation than is currently available here. The illustration in this section is quite famous, widely used, and beloved by all, so I would not be opposed to transferring it to docs.theforeman.org :slight_smile: I’d also be open to a redrawn version!

docs.theforeman.org has some attempt at an architecture/planning guide.

This is far from where we’d like to be, but it is a start, and perhaps has some useful information for the Foreman/Katello scenario

I also have plans to start a Standard Operating Environment guide this month.

Foreman Manual

I have made a table of corresponding areas in the Foreman Manual vs docs.theforeman.org. I am making this post a wiki where people can add comments to this table.

I will leave this open for a while, and if there are no objections to my suggestions, I will start removing sections of the Foreman Manual and linking to the docs.theforeman.org.

Where I’ve written “review & merge”, I mean that I will review the section, ensure nothing is missing from docs.theforeman.org, and make changes, then deprecate that section from the Foreman Manual and link to the new area from there. Eventually, we would have a deprecation notice on the Foreman Manual.

Foreman Manual Corresponding docs.theforeman.org entry Comments?
4.1.1 LDAP Authentication Using LDAP Done
4.1.2 Roles and Permissions Managing Users and Roles Done
4.1.3 Trends Monitoring Trends Removed functionality?
4.1.4 Auditing Maintaining Foreman Server Definite gap here in the procedures that should be added to docs.theforeman.org
4.1.5 Searching Searching and Bookmarking Review and merge?
4.1.6 User Management User Management This section seems poorly described in the manual
4.2 Managing Puppet no such guide Provided by ATIX
4.3 Smart Proxies Installing Smart Proxy on Red Hat I would think this needs a separate review. The Smart Proxy docs vs Smart Proxy/Katello docs as is seem to cause confusion.
Provisioning Provisioning Guide Better and more up to date on docs.theforeman.org
4.5 Command Line Interface There is a Satellite Hammer guide we can grab if we think it is useful We can use a script to generate most of this guide
4.6 Email Management 5.1.6. Email Notifications Email chapter added
4.7 Managing Ansible Configuring Foreman to Use Ansible Replace link
4.8 Managing Chef no equivalent to link to Chef docs will need to be migrated
4.9 Managing Salt No equivalent to link to Salt docs will need to be migrated
4.10 Monitoring Added here Done
4.11 Reports Using Report Templates to Monitor Hosts Reviewed and docs.theforeman.org has more content than Foreman manual.
5.1API no equivalent but there is a Satellite API guide we could take We could also add this to Planning guide
5.2 Compute Resources Comprehensive coverage per Compute resource in Provisioning Guide Review & Merge
Install location Missing Content? no idea what this should be
5.4 Securing Communications with SSL Configuring Smart Proxy with SSL No idea if this is enough. Would appreciate any guidance
5.5 Backup, Recovery and Migration Backing up Foreman server & Smart Proxy, Restoring Foreman server or Smart Proxy from a Backup, I don’t see any migration although Foreman Manual has. docs.theforeman.org has this. All these probably need Foreman-only versions.
5.6 Rails Console no equivalent need to migrate
# 5.7 External Authentication Configuring External Authentication Review and merge?
5.8 Multiple Foreman instances no equivalent Need to migrate
5.9 HTTP(S) Proxy Using an HTTP Smart Proxy Review & Merge

Troubleshooting

In general, I think that troubleshooting guides become a dumping ground. If a topic is well described, you can include the troubleshooting information that relates to that topic at the bottom, or in a specific appendix to a specific scenario.

I wouldn’t be opposed to some form of orderly indexing of troubleshooting information, or advanced information. However, my instinct is that “advanced” information could be added as an appendix to a well described end-to-end scenario in a guide.

Reference materials/lists

I remember having a discussion about including printouts of all the available options, IIRC for the foreman-installer with @ekohl and my main concern is whether or not this will be refreshed and maintained. It would be ideal if we were to create tables or outputs like that, that they would be automated with some script for each new Foreman branch. These quickly go out of date and are rarely maintained manually.

Let me know if you are williing to look at a component that you are familiar with and help me improve the content for docs.theforeman.org

Let me know your opinions about this approach.

Thanks,

Melanie

5 Likes
2

Seems to give better overview on what Permissions and Filters are, we should adopt this information into the introduction of the chapter or somewhere else.

This could be also added to the Planning Guide, I mean the part that describes what Audit is and where to get it. It’s an important aspect of a system that should be taken into consideration when planning.

Good find, I had no idea downstream had so little info about Puppet.

A lot of smart-proxy related installation info is scattered on docs.theforeman.org. As part of this migration we should probably modularize these sections and put all those instructions to the single place - Installing an External Smart Proxy Server.

Yes, this can be dropped we should have everything and more at docs.tf.org.

Do you think all configuration management systems should live in a single guide?

This is farily new (sub-statuses), so chances are this hasn’t been done for downstream yet. However the title is a bit misleading - this is probably not monitoring that users would be looking for (service availability monitoring or performance monitoring). We need to figure out better term I think - status is probably the one to stick with.

I think we should upstreamize both the auto-generated guides - API and CLI. Makes sense. I can volulteer to do that if needed.

Uh that’s old. Just drop this.

This part is different for Foreman core (where Puppet CA is used) and Katello. I think we need to move this content to an appropriate place, perhaps into Installing Foreman guide (without Katello). Katello users should not be interested in this I think.

Also foreman-maintain is I think not availble for Debian, instructions might be different for Ubuntu/Debian users. This needs an extra review.

I tried my best to find what’s missing but I could not find anything. Probably needs a review from maintainers but IMHO this is okay.

Yes, new chapter somewhere. Probably only upstream we don’t support this downstream I think.

I think this is worth pulling out of Provisioning Guide. HTTP proxy is very important to have exposed somewhere in the Installation guide or Deployment guide, specifically the Administer - Settings one which is used for proxy communication.

The other one for Compute Resources can stay as is and we can reference each other.

I think we can put these as release process items. We can automate most of the actions.

I like this, we should probably take the opposite order and start with easy ones. The migrated content can be easily linked and changes can be merged as we go.

Thanks

2 Likes
3

Not sure if it was sarcasm, but I opened an issue about it being incorrect about 6 years ago:

AFAIK nobody has the source anymore and needs to be redrawn.

This was already mentioned in the initial manual pages but was never created:

My guess would be that it’s a list of where files are installed but I can’t say for sure.

1 Like
4

It’s hard to find the words to describe what about this diagram has led people to include it in presentations for such a long period of time! It is inaccurate, outdated, and lacking modern aesthetics, yet it must have some good qualities - perhaps the parts of it that are able to communicate core concepts on a high level - for the very fact that it continued to circulate. I didn’t want to disrespect it, while acknowledging that maybe we need to look at allowing it to retire. I didn’t mean to be sarcastic, but perhaps I wasn’t adequately clear either :wink:

I was discussing this with @tbrisker earlier, and perhaps the best approach is to aim for an update of this that is sufficiently high level and then create more detailed charts to provide an overview to specific workflows.

5

It’s good quality is that it exists while the alternatives, which do not have problems, have the major problem that they don’t exist.

1 Like
6

While feeling like coming late to a party, I want to add my two cents (or more)! :wink:

I am one of those using the architecture picture quite often. Main reason for this is that it exists and I do not have to draw myself as I am lazy, but another one is that using the upstream image makes it much more recognizable when coming for one of my talks or trainings to official docs. But a big :+1: for redrawing and modernizing it.

For the term “monitoring” used by Foreman I am biased because of the other project in my work life, but I am arguing for replacing it since I am involved in the project. I would use “Reporting” or “Status reporting” as it is no active monitoring for nearly all status information in Foreman, but features reporting the status to Foreman in some way, and while monitoring has many facets most think about availability monitoring first.

I would say every Configuration management should have its own guide and configuration management integration in general should be covered in the planning guide. This would allow for a central section with capabilities for those to make a decision or just learn about options on the one hand. On the other hand it will give everyone who has decided on a solution a streamlined guide without any bloat.

On all the other topics I am lacking time at the moment to get involved to much, but if you want one “external” or “technical” view feel free to get in touch with me and I will do my best.

Furthermore I have put on our (so not only my personal one, but NETWAYS) ToDo list to put some effort into the monitoring integration again and to create a guide for it to go into the new docs. Not sure when someone is able to pick it up, but at least it is addressed! :wink:

3 Likes
7

When you say monitoring, what context do you mean exactly in this paragraph?

I am asking because I also have a TODO item: to rewrite this guide completely from scratch: Monitoring Red Hat Satellite Red Hat Satellite 6.5 | Red Hat Customer Portal

There are some technical problems which are holding me back, Prometheus Ruby client library does not work with multiple server instances, the only viable option is statsd but since Foreman does not work on EL8 yet (which is required by PCP) I need to figure out. Perhaps some kind of statsd TCP relay. Or maybe statsd_exporter is the best way to go, the only reason why I am trying to avoid it is that it’s not in RHEL.

Anyway, if this is on your TODO then make sure to talk to me once you start so we can combine our efforts - the guide should definitely describe multiple monitoring solutions.

8

I was referring to Foreman Monitoring and Smart Proxy Monitoring.

But combining efforts and knowledge for a guide on how to monitor Foreman and its components is a good idea. I can definitely provide a guide on monitoring availability with Icinga 2 which would also be a good starting point for Nagios and other forks. For metrics I can provide Graphite and Grafana knowledge and to some degree InfluxDB, for logs and other events Elastic stack and through colleagues Graylog. So it would be the “not in RHEL” way, but as these tools are widely used it could be useful for the community and hopefully some Red Hat customers, too.

1 Like
9

I’ve created a PR to add using Salt for configuration management to the managing hosts guide.

2 Likes
10

In the past 2 or 3 weeks I spent a lot of time on the docs. Going through the guides, fixing easy things, making lists of broken things. Overall I’d say there’s still a HEAVY Katello focus. This can be explained by the fact that it came from Satellite which can assume that Katello is present. That means I think the documentation for vanilla Foreman (so without Katello) is really a long way away from being usable. I’d call it pre-alpha. Here’s what I think would be the roadmap to get to beta quality.

(Re)build the nagivation

To make it clear to users that there are multiple “flavors” and which guides are ready, I think the navigation should be changed. To me this is a blocker to many other changes. If we hide Katello content in the foreman-el build and guides link to foreman-el, the user experience gets very confusing.

https://github.com/theforeman/foreman-documentation/pull/678

Proper release note guides

I think we need real guides for release notes. There was a longer discussion in this PR:

https://github.com/theforeman/foreman-documentation/pull/618

I then took this in and in 2 PRs I took a stab at implementing this:

https://github.com/theforeman/foreman-documentation/pull/698
https://github.com/theforeman/foreman-documentation/pull/699

Upgrade guide

This guide needs work for all flavors. Mostly foreman-deb, but also foreman-el and even some katello. I spotted at least one instruction that only works on Satellite. There is an issue, but it doesn’t describe the concrete problems.

Upgrading is a crucial thing that users go through a lot. Incorrect instructions mean a lot of support questions.

https://github.com/theforeman/foreman-documentation/issues/648

Merge {Installing,Upgrading} on {EL,Debian} guides

These guides are pretty much the same and we already build multiple versions via flavors. Why do we need separate guides?

(no issue/PR)

Consider it beta quality

Once we reach this point resolved, I feel like we could start advertising it as beta quality documentation.

Link from Foreman

The next steps would be to make it easy in Foreman itself to link to our new documentation. Foreman itself has a lot of links to our documentation. Some work started on this:

Build out a list of further work

Once we have this in place, we’re only at the beginning. Then there is still work to do to decide on the remaining guides already in foreman-documentation. There is also a ton of documentation in the existing site that we need to go over and decide what needs to be moved over and what can be dropped. This means both the regular manual, but also plugin pages.

1 Like
11

I think we made tremendous progress in the last couple of months and should finally refer users to the Foreman Documentation instead of Foreman Manual. Considering both Satellite and orcharhino deem the docs “good enough” for their very special and enterprise-y downstream docs, I think we should finally make the move!

PR in ‘theforeman.org’ to replace link in navigation

1 Like
12

Recently a lot of work has been done to use foreman-maintain in both RPM and deb contexts. @upadhyeammit is working on making sure it can be used to upgrade. Until then we don’t have any upgrade guides on the new docs, which is IMHO still a blocker.

1 Like
13

Agree, as I upgraded a Katello system and if I did not know about the modularity in the repos already, I would have been lost with the docs. So before switching I see a last round of QA needed.

14

Thanks Ewoud and Dirk.

Happy to hear that this blocker will be resolved with Foreman 3.4 if I understand this correctly: The foreman-maintain 1.1.3 is now available on Debian! & Tracker #34194: Foreman Maintain support on Debian - Foreman Maintain - Foreman

regarding “last round of QA”: I am unsure what you mean exactly, but as said before, downstream is already using the docs & we are heavily improving, adding, fixing, and extending stuff; there’s a lot of activity in foreman-documentation right now.

1 Like
15

The problem is that downstream is always behind, so they will not catch the latest changes like the modularity introduced before community stumbles over it. So I have the feeling we should find someone to review the documentation for completeness before merge the PR to switch from the manual. So do not get me wrong I like the new documentation and in most cases it really feels like an improvement and over all more complete, but I found already some corner cases like the one mentioned.

16

Yes, but most PRs in foreman-documentation are against master and then cherry-picked to 3.1 (and versions in between).

On the other hand, there isn’t much activity with the Foreman manual: Commits · theforeman/theforeman.org · GitHub

I am just wondering who can/will do a (complete) review before making the move.

17

There are some other navigation issues that we should address. In particular the listing of releases. Take Foreman documentation: nightly as an example. You may already see here that the page title doesn’t render, which isn’t good for search engines. On the page itself it’s also very hard to distinguish what’s what. It lists Configuring Hosts Using Puppet three times, once for each guide.

What I’d propose is to only list the bare minimum: quickstart, install & upgrade. Possibly planning, but otherwise refer to the navigation on the top. I’ve implemented this as a start:
https://github.com/theforeman/foreman-documentation/pull/1470