What is the problem being solved?
Search engines (like Google) frequently route users to outdated documentation for unsupported versions of Foreman. Older pages often outrank newer ones due to accumulated link history.
How does this affect the user or developer?
It creates a frustrating user experience. Users get confused trying to follow outdated instructions, resulting in broken configurations and avoidable support questions in the community.
Background:
Currently, search engines are allowed to index every historical version of our docs site without restriction.
Technical objectives:
Stop search engines from indexing End of Life (EOL) documentation pages so that current, supported versions rank highest.
Non-goals:
We are not deleting old documentation. It will remain fully accessible via direct links and the site’s version dropdown menu.
Proposal
Details of the solution:
Add the <meta name="robots" content="noindex"> tag to the HTML <head> of all documentation pages belonging to unsupported versions. We can update the documentation site generator so that when a version reaches EOL, a simple configuration toggle injects this tag into that version’s build.
Technical and project level impacts:
Requires a one-time minor update to the documentation build templates/scripts.
Alternative Designs
Relying solely on “Outdated” warning banners:
While helpful for users who are already on the page, this doesn’t solve the SEO issue. Search engines will still send users to the old pages first.
Deleting old docs:
Not preferred. Users stuck on older versions still need a reference, even if it is no longer supported.
Decision Outcome
Pending community discussion and consensus.
Impacts
User facing consequences: External web searches will reliably point users to up-to-date, supported documentation, greatly reducing confusion.
Processes: A minor step will be added to the release/EOL workflow checklist: maintainers will need to toggle the noindex flag for a version once its support window ends.
Architecture: Minimal impact; just a small tweak to the site generator logic.
More question-like google how do I install foreman gets me AI summary with a link to N-1 version, that is slightly better.
I was wondering how that could be implemented so I generated this, I thought I would push this as a draft. Not nice, but works across all builds if that is what we want.
@maximilian had a very good point that we should consider noindex for everything except nightlies and I think that is the best approach. Google is always lagging behind a bit, it is always better to direct users to nightlies which is the latest and greatest rather than “at some point was stable” directories which might be completely misleading depending on how often search engines index our page.
And the PR was merged, we will have noindex on all versions but nightly for better Google/AI results. Great idea and job @lstejska and everyone involved!
I like that we only allow robots to index nightly, because that’s what we work on. It fits the community project best. It’s not like creating a version/branching the docs makes them suddenly stable/functional in any way.
Misleading: The ribbon says “pre-release version”, not unsupported. Do you wish it had a different label?
I actually only like indexing nightly versions exclusively. Let me explain.
This is an instruction for search engines, not people. It does not prevent readers from visiting our pages and navigating to the correct version. What this solves is a terrible mess in the search engine results when the same content is repeated many times. By noindexing only unsupported versions, we improve the results but only slightly. With nightly only, search engines should only report one version of each result. This is what we want.
With indexing only nightly versions, we can get readers easier to find and click on a page. Then, they are clearly informed about the fact that this is a nightly version and there is a selector. Do not underestimate readers, also, reading nightly version is not the worst thing in the world. More often than not, it is the best version of the content out there.
I have a proposal - let’s let that sink in, wait until Google revisits the site and rebuilds its catalogue. I just tested it and it still renders nonsense results. After some time (2 months from now?) let’s return back and review results if they make more sense.
Once we have sorted out search results, we will have the people landing on the correct chapters, then we can do something with it. Maybe a welcome modal dialog is possible that is skipped for Google bots, but shown to the users:
Welcome to Foreman documentation, which version of Foreman do you want to read about?
