Converting our manual to AsciiDoc (preview)

Hello,

I wrote a simple tool that interprets liquid templates from jekyll, fixes some things and generates Foreman manual as a markdown single page. That can be then turned into asciidoc via pandoc and converted to HTML/PDF as usual:

scripts/md2adoc

I would like to hear feedback on converting Foreman manual to asciidoc and putting it to our site The Foreman Documentation | Foreman and Katello documentation making it the official manual for 3.1.

Here is the preview, I did not upload images but they work fine (try it yourself):

https://lzap.fedorapeople.org/temp/foreman_manual/foreman-manual-nightly.html

As you can see, it looks fine except upgrade/install sections which use bunch of javascript. I propose to move this out into a separate page which we can host on our project site so it can stay interactive and turning the relevant chapters to normal text instead that is book-friendly (HTML is not the only format asciidoc supports).

The script is here:

Things TODO:

  • tables which are in HTML are not converted well
  • IDs and refs are broken - FIXED
  • variables - FIXED
  • code blocks via md and liquid - FIXED
  • images - FIXED
  • liquid incuding of files - FIXED
3 Likes

Lukáš,

IMHO, it looks very nice! :smiley: Much better than a lot of online documentation.

The first thing I noticed, is the lack of global navigation, but I’m sure that is on the TODO list.

The issue I want to raise is internal linking - Clicking the very first link in the document, which is an internal link fails. The href is
href="manuals/3.1/index.html#4.3SmartProxies"
Which causes a page load and a 404. I’m not sure if a multi-page web format will be generated, but if not, maybe a simple href="#ID".

The second issue, is the ID value itself should be to
<h3 id="_smart_proxy">Smart-Proxy</h3>
I’m guessing. So, internal ID generation needs to addressed, but given the existing href looks identical to the href values in the pre-3.1 documentation (from memory - when fixing an earlier link issue), so I’m guessing this is also probably on your TODO list already.

Looking good,
Peter

1 Like

Thanks for looing into this.

Yes, I can confirm that site navigation is still something we want to look into. The plan is definitely to offer something, what we have today is a work in progress.

I’ve added incorrect IDs to the TODO, I noticed it as well. Also (HTML) tables do not look correct. I am going to take a look right now.

So I have a new version, I spent the whole weekend on this. It is almost perfect, I would like to have a review of the document, please comment on issues. There are couple of minor things, some are actually incorrect in the manual itself, we can fix them along the way.

https://lzap.fedorapeople.org/temp/foreman_manual/foreman-manual-nightly.html

The last remaining problem I am aware of are tables - we use HTML tables because markdown tables are pain to work with, unfortunately this cannot be converted. For now, I detect tables and turn them into code blocks and I am looking into converting them to markdown.

In asciidoc, there are multiple ways of how to write tables, it is much more user-friendly format. So I expect we will use that:

[%header,cols=2*]
|===
|Name of Column 1
|Name of Column 2

|Cell in column 1, row 1
|Cell in column 2, row 1

|Cell in column 1, row 2
|Cell in column 2, row 2
|===

So I made more changes and even converted the tables from HTML to markdown. We can reformat them to more friendly form if needed. The biggest tables are generated from kafo, that is an easy change to do.

  • Tables are rendered as HTML code blocks
  • This code is incorrect: <pre>foreman-rake audits:expire</pre> (and three more)
  • Sixth header level is too deep for asciidoc (######) and appears rendered as plain
  • Links in 5.2 Compute Resources table are broken (needs to be fixed manually)
  • This (C) is rendered as Copyright unicode character (needs to be fixed manually)
  • Test browser reader mode
  • Generate a TOC

If you find a problem, please add a TODO item into this post, this one is a WIKI page.

Lukáš,

I don’t have time at the moment to read the document, but I thought of something last night — Reader mode. In both the latest (Windows) and an older (Fedora) Firefox and an older Apple macOS (10.15) Safari — reader mode seems to start halfway through the document.

For Firefox, it starts at authorize_login_delegation followed by a rendered hr tag and --foreman-apache. While in Safari, it starts at section 3.5.1 Initial Setup.

So something about the structure of the source or generated HTML is confusing these browsers. I tested Reader mode because I didn’t want to ask for a dark mode version.

I’ll try to find time to read the whole thing later today or tomorrow.

Regards,
Peter

1 Like

Due to the inline HTML and Javascript this has some issues and makes various sections rather hard to read. Without an index it’s also hard to navigate. However, I do like the idea of automatic conversion of markdown to asciidoc since it can help with converting some of our existing manual.

I opened a PR to replace more of our HTML usage with real markdown:
https://github.com/theforeman/theforeman.org/pull/1918

We should also consider dropping some sections. In the past we’ve talked about dropping the installation from source sections. IMHO that greatly simplifies things. First of all, it’s not really a scenario we support. In the past months/years I don’t recall seeing a question about it. If we do, it can be considered a developer setup.

Something that we should also consider is making use of appendices. All the installer options would be a good appendix. The same is true for all settings.

1 Like

Can you elaborate what you mean?

To give you more context, I looked into Antora documentation site builder, but that would require substantial changes to all our guides as Antora requires specific file structure and also it has a xref mechanism that would require to change all xrefs. This is something we can look into later, it is not good timing for doing bigger changes due to keeping compatibility with Red Hat docs build system which is currently not based on Antora.

For this reason, I want to stick with calling plan asciidoctor and it inly supports inline (embedded) CSS. On the other hand, this has one big advantage - HTML files are self-contained. You can copy them, they will render in the same way regardless of where they are opened.

Creating an index is just a single line of code, I just never enabled it. Making a TODO item.

Nice, I merged and I will regenerate my preview.

Absolutely, and asciidoc will make this much easier. In markdown, moving/dropping/adding chapters was a huge pain because of the filenaming conventions. This is a no-brainer now with asciidoc.

Yes, not only that, asciidoc supports file inclusion. Meaning we can literally include the output from the stdout from the kafo from a file. Many possibilities now open up!

I am working hard on a new navigation for the site, the same navigation bar will be present both on the welcome page and on each generated guide: