Foreman CLI - design proposal the Ruby way

Hi,

as Tomas announced a couple of days ago, here is my proposal for CLI design implemented in Ruby.

For better reading experience and code to play with check https://github.com/mbacovsky/foreman_cli_draft

Comments and questions are wellcome!

Regards,

Martin

Foreman CLI design draft

··· ========================

This is my attempt to create outline of CLI client for the Foreman. The Foreman project has Ruby as its main langauage, so I choosed to explore the options Ruby world is offering to complete this task.

My initial requirements are as follows:

  • [x] implemented in Ruby
  • [x] Git-like subcomands in CLI app
  • [x] system shell autocompletion of commands and options
  • [x] shell-like environment with autocompletion and history where the commands can be run directly
  • [x] commands extensible via plugins
  • [x] some way to wrap current Katello CLI commands and make it possible to have one CLI client for both
  • [x] simillar UX as from katello CLI

All the requirements were addressed by the design. Though there is plenty of room for clean up and optimisation, the draft can serve as a proof of concept.

Design

Design draft

As the diagram shows, the CLI consist of almost generic framework (shell-like environment, autocompletion, command help text, option evaluation and command invocation) and set of plugins defining the actual commands. This setup is flexible and allows us to easily install different sets of commands for different products. The plugins are independant and can implement any action as an command, so that besides commands calling Foreman API you can have commands calling varius admin tasks on the server, etc.

For real implementation of the plugins there would need to be some decision taken on how to approach the development of the actual commands.

For Katello commands, there can be great deal of work done automatically (using Katello CLI autocompletion feature and templates). However calling Katello CLI via system call is not ideal solution and should be replaced by direct calls to Katello API in the future.

For Foreman Commands development I can see three options

  • generate and sync some generic CLI commands (apipie approach) with option to inherit and redefine individual commands
  • manually create everything (Katello CLI approach)
  • dynamic creation on the fly with option to exclude and manually define specific commands

Sample working implementation you can find in this repo.

Pros and Cons

  • [+] same implementation language as the rest of the project
  • [+] generic CLI, easy to extend by adding gems with commands
  • [+] lightweight design
  • [-] calling katello via system or need to create native Katello plugin

Technologies

CLI Framework

There is plenty of CLI framework gems with varying features and quality on the Internet. I took into consideration Thor for its wide spread, Boson for its features being very close to our requirements and Clamp for its simplicity.

To compare them I played with each of them for a while - Thor has well docummented features and good suport in the community. However the codebase was for me quite cryptic and really dificult to understand and extend. It seems good when you need what is built in. If you need something extra adding is time consuming or requires skills beyond mine.
Boson has good documentation, nice set of plugins available, live upstream and what makes it special for us is built in shell with autocompletion. What put it out of the game is lack of subcommands.
Clamp has very similar features as Thor has but it has minimalistic, clean and straghtforward codebase. It uses one class per command which can come handy if you need similar behavior for subset of commands. It was quite easy to add autocompletion and shell. And yes it is the winner for my CLI draft.

Shell

For shell there is not so much options. There is IRB and similar like Ripl but I started with minimalistic and Readline. It went quite smooth so I kept it and didn’t tried the others

Autocompletion

I wanted to have one solution for Bash and shell autocompletion and started with simple method recursively looking for endings in command tree. Simple bash wrapper call made it ready for use in bash. In shell-like env based on Readline I had dificulties with the fact that readline support autocompletion based on last word and there is no apparent way to take the context into account. I found some workaround but still looking for cleaner solution. Maybe Ripl + Bond are worth trying

Input/Output

I didn’t focus much on processing of input and output in the commands yet. For Foreman commands I’d like to keep it as similar to Katello output as possible. There are gems like highline and table_print that could make it easy.

Usage

And here follows short demo of the features this CLI draft offers

Help

 $ kartit -u admin -p admin -h
 Usage:
     kartit [OPTIONS] SUBCOMMAND [ARGS] ...

 Subcommands:
     shell                         Interactive Shell
     architecture                  Manipulate Foreman's architectures.
     environment                   Manipulate Katello's environments.
     ping                          Get the status of the katello server

 Options:
     -v, --verbose                 be verbose
     -u, --username USERNAME       username to access the remote system
     -p, --password PASSWORD       password to access the remote system
     --version                     show version
     --autocomplete LINE           Get list of possible endings
     -h, --help                    print help


 $ kartit -u admin -p admin architecture -h
 Usage:
     kartit architecture [OPTIONS] SUBCOMMAND [ARGS] ...

 Subcommands:
     list                          List foreman's architectures.

 Options:
     -h, --help                    print help

Foreman call

As it was said above output formating for foreman commands is still missing and will be added later

 $ kartit -u admin -p admin architecture list
 [
     [0] {
         "architecture" => {
                            "name" => "i386",
                              "id" => 5,
                      "created_at" => "2012-06-27T17:21:16Z",
             "operatingsystem_ids" => [],
                      "updated_at" => "2012-06-27T17:21:16Z"
         }
     },
     [1] {
         "architecture" => {
                            "name" => "ppc",
                              "id" => 9,
                      "created_at" => "2012-07-31T12:19:26Z",
             "operatingsystem_ids" => [],
                      "updated_at" => "2012-07-31T12:19:26Z"
         }
     },
     [2] {
         "architecture" => {
                            "name" => "x86_64",
                              "id" => 14,
                      "created_at" => "2012-08-06T15:51:21Z",
             "operatingsystem_ids" => [],
                      "updated_at" => "2012-08-06T15:51:21Z"
         }
     }
 ]

Katello call

 $ kartit -u admin -p admin environment list --org ACME_Corporation
 katello -u admin -p admin environment list --org ACME_Corporation
 ------------------------------------------------------------------------------------------------------------
                                           Environment List

 ID Name    Label   Description Org              Prior Environment
 ------------------------------------------------------------------------------------------------------------
 1  Library Library None        ACME_Corporation None
 2  Dev     Dev     None        ACME_Corporation Library
 3  Prod    Prod    None        ACME_Corporation Dev

Autocompletion

 $ kartit architecture
 -h      --help  list
 $ kartit architecture list -
 -h      --help

Shell

 $ kartit -u admin -p admin shell
 > -h
 Usage:
     kartit [OPTIONS] SUBCOMMAND [ARGS] ...

 Subcommands:
     shell                         Interactive Shell
     architecture                  Manipulate Foreman's architectures.
     environment                   Manipulate Katello's environments.
     ping                          Get the status of the katello server

 Options:
     -v, --verbose                 be verbose
     -u, --username USERNAME       username to access the remote system
     -p, --password PASSWORD       password to access the remote system
     --version                     show version
     --autocomplete LINE           Get list of possible endings
     -h, --help                    print help
 > ping
 katello -u admin -p admin ping
 -----------------------------------------------------------------------------------------------------------
                                               Katello Status

 Status Service        Result Duration Message
 -----------------------------------------------------------------------------------------------------------

 FAIL
 candlepin      ok     141ms
 candlepin_auth ok     225ms
 elasticsearch  ok     795ms
 katello_jobs   FAIL   katello-jobs service not running
 pulp           ok     1706ms
 pulp_auth      ok     618ms

For what it's worth, when I evaluated many CLI frameworks, to write a
next gen foreman cli in, I also ended up selecting clamp:


Here were some of my notes from that evaluation:

NYC.rb

New York, NY
5,253 Rubyists

NYC.rb is the place for new and experienced Ruby and Rails programmers in New York City. MEETUPS We meet on the second Tuesday of the month. TALK SUBMISSION Submit a talk prop...

Next Meetup

The Bug that Forced Me to Understand Memory Compaction by Em...

Wednesday, Nov 11, 2020, 5:30 PM
44 Attending

Check out this Meetup Group →

Thor was difficult to extend, and the big showstopper for me on boson
was a hard dependency on ruby 1.9. (I evaluated many others and ended
up in the same place.)

Here's where I started speccing it out:
http://projects.theforeman.org/projects/1/wiki/Hammer (I chose the
name Hammer as a tongue in cheek play on the name of Chef's knife
tool. IE: Knife is to chef as Hammer is to foreman.)

I like that you are adding interactive shell, but am wondering if a
single CLI that interacts with katello, and other projects is the way
to go? (vs. a purely foreman cli?)

In any case good luck!! (I lost the free time to work on this, but
there's definitely strong demand for a fully featured foreman cli.)

Cheers,
Brian

··· On Thu, May 2, 2013 at 11:34 AM, Martin Bačovský wrote: > Hi, > > as Tomas announced a couple of days ago, here is my proposal for CLI design > implemented in Ruby. > > For better reading experience and code to play with check > https://github.com/mbacovsky/foreman_cli_draft > > Comments and questions are wellcome! > > Regards, > > Martin > > Foreman CLI design draft > ======================== > > This is my attempt to create outline of CLI client for the Foreman. The > Foreman project has Ruby as its main langauage, so I choosed to explore the > options Ruby world is offering to complete this task. > > My initial requirements are as follows: > > - [x] implemented in Ruby > - [x] Git-like subcomands in CLI app > - [x] system shell autocompletion of commands and options > - [x] shell-like environment with autocompletion and history where the > commands can be run directly > - [x] commands extensible via plugins > - [x] some way to wrap current Katello CLI commands and make it possible to > have one CLI client for both > - [x] simillar UX as from katello CLI > > All the requirements were addressed by the design. Though there is plenty > of room for clean up and optimisation, the draft can serve as a proof of > concept. > > > Design > ------ > > ![Design draft](design.png) > > As the diagram shows, the CLI consist of almost generic framework > (shell-like environment, autocompletion, command help text, option > evaluation and command invocation) and set of plugins defining the actual > commands. This setup is flexible and allows us to easily install different > sets of commands for different products. The plugins are independant and can > implement any action as an command, so that besides commands calling Foreman > API you can have commands calling varius admin tasks on the server, etc. > > For real implementation of the plugins there would need to be some decision > taken on how to approach the development of the actual commands. > > For *Katello* commands, there can be great deal of work done automatically > (using Katello CLI autocompletion feature and templates). However calling > Katello CLI via system call is not ideal solution and should be replaced by > direct calls to Katello API in the future. > > For *Foreman* Commands development I can see three options > - generate and sync some generic CLI commands (apipie approach) with option > to inherit and redefine individual commands > - manually create everything (Katello CLI approach) > - dynamic creation on the fly with option to exclude and manually define > specific commands > > Sample working implementation you can find in this repo. > > - [kartit] > (https://github.com/mbacovsky/foreman_cli_draft/tree/master/kartit) gem > contains generic CLI client > - [kartit-foreman] > (https://github.com/mbacovsky/foreman_cli_draft/tree/master/kartit-foreman) > gem contains Foreman related commands definition > - [kartit-katello] > (https://github.com/mbacovsky/foreman_cli_draft/tree/master/kartit-katello) > gem contains Katello related commands definition > > > Pros and Cons > -------------- > - [+] same implementation language as the rest of the project > - [+] generic CLI, easy to extend by adding gems with commands > - [+] lightweight design > - [-] calling katello via *system* or need to create native Katello plugin > > > Technologies > ------------ > > ### CLI Framework > > There is plenty of CLI framework gems with varying features and quality on > the Internet. I took into consideration [Thor][thor] for its wide spread, > [Boson][boson] for its features being very close to our requirements and > [Clamp][clamp] for its simplicity. > > To compare them I played with each of them for a while - _Thor_ has well > docummented features and good suport in the community. However the codebase > was for me quite cryptic and really dificult to understand and extend. It > seems good when you need what is built in. If you need something extra > adding is time consuming or requires skills beyond mine. > _Boson_ has good documentation, nice set of plugins available, live upstream > and what makes it special for us is built in shell with autocompletion. What > put it out of the game is lack of subcommands. > _Clamp_ has very similar features as Thor has but it has minimalistic, clean > and straghtforward codebase. It uses one class per command which can come > handy if you need similar behavior for subset of commands. It was quite easy > to add autocompletion and shell. And yes it is the winner for my CLI draft. > > > ### Shell > For shell there is not so much options. There is IRB and similar like Ripl > but I started with minimalistic and Readline. It went quite smooth so I > kept it and didn't tried the others > > > ### Autocompletion > I wanted to have one solution for Bash and shell autocompletion and started > with simple method recursively looking for endings in command tree. Simple > bash wrapper call made it ready for use in bash. In shell-like env based on > Readline I had dificulties with the fact that readline support > autocompletion based on last word and there is no apparent way to take the > context into account. I found some workaround but still looking for cleaner > solution. Maybe Ripl + Bond are worth trying > > > ### Input/Output > I didn't focus much on processing of input and output in the commands yet. > For Foreman commands I'd like to keep it as similar to Katello output as > possible. There are gems like highline and [table_print][table-p] that could > make it easy. > > > Usage > ----- > And here follows short demo of the features this CLI draft offers > > ### Help > > $ kartit -u admin -p admin -h > Usage: > kartit [OPTIONS] SUBCOMMAND [ARGS] ... > > Subcommands: > shell Interactive Shell > architecture Manipulate Foreman's architectures. > environment Manipulate Katello's environments. > ping Get the status of the katello server > > Options: > -v, --verbose be verbose > -u, --username USERNAME username to access the remote system > -p, --password PASSWORD password to access the remote system > --version show version > --autocomplete LINE Get list of possible endings > -h, --help print help > > > $ kartit -u admin -p admin architecture -h > Usage: > kartit architecture [OPTIONS] SUBCOMMAND [ARGS] ... > > Subcommands: > list List foreman's architectures. > > Options: > -h, --help print help > > > ### Foreman call > > As it was said above output formating for foreman commands is still missing > and will be added later > > $ kartit -u admin -p admin architecture list > [ > [0] { > "architecture" => { > "name" => "i386", > "id" => 5, > "created_at" => "2012-06-27T17:21:16Z", > "operatingsystem_ids" => [], > "updated_at" => "2012-06-27T17:21:16Z" > } > }, > [1] { > "architecture" => { > "name" => "ppc", > "id" => 9, > "created_at" => "2012-07-31T12:19:26Z", > "operatingsystem_ids" => [], > "updated_at" => "2012-07-31T12:19:26Z" > } > }, > [2] { > "architecture" => { > "name" => "x86_64", > "id" => 14, > "created_at" => "2012-08-06T15:51:21Z", > "operatingsystem_ids" => [], > "updated_at" => "2012-08-06T15:51:21Z" > } > } > ] > > > ### Katello call > > $ kartit -u admin -p admin environment list --org ACME_Corporation > katello -u admin -p admin environment list --org ACME_Corporation > > ------------------------------------------------------------------------------------------------------------ > Environment List > > ID Name Label Description Org Prior Environment > > ------------------------------------------------------------------------------------------------------------ > 1 Library Library None ACME_Corporation None > 2 Dev Dev None ACME_Corporation Library > 3 Prod Prod None ACME_Corporation Dev > > ### Autocompletion > > $ kartit architecture > -h --help list > $ kartit architecture list - > -h --help > > ### Shell > > $ kartit -u admin -p admin shell > > -h > Usage: > kartit [OPTIONS] SUBCOMMAND [ARGS] ... > > Subcommands: > shell Interactive Shell > architecture Manipulate Foreman's architectures. > environment Manipulate Katello's environments. > ping Get the status of the katello server > > Options: > -v, --verbose be verbose > -u, --username USERNAME username to access the remote system > -p, --password PASSWORD password to access the remote system > --version show version > --autocomplete LINE Get list of possible endings > -h, --help print help > > ping > katello -u admin -p admin ping > > ----------------------------------------------------------------------------------------------------------- > Katello Status > > Status Service Result Duration Message > > ----------------------------------------------------------------------------------------------------------- > > FAIL > candlepin ok 141ms > candlepin_auth ok 225ms > elasticsearch ok 795ms > katello_jobs FAIL katello-jobs service not running > pulp ok 1706ms > pulp_auth ok 618ms > > > [cli-gems]: http://www.awesomecommandlineapps.com/gems.html "CLI related > gems" > [thor]: https://github.com/wycats/thor "Thor homepage" > [boson]: https://github.com/cldwalker/boson "Boson homepage" > [clamp]: https://github.com/mdub/clamp "Clamp homepage" > [table-p]: https://github.com/arches/table_print "table-print homepage" > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > >

Thanks for sharing your experience. I'm quite new to the project so I
missed your work on CLI. As I looked through your links I found out that
our specs and findings from framework evaluation pretty much overlap. I
take it as a good sign :slight_smile:

I have some experience with Katello project and it makes sense to use
the two together. There was some demand to use single CLI to manage both
of them. My plan is to create one common CLI "evaluator" and feed it
with commands defined in plugins. So for separate foreman the CLI would
consist of the evaluator + Foreman commands. When there is Katello
present in the setup you could just install Katello commands plugin end
be able to manage everything from one place.

Martin

··· On 05/02/2013 10:38 PM, Brian Gupta wrote: > For what it's worth, when I evaluated many CLI frameworks, to write a > next gen foreman cli in, I also ended up selecting clamp: > https://github.com/theforeman/foreman-hammer > https://groups.google.com/forum/?fromgroups=#!topic/foreman-dev/FZT4kAHjMaE > > Here were some of my notes from that evaluation: > http://www.meetup.com/NYC-rb/messages/32146062/ > > Thor was difficult to extend, and the big showstopper for me on boson > was a hard dependency on ruby 1.9. (I evaluated many others and ended > up in the same place.) > > Here's where I started speccing it out: > http://projects.theforeman.org/projects/1/wiki/Hammer (I chose the > name Hammer as a tongue in cheek play on the name of Chef's knife > tool. IE: Knife is to chef as Hammer is to foreman.) > > I like that you are adding interactive shell, but am wondering if a > single CLI that interacts with katello, and other projects is the way > to go? (vs. a purely foreman cli?) > > In any case good luck!! (I lost the free time to work on this, but > there's definitely strong demand for a fully featured foreman cli.) > > Cheers, > Brian >