I've been tasked with creating a tool that will sit on a foreman-katello
server that contains a few commands that might be very helpful to its admin.
The following list of tasks are already slated to be added to foreman-admin:
backing up the server
restoring the server from backup
generating a foreman-debug archive
exporting the server's task data
updating the server (not upgrading)
I've just built the structure of the tool so far. My next step is to have
foreman-admin simply execute the various scripts and/or rake tasks located in
their current places, but in the long run, I'd prefer we moved them into
foreman-admin. Once foreman-admin functions as we want it, then I'll proceed to
pull scripts from other places and move them to foreman-admin (and translate
them to ruby if necessary).
> - backing up the server
> - restoring the server from backup
> - generating a foreman-debug archive
Is this only as an "alias" or do you plan to integrate some logic? I'd
like to stick with keeping everything in foreman-debug and also advising
users to use it directly.
> - exporting the server's task data
> - updating the server (not upgrading)
exporting database into readable format (good for advanced debugging,
we often need to see subnets, domains, hosts but this is out of scope
of foreman-debug as this is too sensitive data - good idea for
foreman-admin)
foreman-tasks shortcut (?)
Make sure "clamp" is in dependencies, otherwise you will have a packaging task
as well
Why are we not simply adding these as rake tasks (for the ones that
don't exist as tasks already) and then asking people to simply call
"foreman-rake <task>" ? That would seem to be the simplest solution,
unless I'm missing something obvious?
Greg
···
On 17 April 2015 at 18:45, Adam Price wrote:
> hi folks!
>
> I've been tasked with creating a tool that will sit on a foreman-katello
> server that contains a few commands that might be very helpful to its admin.
>
> The following list of tasks are already slated to be added to foreman-admin:
>
> - backing up the server
> - restoring the server from backup
> - generating a foreman-debug archive
> - exporting the server's task data
> - updating the server (not upgrading)
additional note. I've successfully used ruby-newt library (after some
patches I intend to push upstream). If you want to do some interactive
stuff, go ahead with this library as I plan to package this into foreman
repositories. Example screens:
IMO, rake is a horrible user experience and offers no assistance to the
admin as to how things with parameters are expected to be run.
foreman-rake
rake aborted!
Please install the sqlite3 adapter: …
foreman-rake --help
rake [-f rakefile] {options} targets…
Options are …
-C, --classic-namespace Put Task and FileTask in the top
level namespace
-D, --describe [PATTERN] Describe the tasks (matching
optional PATTERN), then exit.
-n, --dry-run Do a dry run without executing
actions.
-e, --execute CODE Execute some Ruby code and exit.
-p, --execute-print CODE Execute some Ruby code, print the
result, then exit.
-E, --execute-continue CODE Execute some Ruby code, then
continue with normal task processing.
…
completely obtuse and useless for an admin trying to execute supported
operations on the Foreman server.
we can re-use all the rake tasks we have now, just have 'foreman-admin'
be a wrapper to call them if and when we wish to do so. If the task has
parameters, then have them as documented params in the foreman-admin
command.
Mike
···
On 04/19/2015 02:12 PM, Greg Sutcliffe wrote:
> On 17 April 2015 at 18:45, Adam Price wrote:
>> hi folks!
>>
>> I've been tasked with creating a tool that will sit on a foreman-katello
>> server that contains a few commands that might be very helpful to its admin.
>>
>> The following list of tasks are already slated to be added to foreman-admin:
>>
>> - backing up the server
>> - restoring the server from backup
>> - generating a foreman-debug archive
>> - exporting the server's task data
>> - updating the server (not upgrading)
>
> Why are we not simply adding these as rake tasks (for the ones that
> don't exist as tasks already) and then asking people to simply call
> "foreman-rake" ? That would seem to be the simplest solution,
> unless I'm missing something obvious?
>
> Greg
>
–
Mike McCune
mmccune AT redhat.com
Red Hat Engineering | Portland, OR
Systems Management | 650-254-4248
I think one argument could be that foreman-rake is sooooo slow because
it has to load the entire rails app. Not all of the suggested features
need that.
Stephen
···
On Sun, Apr 19, 2015 at 10:12:06PM +0100, Greg Sutcliffe wrote:
> On 17 April 2015 at 18:45, Adam Price wrote:
> > hi folks!
> >
> > I've been tasked with creating a tool that will sit on a foreman-katello
> > server that contains a few commands that might be very helpful to its admin.
> >
> > The following list of tasks are already slated to be added to foreman-admin:
> >
> > - backing up the server
> > - restoring the server from backup
> > - generating a foreman-debug archive
> > - exporting the server's task data
> > - updating the server (not upgrading)
>
> Why are we not simply adding these as rake tasks (for the ones that
> don't exist as tasks already) and then asking people to simply call
> "foreman-rake " ? That would seem to be the simplest solution,
> unless I'm missing something obvious?
>
> Greg
> > > hi folks!
> > >
> > > I've been tasked with creating a tool that will sit on a
> foreman-katello
> > > server that contains a few commands that might be very helpful to its
> admin.
> > >
> > > The following list of tasks are already slated to be added to
> foreman-admin:
> > >
> > > - backing up the server
> > > - restoring the server from backup
> > > - generating a foreman-debug archive
> > > - exporting the server's task data
> > > - updating the server (not upgrading)
>
What does updating the server mean if not upgrading?
> >
> > Why are we not simply adding these as rake tasks (for the ones that
> > don't exist as tasks already) and then asking people to simply call
> > "foreman-rake <task>" ? That would seem to be the simplest solution,
> > unless I'm missing something obvious?
>
Or hammer? I can understand if we are limiting hammer to actions that can
be performed through the API. However, keeping track of all of our
interfaces and tools and scripts is starting to grow:
hammer
foreman-rake
foreman-debug
foreman-admin*
Something else to consider: how do plugins add to these scripts? This may
be a Katello only concern at the moment, but worth thinking about.
> >
> > Greg
>
> I think one argument could be that foreman-rake is sooooo slow because
> it has to load the entire rails app. Not all of the suggested features
> need that.
>
How many users are concerned with speed when running tasks like these?
Eric
···
On Mon, Apr 20, 2015 at 4:47 AM, Stephen Benjamin wrote:
> On Sun, Apr 19, 2015 at 10:12:06PM +0100, Greg Sutcliffe wrote:
> > On 17 April 2015 at 18:45, Adam Price wrote:
> IMO, rake is a horrible user experience and offers no assistance to the
> admin as to how things with parameters are expected to be run.
That's fair, however…
> # foreman-rake
> rake aborted!
> Please install the sqlite3 adapter: …
> completely obtuse and useless for an admin trying to execute supported
> operations on the Foreman server.
foreman-rake, on the other hand, is just a shellscript wrapper
around rake, and could be made better. Wilst I'd suggest keeping
functionality within rake, it could certainly do better input
validation and/or error handling. I don't see why we need a 3rd tool
(or 4th if you count Hammer), when we can improve the ones we have.
hammer is for interacting via REST to the Foreman API. This seems
entirely unsuited to tasks related to manipulating the Foreman server's
filesystem, running services or other 'root' level tasks.
One option would be:
enhance foreman-rake to be what we are proposing for foreman-admin and
unify them into one thing. The utility would need to first check for
commands that are formalized with specific params/documented help text
and then fall back to just running any pre-defined rake task if no
command was found.
Running it without args would just show the list of all known commands
and perhaps defined set of rake tasks.
Mike
···
On 04/20/2015 11:46 AM, Greg Sutcliffe wrote:
> On 20 April 2015 at 17:18, Mike McCune wrote:
>> IMO, rake is a horrible user experience and offers no assistance to the
>> admin as to how things with parameters are expected to be run.
>
> That's fair, however...
>
>> # foreman-rake
>> rake aborted!
>> Please install the sqlite3 adapter: ...
>> completely obtuse and useless for an admin trying to execute supported
>> operations on the Foreman server.
>
> *foreman*-rake, on the other hand, is just a shellscript wrapper
> around rake, and could be made better. Wilst I'd suggest keeping
> functionality within rake, it could certainly do better input
> validation and/or error handling. I don't see why we need a 3rd tool
> (or 4th if you count Hammer), when we can improve the ones we have.
–
Mike McCune
mmccune AT redhat.com
Red Hat Engineering | Portland, OR
Systems Management | 650-254-4248
>
>
>
> > > hi folks!
> > >
> > > I've been tasked with creating a tool that will sit on a
> foreman-katello
> > > server that contains a few commands that might be very helpful
> to its admin.
> > >
> > > The following list of tasks are already slated to be added to
> foreman-admin:
> > >
> > > - backing up the server
> > > - restoring the server from backup
> > > - generating a foreman-debug archive
> > > - exporting the server's task data
> > > - updating the server (not upgrading)
>
>
> What does updating the server mean if not upgrading?
>
For Katello we had situations in the past where we needed to in a very
specific order:
stop all services
yum update packages
start Elasticsearch
reindex the Elasticsearch
start services
this is a bit less than a full 'katello-installer --upgrade' which does
database migration.
Perhaps as the project matures and we remove Elasticsearch the
difference between 'update' and 'upgrade' will be different such that we
don't need this scenario automated.
> >
> > Why are we not simply adding these as rake tasks (for the ones that
> > don't exist as tasks already) and then asking people to simply call
> > "foreman-rake <task>" ? That would seem to be the simplest solution,
> > unless I'm missing something obvious?
>
>
> Or hammer? I can understand if we are limiting hammer to actions that
> can be performed through the API. However, keeping track of all of our
> interfaces and tools and scripts is starting to grow:
>
> hammer
> foreman-rake
> foreman-debug
> foreman-admin*
>
see my reply to Greg on this.
> Something else to consider: how do plugins add to these scripts? This
> may be a Katello only concern at the moment, but worth thinking about.
>
> >
> > Greg
>
> I think one argument could be that foreman-rake is sooooo slow because
> it has to load the entire rails app. Not all of the suggested features
> need that.
>
>
> How many users are concerned with speed when running tasks like these?
>
>
IMO, speed isn't that big of a deal, most of these things are 'run once'
and wait for it to finish.
···
On 04/20/2015 08:03 AM, Eric D Helms wrote:
> On Mon, Apr 20, 2015 at 4:47 AM, Stephen Benjamin > wrote:
> On Sun, Apr 19, 2015 at 10:12:06PM +0100, Greg Sutcliffe wrote:
> > On 17 April 2015 at 18:45, Adam Price > wrote:
–
Mike McCune
mmccune AT redhat.com
Red Hat Engineering | Portland, OR
Systems Management | 650-254-4248
> hammer is for interacting via REST to the Foreman API. This seems entirely
> unsuited to tasks related to manipulating the Foreman server's filesystem,
> running services or other 'root' level tasks.
Indeed, I wasn't suggesting that we extend Hammer, only pointing out
that we already have multiple cmdline tools
> One option would be:
>
> enhance foreman-rake to be what we are proposing for foreman-admin and unify
> them into one thing. The utility would need to first check for commands that
> are formalized with specific params/documented help text and then fall back
> to just running any pre-defined rake task if no command was found.
>
> foreman-rake backup
> foreman-rake restore
> foreman-rake export-tasks
> …
> foreman-rake <rake task>
>
> Running it without args would just show the list of all known commands and
> perhaps defined set of rake tasks.
That doesn't sound unreasonable. A longer-range task might also be to
bring foreman-debug in here, leaving us with one tool for
on-the-server stuff and one tool for over-the-api stuff. That feels
nice and clean.
My end vision for this was one tool for on-the-server tasks and one tool for over-the-api tasks.
Apologies for articulating that poorly.
···
----- Original Message -----
> On 20 April 2015 at 22:07, Mike McCune wrote:
> > hammer is for interacting via REST to the Foreman API. This seems entirely
> > unsuited to tasks related to manipulating the Foreman server's filesystem,
> > running services or other 'root' level tasks.
>
> Indeed, I wasn't suggesting that we extend Hammer, only pointing out
> that we already have multiple cmdline tools
>
> > One option would be:
> >
> > enhance foreman-rake to be what we are proposing for foreman-admin and
> > unify
> > them into one thing. The utility would need to first check for commands
> > that
> > are formalized with specific params/documented help text and then fall back
> > to just running any pre-defined rake task if no command was found.
> >
> > foreman-rake backup
> > foreman-rake restore
> > foreman-rake export-tasks
> > ...
> > foreman-rake
> >
> > Running it without args would just show the list of all known commands and
> > perhaps defined set of rake tasks.
>
> That doesn't sound unreasonable. A longer-range task might also be to
> bring foreman-debug in here, leaving us with one tool for
> on-the-server stuff and one tool for over-the-api stuff. That feels
> nice and clean.
>
> Greg
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
> My end vision for this was one tool for on-the-server tasks and one tool for over-the-api tasks.
> Apologies for articulating that poorly.
Please don't kill my foreman-debug. It's a wonderful piece of shell
script we were sharpening in the last couple of releases and it has been
also already integrated with sosreport which ships with RHEL
definitely wouldn't kill it. we would just call it … No desire to
rewrite it.
Mike
···
On 04/21/2015 08:06 AM, Lukas Zapletal wrote:
>> My end vision for this was one tool for on-the-server tasks and one tool for over-the-api tasks.
>> Apologies for articulating that poorly.
>
> Please don't kill my foreman-debug. It's a wonderful piece of shell
> script we were sharpening in the last couple of releases and it has been
> also already integrated with sosreport which ships with RHEL :-)
>
–
Mike McCune
mmccune AT redhat.com
Red Hat Engineering | Portland, OR
Systems Management | 650-254-4248
Why is a rewrite completely out of the question? I'm in favor of consolidating
our tools and scripts into as few pieces as possible, preferably in as much
Ruby as possible.
Foreman-debug is indeed a nice utility, but I don't think there should be an
aversion to rewriting it.
I guess what I have in the back of my mind is that there are so many
inconsistencies in our command-line interfaces that I would prefer we
consolidate tooling to share code so that the user-experience (even from the
command line) feels consistent. My goal is to have a tool that does a set
number of things, and it does them well, and it does them consistently, as
opposed to having a handful of scripts laying around that each operate
differently and share chunks of copied code.
I want for us to have awesome command line tools! Because Foreman and
Katello are awesome.
···
----- Original Message -----
> On 04/21/2015 08:06 AM, Lukas Zapletal wrote:
> >> My end vision for this was one tool for on-the-server tasks and one tool
> >> for over-the-api tasks.
> >> Apologies for articulating that poorly.
> >
> > Please don't kill my foreman-debug. It's a wonderful piece of shell
> > script we were sharpening in the last couple of releases and it has been
> > also already integrated with sosreport which ships with RHEL :-)
> >
>
> definitely wouldn't kill it. we would just call it .. No desire to
> rewrite it.
>
> Mike
> --
> Mike McCune
> mmccune AT redhat.com
> Red Hat Engineering | Portland, OR
> Systems Management | 650-254-4248
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
> Why is a rewrite completely out of the question? I'm in favor of consolidating
> our tools and scripts into as few pieces as possible, preferably in as much
> Ruby as possible.
>
> Foreman-debug is indeed a nice utility, but I don't think there should be an
> aversion to rewriting it.
I don't it's out of question, but I would like to add that:
The tool is successfully used by our community for several releases.
We have been tuning it adding various password filters, files, OS
support (RHELs, Fedoras, Debians).
It was integrated with sosreport in already released versions.
It works on the following scenarios:
Foreman
Foreman with Katello
Foreman Proxy
Satellite 6 Server
Satellite 6 Capsule
Packages were made for all supported systems upstream (Debian etc)
The package was broken into subpackages so it is possible to install
it on Foreman/Satellite as well as on Foreman Proxy/Capsule.
It has manual page, documentation upstream/downstream
It has plugin architecture several plugins already leverage (not only
Katello)
It has literally zero dependencies (Bash, binutils) (*)
If the idea of rewriting is in your head, keep in mind not to break
everything I listed here (plus all I forgot). The CLI would definitely
need to be backward compatible so older sosreport versions we support
work with it.
(*) If you want to use Ruby keep in mind we want to support Ruby
version shipped with all the OS we support (even the older ones). Not
sure if this should be 1.8 or 1.9, check that first. Debian can ruin
your plans in regard to modern Ruby features.
So yes, it's 10 pages long shell script with some plugins. But I
personally invested huge amount of time into it. Because I know hard
difficult is to debug issues sometimes and I find this tool
super-important. That's why I was trying my best to get it where it is.
Don't get me wrong, if you do better job in Ruby, I will be happy. But
in all my reviews, I will most likely bring the topics above
> > >> My end vision for this was one tool for on-the-server tasks and one tool
> > >> for over-the-api tasks.
> > >> Apologies for articulating that poorly.
> > >
> > > Please don't kill my foreman-debug. It's a wonderful piece of shell
> > > script we were sharpening in the last couple of releases and it has been
> > > also already integrated with sosreport which ships with RHEL
> > >
> >
> > definitely wouldn't kill it. we would just call it … No desire to
> > rewrite it.
> >
> > Mike
> > –
> > Mike McCune
> > mmccune AT redhat.com
> > Red Hat Engineering | Portland, OR
> > Systems Management | 650-254-4248
> >
> > –
> > You received this message because you are subscribed to the Google Groups
> > "foreman-dev" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to foreman-dev+unsubscribe@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
> >
>
> Why is a rewrite completely out of the question? I'm in favor of consolidating
> our tools and scripts into as few pieces as possible, preferably in as much
> Ruby as possible.
I think we should use the language most appropriate to the task, and
IMHO bash is more suited for rounding up files and spitting out a
tarball. foreman-debug works and is not broken, and there's the old
adage about not fixing what's not broken…
foreman-rake's not really easy to use (i.e. getting help out of it),
that's certainly worth addressing.
···
On Tue, Apr 21, 2015 at 02:15:47PM -0400, Adam Price wrote:
> ----- Original Message -----
> > On 04/21/2015 08:06 AM, Lukas Zapletal wrote:
Foreman-debug is indeed a nice utility, but I don’t think there should be an
aversion to rewriting it.
I guess what I have in the back of my mind is that there are so many
inconsistencies in our command-line interfaces that I would prefer we
consolidate tooling to share code so that the user-experience (even from the
command line) feels consistent. My goal is to have a tool that does a set
number of things, and it does them well, and it does them consistently, as
opposed to having a handful of scripts laying around that each operate
differently and share chunks of copied code.
I want for us to have awesome command line tools! Because Foreman and
Katello are awesome.
Indeed, and can be fixed with a small amount of effort. Exposing the
'test' task as the default is a poor choice for production, and not even
really necessary in development.
Having it respond to --help for individual tasks and when no task is
specified, passing it through to -T would be a quick fix also.
···
On 22/04/15 18:51, Stephen Benjamin wrote:
> On Tue, Apr 21, 2015 at 02:15:47PM -0400, Adam Price wrote:
>> ----- Original Message -----
>>> On 04/21/2015 08:06 AM, Lukas Zapletal wrote:
>>>>> My end vision for this was one tool for on-the-server tasks and one tool
>>>>> for over-the-api tasks.
>>>>> Apologies for articulating that poorly.
>>>>
>>>> Please don't kill my foreman-debug. It's a wonderful piece of shell
>>>> script we were sharpening in the last couple of releases and it has been
>>>> also already integrated with sosreport which ships with RHEL :-)
>>>>
>>>
>>> definitely wouldn't kill it. we would just call it .. No desire to
>>> rewrite it.
>>>
>>> Mike
>>> --
>>> Mike McCune
>>> mmccune AT redhat.com
>>> Red Hat Engineering | Portland, OR
>>> Systems Management | 650-254-4248
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "foreman-dev" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to foreman-dev+unsubscribe@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>> Why is a rewrite completely out of the question? I'm in favor of consolidating
>> our tools and scripts into as few pieces as possible, preferably in as much
>> Ruby as possible.
>
> I think we should use the language most appropriate to the task, and
> IMHO bash is more suited for rounding up files and spitting out a
> tarball. foreman-debug works and is not broken, and there's the old
> adage about not fixing what's not broken...
>
> foreman-rake's not really easy to use (i.e. getting help out of it),
> that's certainly worth addressing.
> >>>>> My end vision for this was one tool for on-the-server tasks and one
> tool
> >>>>> for over-the-api tasks.
> >>>>> Apologies for articulating that poorly.
> >>>>
> >>>> Please don't kill my foreman-debug. It's a wonderful piece of shell
> >>>> script we were sharpening in the last couple of releases and it has
> been
> >>>> also already integrated with sosreport which ships with RHEL
> >>>>
> >>>
> >>> definitely wouldn't kill it. we would just call it … No desire to
> >>> rewrite it.
> >>>
> >>> Mike
> >>> –
> >>> Mike McCune
> >>> mmccune AT redhat.com
> >>> Red Hat Engineering | Portland, OR
> >>> Systems Management | 650-254-4248
> >>>
> >>> –
> >>> You received this message because you are subscribed to the Google
> Groups
> >>> "foreman-dev" group.
> >>> To unsubscribe from this group and stop receiving emails from it, send
> an
> >>> email to foreman-dev+unsubscribe@googlegroups.com.
> >>> For more options, visit https://groups.google.com/d/optout.
> >>>
> >>
> >> Why is a rewrite completely out of the question? I'm in favor of
> consolidating
> >> our tools and scripts into as few pieces as possible, preferably in as
> much
> >> Ruby as possible.
> >
>
It sounds to me like we need to introduce a single entry tool, but not
rewrite it all…
> > I think we should use the language most appropriate to the task, and
> > IMHO bash is more suited for rounding up files and spitting out a
> > tarball. foreman-debug works and is not broken, and there's the old
> > adage about not fixing what's not broken…
> >
> > foreman-rake's not really easy to use (i.e. getting help out of it),
> > that's certainly worth addressing.
>
> Indeed, and can be fixed with a small amount of effort. Exposing the
> 'test' task as the default is a poor choice for production, and not even
> really necessary in development.
>
> Having it respond to --help for individual tasks and when no task is
> specified, passing it through to -T would be a quick fix also.
>
I do see the value of not forcing rake as the 'first entry' tool, there are
many tasks which are rails or developer specific…
Ohad
···
On Thu, Apr 23, 2015 at 10:05 AM, Dominic Cleal wrote:
> On 22/04/15 18:51, Stephen Benjamin wrote:
> > On Tue, Apr 21, 2015 at 02:15:47PM -0400, Adam Price wrote:
> >> ----- Original Message -----
> >>> On 04/21/2015 08:06 AM, Lukas Zapletal wrote:
