I've recorded a screencast on a proof of concept of being able to use
Dynflow to manage MCollective commands: sending both single and multiple
commands and receiving the results that can be further processed.
The demo shows a simple sinatra application that runs Dynflow on background
and exposes simple UI for running mcollective commands. The progress of the tasks
is shown using the built-in Dynflow web console.
The second part of the demo shows how that work can be reused in Katello (my branch
already running Dynflow on background). In the Katello, there is already a remote
commands feature and it's really easy to plug more implementations.
One of the purposes of this demo is to explain the Dynflow's approach to this kind of integration.
The idea is the dynflow controls the state of the task and sends the individual
commands to mcollective, the Dynflow itself runs on Foreman server, either sending the commands
directly to the message bus, or using smart-proxy as a web proxy to the message bus (on
the proxy side, it would translate the API calls to mcollective messages and listen on the
bus for replies, sending it back via a callback, similarly as it's done in the current mcollective
approach.
My plan is to create a foreman-dynflow plugin, that will extract the dynflow infrastructure code
that is now part of Katello, so that it could be used by other plugins/maybe the core, if
they find it interesting.
Another possible use-case is leveraging Dynflow for background tasks, as
it's already async by default, either directly using the thread pool Dynflow has inside, or
abstracting other async task tools in case it's appropriate (from this perspective other async
execution tool is on the same level of abstraction as mcollective or Pulp task).
You can also watch more on how Dynflow is used in Katello in this deep-dive
Thanks for sharing your proof of concept. Nice work!
I liked the way dynflow could potentially be used to combine and
orchestrate multiple mcollective actions. I also really liked the
latter part that showed how the existing Katello UI could be leveraged
as a front-end for several of the actions that it already supports. I
can see where that could help a lot with supporting the actions on
single system as you have illustrated, but also with supporting bulk
actions on a set of systems or system group as well as with reviewing
the history of those actions after they have been completed. Note, this
might also easily extend to using the existing Katello APIs.
cheers,
Brad
···
On 12/04/2013 12:09 PM, Ivan Necas wrote:
> Hi,
>
> I've recorded a screencast on a proof of concept of being able to use
> Dynflow to manage MCollective commands: sending both single and multiple
> commands and receiving the results that can be further processed.
>
> The demo shows a simple sinatra application that runs Dynflow on background
> and exposes simple UI for running mcollective commands. The progress of the tasks
> is shown using the built-in Dynflow web console.
>
> The second part of the demo shows how that work can be reused in Katello (my branch
> already running Dynflow on background). In the Katello, there is already a remote
> commands feature and it's really easy to plug more implementations.
>
> The video is available here:
>
> http://youtu.be/UyO4R7VUFZ4
>
> The code is available here:
>
> https://github.com/iNecas/mcoflow
>
> https://github.com/iNecas/katello/tree/dynflow-mcollective (the Katello's branch reusing the MCOflow in their existing remote execution feature)
>
> One of the purposes of this demo is to explain the Dynflow's approach to this kind of integration.
> The idea is the dynflow controls the state of the task and sends the individual
> commands to mcollective, the Dynflow itself runs on Foreman server, either sending the commands
> directly to the message bus, or using smart-proxy as a web proxy to the message bus (on
> the proxy side, it would translate the API calls to mcollective messages and listen on the
> bus for replies, sending it back via a callback, similarly as it's done in the current mcollective
> approach.
>
> My plan is to create a foreman-dynflow plugin, that will extract the dynflow infrastructure code
> that is now part of Katello, so that it could be used by other plugins/maybe the core, if
> they find it interesting.
>
> Another possible use-case is leveraging Dynflow for background tasks, as
> it's already async by default, either directly using the thread pool Dynflow has inside, or
> abstracting other async task tools in case it's appropriate (from this perspective other async
> execution tool is on the same level of abstraction as mcollective or Pulp task).
>
> You can also watch more on how Dynflow is used in Katello in this deep-dive
>
> http://www.youtube.com/watch?v=cpUz5Ylhkr0#t=0
>
> Feedback welcome,
>
> -- Ivan
>
> Hi,
>
> I've recorded a screencast on a proof of concept of being able to use
> Dynflow to manage MCollective commands: sending both single and multiple
> commands and receiving the results that can be further processed.
>
> The demo shows a simple sinatra application that runs Dynflow on background
> and exposes simple UI for running mcollective commands. The progress of the tasks
> is shown using the built-in Dynflow web console.
>
> The second part of the demo shows how that work can be reused in Katello (my branch
> already running Dynflow on background). In the Katello, there is already a remote
> commands feature and it's really easy to plug more implementations.
>
> The video is available here:
>
> http://youtu.be/UyO4R7VUFZ4
>
> The code is available here:
>
> https://github.com/iNecas/mcoflow
>
> https://github.com/iNecas/katello/tree/dynflow-mcollective (the Katello's branch reusing the MCOflow in their existing remote execution feature)
>
> One of the purposes of this demo is to explain the Dynflow's approach to this kind of integration.
> The idea is the dynflow controls the state of the task and sends the individual
> commands to mcollective, the Dynflow itself runs on Foreman server, either sending the commands
> directly to the message bus, or using smart-proxy as a web proxy to the message bus (on
> the proxy side, it would translate the API calls to mcollective messages and listen on the
> bus for replies, sending it back via a callback, similarly as it's done in the current mcollective
> approach.
>
> My plan is to create a foreman-dynflow plugin, that will extract the dynflow infrastructure code
> that is now part of Katello, so that it could be used by other plugins/maybe the core, if
> they find it interesting.
>
> Another possible use-case is leveraging Dynflow for background tasks, as
> it's already async by default, either directly using the thread pool Dynflow has inside, or
> abstracting other async task tools in case it's appropriate (from this perspective other async
> execution tool is on the same level of abstraction as mcollective or Pulp task).
>
> You can also watch more on how Dynflow is used in Katello in this deep-dive
>
> http://www.youtube.com/watch?v=cpUz5Ylhkr0#t=0
>
> Feedback welcome,
Awesome!
I would like to see a/sync back/foreground tasks unified by Dynflow in
Katello and Foreman. I think it would help a lot.
It's nice indeed. I like the idea of foreman plugin with dynflow engine.
For plain MCO actions like install/remove package which are usually
idempotent orchestration engine is not added value, but once we give our
users possibility to write their own actions this could evolve to
something interesting (Heat-like orchestrator).
LZ
···
On Wed, Dec 04, 2013 at 12:09:53PM -0500, Ivan Necas wrote:
> Hi,
>
> I've recorded a screencast on a proof of concept of being able to use
> Dynflow to manage MCollective commands: sending both single and multiple
> commands and receiving the results that can be further processed.
>
> The demo shows a simple sinatra application that runs Dynflow on background
> and exposes simple UI for running mcollective commands. The progress of the tasks
> is shown using the built-in Dynflow web console.
>
> The second part of the demo shows how that work can be reused in Katello (my branch
> already running Dynflow on background). In the Katello, there is already a remote
> commands feature and it's really easy to plug more implementations.
>
> The video is available here:
>
> http://youtu.be/UyO4R7VUFZ4
>
> The code is available here:
>
> https://github.com/iNecas/mcoflow
>
> https://github.com/iNecas/katello/tree/dynflow-mcollective (the Katello's branch reusing the MCOflow in their existing remote execution feature)
>
> One of the purposes of this demo is to explain the Dynflow's approach to this kind of integration.
> The idea is the dynflow controls the state of the task and sends the individual
> commands to mcollective, the Dynflow itself runs on Foreman server, either sending the commands
> directly to the message bus, or using smart-proxy as a web proxy to the message bus (on
> the proxy side, it would translate the API calls to mcollective messages and listen on the
> bus for replies, sending it back via a callback, similarly as it's done in the current mcollective
> approach.
>
> My plan is to create a foreman-dynflow plugin, that will extract the dynflow infrastructure code
> that is now part of Katello, so that it could be used by other plugins/maybe the core, if
> they find it interesting.
>
> Another possible use-case is leveraging Dynflow for background tasks, as
> it's already async by default, either directly using the thread pool Dynflow has inside, or
> abstracting other async task tools in case it's appropriate (from this perspective other async
> execution tool is on the same level of abstraction as mcollective or Pulp task).
>
> You can also watch more on how Dynflow is used in Katello in this deep-dive
>
> http://www.youtube.com/watch?v=cpUz5Ylhkr0#t=0
>
> Feedback welcome,
>
> -- Ivan
>
> --
> You received this message because you are subscribed to the Google Groups "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
Sorry for not responding sooner, I was out for most of last week. Some comments inline.
> From: "Ivan Necas" <inecas@redhat.com>
> To: "foreman-dev" <foreman-dev@googlegroups.com>
> Sent: Wednesday, December 4, 2013 12:09:53 PM
> Subject: [foreman-dev] Proof of concept: MCollective over Dynflow
>
> Hi,
>
> I've recorded a screencast on a proof of concept of being able to use
> Dynflow to manage MCollective commands: sending both single and multiple
> commands and receiving the results that can be further processed.
>
> The demo shows a simple sinatra application that runs Dynflow on background
> and exposes simple UI for running mcollective commands. The progress of the
> tasks
> is shown using the built-in Dynflow web console.
>
> The second part of the demo shows how that work can be reused in Katello (my
> branch
> already running Dynflow on background). In the Katello, there is already a
> remote
> commands feature and it's really easy to plug more implementations.
>
> The video is available here:
>
> http://youtu.be/UyO4R7VUFZ4
>
> The code is available here:
>
> https://github.com/iNecas/mcoflow
>
> https://github.com/iNecas/katello/tree/dynflow-mcollective (the Katello's
> branch reusing the MCOflow in their existing remote execution feature)
>
> One of the purposes of this demo is to explain the Dynflow's approach to this
> kind of integration.
> The idea is the dynflow controls the state of the task and sends the
> individual
> commands to mcollective, the Dynflow itself runs on Foreman server, either
> sending the commands
> directly to the message bus, or using smart-proxy as a web proxy to the
> message bus (on
> the proxy side, it would translate the API calls to mcollective messages and
> listen on the
> bus for replies, sending it back via a callback, similarly as it's done in
> the current mcollective
> approach.
How do you imagine displaying data for larger numbers of hosts? One of the big UI's that we have left to design is the display of task status; which tasks failed on which hosts. We talked a little bit about this in the deep dive and it seems like dynflow would need some extension to be able to show data about which the execution of steps and where/when failures occurred.
>
> My plan is to create a foreman-dynflow plugin, that will extract the dynflow
> infrastructure code
> that is now part of Katello, so that it could be used by other plugins/maybe
> the core, if
> they find it interesting.
If we could keep all the code outside core that'd be ideal IMO. At some point we're going to move the proxy implementation out to a plugin to prevent bloat. That being said, if foretello wants to move to mco for management remote execution then it might make sense to move the base code into core. We've have to play that by ear.
>
> Another possible use-case is leveraging Dynflow for background tasks, as
> it's already async by default, either directly using the thread pool Dynflow
> has inside, or
> abstracting other async task tools in case it's appropriate (from this
> perspective other async
> execution tool is on the same level of abstraction as mcollective or Pulp
> task).
Could we wrap execution in sidekiq so the existing worker model will continue working?
···
----- Original Message -----
You can also watch more on how Dynflow is used in Katello in this deep-dive
How does this relate to the work which SAM and Dmitri have been doing?
– bk
···
On 12/04/2013 12:09 PM, Ivan Necas wrote:
> Hi,
>
> The video is available here:
>
> http://youtu.be/UyO4R7VUFZ4
>
> The code is available here:
>
> https://github.com/iNecas/mcoflow
>
> https://github.com/iNecas/katello/tree/dynflow-mcollective (the Katello's branch reusing the MCOflow in their existing remote execution feature)
Just curious, but how does open shift accomplish this feat? Open shift
uses mcollective as the orchastration tool and has a cli, web and rest api
built on top of it. Surely we can reuse some of that code?
Corey
···
On Wednesday, December 4, 2013 9:09:53 AM UTC-8, Ivan Necas wrote:
>
> Hi,
>
> I've recorded a screencast on a proof of concept of being able to use
> Dynflow to manage MCollective commands: sending both single and multiple
> commands and receiving the results that can be further processed.
>
> The demo shows a simple sinatra application that runs Dynflow on
> background
> and exposes simple UI for running mcollective commands. The progress of
> the tasks
> is shown using the built-in Dynflow web console.
>
> The second part of the demo shows how that work can be reused in Katello
> (my branch
> already running Dynflow on background). In the Katello, there is already a
> remote
> commands feature and it's really easy to plug more implementations.
>
> The video is available here:
>
> http://youtu.be/UyO4R7VUFZ4
>
> The code is available here:
>
> https://github.com/iNecas/mcoflow
>
> https://github.com/iNecas/katello/tree/dynflow-mcollective (the Katello's
> branch reusing the MCOflow in their existing remote execution feature)
>
> One of the purposes of this demo is to explain the Dynflow's approach to
> this kind of integration.
> The idea is the dynflow controls the state of the task and sends the
> individual
> commands to mcollective, the Dynflow itself runs on Foreman server, either
> sending the commands
> directly to the message bus, or using smart-proxy as a web proxy to the
> message bus (on
> the proxy side, it would translate the API calls to mcollective messages
> and listen on the
> bus for replies, sending it back via a callback, similarly as it's done in
> the current mcollective
> approach.
>
> My plan is to create a foreman-dynflow plugin, that will extract the
> dynflow infrastructure code
> that is now part of Katello, so that it could be used by other
> plugins/maybe the core, if
> they find it interesting.
>
> Another possible use-case is leveraging Dynflow for background tasks, as
> it's already async by default, either directly using the thread pool
> Dynflow has inside, or
> abstracting other async task tools in case it's appropriate (from this
> perspective other async
> execution tool is on the same level of abstraction as mcollective or Pulp
> task).
>
> You can also watch more on how Dynflow is used in Katello in this
> deep-dive
>
> http://www.youtube.com/watch?v=cpUz5Ylhkr0#t=0
>
> Feedback welcome,
>
> -- Ivan
>
This would be the underlying workflow engine for building multi-step processes that rely on the output of an earlier action to proceed. Upgrading the kernel/glibc is a good example of something like this; run yum upgrade on certain packages, check if they recommend restart, and if so reboot the machine.
We're not quite at the point where we can start implementing multi-step workflows, but we should be in the next couple weeks.
···
----- Original Message -----
> On 12/04/2013 12:09 PM, Ivan Necas wrote:
> Sorry for not responding sooner, I was out for most of last week. Some
> comments inline.
>
> > From: "Ivan Necas" <inecas@redhat.com>
> > To: "foreman-dev" <foreman-dev@googlegroups.com>
> > Sent: Wednesday, December 4, 2013 12:09:53 PM
> > Subject: [foreman-dev] Proof of concept: MCollective over Dynflow
> >
> > Hi,
> >
> > I've recorded a screencast on a proof of concept of being able to use
> > Dynflow to manage MCollective commands: sending both single and multiple
> > commands and receiving the results that can be further processed.
> >
> > The demo shows a simple sinatra application that runs Dynflow on background
> > and exposes simple UI for running mcollective commands. The progress of the
> > tasks
> > is shown using the built-in Dynflow web console.
> >
> > The second part of the demo shows how that work can be reused in Katello
> > (my
> > branch
> > already running Dynflow on background). In the Katello, there is already a
> > remote
> > commands feature and it's really easy to plug more implementations.
> >
> > The video is available here:
> >
> > http://youtu.be/UyO4R7VUFZ4
> >
> > The code is available here:
> >
> > https://github.com/iNecas/mcoflow
> >
> > https://github.com/iNecas/katello/tree/dynflow-mcollective (the Katello's
> > branch reusing the MCOflow in their existing remote execution feature)
> >
> > One of the purposes of this demo is to explain the Dynflow's approach to
> > this
> > kind of integration.
> > The idea is the dynflow controls the state of the task and sends the
> > individual
> > commands to mcollective, the Dynflow itself runs on Foreman server, either
> > sending the commands
> > directly to the message bus, or using smart-proxy as a web proxy to the
> > message bus (on
> > the proxy side, it would translate the API calls to mcollective messages
> > and
> > listen on the
> > bus for replies, sending it back via a callback, similarly as it's done in
> > the current mcollective
> > approach.
>
> How do you imagine displaying data for larger numbers of hosts? One of the
> big UI's that we have left to design is the display of task status; which
> tasks failed on which hosts. We talked a little bit about this in the deep
> dive and it seems like dynflow would need some extension to be able to show
> data about which the execution of steps and where/when failures occurred.
We've already partly addressed that part in Katello (more details in this
screencast http://www.youtube.com/watch?v=cpUz5Ylhkr0). I'm in the middle
of extracting the common logic to separate foreman engine (foreman-tasks),
that will introduce the Task model, as well as optionally the Dynflow runtime
(that Katello will use for start). I should have published first preview
by tomorrow.
>
> >
> > My plan is to create a foreman-dynflow plugin, that will extract the
> > dynflow
> > infrastructure code
> > that is now part of Katello, so that it could be used by other
> > plugins/maybe
> > the core, if
> > they find it interesting.
>
> If we could keep all the code outside core that'd be ideal IMO. At some point
> we're going to move the proxy implementation out to a plugin to prevent
> bloat. That being said, if foretello wants to move to mco for management
> remote execution then it might make sense to move the base code into core.
> We've have to play that by ear.
For Katello, the plan is to use Dynflow for all orchestrations that happne in the
system (managing calls to Candlepin and Pulp to provide more stability on overview
of what's happning in the system). I believe having that part of Foreman core
would have some interesting potential (some ideas sketched here https://github.com/iNecas/mcoflow/issues/1). But I think it makes sense to start
as a plugin to not mess the foreman core till we know we're stable enough. That
way, we can start with stabilizing in smaller scope and moving into core once
everyone is confident about that)
>
> >
> > Another possible use-case is leveraging Dynflow for background tasks, as
> > it's already async by default, either directly using the thread pool
> > Dynflow
> > has inside, or
> > abstracting other async task tools in case it's appropriate (from this
> > perspective other async
> > execution tool is on the same level of abstraction as mcollective or Pulp
> > task).
>
> Could we wrap execution in sidekiq so the existing worker model will continue
> working?
The communication to MCollective itself is implmeneted by this class
So it would need to implement the mco_run method to call the smart proxy instead of talking
on the message bus directly. The second part is to wake up the dynflow action when the task
is finished on the smart-proxy side (propagating the smart-proxy callback to dynflow).
We're improving the Dynflow to support this propagation and should be possible soon (well,
Christmas closing will influence that a bit and we need to finish the whole Katello orchestration
as well, but a lot of work is shared here: when we improve the Katello orchestration, the Dynflow
gets better as well)
···
----- Original Message -----
> ----- Original Message -----
You can also watch more on how Dynflow is used in Katello in this deep-dive
I will try to reach to some OpenShift folks for brain-picking.
– Ivan
···
----- Original Message -----
Just curious, but how does open shift accomplish this feat? Open shift uses
mcollective as the orchastration tool and has a cli, web and rest api built
on top of it. Surely we can reuse some of that code?
Corey
On Wednesday, December 4, 2013 9:09:53 AM UTC-8, Ivan Necas wrote:
Hi,
I’ve recorded a screencast on a proof of concept of being able to use
Dynflow to manage MCollective commands: sending both single and multiple
commands and receiving the results that can be further processed.
The demo shows a simple sinatra application that runs Dynflow on background
and exposes simple UI for running mcollective commands. The progress of the
tasks
is shown using the built-in Dynflow web console.
The second part of the demo shows how that work can be reused in Katello
(my
branch
already running Dynflow on background). In the Katello, there is already a
remote
commands feature and it’s really easy to plug more implementations.
>
>
> > From: "Bryan Kearney" <bryan.kearney@gmail.com>
> > To: foreman-dev@googlegroups.com
> > Sent: Monday, December 9, 2013 1:35:25 PM
> > Subject: Re: [foreman-dev] Proof of concept: MCollective over Dynflow
> >
> > > Hi,
> > >
> > > The video is available here:
> > >
> > > http://youtu.be/UyO4R7VUFZ4
> > >
> > > The code is available here:
> > >
> > > https://github.com/iNecas/mcoflow
> > >
> > > https://github.com/iNecas/katello/tree/dynflow-mcollective (the Katello's
> > > branch reusing the MCOflow in their existing remote execution feature)
> >
> > >
> > > – Ivan
> > >
> > How does this relate to the work which SAM and Dmitri have been doing?
>
> This would be the underlying workflow engine for building multi-step
> processes that rely on the output of an earlier action to proceed. Upgrading
> the kernel/glibc is a good example of something like this; run yum upgrade
> on certain packages, check if they recommend restart, and if so reboot the
> machine.
>
> We're not quite at the point where we can start implementing multi-step
> workflows, but we should be in the next couple weeks.
Yes, the work done so far on Foreman side was more about support mcollective
in smart-proxy, basic UI for triggering the actions and showing the reports.
This work is more about putting more actions together, reporing the tasks progress,
and possibly having some more sofisticated orchestration mechanism, as discussed
here https://github.com/iNecas/mcoflow/issues/1
It also puts the mcollective integration in context of other integration we have
already in place.
– Ivan
···
----- Original Message -----
> ----- Original Message -----
> > On 12/04/2013 12:09 PM, Ivan Necas wrote:
