Foreman Tasks: plugin for the Foreman tasks management

Hi,

As part of the work of integrating Dynflow into Katello, we needed models
for keeping the track of tasks run in the system and their assignment to
related resources. Since the work is not specific to Katello only and other
parts of Foreman might be interested in it (especially with growing number
of features that require async processing), I've extracted the common logic
into a plugin. For now, you can find it here:

https://github.com/iNecas/foreman-tasks

I've recorded a screencast on that as well, explaining the main features
and usage:

http://www.youtube.com/watch?v=AqZkoSt8vzA

If you're interested in showing progress of background tasks, auditing,
showing CLI examples for actions of Dynflow integration, you might want
to see that video (sorry for all the coughing, I haven't felt very well
when recording that:)

The idea is this gem would provide the common API for getting the status
of running tasks. The code should be backround-processing-tool agnostic.
For now, we have adapter for Dynflow, as it's plan for using that in Katello's
orchestration.

The Dynflow part there is optional, but it should make it really
easy to start with that if anyone interested (no additional service needed, just
database migration). It might also be interesting to look into the auditing aspect
of this approach: I've spent some time on that in the README and screencast, as well
as some example in the plugin itself (tracking the actions on architecture resource)

See README for more details, I will be offline for some time till end of this year,
therefore I wanted to get something out before that (sorry for missing tests,
that's one of my new years resolutions:)

You can also see the Katello's part in action here

http://www.youtube.com/watch?v=cpUz5Ylhkr0.

There is also a wiki page started for discussions and design around the tasks API.

http://projects.theforeman.org/projects/foreman/wiki/_foreman_tasks

Ad. relation to foreman-background plugin (https://github.com/ohadlevy/foreman-background):
The foreman-background provides just basic wrapper around Sidekiq for asynchronous processing
of some operations. The sidekiq part might be probably part of Foreman Tasks as well (on the
same level as Dynflow). It would be also nice to evaluate, if the Sidekiq is needed and if
Dynflow doesn't already provide enough features around async stuff. I know Sidekiq has some
history that promises being more stable solution at the beginning, but Dynflow is getting
better every commit :). Said that, if anyone is interested in contribution, either in general
API/models stuff, or adapters for other async tools, I will be more than happy to talk to you
and merge the patches.

– Ivan

It looks very promising. Also I like the possibility of auditing all system
actions, not just AR based.

If I understood it correctly, this plugin changes some actions to be async by
monkey patching. Does this mean that foreman-tasks plugin would be requirement
if we'd like to use dynflow solution and all actions would be modified to async
in core? Or would you rather stick with monkey patching which also foreman-
background uses (afaik)?

··· On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote: > Hi, > > As part of the work of integrating Dynflow into Katello, we needed models > for keeping the track of tasks run in the system and their assignment to > related resources. Since the work is not specific to Katello only and other > parts of Foreman might be interested in it (especially with growing number > of features that require async processing), I've extracted the common logic > into a plugin. For now, you can find it here: > > https://github.com/iNecas/foreman-tasks > > I've recorded a screencast on that as well, explaining the main features > and usage: > > http://www.youtube.com/watch?v=AqZkoSt8vzA


Marek

If you’re interested in showing progress of background tasks, auditing,
showing CLI examples for actions of Dynflow integration, you might want
to see that video (sorry for all the coughing, I haven’t felt very well
when recording that:)

The idea is this gem would provide the common API for getting the status
of running tasks. The code should be backround-processing-tool agnostic.
For now, we have adapter for Dynflow, as it’s plan for using that in
Katello’s orchestration.

The Dynflow part there is optional, but it should make it really
easy to start with that if anyone interested (no additional service needed,
just database migration). It might also be interesting to look into the
auditing aspect of this approach: I’ve spent some time on that in the
README and screencast, as well as some example in the plugin itself
(tracking the actions on architecture resource)

See README for more details, I will be offline for some time till end of
this year, therefore I wanted to get something out before that (sorry for
missing tests, that’s one of my new years resolutions:)

You can also see the Katello’s part in action here

http://www.youtube.com/watch?v=cpUz5Ylhkr0.

There is also a wiki page started for discussions and design around the
tasks API.

http://projects.theforeman.org/projects/foreman/wiki/_foreman_tasks

Ad. relation to foreman-background plugin
(https://github.com/ohadlevy/foreman-background): The foreman-background
provides just basic wrapper around Sidekiq for asynchronous processing of
some operations. The sidekiq part might be probably part of Foreman Tasks
as well (on the same level as Dynflow). It would be also nice to evaluate,
if the Sidekiq is needed and if Dynflow doesn’t already provide enough
features around async stuff. I know Sidekiq has some history that promises
being more stable solution at the beginning, but Dynflow is getting better
every commit :). Said that, if anyone is interested in contribution, either
in general API/models stuff, or adapters for other async tools, I will be
more than happy to talk to you and merge the patches.

– Ivan

> > Hi,
> >
> > As part of the work of integrating Dynflow into Katello, we needed models
> > for keeping the track of tasks run in the system and their assignment to
> > related resources. Since the work is not specific to Katello only and other
> > parts of Foreman might be interested in it (especially with growing number
> > of features that require async processing), I've extracted the common logic
> > into a plugin. For now, you can find it here:
> >
> > https://github.com/iNecas/foreman-tasks
> >
> > I've recorded a screencast on that as well, explaining the main features
> > and usage:
> >
> > http://www.youtube.com/watch?v=AqZkoSt8vzA
>
> It looks very promising. Also I like the possibility of auditing all system
> actions, not just AR based.
>
> If I understood it correctly, this plugin changes some actions to be async by
> monkey patching. Does this mean that foreman-tasks plugin would be
> requirement
> if we'd like to use dynflow solution and all actions would be modified to
> async
> in core? Or would you rather stick with monkey patching which also foreman-
> background uses (afaik)?

One note: using Dynflow does not always mean the operation will be async. For
example, for the architectures example, there is nothing happening asynchronously.
Dynflow process has three phases: plan, run, finalize. The plan phase is happening
synchronously, so if you don't have any run,finalize methods in the involved actions,
it's all sync. But it still get's the task, uuid and everything else assigned. It also
gives you an opportunity hooking something that should happen asynchronously later.

Ideally I would like to see bacoming part of Foreman core later. On the other hand,
Foreman has gone quite long way without async oprations and we might work on stabilization
of the solution outside of the core, not affecting the next release of Foreman but allowing
folks to use that if they have a use-case for that (as Katello does). Therefore I take
monkey-patching as temporary solution till we're satisfied with the API/backend stuff.
Katello's integration should help a lot with that.

For the actions as facts import, it probably makes sense making that async in long term,
just for sake of scalability and let the people wait for the task to be finished in the client
tools (such as ENC script). So long term, it would be nice not to have both sync and async version
of the operations, but we need some way to get there.

– Ivan

··· ----- Original Message ----- > On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote:


Marek

If you’re interested in showing progress of background tasks, auditing,
showing CLI examples for actions of Dynflow integration, you might want
to see that video (sorry for all the coughing, I haven’t felt very well
when recording that:)

The idea is this gem would provide the common API for getting the status
of running tasks. The code should be backround-processing-tool agnostic.
For now, we have adapter for Dynflow, as it’s plan for using that in
Katello’s orchestration.

The Dynflow part there is optional, but it should make it really
easy to start with that if anyone interested (no additional service needed,
just database migration). It might also be interesting to look into the
auditing aspect of this approach: I’ve spent some time on that in the
README and screencast, as well as some example in the plugin itself
(tracking the actions on architecture resource)

See README for more details, I will be offline for some time till end of
this year, therefore I wanted to get something out before that (sorry for
missing tests, that’s one of my new years resolutions:)

You can also see the Katello’s part in action here

http://www.youtube.com/watch?v=cpUz5Ylhkr0.

There is also a wiki page started for discussions and design around the
tasks API.

http://projects.theforeman.org/projects/foreman/wiki/_foreman_tasks

Ad. relation to foreman-background plugin
(https://github.com/ohadlevy/foreman-background): The foreman-background
provides just basic wrapper around Sidekiq for asynchronous processing of
some operations. The sidekiq part might be probably part of Foreman Tasks
as well (on the same level as Dynflow). It would be also nice to evaluate,
if the Sidekiq is needed and if Dynflow doesn’t already provide enough
features around async stuff. I know Sidekiq has some history that promises
being more stable solution at the beginning, but Dynflow is getting better
every commit :). Said that, if anyone is interested in contribution, either
in general API/models stuff, or adapters for other async tools, I will be
more than happy to talk to you and merge the patches.

– Ivan


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

> For the sake of argument/completeness: it's possible to use custom
> ActiveSupport::Notifications to implement (among other things)
> auditing/logging. Any reason not to use those? Arguably this would a
> better integration point, as third-part tools and developers are more
> likely to use these.

Could you elaborate a bit more on this approach? Some concerns I have
about using that as integration points is the lack of transparency:
it's not trivial to find what out what happens when you do something:
no one place to see what's going on in the hooks, especially as more
hooks are getting involved. It's also kind hard to
handle "something went wrong" situations, especially if you do something
unrevertable. That's why dynflow has the console to look what's going on in
the task and also persists the state of the process so that one can
fix that and continue (or revert if possible): the persistence is important IMO.

IMO, what's more likely to be used is more question of documentation on how to write
a plugin, than anything else.

– Ivan

··· ----- Original Message -----

-d

On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan mhulan@redhat.com wrote:

On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote:

Hi,

As part of the work of integrating Dynflow into Katello, we needed models
for keeping the track of tasks run in the system and their assignment to
related resources. Since the work is not specific to Katello only and
other
parts of Foreman might be interested in it (especially with growing number
of features that require async processing), I’ve extracted the common
logic
into a plugin. For now, you can find it here:

https://github.com/iNecas/foreman-tasks

I’ve recorded a screencast on that as well, explaining the main features
and usage:

http://www.youtube.com/watch?v=AqZkoSt8vzA

It looks very promising. Also I like the possibility of auditing all system
actions, not just AR based.

If I understood it correctly, this plugin changes some actions to be async
by
monkey patching. Does this mean that foreman-tasks plugin would be
requirement
if we’d like to use dynflow solution and all actions would be modified to
async
in core? Or would you rather stick with monkey patching which also foreman-
background uses (afaik)?


Marek

If you’re interested in showing progress of background tasks, auditing,
showing CLI examples for actions of Dynflow integration, you might want
to see that video (sorry for all the coughing, I haven’t felt very well
when recording that:)

The idea is this gem would provide the common API for getting the status
of running tasks. The code should be backround-processing-tool agnostic.
For now, we have adapter for Dynflow, as it’s plan for using that in
Katello’s orchestration.

The Dynflow part there is optional, but it should make it really
easy to start with that if anyone interested (no additional service
needed,
just database migration). It might also be interesting to look into the
auditing aspect of this approach: I’ve spent some time on that in the
README and screencast, as well as some example in the plugin itself
(tracking the actions on architecture resource)

See README for more details, I will be offline for some time till end of
this year, therefore I wanted to get something out before that (sorry for
missing tests, that’s one of my new years resolutions:)

You can also see the Katello’s part in action here

http://www.youtube.com/watch?v=cpUz5Ylhkr0.

There is also a wiki page started for discussions and design around the
tasks API.

http://projects.theforeman.org/projects/foreman/wiki/_foreman_tasks

Ad. relation to foreman-background plugin
(https://github.com/ohadlevy/foreman-background): The foreman-background
provides just basic wrapper around Sidekiq for asynchronous processing of
some operations. The sidekiq part might be probably part of Foreman Tasks
as well (on the same level as Dynflow). It would be also nice to evaluate,
if the Sidekiq is needed and if Dynflow doesn’t already provide enough
features around async stuff. I know Sidekiq has some history that promises
being more stable solution at the beginning, but Dynflow is getting better
every commit :). Said that, if anyone is interested in contribution,
either
in general API/models stuff, or adapters for other async tools, I will be
more than happy to talk to you and merge the patches.

– Ivan


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

For the sake of argument/completeness: it's possible to use custom
ActiveSupport::Notifications to implement (among other things)
auditing/logging. Any reason not to use those? Arguably this would a
better integration point, as third-part tools and developers are more
likely to use these.

-d

··· On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan wrote: > On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote: >> Hi, >> >> As part of the work of integrating Dynflow into Katello, we needed models >> for keeping the track of tasks run in the system and their assignment to >> related resources. Since the work is not specific to Katello only and other >> parts of Foreman might be interested in it (especially with growing number >> of features that require async processing), I've extracted the common logic >> into a plugin. For now, you can find it here: >> >> https://github.com/iNecas/foreman-tasks >> >> I've recorded a screencast on that as well, explaining the main features >> and usage: >> >> http://www.youtube.com/watch?v=AqZkoSt8vzA > > It looks very promising. Also I like the possibility of auditing all system > actions, not just AR based. > > If I understood it correctly, this plugin changes some actions to be async by > monkey patching. Does this mean that foreman-tasks plugin would be requirement > if we'd like to use dynflow solution and all actions would be modified to async > in core? Or would you rather stick with monkey patching which also foreman- > background uses (afaik)? > > -- > Marek > >> >> If you're interested in showing progress of background tasks, auditing, >> showing CLI examples for actions of Dynflow integration, you might want >> to see that video (sorry for all the coughing, I haven't felt very well >> when recording that:) >> >> The idea is this gem would provide the common API for getting the status >> of running tasks. The code should be backround-processing-tool agnostic. >> For now, we have adapter for Dynflow, as it's plan for using that in >> Katello's orchestration. >> >> The Dynflow part there is optional, but it should make it really >> easy to start with that if anyone interested (no additional service needed, >> just database migration). It might also be interesting to look into the >> auditing aspect of this approach: I've spent some time on that in the >> README and screencast, as well as some example in the plugin itself >> (tracking the actions on architecture resource) >> >> See README for more details, I will be offline for some time till end of >> this year, therefore I wanted to get something out before that (sorry for >> missing tests, that's one of my new years resolutions:) >> >> You can also see the Katello's part in action here >> >> http://www.youtube.com/watch?v=cpUz5Ylhkr0. >> >> There is also a wiki page started for discussions and design around the >> tasks API. >> >> http://projects.theforeman.org/projects/foreman/wiki/_foreman_tasks >> >> Ad. relation to foreman-background plugin >> (https://github.com/ohadlevy/foreman-background): The foreman-background >> provides just basic wrapper around Sidekiq for asynchronous processing of >> some operations. The sidekiq part might be probably part of Foreman Tasks >> as well (on the same level as Dynflow). It would be also nice to evaluate, >> if the Sidekiq is needed and if Dynflow doesn't already provide enough >> features around async stuff. I know Sidekiq has some history that promises >> being more stable solution at the beginning, but Dynflow is getting better >> every commit :). Said that, if anyone is interested in contribution, either >> in general API/models stuff, or adapters for other async tools, I will be >> more than happy to talk to you and merge the patches. >> >> -- Ivan > > -- > You received this message because you are subscribed to the Google Groups "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out.

> It looks very promising. Also I like the possibility of auditing all system
> actions, not just AR based.
>

Agreed - we've spoken before about storing the exact
templates/parameters/etc that was used to provision a machine at a certain
point in time. This might be a solution for that problem.

> If I understood it correctly, this plugin changes some actions to be async
> by
> monkey patching. Does this mean that foreman-tasks plugin would be
> requirement
> if we'd like to use dynflow solution and all actions would be modified to
> async
> in core? Or would you rather stick with monkey patching which also foreman-
> background uses (afaik)?
>

As I understand it, it's proof-of-concept, hence the patching. I'm going to
try adding the stuff I'm working on to the gem, as a further
proof-of-concept. Should be fun :stuck_out_tongue:

Long-term, I thing it makes sense to implement this in core, and have the
internals of foreman_tasks be able to select a background work, or execute
synchronously if no background worker is available.

This is really cool - inecas++ :slight_smile:

Greg

··· On 16 December 2013 10:43, Marek Hulan wrote:

> > It looks very promising. Also I like the possibility of auditing all system
>
> > actions, not just AR based.
>

> Agreed - we've spoken before about storing the exact templates/parameters/etc
> that was used to provision a machine at a certain point in time. This might
> be a solution for that problem.

Really interesting idea! Also, if we modeled the provisioning process using the Dynflow
style (breaking the whole provisioning process into more steps,
we would get a lot of the stored state for free (+ better overview of what's
going on while provisioning and ability to deal with situations, when something
goes wrong: I've talked about that plan some time ago here
https://github.com/iNecas/mcoflow/issues/1#issuecomment-30316535 ).

Even more interesting it's getting in combination with this idea:
https://github.com/iNecas/mcoflow/issues/1#issuecomment-30316535. Let's say
you would be able to take the serialized state of various hosts from the audit, put it into
a template with mcollective actions as well and hit the magic run button:
might be quite interesting feature for Foreman 2.0 :slight_smile:

> > If I understood it correctly, this plugin changes some actions to be async
> > by
>
> > monkey patching. Does this mean that foreman-tasks plugin would be
> > requirement
>
> > if we'd like to use dynflow solution and all actions would be modified to
> > async
>
> > in core? Or would you rather stick with monkey patching which also foreman-
>
> > background uses (afaik)?
>

> As I understand it, it's proof-of-concept, hence the patching. I'm going to
> try adding the stuff I'm working on to the gem, as a further
> proof-of-concept. Should be fun :stuck_out_tongue:

> Long-term, I thing it makes sense to implement this in core, and have the
> internals of foreman_tasks be able to select a background work, or execute
> synchronously if no background worker is available.

That's right. There are main mottos of Dynflow: no global state and don't stick
to one implementation: we use a concept of a world for both of them, that determines what's going to
be used for persistence, execution, transactions, logging etc. There can be (read there soon will be)
a world for testing that will make it easier to write tests without much stubbing.
Therefore the thing that actually executes the workflow (executor) can be changed as well, it just
needs to know, how to process the execution plan.

We started with really simple,
synchronous one. As the time went and features were added, we added another on: parallel executor,
which is able to run the actions concurrently, if they allow that. The async executor
also looks like good tool for dealing with syncing witch external tasks, such as Pulp or Mcollective
is. So, for some features, the parallel executor is needed, but for the basic one, even
the sync one could be enough, although you would loose some features and therefore not
everything would work.

We have abandoned the sync executor as the parallel one get more stable.
We could either reintroduce it for a subset of tasks, but personally I would vote for having
keeping always the parallel one and just make sure it works even without an external worker
in production: it already does in development (therefore you don't need any additional worker
to run async tasks in foreman-tasks, even the code reloading works so you don't have to
restart the server when you edit the dynflow action), for now there is a limitation of running
only one executor, so in production, where there are multiple processes handling the web requests,
the executor needs to run in separate worker, but I want to get rid of this limitation soon. On
the other hand, having the workers separated from the web server part has it's advantages as well (such as being
able to restart the web server without worrying much about the async tasks, although we
try to make sure we're able to recover from the situations when the executor is killed prematurely)
and I would still prefer that one).

··· ----- Original Message ----- > On 16 December 2013 10:43, Marek Hulan < mhulan@redhat.com > wrote:

This is really cool - inecas++ :slight_smile:

Greg


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out .

Not sure what you mean by lack of transparency: this is a rather
standard publisher-subscriber system (independent of AR callbacks)
that Rails relies on for stuff like performance monitoring, sql
queries monitoring, etc. It's pretty trivial to create custom messages
and listeners for those.

I'd think this is a better integration route as far as
logging/auditing goes, as we are more likely to see developers use
existing api (for example that's what Relic uses), etc, etc.
-d

··· On Mon, Dec 16, 2013 at 1:46 PM, Ivan Necas wrote: > > > ----- Original Message ----- >> For the sake of argument/completeness: it's possible to use custom >> ActiveSupport::Notifications to implement (among other things) >> auditing/logging. Any reason not to use those? Arguably this would a >> better integration point, as third-part tools and developers are more >> likely to use these. > > Could you elaborate a bit more on this approach? Some concerns I have > about using that as integration points is the lack of transparency: > it's not trivial to find what out what happens when you do something: > no one place to see what's going on in the hooks, especially as more > hooks are getting involved. It's also kind hard to > handle "something went wrong" situations, especially if you do something > unrevertable. That's why dynflow has the console to look what's going on in > the task and also persists the state of the process so that one can > fix that and continue (or revert if possible): the persistence is important IMO. > > IMO, what's more likely to be used is more question of documentation on how to write > a plugin, than anything else. > > -- Ivan > >> >> -d >> >> On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan wrote: >> > On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote: >> >> Hi, >> >> >> >> As part of the work of integrating Dynflow into Katello, we needed models >> >> for keeping the track of tasks run in the system and their assignment to >> >> related resources. Since the work is not specific to Katello only and >> >> other >> >> parts of Foreman might be interested in it (especially with growing number >> >> of features that require async processing), I've extracted the common >> >> logic >> >> into a plugin. For now, you can find it here: >> >> >> >> https://github.com/iNecas/foreman-tasks >> >> >> >> I've recorded a screencast on that as well, explaining the main features >> >> and usage: >> >> >> >> http://www.youtube.com/watch?v=AqZkoSt8vzA >> > >> > It looks very promising. Also I like the possibility of auditing all system >> > actions, not just AR based. >> > >> > If I understood it correctly, this plugin changes some actions to be async >> > by >> > monkey patching. Does this mean that foreman-tasks plugin would be >> > requirement >> > if we'd like to use dynflow solution and all actions would be modified to >> > async >> > in core? Or would you rather stick with monkey patching which also foreman- >> > background uses (afaik)? >> > >> > -- >> > Marek >> > >> >> >> >> If you're interested in showing progress of background tasks, auditing, >> >> showing CLI examples for actions of Dynflow integration, you might want >> >> to see that video (sorry for all the coughing, I haven't felt very well >> >> when recording that:) >> >> >> >> The idea is this gem would provide the common API for getting the status >> >> of running tasks. The code should be backround-processing-tool agnostic. >> >> For now, we have adapter for Dynflow, as it's plan for using that in >> >> Katello's orchestration. >> >> >> >> The Dynflow part there is optional, but it should make it really >> >> easy to start with that if anyone interested (no additional service >> >> needed, >> >> just database migration). It might also be interesting to look into the >> >> auditing aspect of this approach: I've spent some time on that in the >> >> README and screencast, as well as some example in the plugin itself >> >> (tracking the actions on architecture resource) >> >> >> >> See README for more details, I will be offline for some time till end of >> >> this year, therefore I wanted to get something out before that (sorry for >> >> missing tests, that's one of my new years resolutions:) >> >> >> >> You can also see the Katello's part in action here >> >> >> >> http://www.youtube.com/watch?v=cpUz5Ylhkr0. >> >> >> >> There is also a wiki page started for discussions and design around the >> >> tasks API. >> >> >> >> http://projects.theforeman.org/projects/foreman/wiki/_foreman_tasks >> >> >> >> Ad. relation to foreman-background plugin >> >> (https://github.com/ohadlevy/foreman-background): The foreman-background >> >> provides just basic wrapper around Sidekiq for asynchronous processing of >> >> some operations. The sidekiq part might be probably part of Foreman Tasks >> >> as well (on the same level as Dynflow). It would be also nice to evaluate, >> >> if the Sidekiq is needed and if Dynflow doesn't already provide enough >> >> features around async stuff. I know Sidekiq has some history that promises >> >> being more stable solution at the beginning, but Dynflow is getting better >> >> every commit :). Said that, if anyone is interested in contribution, >> >> either >> >> in general API/models stuff, or adapters for other async tools, I will be >> >> more than happy to talk to you and merge the patches. >> >> >> >> -- Ivan >> > >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "foreman-dev" group. >> > To unsubscribe from this group and stop receiving emails from it, send an >> > email to foreman-dev+unsubscribe@googlegroups.com. >> > For more options, visit https://groups.google.com/groups/opt_out. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "foreman-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to foreman-dev+unsubscribe@googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- > You received this message because you are subscribed to the Google Groups "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out.

My two cents. I'll describe how I see these layers / systems logically
structured and what would be the main benefits of this approach.

It is touching the following:

  • plugins
  • user notifications
  • auditing
  • business logic (processes)
  • ActiveRecord models

Business Logic and ActiveRecord models

Dynflow is a kind of workflow engine where business processes are
composed from dynflow-actions. It would be beneficial approach to have
all actions modifying application's data captured as Dynflow actions
where ActiveRecord models would serve only as a lower-level layer for
reading and modifying data but would be business-logic-free (which would
help with fat models - got objects a lot).

Any change in the application would be triggered by dynflow-action, for
data reading ActiveRecord models would be used as usual.

Let's assume we have logically divided responsibilities as above and
whole business logic is modeled with Dynflow actions form now on.

Plugins

Plug-ability is a base feature of Dynflow and its actions. Plugins can
subscribe to any Dynflow Action and since there are actions for
everything it is creating complete API for plugins.

Actions also have in/output format definited to be able to document it
and validate. It will have to keep compatibility. (Validation is not
enabled yet.)

Audit

Combination of stored Dynflow ExecutionPlans and foreman-tasks's Locks
provides complete human-readable (and also very detailed if needed)
audit of all actions.

User Notifications

Action triggering creates natural point to integrate notifications
reusing audit humanized messages. I think I've mentioned that in some
previous email. It would be very easy to slip in a layer to be able to
tie notifications to actions saying:

  • don't show (e.g. when coming form CLI)
  • show (e.g. when sync task coming from UI)
  • show only when failed (e.g. background task form UI)

Summary:

  • there is layer representing whole applications business logic
  • light ActiveRecord models
  • complete documented plugin API
  • complete audit of all actions
  • easy integration of notification

A/sync processing Note

Plan phase is always synchronous and it always happens in same Thread as
called. Plan phase is very short. Run and Finalize phase is always
executed in Dynflow thread pool and can be executed a/synchronously
depending on what is needed - basically when triggering action a
developer can choose if he wants trigger call to block or return
immediately after plan phase.

Scheduling Note

Let pulp to handle scheduling should be only temporary. Foreman-tasks
provides sophisticated locking mechanism to avoid race conditions on
involved resources. If it runs outside the dynflow system it can cause
unexpected problems.

I've already implemented effective timer in Dynflow to able to schedule
pooling of suspended actions. If it's used it should be matter of one or
two days to write more general task scheduler for Foreman.

In conclusion

So I would see the Architecture integration in foreman-tasks as
temporary solution to connect architecture model with actions.
Integration via callbacks gives quick way how to connect it together
providing most of the benefits mentioned above (complete plugin API,
audit, notifications). It is missing the business shift from models to
actions. This does not probably matter much for simple sets like
Architecture but it should not be ever used in any of the more
complicated models.

Personally I think it would be not a good approach to brake this rule
even for simple sets like Architectures. I would be for creating
abstract Actions for CRUD operations over simple models like
Architecture obeying the rule about business logic shift. (I know this
was not possible for this screencast because there was little time.)

One of the possible problems with this rule violation could be: Let's
have a more complicated action called 'InTrouble' which needs to create
architecture. A developer would naturally try to plan action
Architecture::Create but it won't do anything. The action server only as
an event holder it does not do anything by itself. To make it work a
developer would have to create new architecture in run phase of action
InTrouble, which would trigger this event on its own without connecting
Architecture::Create to InTrouble. This will result in unreadable
shattered audits, unwanted notifications and probably other problems.

Sorry for the long email I wanted to describe how I perceive it, how
this could all work together so we can discuss more and come to
conclusions what will be the best.

Petr

··· On 16.12.13 14:46, Ivan Necas wrote: > > > ----- Original Message ----- >> For the sake of argument/completeness: it's possible to use custom >> ActiveSupport::Notifications to implement (among other things) >> auditing/logging. Any reason not to use those? Arguably this would a >> better integration point, as third-part tools and developers are more >> likely to use these. > > Could you elaborate a bit more on this approach? Some concerns I have > about using that as integration points is the lack of transparency: > it's not trivial to find what out what happens when you do something: > no one place to see what's going on in the hooks, especially as more > hooks are getting involved. It's also kind hard to > handle "something went wrong" situations, especially if you do something > unrevertable. That's why dynflow has the console to look what's going on in > the task and also persists the state of the process so that one can > fix that and continue (or revert if possible): the persistence is important IMO. > > IMO, what's more likely to be used is more question of documentation on how to write > a plugin, than anything else. > > -- Ivan > >> >> -d >> >> On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan wrote: >>> On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote: >>>> Hi, >>>> >>>> As part of the work of integrating Dynflow into Katello, we needed models >>>> for keeping the track of tasks run in the system and their assignment to >>>> related resources. Since the work is not specific to Katello only and >>>> other >>>> parts of Foreman might be interested in it (especially with growing number >>>> of features that require async processing), I've extracted the common >>>> logic >>>> into a plugin. For now, you can find it here: >>>> >>>> https://github.com/iNecas/foreman-tasks >>>> >>>> I've recorded a screencast on that as well, explaining the main features >>>> and usage: >>>> >>>> http://www.youtube.com/watch?v=AqZkoSt8vzA >>> >>> It looks very promising. Also I like the possibility of auditing all system >>> actions, not just AR based. >>> >>> If I understood it correctly, this plugin changes some actions to be async >>> by >>> monkey patching. Does this mean that foreman-tasks plugin would be >>> requirement >>> if we'd like to use dynflow solution and all actions would be modified to >>> async >>> in core? Or would you rather stick with monkey patching which also foreman- >>> background uses (afaik)? >>> >>> -- >>> Marek >>> >>>> >>>> If you're interested in showing progress of background tasks, auditing, >>>> showing CLI examples for actions of Dynflow integration, you might want >>>> to see that video (sorry for all the coughing, I haven't felt very well >>>> when recording that:) >>>> >>>> The idea is this gem would provide the common API for getting the status >>>> of running tasks. The code should be backround-processing-tool agnostic. >>>> For now, we have adapter for Dynflow, as it's plan for using that in >>>> Katello's orchestration. >>>> >>>> The Dynflow part there is optional, but it should make it really >>>> easy to start with that if anyone interested (no additional service >>>> needed, >>>> just database migration). It might also be interesting to look into the >>>> auditing aspect of this approach: I've spent some time on that in the >>>> README and screencast, as well as some example in the plugin itself >>>> (tracking the actions on architecture resource) >>>> >>>> See README for more details, I will be offline for some time till end of >>>> this year, therefore I wanted to get something out before that (sorry for >>>> missing tests, that's one of my new years resolutions:) >>>> >>>> You can also see the Katello's part in action here >>>> >>>> http://www.youtube.com/watch?v=cpUz5Ylhkr0. >>>> >>>> There is also a wiki page started for discussions and design around the >>>> tasks API. >>>> >>>> http://projects.theforeman.org/projects/foreman/wiki/_foreman_tasks >>>> >>>> Ad. relation to foreman-background plugin >>>> (https://github.com/ohadlevy/foreman-background): The foreman-background >>>> provides just basic wrapper around Sidekiq for asynchronous processing of >>>> some operations. The sidekiq part might be probably part of Foreman Tasks >>>> as well (on the same level as Dynflow). It would be also nice to evaluate, >>>> if the Sidekiq is needed and if Dynflow doesn't already provide enough >>>> features around async stuff. I know Sidekiq has some history that promises >>>> being more stable solution at the beginning, but Dynflow is getting better >>>> every commit :). Said that, if anyone is interested in contribution, >>>> either >>>> in general API/models stuff, or adapters for other async tools, I will be >>>> more than happy to talk to you and merge the patches. >>>> >>>> -- Ivan >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "foreman-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to foreman-dev+unsubscribe@googlegroups.com. >>> For more options, visit https://groups.google.com/groups/opt_out. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "foreman-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to foreman-dev+unsubscribe@googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> >

> My two cents. I'll describe how I see these layers / systems logically
> structured and what would be the main benefits of this approach.
>
> # It is touching the following:
> - plugins
> - user notifications
> - auditing
> - business logic (processes)
> - ActiveRecord models

Auditing/logging shouldn't be a layer – it's orthogonal to the rest
of the system (a concern in AOP terminology).

>
> ## Business Logic and ActiveRecord models
>
> Dynflow is a kind of workflow engine where business processes are composed
> from dynflow-actions. It would be beneficial approach to have all actions
> modifying application's data captured as Dynflow actions where ActiveRecord
> models would serve only as a lower-level layer for reading and modifying
> data but would be business-logic-free (which would help with fat models -
> got objects a lot).
>
> Any change in the application would be triggered by dynflow-action, for data
> reading ActiveRecord models would be used as usual.
>
> Let's assume we have logically divided responsibilities as above and whole
> business logic is modeled with Dynflow actions form now on.
>
> ## Plugins
>
> Plug-ability is a base feature of Dynflow and its actions. Plugins can
> subscribe to any Dynflow Action and since there are actions for everything
> it is creating complete API for plugins.
>
> Actions also have in/output format definited to be able to document it and
> validate. It will have to keep compatibility. (Validation is not enabled
> yet.)
>
> ## Audit
>
> Combination of stored Dynflow ExecutionPlans and foreman-tasks's Locks
> provides complete human-readable (and also very detailed if needed) audit of
> all actions.
>
> ## User Notifications
>
> Action triggering creates natural point to integrate notifications reusing
> audit humanized messages. I think I've mentioned that in some previous
> email. It would be very easy to slip in a layer to be able to tie
> notifications to actions saying:
> - don't show (e.g. when coming form CLI)
> - show (e.g. when sync task coming from UI)
> - show only when failed (e.g. background task form UI)
>
> ## Summary:
>
> - there is layer representing whole applications business logic
> - light ActiveRecord models
> - complete documented plugin API
> - complete audit of all actions
> - easy integration of notification
>
> # A/sync processing Note
>
> Plan phase is always synchronous and it always happens in same Thread as
> called. Plan phase is very short. Run and Finalize phase is always executed
> in Dynflow thread pool and can be executed a/synchronously depending on what
> is needed - basically when triggering action a developer can choose if he
> wants trigger call to block or return immediately after plan phase.
>
> # Scheduling Note
>
> Let pulp to handle scheduling should be only temporary. Foreman-tasks
> provides sophisticated locking mechanism to avoid race conditions on
> involved resources. If it runs outside the dynflow system it can cause
> unexpected problems.

I don't think foreman should be controlling access to resources
controlled by other systems, this severely impacts both systems, how
they are deployed and used.
-d

··· On Tue, Dec 17, 2013 at 11:59 AM, Petr Chalupa wrote:

I’ve already implemented effective timer in Dynflow to able to schedule
pooling of suspended actions. If it’s used it should be matter of one or two
days to write more general task scheduler for Foreman.

In conclusion

So I would see the Architecture integration in foreman-tasks as temporary
solution to connect architecture model with actions. Integration via
callbacks gives quick way how to connect it together providing most of the
benefits mentioned above (complete plugin API, audit, notifications). It is
missing the business shift from models to actions. This does not probably
matter much for simple sets like Architecture but it should not be ever used
in any of the more complicated models.

Personally I think it would be not a good approach to brake this rule even
for simple sets like Architectures. I would be for creating abstract Actions
for CRUD operations over simple models like Architecture obeying the rule
about business logic shift. (I know this was not possible for this
screencast because there was little time.)

One of the possible problems with this rule violation could be: Let’s have a
more complicated action called ‘InTrouble’ which needs to create
architecture. A developer would naturally try to plan action
Architecture::Create but it won’t do anything. The action server only as an
event holder it does not do anything by itself. To make it work a developer
would have to create new architecture in run phase of action InTrouble,
which would trigger this event on its own without connecting
Architecture::Create to InTrouble. This will result in unreadable shattered
audits, unwanted notifications and probably other problems.

Sorry for the long email I wanted to describe how I perceive it, how this
could all work together so we can discuss more and come to conclusions what
will be the best.

Petr

On 16.12.13 14:46, Ivan Necas wrote:

----- Original Message -----

For the sake of argument/completeness: it’s possible to use custom
ActiveSupport::Notifications to implement (among other things)
auditing/logging. Any reason not to use those? Arguably this would a
better integration point, as third-part tools and developers are more
likely to use these.

Could you elaborate a bit more on this approach? Some concerns I have
about using that as integration points is the lack of transparency:
it’s not trivial to find what out what happens when you do something:
no one place to see what’s going on in the hooks, especially as more
hooks are getting involved. It’s also kind hard to
handle “something went wrong” situations, especially if you do something
unrevertable. That’s why dynflow has the console to look what’s going on
in
the task and also persists the state of the process so that one can
fix that and continue (or revert if possible): the persistence is
important IMO.

IMO, what’s more likely to be used is more question of documentation on
how to write
a plugin, than anything else.

– Ivan

-d

On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan mhulan@redhat.com wrote:

On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote:

Hi,

As part of the work of integrating Dynflow into Katello, we needed
models
for keeping the track of tasks run in the system and their assignment
to
related resources. Since the work is not specific to Katello only and
other
parts of Foreman might be interested in it (especially with growing
number
of features that require async processing), I’ve extracted the common
logic
into a plugin. For now, you can find it here:

 https://github.com/iNecas/foreman-tasks

I’ve recorded a screencast on that as well, explaining the main
features
and usage:

 http://www.youtube.com/watch?v=AqZkoSt8vzA

It looks very promising. Also I like the possibility of auditing all
system
actions, not just AR based.

If I understood it correctly, this plugin changes some actions to be
async
by
monkey patching. Does this mean that foreman-tasks plugin would be
requirement
if we’d like to use dynflow solution and all actions would be modified
to
async
in core? Or would you rather stick with monkey patching which also
foreman-
background uses (afaik)?


Marek

If you’re interested in showing progress of background tasks, auditing,
showing CLI examples for actions of Dynflow integration, you might want
to see that video (sorry for all the coughing, I haven’t felt very well
when recording that:)

The idea is this gem would provide the common API for getting the
status
of running tasks. The code should be backround-processing-tool
agnostic.
For now, we have adapter for Dynflow, as it’s plan for using that in
Katello’s orchestration.

The Dynflow part there is optional, but it should make it really
easy to start with that if anyone interested (no additional service
needed,
just database migration). It might also be interesting to look into the
auditing aspect of this approach: I’ve spent some time on that in the
README and screencast, as well as some example in the plugin itself
(tracking the actions on architecture resource)

See README for more details, I will be offline for some time till end
of
this year, therefore I wanted to get something out before that (sorry
for
missing tests, that’s one of my new years resolutions:)

You can also see the Katello’s part in action here

 http://www.youtube.com/watch?v=cpUz5Ylhkr0.

There is also a wiki page started for discussions and design around the
tasks API.

foreman tasks - Foreman

Ad. relation to foreman-background plugin
(https://github.com/ohadlevy/foreman-background): The
foreman-background
provides just basic wrapper around Sidekiq for asynchronous processing
of
some operations. The sidekiq part might be probably part of Foreman
Tasks
as well (on the same level as Dynflow). It would be also nice to
evaluate,
if the Sidekiq is needed and if Dynflow doesn’t already provide enough
features around async stuff. I know Sidekiq has some history that
promises
being more stable solution at the beginning, but Dynflow is getting
better
every commit :). Said that, if anyone is interested in contribution,
either
in general API/models stuff, or adapters for other async tools, I will
be
more than happy to talk to you and merge the patches.

– Ivan


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

>> My two cents. I'll describe how I see these layers / systems logically
>> structured and what would be the main benefits of this approach.
>>
>> # It is touching the following:
>> - plugins
>> - user notifications
>> - auditing
>> - business logic (processes)
>> - ActiveRecord models
>
> Auditing/logging shouldn't be a layer – it's orthogonal to the rest
> of the system (a concern in AOP terminology).

Could you elaborate? I am aware of AOP but we do not generally use it. I
would like to avoid wild guessing what issue you are trying to point out
or what concept you would like to apply. Thanks.

>
>>
>> ## Business Logic and ActiveRecord models
>>
>> Dynflow is a kind of workflow engine where business processes are composed
>> from dynflow-actions. It would be beneficial approach to have all actions
>> modifying application's data captured as Dynflow actions where ActiveRecord
>> models would serve only as a lower-level layer for reading and modifying
>> data but would be business-logic-free (which would help with fat models -
>> got objects a lot).
>>
>> Any change in the application would be triggered by dynflow-action, for data
>> reading ActiveRecord models would be used as usual.
>>
>> Let's assume we have logically divided responsibilities as above and whole
>> business logic is modeled with Dynflow actions form now on.
>>
>> ## Plugins
>>
>> Plug-ability is a base feature of Dynflow and its actions. Plugins can
>> subscribe to any Dynflow Action and since there are actions for everything
>> it is creating complete API for plugins.
>>
>> Actions also have in/output format definited to be able to document it and
>> validate. It will have to keep compatibility. (Validation is not enabled
>> yet.)
>>
>> ## Audit
>>
>> Combination of stored Dynflow ExecutionPlans and foreman-tasks's Locks
>> provides complete human-readable (and also very detailed if needed) audit of
>> all actions.
>>
>> ## User Notifications
>>
>> Action triggering creates natural point to integrate notifications reusing
>> audit humanized messages. I think I've mentioned that in some previous
>> email. It would be very easy to slip in a layer to be able to tie
>> notifications to actions saying:
>> - don't show (e.g. when coming form CLI)
>> - show (e.g. when sync task coming from UI)
>> - show only when failed (e.g. background task form UI)
>>
>> ## Summary:
>>
>> - there is layer representing whole applications business logic
>> - light ActiveRecord models
>> - complete documented plugin API
>> - complete audit of all actions
>> - easy integration of notification
>>
>> # A/sync processing Note
>>
>> Plan phase is always synchronous and it always happens in same Thread as
>> called. Plan phase is very short. Run and Finalize phase is always executed
>> in Dynflow thread pool and can be executed a/synchronously depending on what
>> is needed - basically when triggering action a developer can choose if he
>> wants trigger call to block or return immediately after plan phase.
>>
>> # Scheduling Note
>>
>> Let pulp to handle scheduling should be only temporary. Foreman-tasks
>> provides sophisticated locking mechanism to avoid race conditions on
>> involved resources. If it runs outside the dynflow system it can cause
>> unexpected problems.
>
> I don't think foreman should be controlling access to resources
> controlled by other systems, this severely impacts both systems, how
> they are deployed and used.
> -d

I am not sure what you mean exactly. Foreman would not be controlling
access a Lock prevents resource-changes, resource can be read freely.
How does that severely impacts the systems? Please share an example. My
note was about triggering scheduled tasks (e.g. repo-synchronizations).
Only change is that a repo-synchronization would be triggered by Katello
instead by Pulp.

··· On 17.12.13 13:26, Dmitri Dolguikh wrote: > On Tue, Dec 17, 2013 at 11:59 AM, Petr Chalupa wrote:

I’ve already implemented effective timer in Dynflow to able to schedule
pooling of suspended actions. If it’s used it should be matter of one or two
days to write more general task scheduler for Foreman.

In conclusion

So I would see the Architecture integration in foreman-tasks as temporary
solution to connect architecture model with actions. Integration via
callbacks gives quick way how to connect it together providing most of the
benefits mentioned above (complete plugin API, audit, notifications). It is
missing the business shift from models to actions. This does not probably
matter much for simple sets like Architecture but it should not be ever used
in any of the more complicated models.

Personally I think it would be not a good approach to brake this rule even
for simple sets like Architectures. I would be for creating abstract Actions
for CRUD operations over simple models like Architecture obeying the rule
about business logic shift. (I know this was not possible for this
screencast because there was little time.)

One of the possible problems with this rule violation could be: Let’s have a
more complicated action called ‘InTrouble’ which needs to create
architecture. A developer would naturally try to plan action
Architecture::Create but it won’t do anything. The action server only as an
event holder it does not do anything by itself. To make it work a developer
would have to create new architecture in run phase of action InTrouble,
which would trigger this event on its own without connecting
Architecture::Create to InTrouble. This will result in unreadable shattered
audits, unwanted notifications and probably other problems.

Sorry for the long email I wanted to describe how I perceive it, how this
could all work together so we can discuss more and come to conclusions what
will be the best.

Petr

On 16.12.13 14:46, Ivan Necas wrote:

----- Original Message -----

For the sake of argument/completeness: it’s possible to use custom
ActiveSupport::Notifications to implement (among other things)
auditing/logging. Any reason not to use those? Arguably this would a
better integration point, as third-part tools and developers are more
likely to use these.

Could you elaborate a bit more on this approach? Some concerns I have
about using that as integration points is the lack of transparency:
it’s not trivial to find what out what happens when you do something:
no one place to see what’s going on in the hooks, especially as more
hooks are getting involved. It’s also kind hard to
handle “something went wrong” situations, especially if you do something
unrevertable. That’s why dynflow has the console to look what’s going on
in
the task and also persists the state of the process so that one can
fix that and continue (or revert if possible): the persistence is
important IMO.

IMO, what’s more likely to be used is more question of documentation on
how to write
a plugin, than anything else.

– Ivan

-d

On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan mhulan@redhat.com wrote:

On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote:

Hi,

As part of the work of integrating Dynflow into Katello, we needed
models
for keeping the track of tasks run in the system and their assignment
to
related resources. Since the work is not specific to Katello only and
other
parts of Foreman might be interested in it (especially with growing
number
of features that require async processing), I’ve extracted the common
logic
into a plugin. For now, you can find it here:

  https://github.com/iNecas/foreman-tasks

I’ve recorded a screencast on that as well, explaining the main
features
and usage:

  http://www.youtube.com/watch?v=AqZkoSt8vzA

It looks very promising. Also I like the possibility of auditing all
system
actions, not just AR based.

If I understood it correctly, this plugin changes some actions to be
async
by
monkey patching. Does this mean that foreman-tasks plugin would be
requirement
if we’d like to use dynflow solution and all actions would be modified
to
async
in core? Or would you rather stick with monkey patching which also
foreman-
background uses (afaik)?


Marek

If you’re interested in showing progress of background tasks, auditing,
showing CLI examples for actions of Dynflow integration, you might want
to see that video (sorry for all the coughing, I haven’t felt very well
when recording that:)

The idea is this gem would provide the common API for getting the
status
of running tasks. The code should be backround-processing-tool
agnostic.
For now, we have adapter for Dynflow, as it’s plan for using that in
Katello’s orchestration.

The Dynflow part there is optional, but it should make it really
easy to start with that if anyone interested (no additional service
needed,
just database migration). It might also be interesting to look into the
auditing aspect of this approach: I’ve spent some time on that in the
README and screencast, as well as some example in the plugin itself
(tracking the actions on architecture resource)

See README for more details, I will be offline for some time till end
of
this year, therefore I wanted to get something out before that (sorry
for
missing tests, that’s one of my new years resolutions:)

You can also see the Katello’s part in action here

  http://www.youtube.com/watch?v=cpUz5Ylhkr0.

There is also a wiki page started for discussions and design around the
tasks API.

foreman tasks - Foreman

Ad. relation to foreman-background plugin
(https://github.com/ohadlevy/foreman-background): The
foreman-background
provides just basic wrapper around Sidekiq for asynchronous processing
of
some operations. The sidekiq part might be probably part of Foreman
Tasks
as well (on the same level as Dynflow). It would be also nice to
evaluate,
if the Sidekiq is needed and if Dynflow doesn’t already provide enough
features around async stuff. I know Sidekiq has some history that
promises
being more stable solution at the beginning, but Dynflow is getting
better
every commit :). Said that, if anyone is interested in contribution,
either
in general API/models stuff, or adapters for other async tools, I will
be
more than happy to talk to you and merge the patches.

– Ivan


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

> Not sure what you mean by lack of transparency: this is a rather
> standard publisher-subscriber system (independent of AR callbacks)
> that Rails relies on for stuff like performance monitoring, sql
> queries monitoring, etc. It's pretty trivial to create custom messages
> and listeners for those.

As when you subscribe more listeners to one message, it's quite hard
to figure out what's going on. Real hard times come when two listeners
do some modifying action on external system, and one fails.

We should think about what kind of operations we expect to be performed
on the hooks. Also this should not be limited only to the third-party
extensions. We should be able to apply the same model for the orchestration
itself (such as smart-proxy calls, pulp/candlepin orchestration etc.)

Let's take a simple (but real world) example of deleting a user. In Foreman core, there
is no complicated logic in that. Basically just user.destroy!.

The fun comes, when Katello is plugged in, and it needs to delete users
in Candlepin and Pulp as well. What happens when the Cadnlepin call suceeds
but Pulp call fails? How to deal with a situation, when yet another plugin
subscribes to the custom message (e.g. IdM integration),
performing yet another modifying action.

From Foreman perspective, Katello is just another third-party extension. That
doesn't mean Foreman should not care about consistency in the system.
The administrator should be able to find out something went wrong. There
should be also tools to be able to fix the inconsistency.

>
> I'd think this is a better integration route as far as
> logging/auditing goes, as we are more likely to see developers use
> existing api (for example that's what Relic uses), etc, etc.

This is not relevant for us, as it solves different problem. If we needed
the extension points just to perform some monitoring/logging, than it
would be fine/preferable solution. But we do more than that. And
I've been on Katello project too long to think this lightweight approach
would work for us. That's not that different from what's currently in Katello
(and Foreman) and we learned hard way, that this approach is not sustainable
for this kind of project.

Said that, I don't have problem with having more extension points in the
Foreman (at least till it's clear which approach makes more sense here).
We know Katello is going to use Dynflow as orchestration layer. And if
we can subscribe to custom messages in Foreman instead of monkey-patching,
I have not issues with that.

On the other hand, given all the features that come for free with Dynflow
in place (I will not repeat myself here, Petr also made really nice
summary here https://groups.google.com/d/msg/foreman-dev/K5BYuhmrpJU/hQgsa5RW5FkJ),
it would look strange to me going another direction to solve that issues just because
there are more ways how to do hook into the code to run additional code.

Of course, there is plenty of space for improving Dynflow (such as one
executor limit), but I'm not aware of any unsolvable issue.

– Ivan

··· ----- Original Message -----

-d

On Mon, Dec 16, 2013 at 1:46 PM, Ivan Necas inecas@redhat.com wrote:

----- Original Message -----

For the sake of argument/completeness: it’s possible to use custom
ActiveSupport::Notifications to implement (among other things)
auditing/logging. Any reason not to use those? Arguably this would a
better integration point, as third-part tools and developers are more
likely to use these.

Could you elaborate a bit more on this approach? Some concerns I have
about using that as integration points is the lack of transparency:
it’s not trivial to find what out what happens when you do something:
no one place to see what’s going on in the hooks, especially as more
hooks are getting involved. It’s also kind hard to
handle “something went wrong” situations, especially if you do something
unrevertable. That’s why dynflow has the console to look what’s going on in
the task and also persists the state of the process so that one can
fix that and continue (or revert if possible): the persistence is important
IMO.

IMO, what’s more likely to be used is more question of documentation on how
to write
a plugin, than anything else.

– Ivan

-d

On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan mhulan@redhat.com wrote:

On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote:

Hi,

As part of the work of integrating Dynflow into Katello, we needed
models
for keeping the track of tasks run in the system and their assignment
to
related resources. Since the work is not specific to Katello only and
other
parts of Foreman might be interested in it (especially with growing
number
of features that require async processing), I’ve extracted the common
logic
into a plugin. For now, you can find it here:

https://github.com/iNecas/foreman-tasks

I’ve recorded a screencast on that as well, explaining the main
features
and usage:

http://www.youtube.com/watch?v=AqZkoSt8vzA

It looks very promising. Also I like the possibility of auditing all
system
actions, not just AR based.

If I understood it correctly, this plugin changes some actions to be
async
by
monkey patching. Does this mean that foreman-tasks plugin would be
requirement
if we’d like to use dynflow solution and all actions would be modified
to
async
in core? Or would you rather stick with monkey patching which also
foreman-
background uses (afaik)?


Marek

If you’re interested in showing progress of background tasks, auditing,
showing CLI examples for actions of Dynflow integration, you might want
to see that video (sorry for all the coughing, I haven’t felt very well
when recording that:)

The idea is this gem would provide the common API for getting the
status
of running tasks. The code should be backround-processing-tool
agnostic.
For now, we have adapter for Dynflow, as it’s plan for using that in
Katello’s orchestration.

The Dynflow part there is optional, but it should make it really
easy to start with that if anyone interested (no additional service
needed,
just database migration). It might also be interesting to look into the
auditing aspect of this approach: I’ve spent some time on that in the
README and screencast, as well as some example in the plugin itself
(tracking the actions on architecture resource)

See README for more details, I will be offline for some time till end
of
this year, therefore I wanted to get something out before that (sorry
for
missing tests, that’s one of my new years resolutions:)

You can also see the Katello’s part in action here

http://www.youtube.com/watch?v=cpUz5Ylhkr0.

There is also a wiki page started for discussions and design around the
tasks API.

http://projects.theforeman.org/projects/foreman/wiki/_foreman_tasks

Ad. relation to foreman-background plugin
(https://github.com/ohadlevy/foreman-background): The
foreman-background
provides just basic wrapper around Sidekiq for asynchronous processing
of
some operations. The sidekiq part might be probably part of Foreman
Tasks
as well (on the same level as Dynflow). It would be also nice to
evaluate,
if the Sidekiq is needed and if Dynflow doesn’t already provide enough
features around async stuff. I know Sidekiq has some history that
promises
being more stable solution at the beginning, but Dynflow is getting
better
every commit :). Said that, if anyone is interested in contribution,
either
in general API/models stuff, or adapters for other async tools, I will
be
more than happy to talk to you and merge the patches.

– Ivan


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

As logging/auditing can potentially apply to any part or a layer in a
system. It cannot itself be a layer, as in this case anything below
won't be able to use it. A some sort of publisher/subscriber (Rails 3
ActiveSupport::Notification) or a queue (Rails 4
ActiveSupport::Notification) would work well, as they allow to have
multiple event sinks and event processors.

-d

··· On Wed, Dec 18, 2013 at 10:09 AM, Petr Chalupa wrote: > > > On 17.12.13 13:26, Dmitri Dolguikh wrote: >> >> On Tue, Dec 17, 2013 at 11:59 AM, Petr Chalupa >> wrote: >>> >>> My two cents. I'll describe how I see these layers / systems logically >>> structured and what would be the main benefits of this approach. >>> >>> # It is touching the following: >>> - plugins >>> - user notifications >>> - auditing >>> - business logic (processes) >>> - ActiveRecord models >> >> >> Auditing/logging shouldn't be a layer -- it's orthogonal to the rest >> of the system (a concern in AOP terminology). > > > Could you elaborate? I am aware of AOP but we do not generally use it. I > would like to avoid wild guessing what issue you are trying to point out or > what concept you would like to apply. Thanks. > > >> >>> >>> ## Business Logic and ActiveRecord models >>> >>> Dynflow is a kind of workflow engine where business processes are >>> composed >>> from dynflow-actions. It would be beneficial approach to have all actions >>> modifying application's data captured as Dynflow actions where >>> ActiveRecord >>> models would serve only as a lower-level layer for reading and modifying >>> data but would be business-logic-free (which would help with fat models - >>> got objects a lot). >>> >>> Any change in the application would be triggered by dynflow-action, for >>> data >>> reading ActiveRecord models would be used as usual. >>> >>> Let's assume we have logically divided responsibilities as above and >>> whole >>> business logic is modeled with Dynflow actions form now on. >>> >>> ## Plugins >>> >>> Plug-ability is a base feature of Dynflow and its actions. Plugins can >>> subscribe to any Dynflow Action and since there are actions for >>> everything >>> it is creating complete API for plugins. >>> >>> Actions also have in/output format definited to be able to document it >>> and >>> validate. It will have to keep compatibility. (Validation is not enabled >>> yet.) >>> >>> ## Audit >>> >>> Combination of stored Dynflow ExecutionPlans and foreman-tasks's Locks >>> provides complete human-readable (and also very detailed if needed) audit >>> of >>> all actions. >>> >>> ## User Notifications >>> >>> Action triggering creates natural point to integrate notifications >>> reusing >>> audit humanized messages. I think I've mentioned that in some previous >>> email. It would be very easy to slip in a layer to be able to tie >>> notifications to actions saying: >>> - don't show (e.g. when coming form CLI) >>> - show (e.g. when sync task coming from UI) >>> - show only when failed (e.g. background task form UI) >>> >>> ## Summary: >>> >>> - there is layer representing whole applications business logic >>> - light ActiveRecord models >>> - complete documented plugin API >>> - complete audit of all actions >>> - easy integration of notification >>> >>> # A/sync processing Note >>> >>> Plan phase is always synchronous and it always happens in same Thread as >>> called. Plan phase is very short. Run and Finalize phase is always >>> executed >>> in Dynflow thread pool and can be executed a/synchronously depending on >>> what >>> is needed - basically when triggering action a developer can choose if he >>> wants trigger call to block or return immediately after plan phase. >>> >>> # Scheduling Note >>> >>> Let pulp to handle scheduling should be only temporary. Foreman-tasks >>> provides sophisticated locking mechanism to avoid race conditions on >>> involved resources. If it runs outside the dynflow system it can cause >>> unexpected problems. >> >> >> I don't think foreman should be controlling access to resources >> controlled by other systems, this severely impacts both systems, how >> they are deployed and used. >> -d > > > I am not sure what you mean exactly. Foreman would not be controlling access > a Lock prevents resource-changes, resource can be read freely. How does that > severely impacts the systems? Please share an example. My note was about > triggering scheduled tasks (e.g. repo-synchronizations). Only change is that > a repo-synchronization would be triggered by Katello instead by Pulp. > > >> >>> >>> I've already implemented effective timer in Dynflow to able to schedule >>> pooling of suspended actions. If it's used it should be matter of one or >>> two >>> days to write more general task scheduler for Foreman. >>> >>> # In conclusion >>> >>> So I would see the Architecture integration in foreman-tasks as temporary >>> solution to connect architecture model with actions. Integration via >>> callbacks gives quick way how to connect it together providing most of >>> the >>> benefits mentioned above (complete plugin API, audit, notifications). It >>> is >>> missing the business shift from models to actions. This does not probably >>> matter much for simple sets like Architecture but it should not be ever >>> used >>> in any of the more complicated models. >>> >>> Personally I think it would be not a good approach to brake this rule >>> even >>> for simple sets like Architectures. I would be for creating abstract >>> Actions >>> for CRUD operations over simple models like Architecture obeying the rule >>> about business logic shift. (I know this was not possible for this >>> screencast because there was little time.) >>> >>> One of the possible problems with this rule violation could be: Let's >>> have a >>> more complicated action called 'InTrouble' which needs to create >>> architecture. A developer would naturally try to plan action >>> Architecture::Create but it won't do anything. The action server only as >>> an >>> event holder it does not do anything by itself. To make it work a >>> developer >>> would have to create new architecture in run phase of action InTrouble, >>> which would trigger this event on its own without connecting >>> Architecture::Create to InTrouble. This will result in unreadable >>> shattered >>> audits, unwanted notifications and probably other problems. >>> >>> Sorry for the long email I wanted to describe how I perceive it, how this >>> could all work together so we can discuss more and come to conclusions >>> what >>> will be the best. >>> >>> Petr >>> >>> >>> On 16.12.13 14:46, Ivan Necas wrote: >>>> >>>> >>>> >>>> >>>> ----- Original Message ----- >>>>> >>>>> >>>>> For the sake of argument/completeness: it's possible to use custom >>>>> ActiveSupport::Notifications to implement (among other things) >>>>> auditing/logging. Any reason not to use those? Arguably this would a >>>>> better integration point, as third-part tools and developers are more >>>>> likely to use these. >>>> >>>> >>>> >>>> Could you elaborate a bit more on this approach? Some concerns I have >>>> about using that as integration points is the lack of transparency: >>>> it's not trivial to find what out what happens when you do something: >>>> no one place to see what's going on in the hooks, especially as more >>>> hooks are getting involved. It's also kind hard to >>>> handle "something went wrong" situations, especially if you do something >>>> unrevertable. That's why dynflow has the console to look what's going on >>>> in >>>> the task and also persists the state of the process so that one can >>>> fix that and continue (or revert if possible): the persistence is >>>> important IMO. >>>> >>>> IMO, what's more likely to be used is more question of documentation on >>>> how to write >>>> a plugin, than anything else. >>>> >>>> -- Ivan >>>> >>>>> >>>>> -d >>>>> >>>>> On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan >>>>> wrote: >>>>>> >>>>>> >>>>>> On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote: >>>>>>> >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> As part of the work of integrating Dynflow into Katello, we needed >>>>>>> models >>>>>>> for keeping the track of tasks run in the system and their assignment >>>>>>> to >>>>>>> related resources. Since the work is not specific to Katello only and >>>>>>> other >>>>>>> parts of Foreman might be interested in it (especially with growing >>>>>>> number >>>>>>> of features that require async processing), I've extracted the common >>>>>>> logic >>>>>>> into a plugin. For now, you can find it here: >>>>>>> >>>>>>> https://github.com/iNecas/foreman-tasks >>>>>>> >>>>>>> I've recorded a screencast on that as well, explaining the main >>>>>>> features >>>>>>> and usage: >>>>>>> >>>>>>> http://www.youtube.com/watch?v=AqZkoSt8vzA >>>>>> >>>>>> >>>>>> >>>>>> It looks very promising. Also I like the possibility of auditing all >>>>>> system >>>>>> actions, not just AR based. >>>>>> >>>>>> If I understood it correctly, this plugin changes some actions to be >>>>>> async >>>>>> by >>>>>> monkey patching. Does this mean that foreman-tasks plugin would be >>>>>> requirement >>>>>> if we'd like to use dynflow solution and all actions would be modified >>>>>> to >>>>>> async >>>>>> in core? Or would you rather stick with monkey patching which also >>>>>> foreman- >>>>>> background uses (afaik)? >>>>>> >>>>>> -- >>>>>> Marek >>>>>> >>>>>>> >>>>>>> If you're interested in showing progress of background tasks, >>>>>>> auditing, >>>>>>> showing CLI examples for actions of Dynflow integration, you might >>>>>>> want >>>>>>> to see that video (sorry for all the coughing, I haven't felt very >>>>>>> well >>>>>>> when recording that:) >>>>>>> >>>>>>> The idea is this gem would provide the common API for getting the >>>>>>> status >>>>>>> of running tasks. The code should be backround-processing-tool >>>>>>> agnostic. >>>>>>> For now, we have adapter for Dynflow, as it's plan for using that in >>>>>>> Katello's orchestration. >>>>>>> >>>>>>> The Dynflow part there is optional, but it should make it really >>>>>>> easy to start with that if anyone interested (no additional service >>>>>>> needed, >>>>>>> just database migration). It might also be interesting to look into >>>>>>> the >>>>>>> auditing aspect of this approach: I've spent some time on that in the >>>>>>> README and screencast, as well as some example in the plugin itself >>>>>>> (tracking the actions on architecture resource) >>>>>>> >>>>>>> See README for more details, I will be offline for some time till end >>>>>>> of >>>>>>> this year, therefore I wanted to get something out before that (sorry >>>>>>> for >>>>>>> missing tests, that's one of my new years resolutions:) >>>>>>> >>>>>>> You can also see the Katello's part in action here >>>>>>> >>>>>>> http://www.youtube.com/watch?v=cpUz5Ylhkr0. >>>>>>> >>>>>>> There is also a wiki page started for discussions and design around >>>>>>> the >>>>>>> tasks API. >>>>>>> >>>>>>> >>>>>>> http://projects.theforeman.org/projects/foreman/wiki/_foreman_tasks >>>>>>> >>>>>>> Ad. relation to foreman-background plugin >>>>>>> (https://github.com/ohadlevy/foreman-background): The >>>>>>> foreman-background >>>>>>> provides just basic wrapper around Sidekiq for asynchronous >>>>>>> processing >>>>>>> of >>>>>>> some operations. The sidekiq part might be probably part of Foreman >>>>>>> Tasks >>>>>>> as well (on the same level as Dynflow). It would be also nice to >>>>>>> evaluate, >>>>>>> if the Sidekiq is needed and if Dynflow doesn't already provide >>>>>>> enough >>>>>>> features around async stuff. I know Sidekiq has some history that >>>>>>> promises >>>>>>> being more stable solution at the beginning, but Dynflow is getting >>>>>>> better >>>>>>> every commit :). Said that, if anyone is interested in contribution, >>>>>>> either >>>>>>> in general API/models stuff, or adapters for other async tools, I >>>>>>> will >>>>>>> be >>>>>>> more than happy to talk to you and merge the patches. >>>>>>> >>>>>>> -- Ivan >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups >>>>>> "foreman-dev" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>> an >>>>>> email to foreman-dev+unsubscribe@googlegroups.com. >>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups >>>>> "foreman-dev" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an >>>>> email to foreman-dev+unsubscribe@googlegroups.com. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "foreman-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to foreman-dev+unsubscribe@googlegroups.com. >>> For more options, visit https://groups.google.com/groups/opt_out. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out.

Dmitri,

I think we are getting stuck in abstract discussion here.

Dynflow is a workflow engine (I like orchestration engine more but
nevermind). Every workflow engine needs administration, persistence and
logging (sometimes called as audit or message store) capabilities.

I am not sure how Rails Notifications or Queues fits into this as these
are just simple messaging APIs. Messaging != orchestration. Only thing I
can think of is an "adapter" - just to learn Dynflow to understand Rails
Notifications (e.g. to initiate workflow using a Rails message or to
emit Rails messages from workflow processes).

I also do not understand your point about audit/log layer. If we want to
audit/administer workflow engine, we need to have some bits in there to
allow us to do this. If you think it should be decoupled somehow, I
agree there is still some room for improvement - for example log is
often implemented as a "sink" of the message bus which is used for
orchestration engine and there is a component (log store) that just logs
everything and nothing else. But I am totally fine starting with current
Dynflow and extending it as we go, if we are all fine with it's
architecture and purpose.

Please elaborate your concerns.

LZ

··· On Wed, Dec 18, 2013 at 10:28:36AM +0000, Dmitri Dolguikh wrote: > As logging/auditing can potentially apply to any part or a layer in a > system. It cannot itself be a layer, as in this case anything below > won't be able to use it. A some sort of publisher/subscriber (Rails 3 > ActiveSupport::Notification) or a queue (Rails 4 > ActiveSupport::Notification) would work well, as they allow to have > multiple event sinks and event processors. > > -d > > On Wed, Dec 18, 2013 at 10:09 AM, Petr Chalupa wrote: > > > > > > On 17.12.13 13:26, Dmitri Dolguikh wrote: > >> > >> On Tue, Dec 17, 2013 at 11:59 AM, Petr Chalupa > >> wrote: > >>> > >>> My two cents. I'll describe how I see these layers / systems logically > >>> structured and what would be the main benefits of this approach. > >>> > >>> # It is touching the following: > >>> - plugins > >>> - user notifications > >>> - auditing > >>> - business logic (processes) > >>> - ActiveRecord models > >> > >> > >> Auditing/logging shouldn't be a layer -- it's orthogonal to the rest > >> of the system (a concern in AOP terminology). > > > > > > Could you elaborate? I am aware of AOP but we do not generally use it. I > > would like to avoid wild guessing what issue you are trying to point out or > > what concept you would like to apply. Thanks. > > > > > >> > >>> > >>> ## Business Logic and ActiveRecord models > >>> > >>> Dynflow is a kind of workflow engine where business processes are > >>> composed > >>> from dynflow-actions. It would be beneficial approach to have all actions > >>> modifying application's data captured as Dynflow actions where > >>> ActiveRecord > >>> models would serve only as a lower-level layer for reading and modifying > >>> data but would be business-logic-free (which would help with fat models - > >>> got objects a lot). > >>> > >>> Any change in the application would be triggered by dynflow-action, for > >>> data > >>> reading ActiveRecord models would be used as usual. > >>> > >>> Let's assume we have logically divided responsibilities as above and > >>> whole > >>> business logic is modeled with Dynflow actions form now on. > >>> > >>> ## Plugins > >>> > >>> Plug-ability is a base feature of Dynflow and its actions. Plugins can > >>> subscribe to any Dynflow Action and since there are actions for > >>> everything > >>> it is creating complete API for plugins. > >>> > >>> Actions also have in/output format definited to be able to document it > >>> and > >>> validate. It will have to keep compatibility. (Validation is not enabled > >>> yet.) > >>> > >>> ## Audit > >>> > >>> Combination of stored Dynflow ExecutionPlans and foreman-tasks's Locks > >>> provides complete human-readable (and also very detailed if needed) audit > >>> of > >>> all actions. > >>> > >>> ## User Notifications > >>> > >>> Action triggering creates natural point to integrate notifications > >>> reusing > >>> audit humanized messages. I think I've mentioned that in some previous > >>> email. It would be very easy to slip in a layer to be able to tie > >>> notifications to actions saying: > >>> - don't show (e.g. when coming form CLI) > >>> - show (e.g. when sync task coming from UI) > >>> - show only when failed (e.g. background task form UI) > >>> > >>> ## Summary: > >>> > >>> - there is layer representing whole applications business logic > >>> - light ActiveRecord models > >>> - complete documented plugin API > >>> - complete audit of all actions > >>> - easy integration of notification > >>> > >>> # A/sync processing Note > >>> > >>> Plan phase is always synchronous and it always happens in same Thread as > >>> called. Plan phase is very short. Run and Finalize phase is always > >>> executed > >>> in Dynflow thread pool and can be executed a/synchronously depending on > >>> what > >>> is needed - basically when triggering action a developer can choose if he > >>> wants trigger call to block or return immediately after plan phase. > >>> > >>> # Scheduling Note > >>> > >>> Let pulp to handle scheduling should be only temporary. Foreman-tasks > >>> provides sophisticated locking mechanism to avoid race conditions on > >>> involved resources. If it runs outside the dynflow system it can cause > >>> unexpected problems. > >> > >> > >> I don't think foreman should be controlling access to resources > >> controlled by other systems, this severely impacts both systems, how > >> they are deployed and used. > >> -d > > > > > > I am not sure what you mean exactly. Foreman would not be controlling access > > a Lock prevents resource-changes, resource can be read freely. How does that > > severely impacts the systems? Please share an example. My note was about > > triggering scheduled tasks (e.g. repo-synchronizations). Only change is that > > a repo-synchronization would be triggered by Katello instead by Pulp. > > > > > >> > >>> > >>> I've already implemented effective timer in Dynflow to able to schedule > >>> pooling of suspended actions. If it's used it should be matter of one or > >>> two > >>> days to write more general task scheduler for Foreman. > >>> > >>> # In conclusion > >>> > >>> So I would see the Architecture integration in foreman-tasks as temporary > >>> solution to connect architecture model with actions. Integration via > >>> callbacks gives quick way how to connect it together providing most of > >>> the > >>> benefits mentioned above (complete plugin API, audit, notifications). It > >>> is > >>> missing the business shift from models to actions. This does not probably > >>> matter much for simple sets like Architecture but it should not be ever > >>> used > >>> in any of the more complicated models. > >>> > >>> Personally I think it would be not a good approach to brake this rule > >>> even > >>> for simple sets like Architectures. I would be for creating abstract > >>> Actions > >>> for CRUD operations over simple models like Architecture obeying the rule > >>> about business logic shift. (I know this was not possible for this > >>> screencast because there was little time.) > >>> > >>> One of the possible problems with this rule violation could be: Let's > >>> have a > >>> more complicated action called 'InTrouble' which needs to create > >>> architecture. A developer would naturally try to plan action > >>> Architecture::Create but it won't do anything. The action server only as > >>> an > >>> event holder it does not do anything by itself. To make it work a > >>> developer > >>> would have to create new architecture in run phase of action InTrouble, > >>> which would trigger this event on its own without connecting > >>> Architecture::Create to InTrouble. This will result in unreadable > >>> shattered > >>> audits, unwanted notifications and probably other problems. > >>> > >>> Sorry for the long email I wanted to describe how I perceive it, how this > >>> could all work together so we can discuss more and come to conclusions > >>> what > >>> will be the best. > >>> > >>> Petr > >>> > >>> > >>> On 16.12.13 14:46, Ivan Necas wrote: > >>>> > >>>> > >>>> > >>>> > >>>> ----- Original Message ----- > >>>>> > >>>>> > >>>>> For the sake of argument/completeness: it's possible to use custom > >>>>> ActiveSupport::Notifications to implement (among other things) > >>>>> auditing/logging. Any reason not to use those? Arguably this would a > >>>>> better integration point, as third-part tools and developers are more > >>>>> likely to use these. > >>>> > >>>> > >>>> > >>>> Could you elaborate a bit more on this approach? Some concerns I have > >>>> about using that as integration points is the lack of transparency: > >>>> it's not trivial to find what out what happens when you do something: > >>>> no one place to see what's going on in the hooks, especially as more > >>>> hooks are getting involved. It's also kind hard to > >>>> handle "something went wrong" situations, especially if you do something > >>>> unrevertable. That's why dynflow has the console to look what's going on > >>>> in > >>>> the task and also persists the state of the process so that one can > >>>> fix that and continue (or revert if possible): the persistence is > >>>> important IMO. > >>>> > >>>> IMO, what's more likely to be used is more question of documentation on > >>>> how to write > >>>> a plugin, than anything else. > >>>> > >>>> -- Ivan > >>>> > >>>>> > >>>>> -d > >>>>> > >>>>> On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan > >>>>> wrote: > >>>>>> > >>>>>> > >>>>>> On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote: > >>>>>>> > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> As part of the work of integrating Dynflow into Katello, we needed > >>>>>>> models > >>>>>>> for keeping the track of tasks run in the system and their assignment > >>>>>>> to > >>>>>>> related resources. Since the work is not specific to Katello only and > >>>>>>> other > >>>>>>> parts of Foreman might be interested in it (especially with growing > >>>>>>> number > >>>>>>> of features that require async processing), I've extracted the common > >>>>>>> logic > >>>>>>> into a plugin. For now, you can find it here: > >>>>>>> > >>>>>>> https://github.com/iNecas/foreman-tasks > >>>>>>> > >>>>>>> I've recorded a screencast on that as well, explaining the main > >>>>>>> features > >>>>>>> and usage: > >>>>>>> > >>>>>>> http://www.youtube.com/watch?v=AqZkoSt8vzA > >>>>>> > >>>>>> > >>>>>> > >>>>>> It looks very promising. Also I like the possibility of auditing all > >>>>>> system > >>>>>> actions, not just AR based. > >>>>>> > >>>>>> If I understood it correctly, this plugin changes some actions to be > >>>>>> async > >>>>>> by > >>>>>> monkey patching. Does this mean that foreman-tasks plugin would be > >>>>>> requirement > >>>>>> if we'd like to use dynflow solution and all actions would be modified > >>>>>> to > >>>>>> async > >>>>>> in core? Or would you rather stick with monkey patching which also > >>>>>> foreman- > >>>>>> background uses (afaik)? > >>>>>> > >>>>>> -- > >>>>>> Marek > >>>>>> > >>>>>>> > >>>>>>> If you're interested in showing progress of background tasks, > >>>>>>> auditing, > >>>>>>> showing CLI examples for actions of Dynflow integration, you might > >>>>>>> want > >>>>>>> to see that video (sorry for all the coughing, I haven't felt very > >>>>>>> well > >>>>>>> when recording that:) > >>>>>>> > >>>>>>> The idea is this gem would provide the common API for getting the > >>>>>>> status > >>>>>>> of running tasks. The code should be backround-processing-tool > >>>>>>> agnostic. > >>>>>>> For now, we have adapter for Dynflow, as it's plan for using that in > >>>>>>> Katello's orchestration. > >>>>>>> > >>>>>>> The Dynflow part there is optional, but it should make it really > >>>>>>> easy to start with that if anyone interested (no additional service > >>>>>>> needed, > >>>>>>> just database migration). It might also be interesting to look into > >>>>>>> the > >>>>>>> auditing aspect of this approach: I've spent some time on that in the > >>>>>>> README and screencast, as well as some example in the plugin itself > >>>>>>> (tracking the actions on architecture resource) > >>>>>>> > >>>>>>> See README for more details, I will be offline for some time till end > >>>>>>> of > >>>>>>> this year, therefore I wanted to get something out before that (sorry > >>>>>>> for > >>>>>>> missing tests, that's one of my new years resolutions:) > >>>>>>> > >>>>>>> You can also see the Katello's part in action here > >>>>>>> > >>>>>>> http://www.youtube.com/watch?v=cpUz5Ylhkr0. > >>>>>>> > >>>>>>> There is also a wiki page started for discussions and design around > >>>>>>> the > >>>>>>> tasks API. > >>>>>>> > >>>>>>> > >>>>>>> http://projects.theforeman.org/projects/foreman/wiki/_foreman_tasks > >>>>>>> > >>>>>>> Ad. relation to foreman-background plugin > >>>>>>> (https://github.com/ohadlevy/foreman-background): The > >>>>>>> foreman-background > >>>>>>> provides just basic wrapper around Sidekiq for asynchronous > >>>>>>> processing > >>>>>>> of > >>>>>>> some operations. The sidekiq part might be probably part of Foreman > >>>>>>> Tasks > >>>>>>> as well (on the same level as Dynflow). It would be also nice to > >>>>>>> evaluate, > >>>>>>> if the Sidekiq is needed and if Dynflow doesn't already provide > >>>>>>> enough > >>>>>>> features around async stuff. I know Sidekiq has some history that > >>>>>>> promises > >>>>>>> being more stable solution at the beginning, but Dynflow is getting > >>>>>>> better > >>>>>>> every commit :). Said that, if anyone is interested in contribution, > >>>>>>> either > >>>>>>> in general API/models stuff, or adapters for other async tools, I > >>>>>>> will > >>>>>>> be > >>>>>>> more than happy to talk to you and merge the patches. > >>>>>>> > >>>>>>> -- Ivan > >>>>>> > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> You received this message because you are subscribed to the Google > >>>>>> Groups > >>>>>> "foreman-dev" group. > >>>>>> To unsubscribe from this group and stop receiving emails from it, send > >>>>>> an > >>>>>> email to foreman-dev+unsubscribe@googlegroups.com. > >>>>>> For more options, visit https://groups.google.com/groups/opt_out. > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> You received this message because you are subscribed to the Google > >>>>> Groups > >>>>> "foreman-dev" group. > >>>>> To unsubscribe from this group and stop receiving emails from it, send > >>>>> an > >>>>> email to foreman-dev+unsubscribe@googlegroups.com. > >>>>> For more options, visit https://groups.google.com/groups/opt_out. > >>>>> > >>>> > >>> > >>> -- > >>> You received this message because you are subscribed to the Google Groups > >>> "foreman-dev" group. > >>> To unsubscribe from this group and stop receiving emails from it, send an > >>> email to foreman-dev+unsubscribe@googlegroups.com. > >>> For more options, visit https://groups.google.com/groups/opt_out. > >> > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "foreman-dev" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to foreman-dev+unsubscribe@googlegroups.com. > > For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to the Google Groups "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out.


Later,

Lukas “lzap” Zapletal
irc: lzap #theforeman

> As logging/auditing can potentially apply to any part or a layer in a
> system. It cannot itself be a layer, as in this case anything below
> won't be able to use it. A some sort of publisher/subscriber (Rails 3
> ActiveSupport::Notification) or a queue (Rails 4
> ActiveSupport::Notification) would work well, as they allow to have
> multiple event sinks and event processors.

It depends on what level of abstration we're talking about. The foreman-tasks's mean
of audit is tracking the actions that were related to resources/users.

Of course Dynflow will not audit the sql commands run in the system,
bur the audit sql commands in the system is not the best way to
see what happened to some resource past two days.

The advantage of Dynflow's approach is, that keeps the data
in structured form, not mentioning the metadata about execution time, errors
etc. And as it was shown, this data can be simple used for displaying additional
information to the user, showing the progress of the actions, cli example etc.
If an error occurs on some task, the admin just could take this data and
ask for support. This looks to me like a good way to start investigating
issues, especially when debug messages are usually turned of in production systems.

– Ivan

··· ----- Original Message -----

-d

On Wed, Dec 18, 2013 at 10:09 AM, Petr Chalupa pchalupa@redhat.com wrote:

On 17.12.13 13:26, Dmitri Dolguikh wrote:

On Tue, Dec 17, 2013 at 11:59 AM, Petr Chalupa pchalupa@redhat.com > >> wrote:

My two cents. I’ll describe how I see these layers / systems logically
structured and what would be the main benefits of this approach.

It is touching the following:

  • plugins
  • user notifications
  • auditing
  • business logic (processes)
  • ActiveRecord models

Auditing/logging shouldn’t be a layer – it’s orthogonal to the rest
of the system (a concern in AOP terminology).

Could you elaborate? I am aware of AOP but we do not generally use it. I
would like to avoid wild guessing what issue you are trying to point out or
what concept you would like to apply. Thanks.

Business Logic and ActiveRecord models

Dynflow is a kind of workflow engine where business processes are
composed
from dynflow-actions. It would be beneficial approach to have all actions
modifying application’s data captured as Dynflow actions where
ActiveRecord
models would serve only as a lower-level layer for reading and modifying
data but would be business-logic-free (which would help with fat models -
got objects a lot).

Any change in the application would be triggered by dynflow-action, for
data
reading ActiveRecord models would be used as usual.

Let’s assume we have logically divided responsibilities as above and
whole
business logic is modeled with Dynflow actions form now on.

Plugins

Plug-ability is a base feature of Dynflow and its actions. Plugins can
subscribe to any Dynflow Action and since there are actions for
everything
it is creating complete API for plugins.

Actions also have in/output format definited to be able to document it
and
validate. It will have to keep compatibility. (Validation is not enabled
yet.)

Audit

Combination of stored Dynflow ExecutionPlans and foreman-tasks’s Locks
provides complete human-readable (and also very detailed if needed) audit
of
all actions.

User Notifications

Action triggering creates natural point to integrate notifications
reusing
audit humanized messages. I think I’ve mentioned that in some previous
email. It would be very easy to slip in a layer to be able to tie
notifications to actions saying:

  • don’t show (e.g. when coming form CLI)
  • show (e.g. when sync task coming from UI)
  • show only when failed (e.g. background task form UI)

Summary:

  • there is layer representing whole applications business logic
  • light ActiveRecord models
  • complete documented plugin API
  • complete audit of all actions
  • easy integration of notification

A/sync processing Note

Plan phase is always synchronous and it always happens in same Thread as
called. Plan phase is very short. Run and Finalize phase is always
executed
in Dynflow thread pool and can be executed a/synchronously depending on
what
is needed - basically when triggering action a developer can choose if he
wants trigger call to block or return immediately after plan phase.

Scheduling Note

Let pulp to handle scheduling should be only temporary. Foreman-tasks
provides sophisticated locking mechanism to avoid race conditions on
involved resources. If it runs outside the dynflow system it can cause
unexpected problems.

I don’t think foreman should be controlling access to resources
controlled by other systems, this severely impacts both systems, how
they are deployed and used.
-d

I am not sure what you mean exactly. Foreman would not be controlling
access
a Lock prevents resource-changes, resource can be read freely. How does
that
severely impacts the systems? Please share an example. My note was about
triggering scheduled tasks (e.g. repo-synchronizations). Only change is
that
a repo-synchronization would be triggered by Katello instead by Pulp.

I’ve already implemented effective timer in Dynflow to able to schedule
pooling of suspended actions. If it’s used it should be matter of one or
two
days to write more general task scheduler for Foreman.

In conclusion

So I would see the Architecture integration in foreman-tasks as temporary
solution to connect architecture model with actions. Integration via
callbacks gives quick way how to connect it together providing most of
the
benefits mentioned above (complete plugin API, audit, notifications). It
is
missing the business shift from models to actions. This does not probably
matter much for simple sets like Architecture but it should not be ever
used
in any of the more complicated models.

Personally I think it would be not a good approach to brake this rule
even
for simple sets like Architectures. I would be for creating abstract
Actions
for CRUD operations over simple models like Architecture obeying the rule
about business logic shift. (I know this was not possible for this
screencast because there was little time.)

One of the possible problems with this rule violation could be: Let’s
have a
more complicated action called ‘InTrouble’ which needs to create
architecture. A developer would naturally try to plan action
Architecture::Create but it won’t do anything. The action server only as
an
event holder it does not do anything by itself. To make it work a
developer
would have to create new architecture in run phase of action InTrouble,
which would trigger this event on its own without connecting
Architecture::Create to InTrouble. This will result in unreadable
shattered
audits, unwanted notifications and probably other problems.

Sorry for the long email I wanted to describe how I perceive it, how this
could all work together so we can discuss more and come to conclusions
what
will be the best.

Petr

On 16.12.13 14:46, Ivan Necas wrote:

----- Original Message -----

For the sake of argument/completeness: it’s possible to use custom
ActiveSupport::Notifications to implement (among other things)
auditing/logging. Any reason not to use those? Arguably this would a
better integration point, as third-part tools and developers are more
likely to use these.

Could you elaborate a bit more on this approach? Some concerns I have
about using that as integration points is the lack of transparency:
it’s not trivial to find what out what happens when you do something:
no one place to see what’s going on in the hooks, especially as more
hooks are getting involved. It’s also kind hard to
handle “something went wrong” situations, especially if you do something
unrevertable. That’s why dynflow has the console to look what’s going on
in
the task and also persists the state of the process so that one can
fix that and continue (or revert if possible): the persistence is
important IMO.

IMO, what’s more likely to be used is more question of documentation on
how to write
a plugin, than anything else.

– Ivan

-d

On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan mhulan@redhat.com > >>>>> wrote:

On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote:

Hi,

As part of the work of integrating Dynflow into Katello, we needed
models
for keeping the track of tasks run in the system and their assignment
to
related resources. Since the work is not specific to Katello only and
other
parts of Foreman might be interested in it (especially with growing
number
of features that require async processing), I’ve extracted the common
logic
into a plugin. For now, you can find it here:

  https://github.com/iNecas/foreman-tasks

I’ve recorded a screencast on that as well, explaining the main
features
and usage:

  http://www.youtube.com/watch?v=AqZkoSt8vzA

It looks very promising. Also I like the possibility of auditing all
system
actions, not just AR based.

If I understood it correctly, this plugin changes some actions to be
async
by
monkey patching. Does this mean that foreman-tasks plugin would be
requirement
if we’d like to use dynflow solution and all actions would be modified
to
async
in core? Or would you rather stick with monkey patching which also
foreman-
background uses (afaik)?


Marek

If you’re interested in showing progress of background tasks,
auditing,
showing CLI examples for actions of Dynflow integration, you might
want
to see that video (sorry for all the coughing, I haven’t felt very
well
when recording that:)

The idea is this gem would provide the common API for getting the
status
of running tasks. The code should be backround-processing-tool
agnostic.
For now, we have adapter for Dynflow, as it’s plan for using that in
Katello’s orchestration.

The Dynflow part there is optional, but it should make it really
easy to start with that if anyone interested (no additional service
needed,
just database migration). It might also be interesting to look into
the
auditing aspect of this approach: I’ve spent some time on that in the
README and screencast, as well as some example in the plugin itself
(tracking the actions on architecture resource)

See README for more details, I will be offline for some time till end
of
this year, therefore I wanted to get something out before that (sorry
for
missing tests, that’s one of my new years resolutions:)

You can also see the Katello’s part in action here

  http://www.youtube.com/watch?v=cpUz5Ylhkr0.

There is also a wiki page started for discussions and design around
the
tasks API.

foreman tasks - Foreman

Ad. relation to foreman-background plugin
(https://github.com/ohadlevy/foreman-background): The
foreman-background
provides just basic wrapper around Sidekiq for asynchronous
processing
of
some operations. The sidekiq part might be probably part of Foreman
Tasks
as well (on the same level as Dynflow). It would be also nice to
evaluate,
if the Sidekiq is needed and if Dynflow doesn’t already provide
enough
features around async stuff. I know Sidekiq has some history that
promises
being more stable solution at the beginning, but Dynflow is getting
better
every commit :). Said that, if anyone is interested in contribution,
either
in general API/models stuff, or adapters for other async tools, I
will
be
more than happy to talk to you and merge the patches.

– Ivan


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

> Dmitri,
>
> I think we are getting stuck in abstract discussion here.
>
> Dynflow is a workflow engine (I like orchestration engine more but
> nevermind). Every workflow engine needs administration, persistence and
> logging (sometimes called as audit or message store) capabilities.
>
> I am not sure how Rails Notifications or Queues fits into this as these
> are just simple messaging APIs. Messaging != orchestration. Only thing I
> can think of is an "adapter" - just to learn Dynflow to understand Rails
> Notifications (e.g. to initiate workflow using a Rails message or to
> emit Rails messages from workflow processes).

I don't think I was comparing dynflow with rails notifications;
there's a separate thread about dynflow's internal messaging vs.
rails' activesupport::notifications (the gist is why aren't we using
rails' implementation?).

>
> I also do not understand your point about audit/log layer. If we want to
> audit/administer workflow engine, we need to have some bits in there to
> allow us to do this. If you think it should be decoupled somehow, I
> agree there is still some room for improvement - for example log is
> often implemented as a "sink" of the message bus which is used for
> orchestration engine and there is a component (log store) that just logs
> everything and nothing else. But I am totally fine starting with current
> Dynflow and extending it as we go, if we are all fine with it's
> architecture and purpose.

In my view the scope of auditing is (potentially) the whole of the
application, not just orchestrations managed by dynflow, so yes it
should be decoupled. There's already a nicely decoupled implementation
of publisher/subscriber in Rails, shouldn't we start by looking at
that?

-d

··· On Thu, Dec 19, 2013 at 2:37 PM, Lukas Zapletal wrote:

Please elaborate your concerns.

LZ

On Wed, Dec 18, 2013 at 10:28:36AM +0000, Dmitri Dolguikh wrote:

As logging/auditing can potentially apply to any part or a layer in a
system. It cannot itself be a layer, as in this case anything below
won’t be able to use it. A some sort of publisher/subscriber (Rails 3
ActiveSupport::Notification) or a queue (Rails 4
ActiveSupport::Notification) would work well, as they allow to have
multiple event sinks and event processors.

-d

On Wed, Dec 18, 2013 at 10:09 AM, Petr Chalupa pchalupa@redhat.com wrote:

On 17.12.13 13:26, Dmitri Dolguikh wrote:

On Tue, Dec 17, 2013 at 11:59 AM, Petr Chalupa pchalupa@redhat.com >> >> wrote:

My two cents. I’ll describe how I see these layers / systems logically
structured and what would be the main benefits of this approach.

It is touching the following:

  • plugins
  • user notifications
  • auditing
  • business logic (processes)
  • ActiveRecord models

Auditing/logging shouldn’t be a layer – it’s orthogonal to the rest
of the system (a concern in AOP terminology).

Could you elaborate? I am aware of AOP but we do not generally use it. I
would like to avoid wild guessing what issue you are trying to point out or
what concept you would like to apply. Thanks.

Business Logic and ActiveRecord models

Dynflow is a kind of workflow engine where business processes are
composed
from dynflow-actions. It would be beneficial approach to have all actions
modifying application’s data captured as Dynflow actions where
ActiveRecord
models would serve only as a lower-level layer for reading and modifying
data but would be business-logic-free (which would help with fat models -
got objects a lot).

Any change in the application would be triggered by dynflow-action, for
data
reading ActiveRecord models would be used as usual.

Let’s assume we have logically divided responsibilities as above and
whole
business logic is modeled with Dynflow actions form now on.

Plugins

Plug-ability is a base feature of Dynflow and its actions. Plugins can
subscribe to any Dynflow Action and since there are actions for
everything
it is creating complete API for plugins.

Actions also have in/output format definited to be able to document it
and
validate. It will have to keep compatibility. (Validation is not enabled
yet.)

Audit

Combination of stored Dynflow ExecutionPlans and foreman-tasks’s Locks
provides complete human-readable (and also very detailed if needed) audit
of
all actions.

User Notifications

Action triggering creates natural point to integrate notifications
reusing
audit humanized messages. I think I’ve mentioned that in some previous
email. It would be very easy to slip in a layer to be able to tie
notifications to actions saying:

  • don’t show (e.g. when coming form CLI)
  • show (e.g. when sync task coming from UI)
  • show only when failed (e.g. background task form UI)

Summary:

  • there is layer representing whole applications business logic
  • light ActiveRecord models
  • complete documented plugin API
  • complete audit of all actions
  • easy integration of notification

A/sync processing Note

Plan phase is always synchronous and it always happens in same Thread as
called. Plan phase is very short. Run and Finalize phase is always
executed
in Dynflow thread pool and can be executed a/synchronously depending on
what
is needed - basically when triggering action a developer can choose if he
wants trigger call to block or return immediately after plan phase.

Scheduling Note

Let pulp to handle scheduling should be only temporary. Foreman-tasks
provides sophisticated locking mechanism to avoid race conditions on
involved resources. If it runs outside the dynflow system it can cause
unexpected problems.

I don’t think foreman should be controlling access to resources
controlled by other systems, this severely impacts both systems, how
they are deployed and used.
-d

I am not sure what you mean exactly. Foreman would not be controlling access
a Lock prevents resource-changes, resource can be read freely. How does that
severely impacts the systems? Please share an example. My note was about
triggering scheduled tasks (e.g. repo-synchronizations). Only change is that
a repo-synchronization would be triggered by Katello instead by Pulp.

I’ve already implemented effective timer in Dynflow to able to schedule
pooling of suspended actions. If it’s used it should be matter of one or
two
days to write more general task scheduler for Foreman.

In conclusion

So I would see the Architecture integration in foreman-tasks as temporary
solution to connect architecture model with actions. Integration via
callbacks gives quick way how to connect it together providing most of
the
benefits mentioned above (complete plugin API, audit, notifications). It
is
missing the business shift from models to actions. This does not probably
matter much for simple sets like Architecture but it should not be ever
used
in any of the more complicated models.

Personally I think it would be not a good approach to brake this rule
even
for simple sets like Architectures. I would be for creating abstract
Actions
for CRUD operations over simple models like Architecture obeying the rule
about business logic shift. (I know this was not possible for this
screencast because there was little time.)

One of the possible problems with this rule violation could be: Let’s
have a
more complicated action called ‘InTrouble’ which needs to create
architecture. A developer would naturally try to plan action
Architecture::Create but it won’t do anything. The action server only as
an
event holder it does not do anything by itself. To make it work a
developer
would have to create new architecture in run phase of action InTrouble,
which would trigger this event on its own without connecting
Architecture::Create to InTrouble. This will result in unreadable
shattered
audits, unwanted notifications and probably other problems.

Sorry for the long email I wanted to describe how I perceive it, how this
could all work together so we can discuss more and come to conclusions
what
will be the best.

Petr

On 16.12.13 14:46, Ivan Necas wrote:

----- Original Message -----

For the sake of argument/completeness: it’s possible to use custom
ActiveSupport::Notifications to implement (among other things)
auditing/logging. Any reason not to use those? Arguably this would a
better integration point, as third-part tools and developers are more
likely to use these.

Could you elaborate a bit more on this approach? Some concerns I have
about using that as integration points is the lack of transparency:
it’s not trivial to find what out what happens when you do something:
no one place to see what’s going on in the hooks, especially as more
hooks are getting involved. It’s also kind hard to
handle “something went wrong” situations, especially if you do something
unrevertable. That’s why dynflow has the console to look what’s going on
in
the task and also persists the state of the process so that one can
fix that and continue (or revert if possible): the persistence is
important IMO.

IMO, what’s more likely to be used is more question of documentation on
how to write
a plugin, than anything else.

– Ivan

-d

On Mon, Dec 16, 2013 at 10:43 AM, Marek Hulan mhulan@redhat.com >> >>>>> wrote:

On Sunday 15 of December 2013 14:05:00 Ivan Necas wrote:

Hi,

As part of the work of integrating Dynflow into Katello, we needed
models
for keeping the track of tasks run in the system and their assignment
to
related resources. Since the work is not specific to Katello only and
other
parts of Foreman might be interested in it (especially with growing
number
of features that require async processing), I’ve extracted the common
logic
into a plugin. For now, you can find it here:

  https://github.com/iNecas/foreman-tasks

I’ve recorded a screencast on that as well, explaining the main
features
and usage:

  http://www.youtube.com/watch?v=AqZkoSt8vzA

It looks very promising. Also I like the possibility of auditing all
system
actions, not just AR based.

If I understood it correctly, this plugin changes some actions to be
async
by
monkey patching. Does this mean that foreman-tasks plugin would be
requirement
if we’d like to use dynflow solution and all actions would be modified
to
async
in core? Or would you rather stick with monkey patching which also
foreman-
background uses (afaik)?


Marek

If you’re interested in showing progress of background tasks,
auditing,
showing CLI examples for actions of Dynflow integration, you might
want
to see that video (sorry for all the coughing, I haven’t felt very
well
when recording that:)

The idea is this gem would provide the common API for getting the
status
of running tasks. The code should be backround-processing-tool
agnostic.
For now, we have adapter for Dynflow, as it’s plan for using that in
Katello’s orchestration.

The Dynflow part there is optional, but it should make it really
easy to start with that if anyone interested (no additional service
needed,
just database migration). It might also be interesting to look into
the
auditing aspect of this approach: I’ve spent some time on that in the
README and screencast, as well as some example in the plugin itself
(tracking the actions on architecture resource)

See README for more details, I will be offline for some time till end
of
this year, therefore I wanted to get something out before that (sorry
for
missing tests, that’s one of my new years resolutions:)

You can also see the Katello’s part in action here

  http://www.youtube.com/watch?v=cpUz5Ylhkr0.

There is also a wiki page started for discussions and design around
the
tasks API.

foreman tasks - Foreman

Ad. relation to foreman-background plugin
(https://github.com/ohadlevy/foreman-background): The
foreman-background
provides just basic wrapper around Sidekiq for asynchronous
processing
of
some operations. The sidekiq part might be probably part of Foreman
Tasks
as well (on the same level as Dynflow). It would be also nice to
evaluate,
if the Sidekiq is needed and if Dynflow doesn’t already provide
enough
features around async stuff. I know Sidekiq has some history that
promises
being more stable solution at the beginning, but Dynflow is getting
better
every commit :). Said that, if anyone is interested in contribution,
either
in general API/models stuff, or adapters for other async tools, I
will
be
more than happy to talk to you and merge the patches.

– Ivan


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google
Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send
an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups “foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


Later,

Lukas “lzap” Zapletal
irc: lzap #theforeman


You received this message b