Using foreman-tasks for large deployments

I was wondering, how useful would it be to take advantage of Foreman-Tasks
and use it to support many or heavy-to-process requests?

I'm thinking the user could set a threshold per Foreman host depending on
their processing/memory and we make requests be handled by Foreman-tasks
once requests go over that.

For instance, deleting 100 hosts is a task bound to timeout or fail because
of all the associated data (reports/logs, etc)

Your thoughts?

··· -- Daniel Lobato

@elobatoss
blog.daniellobato.me
https://mailtrack.io/trace/link/657886dff180af4370c51f673bcde0ece76cba93
daniellobato.me
https://mailtrack.io/trace/link/1b6e7a141b5dd01c9f62582aea5ae796d4d88d57

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30

I'm finishing adding bulk actions to Foreman Tasks these days (there are
couple of bulk actions in Katello that need exactly the same treatment).
I can do a deep-dive on this later on.

I'm not a big fan of having two ways to deal with the same scenario
and deciding which on to use based on the load. There are other benefits
for using the Foreman Tasks for deleting just 2 hosts (and even 1, especially
when the orchestration process itself is managed by Dynflow, said that, it's
possible to just call system.destroy! in the plan phase and let the current
orchestration to handle the stuff, so that we could start using Foreman Tasks
even with the old orchestration in place): tracking the status of the bulk action
and having a record about this in the tasks for auditing.

I've started seeing Foreman Tasks/Dynflow being referenced in more discussions
lately: remote execution, multi-host deployments used in Staypuft, better
error-handling in host orchestration, bulk actions. It maybe makes sense to
start thinking about adopting foreman-tasks as a core foreman feature and being
able to rely on it's presence.

I can think about four things to be added to Dynflow or Foreman Tasks
to support the host creation orchestration:

  1. support for multi-host deployments: right now, we rely on the Dynflow executre
    running on the same host as the foreman server. This is a known limitation
    we have to solve sooner or later. We have ideas on how to solve that, it's
    more about prioritization.

  2. rollbacks support: in the Katello's usage, using the task resume
    is the one to go for majority of the cases. It however makes a lot of sense
    in host provisioning. Relying on Dynflow in the orchestration would allow
    us to decide if for particular error, it's better to rollback or
    or let the admin to fix the external issue and continue with the process

  3. backward compatibility with the old orchestration: ideally, there should
    be a way how run the current orchestration steps as part of a Dynflow tasks.
    Although, the core orchestration would be still benefit more from transferring
    to dynflow actions, we need a strategy on how to keep third-party plugins
    (including Foreman hooks) working.

  4. On the Foreman side, it would require to drop the Ruby 1.8.7 support.

Just for the bulk actions, the (1) and (4) should be sufficient to
unblock us from proceeding. The (2) and (3) would be required for provisioning
orchestration.

I believe Foreman would benefit a lot from the adoption (I'm might be a little
biased for obvious reasons though), at least we got really good feedback
so far when using it for Katello orchestration. The Dynflow itself is also
moving forward, with the pitr-ch improved actors implementation getting
merged into concurrent-ruby gem [1] (we have a basic one right now directly in Dynflow).
We will switch to the concurrent-ruby soon, so that we can share the concurrent stuff
with others: it's getting really good feedback so far.

So the questions I have are:

  1. is the Foreman team willing at some point to merge
    foreman-tasks into the core (or make it a hard dependency). What are the other things
    that need to be done before being able to do so?
  2. if (1) holds, what version it would be for (it would need dropping the ruby 1.8.7
    for that version)?
  3. would it be possible to use Dynflow for the host orchestration? I'm sure it would be
    possible from the Dynflow point of view (we have a lot of experience with doing so
    in the Katello codebase), but I'm aware that the orchestration has a long
    history and people might be defensive on rewriting that.

The answers for this questions would help us better prioritize the things to keep
things going forward.

[1] - https://github.com/ruby-concurrency/concurrent-ruby

Cheers,

Ivan

··· ----- Original Message ----- > I was wondering, how useful would it be to take advantage of Foreman-Tasks > and use it to support many or heavy-to-process requests? > > I'm thinking the user could set a threshold per Foreman host depending on > their processing/memory and we make requests be handled by Foreman-tasks > once requests go over that. > > For instance, deleting 100 hosts is a task bound to timeout or fail because > of all the associated data (reports/logs, etc) > > Your thoughts? > > -- > Daniel Lobato > > @elobatoss > blog.daniellobato.me > > daniellobato.me > > > GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30 > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >

> I'm finishing adding bulk actions to Foreman Tasks these days (there are
> couple of bulk actions in Katello that need exactly the same treatment).
> I can do a deep-dive on this later on.
>
> I'm not a big fan of having two ways to deal with the same scenario
> and deciding which on to use based on the load. There are other benefits
> for using the Foreman Tasks for deleting just 2 hosts (and even 1,
> especially when the orchestration process itself is managed by Dynflow,
> said that, it's possible to just call system.destroy! in the plan phase and
> let the current orchestration to handle the stuff, so that we could start
> using Foreman Tasks even with the old orchestration in place): tracking the
> status of the bulk action and having a record about this in the tasks for
> auditing.

+1, let's have one way, I don't see any cons using Dynflow even for non-bulk
action or small number of items

> I've started seeing Foreman Tasks/Dynflow being referenced in more
> discussions lately: remote execution, multi-host deployments used in
> Staypuft, better error-handling in host orchestration, bulk actions. It
> maybe makes sense to start thinking about adopting foreman-tasks as a core
> foreman feature and being able to rely on it's presence.
>
> I can think about four things to be added to Dynflow or Foreman Tasks
> to support the host creation orchestration:
>
> 1. support for multi-host deployments: right now, we rely on the Dynflow
> executre running on the same host as the foreman server. This is a known
> limitation we have to solve sooner or later. We have ideas on how to solve
> that, it's more about prioritization.
>
> 2. rollbacks support: in the Katello's usage, using the task resume
> is the one to go for majority of the cases. It however makes a lot of
> sense in host provisioning. Relying on Dynflow in the orchestration would
> allow us to decide if for particular error, it's better to rollback or or
> let the admin to fix the external issue and continue with the process

I see added value here, rather than rollback automatically I'd like to
investigate what's wrong, fix it and resume from point I encountered an issue.
We require rollback support though so we can start easily from the beginning.

> 3. backward compatibility with the old orchestration: ideally, there should
> be a way how run the current orchestration steps as part of a Dynflow
> tasks. Although, the core orchestration would be still benefit more from
> transferring to dynflow actions, we need a strategy on how to keep
> third-party plugins (including Foreman hooks) working.
>
> 4. On the Foreman side, it would require to drop the Ruby 1.8.7 support.

+1 on dropping in 1.7

··· On Tuesday 12 of August 2014 06:07:45 Ivan Necas wrote:

Just for the bulk actions, the (1) and (4) should be sufficient to
unblock us from proceeding. The (2) and (3) would be required for
provisioning orchestration.

I believe Foreman would benefit a lot from the adoption (I’m might be a
little biased for obvious reasons though), at least we got really good
feedback so far when using it for Katello orchestration. The Dynflow itself
is also moving forward, with the pitr-ch improved actors implementation
getting merged into concurrent-ruby gem [1] (we have a basic one right now
directly in Dynflow). We will switch to the concurrent-ruby soon, so that
we can share the concurrent stuff with others: it’s getting really good
feedback so far.

So the questions I have are:

  1. is the Foreman team willing at some point to merge
    foreman-tasks into the core (or make it a hard dependency). What are the
    other things that need to be done before being able to do so?
  2. if (1) holds, what version it would be for (it would need dropping the
    ruby 1.8.7 for that version)?
  3. would it be possible to use Dynflow for the host orchestration? I’m sure
    it would be possible from the Dynflow point of view (we have a lot of
    experience with doing so in the Katello codebase), but I’m aware that the
    orchestration has a long history and people might be defensive on rewriting
    that.

The answers for this questions would help us better prioritize the things to
keep things going forward.

[1] - https://github.com/ruby-concurrency/concurrent-ruby

Cheers,

Ivan

----- Original Message -----

I was wondering, how useful would it be to take advantage of Foreman-Tasks
and use it to support many or heavy-to-process requests?

I’m thinking the user could set a threshold per Foreman host depending on
their processing/memory and we make requests be handled by Foreman-tasks
once requests go over that.

For instance, deleting 100 hosts is a task bound to timeout or fail
because
of all the associated data (reports/logs, etc)

Your thoughts?


Daniel Lobato

@elobatoss
blog.daniellobato.me
https://mailtrack.io/trace/link/657886dff180af4370c51f673bcde0ece76cba93
daniellobato.me
https://mailtrack.io/trace/link/1b6e7a141b5dd01c9f62582aea5ae796d4d88d57

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Marek

> I'm finishing adding bulk actions to Foreman Tasks these days (there are
> couple of bulk actions in Katello that need exactly the same treatment).
> I can do a deep-dive on this later on.
>
> I'm not a big fan of having two ways to deal with the same scenario
> and deciding which on to use based on the load. There are other benefits
> for using the Foreman Tasks for deleting just 2 hosts (and even 1,
> especially
> when the orchestration process itself is managed by Dynflow, said that,
> it's
> possible to just call system.destroy! in the plan phase and let the current
> orchestration to handle the stuff, so that we could start using Foreman
> Tasks
> even with the old orchestration in place): tracking the status of the bulk
> action
> and having a record about this in the tasks for auditing.
>
> I've started seeing Foreman Tasks/Dynflow being referenced in more
> discussions
> lately: remote execution, multi-host deployments used in Staypuft, better
> error-handling in host orchestration, bulk actions. It maybe makes sense to
> start thinking about adopting foreman-tasks as a core foreman feature and
> being
> able to rely on it's presence.

> I can think about four things to be added to Dynflow or Foreman Tasks
> to support the host creation orchestration:
>
> 1. support for multi-host deployments: right now, we rely on the Dynflow
> executre
> running on the same host as the foreman server. This is a known
> limitation
> we have to solve sooner or later. We have ideas on how to solve that,
> it's
> more about prioritization.
>
What is the actual requirement to run it on the same host? I assume that if
its another server with the code / access to db it should just work?

>
> 2. rollbacks support: in the Katello's usage, using the task resume
> is the one to go for majority of the cases. It however makes a lot of
> sense
> in host provisioning. Relying on Dynflow in the orchestration would
> allow
> us to decide if for particular error, it's better to rollback or
> or let the admin to fix the external issue and continue with the process
>
true, atm, most (all?) of the host orchestration are serial tasks, meaning
you need the output of one to move on to the next step, leaving things like
a blank vm entry is not really useful.

further, the real technical reason imho, is how to present failures to the
user, and I would prefer in the first adoption phase to leave the host
orchestration for a future consolidation attempts.

>
> 3. backward compatibility with the old orchestration: ideally, there should
> be a way how run the current orchestration steps as part of a Dynflow
> tasks.
> Although, the core orchestration would be still benefit more from
> transferring
> to dynflow actions, we need a strategy on how to keep third-party
> plugins
> (including Foreman hooks) working.
>
> 4. On the Foreman side, it would require to drop the Ruby 1.8.7 support.
>

> Just for the bulk actions, the (1) and (4) should be sufficient to
> unblock us from proceeding. The (2) and (3) would be required for
> provisioning
> orchestration.
>
> I believe Foreman would benefit a lot from the adoption (I'm might be a
> little
> biased for obvious reasons though), at least we got really good feedback
> so far when using it for Katello orchestration. The Dynflow itself is also
> moving forward, with the pitr-ch improved actors implementation getting
> merged into concurrent-ruby gem [1] (we have a basic one right now
> directly in Dynflow).
> We will switch to the concurrent-ruby soon, so that we can share the
> concurrent stuff
> with others: it's getting really good feedback so far.
>
> So the questions I have are:
>
> 1. is the Foreman team willing at some point to merge
> foreman-tasks into the core (or make it a hard dependency). What are
> the other things
> that need to be done before being able to do so?
>
I would assume mostly installer, upgrade, docs etc ? potentially a review
of foreman-tasks code itself?

> 2. if (1) holds, what version it would be for (it would need dropping the
> ruby 1.8.7
> for that version)?
>
IMHO its time, can we target it for 1.7 release?

> 3. would it be possible to use Dynflow for the host orchestration? I'm
> sure it would be
> possible from the Dynflow point of view (we have a lot of experience
> with doing so
> in the Katello codebase), but I'm aware that the orchestration has a
> long
> history and people might be defensive on rewriting that.
>
I have no objections, but I see smaller value in that vs adding async
support for long running tasks (e.g. puppet class imports, report
processing, bulk actions etc).

we could consider extracting really long tasks (such as SSH into an
instance).

>
> The answers for this questions would help us better prioritize the things
> to keep
> things going forward.
>
> [1] - https://github.com/ruby-concurrency/concurrent-ruby
>
> Cheers,
>
>
thanks Ivan!

Ohad

··· On Tue, Aug 12, 2014 at 1:07 PM, Ivan Necas wrote:

Ivan

----- Original Message -----

I was wondering, how useful would it be to take advantage of
Foreman-Tasks
and use it to support many or heavy-to-process requests?

I’m thinking the user could set a threshold per Foreman host depending on
their processing/memory and we make requests be handled by Foreman-tasks
once requests go over that.

For instance, deleting 100 hosts is a task bound to timeout or fail
because
of all the associated data (reports/logs, etc)

Your thoughts?


Daniel Lobato

@elobatoss
blog.daniellobato.me
<
https://mailtrack.io/trace/link/657886dff180af4370c51f673bcde0ece76cba93>
daniellobato.me
<
https://mailtrack.io/trace/link/1b6e7a141b5dd01c9f62582aea5ae796d4d88d57>

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

>
>
>
>
>> I'm finishing adding bulk actions to Foreman Tasks these days (there are
>> couple of bulk actions in Katello that need exactly the same treatment).
>> I can do a deep-dive on this later on.
>>
>> I'm not a big fan of having two ways to deal with the same scenario
>> and deciding which on to use based on the load. There are other benefits
>> for using the Foreman Tasks for deleting just 2 hosts (and even 1,
>> especially
>> when the orchestration process itself is managed by Dynflow, said that,
>> it's
>> possible to just call system.destroy! in the plan phase and let the
>> current
>> orchestration to handle the stuff, so that we could start using Foreman
>> Tasks
>> even with the old orchestration in place): tracking the status of the
>> bulk action
>> and having a record about this in the tasks for auditing.
>>
>> I've started seeing Foreman Tasks/Dynflow being referenced in more
>> discussions
>> lately: remote execution, multi-host deployments used in Staypuft, better
>> error-handling in host orchestration, bulk actions. It maybe makes sense
>> to
>> start thinking about adopting foreman-tasks as a core foreman feature and
>> being
>> able to rely on it's presence.
>
>
>> I can think about four things to be added to Dynflow or Foreman Tasks
>> to support the host creation orchestration:
>>
>> 1. support for multi-host deployments: right now, we rely on the Dynflow
>> executre
>> running on the same host as the foreman server. This is a known
>> limitation
>> we have to solve sooner or later. We have ideas on how to solve that,
>> it's
>> more about prioritization.
>>
> What is the actual requirement to run it on the same host? I assume that
> if its another server with the code / access to db it should just work?
>
>>
>> 2. rollbacks support: in the Katello's usage, using the task resume
>> is the one to go for majority of the cases. It however makes a lot of
>> sense
>> in host provisioning. Relying on Dynflow in the orchestration would
>> allow
>> us to decide if for particular error, it's better to rollback or
>> or let the admin to fix the external issue and continue with the
>> process
>>
> true, atm, most (all?) of the host orchestration are serial tasks, meaning
> you need the output of one to move on to the next step, leaving things like
> a blank vm entry is not really useful.
>
> further, the real technical reason imho, is how to present failures to the
> user, and I would prefer in the first adoption phase to leave the host
> orchestration for a future consolidation attempts.
>
>>
>> 3. backward compatibility with the old orchestration: ideally, there
>> should
>> be a way how run the current orchestration steps as part of a Dynflow
>> tasks.
>> Although, the core orchestration would be still benefit more from
>> transferring
>> to dynflow actions, we need a strategy on how to keep third-party
>> plugins
>> (including Foreman hooks) working.
>>
>> 4. On the Foreman side, it would require to drop the Ruby 1.8.7 support.
>>
>
>> Just for the bulk actions, the (1) and (4) should be sufficient to
>> unblock us from proceeding. The (2) and (3) would be required for
>> provisioning
>> orchestration.
>>
>> I believe Foreman would benefit a lot from the adoption (I'm might be a
>> little
>> biased for obvious reasons though), at least we got really good feedback
>> so far when using it for Katello orchestration. The Dynflow itself is also
>> moving forward, with the pitr-ch improved actors implementation getting
>> merged into concurrent-ruby gem [1] (we have a basic one right now
>> directly in Dynflow).
>> We will switch to the concurrent-ruby soon, so that we can share the
>> concurrent stuff
>> with others: it's getting really good feedback so far.
>>
>> So the questions I have are:
>>
>> 1. is the Foreman team willing at some point to merge
>> foreman-tasks into the core (or make it a hard dependency). What are
>> the other things
>> that need to be done before being able to do so?
>>
> I would assume mostly installer, upgrade, docs etc ? potentially a review
> of foreman-tasks code itself?
>
>
>> 2. if (1) holds, what version it would be for (it would need dropping the
>> ruby 1.8.7
>> for that version)?
>>
> IMHO its time, can we target it for 1.7 release?
>
>
>> 3. would it be possible to use Dynflow for the host orchestration? I'm
>> sure it would be
>> possible from the Dynflow point of view (we have a lot of experience
>> with doing so
>> in the Katello codebase), but I'm aware that the orchestration has a
>> long
>> history and people might be defensive on rewriting that.
>>
> I have no objections, but I see smaller value in that vs adding async
> support for long running tasks (e.g. puppet class imports, report
> processing, bulk actions etc).
>
> ah, and of course, API usage should convert to async when it make sense.

> we could consider extracting really long tasks (such as SSH into an
> instance).
>
>>
>> The answers for this questions would help us better prioritize the things
>> to keep
>> things going forward.
>>
>> [1] - https://github.com/ruby-concurrency/concurrent-ruby
>>
>> Cheers,
>>
>>
> thanks Ivan!
>
> Ohad
>
>> Ivan
>>
>> > I was wondering, how useful would it be to take advantage of
>> Foreman-Tasks
>> > and use it to support many or heavy-to-process requests?
>> >
>> > I'm thinking the user could set a threshold per Foreman host depending
>> on
>> > their processing/memory and we make requests be handled by Foreman-tasks
>> > once requests go over that.
>> >
>> > For instance, deleting 100 hosts is a task bound to timeout or fail
>> because
>> > of all the associated data (reports/logs, etc)
>> >
>> > Your thoughts?
>> >
>> > –
>> > Daniel Lobato
>> >
>> > @elobatoss
>> > blog.daniellobato.me
>> > <
>> https://mailtrack.io/trace/link/657886dff180af4370c51f673bcde0ece76cba93>
>> > daniellobato.me
>> > <
>> https://mailtrack.io/trace/link/1b6e7a141b5dd01c9f62582aea5ae796d4d88d57>
>> >
>> > GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30
>> >
>> > –
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "foreman-dev" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an
>> > email to foreman-dev+unsubscribe@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>> >
>>
>> –
>> You received this message because you are subscribed to the Google Groups
>> "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to foreman-dev+unsubscribe@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

··· On Tue, Aug 12, 2014 at 1:26 PM, Ohad Levy wrote: > On Tue, Aug 12, 2014 at 1:07 PM, Ivan Necas wrote: >> ----- Original Message -----

>
> > I'm finishing adding bulk actions to Foreman Tasks these days (there are
> > couple of bulk actions in Katello that need exactly the same treatment).
> > I can do a deep-dive on this later on.
> >
> > I'm not a big fan of having two ways to deal with the same scenario
> > and deciding which on to use based on the load. There are other benefits
> > for using the Foreman Tasks for deleting just 2 hosts (and even 1,
> > especially
> > when the orchestration process itself is managed by Dynflow, said that,
> > it's
> > possible to just call system.destroy! in the plan phase and let the current
> > orchestration to handle the stuff, so that we could start using Foreman
> > Tasks
> > even with the old orchestration in place): tracking the status of the bulk
> > action
> > and having a record about this in the tasks for auditing.
> >
> > I've started seeing Foreman Tasks/Dynflow being referenced in more
> > discussions
> > lately: remote execution, multi-host deployments used in Staypuft, better
> > error-handling in host orchestration, bulk actions. It maybe makes sense to
> > start thinking about adopting foreman-tasks as a core foreman feature and
> > being
> > able to rely on it's presence.
>
>
> > I can think about four things to be added to Dynflow or Foreman Tasks
> > to support the host creation orchestration:
> >
> > 1. support for multi-host deployments: right now, we rely on the Dynflow
> > executre
> > running on the same host as the foreman server. This is a known
> > limitation
> > we have to solve sooner or later. We have ideas on how to solve that,
> > it's
> > more about prioritization.
> >
> What is the actual requirement to run it on the same host? I assume that if
> its another server with the code / access to db it should just work?

The thing is the Dynflow supports suspending actions and waking them up
on events: for example when doing polling, we suspend the action and plan
to send an event to wake it up and perform another polling in some interval.
Between the two events, the action is not consuming thread, letting other work
to be done.

The events are not limited just for the polling, in staypuft, there are events
triggered after provisioning is finished, so the event is produced thought the
Foreman API (/unattended/built action), and we need to propagate the event to
the executor. The same applies to canceling of the tasks.

The communication goes also the other direction: you can trigger the task on web server
and wait for it to finish in the very same request: the executor after the task finishes
sends a message back to the web server to continue working (we don't do polling)

In one-host setup, we use a unix socket for the web-server -> executor communication,
since it requires minimal setup. For multi-host setup, it seems we would need
other form of communication (probably messaging) for the executors and web servers
to talk to each other.

>
> >
> > 2. rollbacks support: in the Katello's usage, using the task resume
> > is the one to go for majority of the cases. It however makes a lot of
> > sense
> > in host provisioning. Relying on Dynflow in the orchestration would
> > allow
> > us to decide if for particular error, it's better to rollback or
> > or let the admin to fix the external issue and continue with the process
> >
> true, atm, most (all?) of the host orchestration are serial tasks, meaning
> you need the output of one to move on to the next step, leaving things like
> a blank vm entry is not really useful.
>
> further, the real technical reason imho, is how to present failures to the
> user, and I would prefer in the first adoption phase to leave the host
> orchestration for a future consolidation attempts.

Not doing everything at once is of course better. As you said, the first candidates
would be the steps that take a while to be executed, such as ssh finish scripts.
I wonder, if the ssh-finish scripts shouldn't be considered as a form of remote execution
rather than limiting that for just provisioning.

>
> >
> > 3. backward compatibility with the old orchestration: ideally, there should
> > be a way how run the current orchestration steps as part of a Dynflow
> > tasks.
> > Although, the core orchestration would be still benefit more from
> > transferring
> > to dynflow actions, we need a strategy on how to keep third-party
> > plugins
> > (including Foreman hooks) working.
> >
> > 4. On the Foreman side, it would require to drop the Ruby 1.8.7 support.
> >
>
> > Just for the bulk actions, the (1) and (4) should be sufficient to
> > unblock us from proceeding. The (2) and (3) would be required for
> > provisioning
> > orchestration.
> >
> > I believe Foreman would benefit a lot from the adoption (I'm might be a
> > little
> > biased for obvious reasons though), at least we got really good feedback
> > so far when using it for Katello orchestration. The Dynflow itself is also
> > moving forward, with the pitr-ch improved actors implementation getting
> > merged into concurrent-ruby gem [1] (we have a basic one right now
> > directly in Dynflow).
> > We will switch to the concurrent-ruby soon, so that we can share the
> > concurrent stuff
> > with others: it's getting really good feedback so far.
> >
> > So the questions I have are:
> >
> > 1. is the Foreman team willing at some point to merge
> > foreman-tasks into the core (or make it a hard dependency). What are
> > the other things
> > that need to be done before being able to do so?
> >
> I would assume mostly installer, upgrade, docs etc ? potentially a review
> of foreman-tasks code itself?

Sounds good to me.

>
>
> > 2. if (1) holds, what version it would be for (it would need dropping the
> > ruby 1.8.7
> > for that version)?
> >
> IMHO its time, can we target it for 1.7 release?

That would be fantastic.

>
>
> > 3. would it be possible to use Dynflow for the host orchestration? I'm
> > sure it would be
> > possible from the Dynflow point of view (we have a lot of experience
> > with doing so
> > in the Katello codebase), but I'm aware that the orchestration has a
> > long
> > history and people might be defensive on rewriting that.
> >
> I have no objections, but I see smaller value in that vs adding async
> support for long running tasks (e.g. puppet class imports, report
> processing, bulk actions etc).
>
> we could consider extracting really long tasks (such as SSH into an
> instance).

+1

··· ----- Original Message ----- > On Tue, Aug 12, 2014 at 1:07 PM, Ivan Necas wrote:

The answers for this questions would help us better prioritize the things
to keep
things going forward.

[1] - https://github.com/ruby-concurrency/concurrent-ruby

Cheers,

thanks Ivan!

Ohad

Ivan

----- Original Message -----

I was wondering, how useful would it be to take advantage of
Foreman-Tasks
and use it to support many or heavy-to-process requests?

I’m thinking the user could set a threshold per Foreman host depending on
their processing/memory and we make requests be handled by Foreman-tasks
once requests go over that.

For instance, deleting 100 hosts is a task bound to timeout or fail
because
of all the associated data (reports/logs, etc)

Your thoughts?


Daniel Lobato

@elobatoss
blog.daniellobato.me
<
https://mailtrack.io/trace/link/657886dff180af4370c51f673bcde0ece76cba93>
daniellobato.me
<
https://mailtrack.io/trace/link/1b6e7a141b5dd01c9f62582aea5ae796d4d88d57>

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.