Provisioning templates disapeared

Went in to Foreman today and all the provisioning templates we had
assigned to groups were gone. Unassigned templates were still there.
Checked the database directly and they are not in the config_templates
table.

What could could have caused this?

Only major difference in the environment that occurred was the
enabling of the cron jobs in the Foreman puppet class last Thursday.
They were installed but failing the last few weeks due to auth
errors. When the foreman account was granted the rights to run cron
jobs, they started running and we started receiving the email
summaries.

These are the cron jobs:

HEADER: This file was autogenerated at Mon Jun 20 12:12:27 -0400

2011 by puppet.

HEADER: While it can still be managed manually, it is definitely not

recommended.

HEADER: Note particularly that the comments starting with 'Puppet

Name' should

HEADER: not be deleted, as doing so could cause duplicate cron jobs.

Puppet Name: populate_hosts

RAILS_ENV=production
*/30 * * * * (cd /usr/share/foreman && rake
puppet:migrate:populate_hosts)

Puppet Name: clear_session_table

RAILS_ENV=production
15 23 * * * (cd /usr/share/foreman && rake db:sessions:clear)

Puppet Name: expire_old_reports

RAILS_ENV=production
30 7 * * * (cd /usr/share/foreman && rake reports:expire)

Puppet Name: daily summary

RAILS_ENV=production
30 7 * * * (cd /usr/share/foreman && rake reports:summarize)

Kind of a bummer. Spent a long time tweaking those kickstart
scripts. Shouldn't be lost though. Think I can get a replay off the
SAN and restore them. But would like to have them stay in place.

Any ideas?

> Went in to Foreman today and all the provisioning templates we had
> assigned to groups were gone. Unassigned templates were still there.
> Checked the database directly and they are not in the config_templates
> table.
>
> What could could have caused this?

Did you change / delete any of your groups?
I can't see any reason of why the associations would be deleted.
In your logs, do you see any calls to the config_template controller ?
(are you trying to create / update those via the API or simply using the
UI?).

>
> Only major difference in the environment that occurred was the
> enabling of the cron jobs in the Foreman puppet class last Thursday.
> They were installed but failing the last few weeks due to auth
> errors. When the foreman account was granted the rights to run cron
> jobs, they started running and we started receiving the email
> summaries.
>
> These are the cron jobs:
>
>
> # HEADER: This file was autogenerated at Mon Jun 20 12:12:27 -0400
> 2011 by puppet.
> # HEADER: While it can still be managed manually, it is definitely not
> recommended.
> # HEADER: Note particularly that the comments starting with 'Puppet
> Name' should
> # HEADER: not be deleted, as doing so could cause duplicate cron jobs.
> # Puppet Name: populate_hosts
> RAILS_ENV=production
> */30 * * * * (cd /usr/share/foreman && rake
> puppet:migrate:populate_hosts)
> # Puppet Name: clear_session_table
> RAILS_ENV=production
> 15 23 * * * (cd /usr/share/foreman && rake db:sessions:clear)
> # Puppet Name: expire_old_reports
> RAILS_ENV=production
> 30 7 * * * (cd /usr/share/foreman && rake reports:expire)
> # Puppet Name: daily summary
> RAILS_ENV=production
> 30 7 * * * (cd /usr/share/foreman && rake reports:summarize)
>
None of these have any relationship to the templates…
>
> Kind of a bummer. Spent a long time tweaking those kickstart
> scripts. Shouldn't be lost though. Think I can get a replay off the
> SAN and restore them. But would like to have them stay in place.

> Any ideas?
>

I'm assuming you mean that only the relationship between a hostgroup to
a template is removed, not the template it self ?
if thats the case, it might be possible to recover it via the logs
(e.g. when they were created).

one of my goals for this version, is to have a better audit / undo
options…

Ohad

··· On Mon, 2011-07-11 at 15:15 -0700, Roger wrote:

The templates themselves are gone. But the only templates that were deleted
were ones that were associated with a host group.

I did delete a host group. It no longer had any hosts associated with it.
I guess that's when the provisioning templates got chopped.

The provisioning templates that are gone had the host group I deleted
associated with them – but they also had 3 other host
groups associated with them.

··· On Tue, Jul 12, 2011 at 2:11 AM, Ohad Levy wrote:

On Mon, 2011-07-11 at 15:15 -0700, Roger wrote:

Went in to Foreman today and all the provisioning templates we had
assigned to groups were gone. Unassigned templates were still there.
Checked the database directly and they are not in the config_templates
table.

What could could have caused this?

Did you change / delete any of your groups?
I can’t see any reason of why the associations would be deleted.
In your logs, do you see any calls to the config_template controller ?
(are you trying to create / update those via the API or simply using the
UI?).

Only major difference in the environment that occurred was the
enabling of the cron jobs in the Foreman puppet class last Thursday.
They were installed but failing the last few weeks due to auth
errors. When the foreman account was granted the rights to run cron
jobs, they started running and we started receiving the email
summaries.

These are the cron jobs:

HEADER: This file was autogenerated at Mon Jun 20 12:12:27 -0400

2011 by puppet.

HEADER: While it can still be managed manually, it is definitely not

recommended.

HEADER: Note particularly that the comments starting with 'Puppet

Name’ should

HEADER: not be deleted, as doing so could cause duplicate cron jobs.

Puppet Name: populate_hosts

RAILS_ENV=production
*/30 * * * * (cd /usr/share/foreman && rake
puppet:migrate:populate_hosts)

Puppet Name: clear_session_table

RAILS_ENV=production
15 23 * * * (cd /usr/share/foreman && rake db:sessions:clear)

Puppet Name: expire_old_reports

RAILS_ENV=production
30 7 * * * (cd /usr/share/foreman && rake reports:expire)

Puppet Name: daily summary

RAILS_ENV=production
30 7 * * * (cd /usr/share/foreman && rake reports:summarize)

None of these have any relationship to the templates…

Kind of a bummer. Spent a long time tweaking those kickstart
scripts. Shouldn’t be lost though. Think I can get a replay off the
SAN and restore them. But would like to have them stay in place.

Any ideas?

I’m assuming you mean that only the relationship between a hostgroup to
a template is removed, not the template it self ?
if thats the case, it might be possible to recover it via the logs
(e.g. when they were created).

one of my goals for this version, is to have a better audit / undo
options…

Ohad


You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To post to this group, send email to foreman-users@googlegroups.com.
To unsubscribe from this group, send email to
foreman-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/foreman-users?hl=en.

> The templates themselves are gone. But the only templates that were
> deleted were ones that were associated with a host group.
>
> I did delete a host group. It no longer had any hosts associated with it.
> I guess that's when the provisioning templates got chopped.
>
> The provisioning templates that are gone had the host group I deleted
> associated with them – but they also had 3 other host
> groups associated with them.
>
brr… sounds like a bug… (sorry about that).

Would you mind doing two things

  1. see if you can reproduce this
  2. open a ticket

I'll try solving it asap…

Ohad

··· On Tue, Jul 12, 2011 at 5:54 PM, Roger Spencer wrote:

On Tue, Jul 12, 2011 at 2:11 AM, Ohad Levy ohadlevy@gmail.com wrote:

On Mon, 2011-07-11 at 15:15 -0700, Roger wrote:

Went in to Foreman today and all the provisioning templates we had
assigned to groups were gone. Unassigned templates were still there.
Checked the database directly and they are not in the config_templates
table.

What could could have caused this?

Did you change / delete any of your groups?
I can’t see any reason of why the associations would be deleted.
In your logs, do you see any calls to the config_template controller ?
(are you trying to create / update those via the API or simply using the
UI?).

Only major difference in the environment that occurred was the
enabling of the cron jobs in the Foreman puppet class last Thursday.
They were installed but failing the last few weeks due to auth
errors. When the foreman account was granted the rights to run cron
jobs, they started running and we started receiving the email
summaries.

These are the cron jobs:

HEADER: This file was autogenerated at Mon Jun 20 12:12:27 -0400

2011 by puppet.

HEADER: While it can still be managed manually, it is definitely not

recommended.

HEADER: Note particularly that the comments starting with 'Puppet

Name’ should

HEADER: not be deleted, as doing so could cause duplicate cron jobs.

Puppet Name: populate_hosts

RAILS_ENV=production
*/30 * * * * (cd /usr/share/foreman && rake
puppet:migrate:populate_hosts)

Puppet Name: clear_session_table

RAILS_ENV=production
15 23 * * * (cd /usr/share/foreman && rake db:sessions:clear)

Puppet Name: expire_old_reports

RAILS_ENV=production
30 7 * * * (cd /usr/share/foreman && rake reports:expire)

Puppet Name: daily summary

RAILS_ENV=production
30 7 * * * (cd /usr/share/foreman && rake reports:summarize)

None of these have any relationship to the templates…

Kind of a bummer. Spent a long time tweaking those kickstart
scripts. Shouldn’t be lost though. Think I can get a replay off the
SAN and restore them. But would like to have them stay in place.

Any ideas?

I’m assuming you mean that only the relationship between a hostgroup to
a template is removed, not the template it self ?
if thats the case, it might be possible to recover it via the logs
(e.g. when they were created).

one of my goals for this version, is to have a better audit / undo
options…

Ohad

You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To post to this group, send email to foreman-users@googlegroups.com.
To unsubscribe from this group, send email to
foreman-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/foreman-users?hl=en.


You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To post to this group, send email to foreman-users@googlegroups.com.
To unsubscribe from this group, send email to
foreman-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/foreman-users?hl=en.

>
>
>
>> The templates themselves are gone. But the only templates that were
>> deleted were ones that were associated with a host group.
>>
>> I did delete a host group. It no longer had any hosts associated with it.
>> I guess that's when the provisioning templates got chopped.
>>
>> The provisioning templates that are gone had the host group I deleted
>> associated with them – but they also had 3 other host
>> groups associated with them.
>>
> brr… sounds like a bug… (sorry about that).
>
> Would you mind doing two things
> 1. see if you can reproduce this
> 2. open a ticket
>
> I'll try solving it asap…
>
> fixed at Bug #1040: deleting a hostgroup, may delete a configuration template too - Foreman

the patch itself is a one liner, so maybe worth applying to your setup.

Ohad

··· On Tue, Jul 12, 2011 at 5:57 PM, Ohad Levy wrote: > On Tue, Jul 12, 2011 at 5:54 PM, Roger Spencer wrote:

Ohad

On Tue, Jul 12, 2011 at 2:11 AM, Ohad Levy ohadlevy@gmail.com wrote:

On Mon, 2011-07-11 at 15:15 -0700, Roger wrote:

Went in to Foreman today and all the provisioning templates we had
assigned to groups were gone. Unassigned templates were still there.
Checked the database directly and they are not in the config_templates
table.

What could could have caused this?

Did you change / delete any of your groups?
I can’t see any reason of why the associations would be deleted.
In your logs, do you see any calls to the config_template controller ?
(are you trying to create / update those via the API or simply using the
UI?).

Only major difference in the environment that occurred was the
enabling of the cron jobs in the Foreman puppet class last Thursday.
They were installed but failing the last few weeks due to auth
errors. When the foreman account was granted the rights to run cron
jobs, they started running and we started receiving the email
summaries.

These are the cron jobs:

HEADER: This file was autogenerated at Mon Jun 20 12:12:27 -0400

2011 by puppet.

HEADER: While it can still be managed manually, it is definitely not

recommended.

HEADER: Note particularly that the comments starting with 'Puppet

Name’ should

HEADER: not be deleted, as doing so could cause duplicate cron jobs.

Puppet Name: populate_hosts

RAILS_ENV=production
*/30 * * * * (cd /usr/share/foreman && rake
puppet:migrate:populate_hosts)

Puppet Name: clear_session_table

RAILS_ENV=production
15 23 * * * (cd /usr/share/foreman && rake db:sessions:clear)

Puppet Name: expire_old_reports

RAILS_ENV=production
30 7 * * * (cd /usr/share/foreman && rake reports:expire)

Puppet Name: daily summary

RAILS_ENV=production
30 7 * * * (cd /usr/share/foreman && rake reports:summarize)

None of these have any relationship to the templates…

Kind of a bummer. Spent a long time tweaking those kickstart
scripts. Shouldn’t be lost though. Think I can get a replay off the
SAN and restore them. But would like to have them stay in place.

Any ideas?

I’m assuming you mean that only the relationship between a hostgroup to
a template is removed, not the template it self ?
if thats the case, it might be possible to recover it via the logs
(e.g. when they were created).

one of my goals for this version, is to have a better audit / undo
options…

Ohad

You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To post to this group, send email to foreman-users@googlegroups.com.
To unsubscribe from this group, send email to
foreman-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/foreman-users?hl=en.


You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To post to this group, send email to foreman-users@googlegroups.com.
To unsubscribe from this group, send email to
foreman-users+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/foreman-users?hl=en.

Thanks for the quick fix. I really appreciate it.

··· On Jul 12, 11:10 am, Ohad Levy wrote: > On Tue, Jul 12, 2011 at 5:57 PM, Ohad Levy wrote: > > > On Tue, Jul 12, 2011 at 5:54 PM, Roger Spencer wrote: > > >> The templates themselves are gone. But the only templates that were > >> deleted were ones that were associated with a host group. > > >> I did delete a host group. It no longer had any hosts associated with it. > >> I guess that's when the provisioning templates got chopped. > > >> The provisioning templates that are gone had the host group I deleted > >> associated with them -- but they also had 3 other host > >> groups associated with them. > > > brr.. sounds like a bug... (sorry about that). > > > Would you mind doing two things > > 1. see if you can reproduce this > > 2. open a ticket > > > I'll try solving it asap... > > > fixed athttp://theforeman.org/issues/1040 > > the patch itself is a one liner, so maybe worth applying to your setup. > > Ohad > > > > > > > > > Ohad > > >> On Tue, Jul 12, 2011 at 2:11 AM, Ohad Levy wrote: > > >>> On Mon, 2011-07-11 at 15:15 -0700, Roger wrote: > >>> > Went in to Foreman today and all the provisioning templates we had > >>> > assigned to groups were gone. Unassigned templates were still there. > >>> > Checked the database directly and they are not in the config_templates > >>> > table. > > >>> > What could could have caused this? > > >>> Did you change / delete any of your groups? > >>> I can't see any reason of why the associations would be deleted. > >>> In your logs, do you see any calls to the config_template controller ? > >>> (are you trying to create / update those via the API or simply using the > >>> UI?). > > >>> > Only major difference in the environment that occurred was the > >>> > enabling of the cron jobs in the Foreman puppet class last Thursday. > >>> > They were installed but failing the last few weeks due to auth > >>> > errors. When the foreman account was granted the rights to run cron > >>> > jobs, they started running and we started receiving the email > >>> > summaries. > > >>> > These are the cron jobs: > > >>> > # HEADER: This file was autogenerated at Mon Jun 20 12:12:27 -0400 > >>> > 2011 by puppet. > >>> > # HEADER: While it can still be managed manually, it is definitely not > >>> > recommended. > >>> > # HEADER: Note particularly that the comments starting with 'Puppet > >>> > Name' should > >>> > # HEADER: not be deleted, as doing so could cause duplicate cron jobs. > >>> > # Puppet Name: populate_hosts > >>> > RAILS_ENV=production > >>> > */30 * * * * (cd /usr/share/foreman && rake > >>> > puppet:migrate:populate_hosts) > >>> > # Puppet Name: clear_session_table > >>> > RAILS_ENV=production > >>> > 15 23 * * * (cd /usr/share/foreman && rake db:sessions:clear) > >>> > # Puppet Name: expire_old_reports > >>> > RAILS_ENV=production > >>> > 30 7 * * * (cd /usr/share/foreman && rake reports:expire) > >>> > # Puppet Name: daily summary > >>> > RAILS_ENV=production > >>> > 30 7 * * * (cd /usr/share/foreman && rake reports:summarize) > > >>> None of these have any relationship to the templates.. > > >>> > Kind of a bummer. Spent a long time tweaking those kickstart > >>> > scripts. Shouldn't be lost though. Think I can get a replay off the > >>> > SAN and restore them. But would like to have them stay in place. > > >>> > Any ideas? > > >>> I'm assuming you mean that only the relationship between a hostgroup to > >>> a template is removed, not the template it self ? > >>> if thats the case, it *might* be possible to recover it via the logs > >>> (e.g. when they were created). > > >>> one of my goals for this version, is to have a better audit / undo > >>> options... > > >>> Ohad > > >>> -- > > >>> You received this message because you are subscribed to the Google Groups > >>> "Foreman users" group. > >>> To post to this group, send email to foreman-users@googlegroups.com. > >>> To unsubscribe from this group, send email to > >>> foreman-users+unsubscribe@googlegroups.com. > >>> For more options, visit this group at > >>>http://groups.google.com/group/foreman-users?hl=en. > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Foreman users" group. > >> To post to this group, send email to foreman-users@googlegroups.com. > >> To unsubscribe from this group, send email to > >> foreman-users+unsubscribe@googlegroups.com. > >> For more options, visit this group at > >>http://groups.google.com/group/foreman-users?hl=en.