Foreman-installer & concat

Gavin_Williams · April 24, 2013, 4:30pm

Afternoon all

I'm currently trying to set-up a new Foreman/Puppet env using the
foreman-installer…

I've pulled down the latest code from Git
(https://github.com/theforeman/foreman-installer), added in the required
modules for mysql support and concat support, and managed to get it
running, to a point…

However I'm currently getting the following error:
echo "class {'::foreman': locations_enabled => true, db_type => 'mysql' }
" |puppet apply -v
Info: Applying configuration version '1366819945'
Error: /Stage[main]/Foreman::Config/Concat_build[foreman_settings]: Could
not evaluate: can't convert nil into String
Notice: /File[/etc/foreman/settings.yaml]: Dependency
Concat_build[foreman_settings] has failures: true
Warning: /File[/etc/foreman/settings.yaml]: Skipping because of failed
dependencies
Notice: /Stage[main]/Foreman::Database/Exec[dbmigrate]: Dependency
Concat_build[foreman_settings] has failures: true
Warning: /Stage[main]/Foreman::Database/Exec[dbmigrate]: Skipping because
of failed dependencies
Notice: /Stage[main]/Foreman::Service/Service[foreman]: Dependency
Concat_build[foreman_settings] has failures: true
Warning: /Stage[main]/Foreman::Service/Service[foreman]: Skipping because
of failed dependencies
Notice: Finished catalog run in 2.45 seconds

I've had a look at the Concat_build provider/type, but couldn't see
anything obvious…

Any ideas???

Cheers
Gavin

Gavin_Williams · April 24, 2013, 4:39pm

As always, just after I posted this, looks like I've solved the problem…

Went back to the puppet-concat module, had a look at a couple of the
examples, and they had a 'target' property.
So I set one, as follows:
foreman/manifests/config.pp:8
concat_build {'foreman_settings':
order => ['*.yaml'],
target => '/etc/foreman/settings.yaml'
}
and re-run the 'puppet apply', and hey-presto, it worked…

echo "class {'::foreman': locations_enabled => true, db_type => 'mysql'

} " |puppet apply -v
Info: Applying configuration version '1366821100'
Notice: /File[/etc/foreman/settings.yaml]/owner: owner changed 'foreman' to
'root'
Info: /File[/etc/foreman/settings.yaml]: Scheduling refresh of
Class[Foreman::Service]
Info: Class[Foreman::Config]: Scheduling refresh of Class[Foreman::Database]
Info: Class[Foreman::Database]: Scheduling refresh of Exec[dbmigrate]
Notice: /Stage[main]/Foreman::Database/Exec[dbmigrate]: Triggered 'refresh'
from 1 events
Info: Class[Foreman::Database]: Scheduling refresh of
Class[Foreman::Service]
Info: Class[Foreman::Service]: Scheduling refresh of Service[foreman]
Notice: /Stage[main]/Foreman::Service/Service[foreman]: Triggered 'refresh'
from 1 events
Notice: Finished catalog run in 12.91 seconds

So not sure if that's the right behaviour, but it works…

Can submit a PR if required.

Cheers
Gavin

···

On Wednesday, 24 April 2013 17:30:25 UTC+1, Gavin Williams wrote: > > Afternoon all > > I'm currently trying to set-up a new Foreman/Puppet env using the > foreman-installer... > > I've pulled down the latest code from Git ( > https://github.com/theforeman/foreman-installer), added in the required > modules for mysql support and concat support, and managed to get it > running, to a point... > > However I'm currently getting the following error: > echo "class {'::foreman': locations_enabled => true, db_type => 'mysql' } > " |puppet apply -v > Info: Applying configuration version '1366819945' > Error: /Stage[main]/Foreman::Config/Concat_build[foreman_settings]: Could > not evaluate: can't convert nil into String > Notice: /File[/etc/foreman/settings.yaml]: Dependency > Concat_build[foreman_settings] has failures: true > Warning: /File[/etc/foreman/settings.yaml]: Skipping because of failed > dependencies > Notice: /Stage[main]/Foreman::Database/Exec[dbmigrate]: Dependency > Concat_build[foreman_settings] has failures: true > Warning: /Stage[main]/Foreman::Database/Exec[dbmigrate]: Skipping because > of failed dependencies > Notice: /Stage[main]/Foreman::Service/Service[foreman]: Dependency > Concat_build[foreman_settings] has failures: true > Warning: /Stage[main]/Foreman::Service/Service[foreman]: Skipping because > of failed dependencies > Notice: Finished catalog run in 2.45 seconds > > I've had a look at the Concat_build provider/type, but couldn't see > anything obvious... > > Any ideas??? > > Cheers > Gavin >

Gwmngilfen · April 24, 2013, 4:42pm

I was just about to reply to the earlier message, that saves some time

As I recall, the target is left out of that resource for a reason -
specifically, concat_build doesn't do filebucketing when there is a change
to the file. So, instead, we simply build the file, and then let the file
resource a few lines further down handle copying it out of $vardir and into
the correct place on the filesystem.

What's interesting is that I don't hit this in my testing - what OS /
puppet version / ruby version are you running?

Greg

···

On 24 April 2013 17:39, Gavin Williams wrote:

As always, just after I posted this, looks like I’ve solved the problem…

Gavin_Williams · April 24, 2013, 4:46pm

Greg,

Yeh, did notice that concat_output function was there…

Env is:
Centos 6.3
Puppet v3.1.1
ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]

Happy to run any more tests as needed

Cheers
Gavin

···

On 24 April 2013 17:42, Greg Sutcliffe wrote:

On 24 April 2013 17:39, Gavin Williams fatmcgav@gmail.com wrote:

As always, just after I posted this, looks like I’ve solved the
problem…

I was just about to reply to the earlier message, that saves some time

As I recall, the target is left out of that resource for a reason -
specifically, concat_build doesn’t do filebucketing when there is a change
to the file. So, instead, we simply build the file, and then let the file
resource a few lines further down handle copying it out of $vardir and into
the correct place on the filesystem.

What’s interesting is that I don’t hit this in my testing - what OS /
puppet version / ruby version are you running?

Greg

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/7MsbECLOXrc/unsubscribe?hl=en
.
To unsubscribe from this group and all its topics, send an email to
foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Gwmngilfen · April 24, 2013, 4:49pm

Hmm, thats not far different to my setup, Debian Squeeze/3.1.1/1.8.7. I'll
see if I can replicate it later this week on your OS. In the meantime,
should you have any spare time, feel free to see if there's a bug in the
type/provider

···

On 24 April 2013 17:46, fatmcgav wrote:

Greg,

Yeh, did notice that concat_output function was there…

Env is:
Centos 6.3
Puppet v3.1.1
ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]

Gavin_Williams · April 25, 2013, 8:08am

Ah, think I've found it…

Mocked up a test case which gave me the same error:

class scratch {
>
> concat_build { 'test':
> order => ['*.tmp']
> }
>
> concat_fragment { 'test+01.tmp':
> content => 'this is test contents'
> }
>
> }
>
>
echo 'include scratch' |puppet apply -v
> Info: Applying configuration version '1366876755'
> Error: /Stage[main]/Scratch/Concat_build[test]: Could not evaluate: can't
> convert nil into String
> Notice: Finished catalog run in 0.12 seconds
>

Ran with '-trace', and got this:

echo 'include scratch' |puppet apply -v --trace
> Info: Applying configuration version '1366876780'
> Error: /Stage[main]/Scratch/Concat_build[test]: Could not evaluate: can't
> convert nil into String
> /etc/puppet/modules/common/concat/lib/puppet/type/concat_build.rb:171:in
> exists?' > /etc/puppet/modules/common/concat/lib/puppet/type/concat_build.rb:171:in >insync?'
> /usr/lib/ruby/site_ruby/1.8/puppet/property.rb:305:in safe_insync?' > /usr/lib/ruby/site_ruby/1.8/puppet/transaction/resource_harness.rb:61:in >perform_changes'
> /usr/lib/ruby/site_ruby/1.8/puppet/transaction/resource_harness.rb:60:in
> each' > /usr/lib/ruby/site_ruby/1.8/puppet/transaction/resource_harness.rb:60:in >perform_changes'
> /usr/lib/ruby/site_ruby/1.8/puppet/transaction/resource_harness.rb:133:in
> evaluate' > /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:48:inapply'
> /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:83:in eval_resource' > /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:103:inevaluate'
> /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:351:in thinmark' > /usr/lib/ruby/1.8/benchmark.rb:308:inrealtime'
> /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:350:in thinmark' > /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:103:inevaluate'
> /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:382:in traverse' > /usr/lib/ruby/site_ruby/1.8/puppet/transaction.rb:98:inevaluate'
> /usr/lib/ruby/site_ruby/1.8/puppet/resource/catalog.rb:144:in apply' > /usr/lib/ruby/site_ruby/1.8/puppet/configurer.rb:122:inapply_catalog'
> /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:179:in benchmark' > /usr/lib/ruby/1.8/benchmark.rb:308:inrealtime'
> /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:178:in benchmark' > /usr/lib/ruby/site_ruby/1.8/puppet/configurer.rb:121:inapply_catalog'
> /usr/lib/ruby/site_ruby/1.8/puppet/configurer.rb:179:in run' > /usr/lib/ruby/site_ruby/1.8/puppet/application/apply.rb:265:in >apply_catalog'
> /usr/lib/ruby/site_ruby/1.8/puppet/application/apply.rb:213:in main' > /usr/lib/ruby/site_ruby/1.8/puppet/application/apply.rb:146:in >run_command'
> /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:364:in run' > /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:456:inplugin_hook'
> /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:364:in run' > /usr/lib/ruby/site_ruby/1.8/puppet/util.rb:504:inexit_on_fail'
> /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:364:in run' > /usr/lib/ruby/site_ruby/1.8/puppet/util/command_line.rb:132:inrun'
> /usr/lib/ruby/site_ruby/1.8/puppet/util/command_line.rb:86:in `execute'
> /usr/bin/puppet:4
> Notice: Finished catalog run in 0.12 seconds
>
>
Line 171 is:

return false unless File.exists?(@resource[:target])
>

So I'm guessing that File.exists? doesn't like a nil value…

So, added the following line prior to 171, and it seems to work as expected:

> return false if @resource[:target].nil?
>

And lo and behold, it now works

echo 'include scratch' |puppet apply -v --trace
> Info: Applying configuration version '1366877049'
> Notice: /Stage[main]/Scratch/Concat_build[test]/order: *.tmp used for
> ordering
> Notice: Finished catalog run in 0.13 seconds
>

Happy to submit a PR if it helps

Cheers
Gavin

···

On Wednesday, 24 April 2013 17:49:11 UTC+1, Greg Sutcliffe wrote: > > On 24 April 2013 17:46, fatmcgav <fatm...@gmail.com > wrote: > >> Greg, >> >> Yeh, did notice that concat_output function was there... >> >> Env is: >> Centos 6.3 >> Puppet v3.1.1 >> ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux] >> > > Hmm, thats not far different to my setup, Debian Squeeze/3.1.1/1.8.7.I'll see if I can replicate it later this week on your OS. In the meantime, > should you have any spare time, feel free to see if there's a bug in the > type/provider ;) >

Gwmngilfen · April 25, 2013, 9:31am

> Ah, think I've found it…
>
> Mocked up a test case which gave me the same error:
>

Awesome!

> Happy to submit a PR if it helps

It's not our module, we use OnyxPoint's concat module. Submit it there, or
I will if you like.

Thanks for the debug

Greg

···

On 25 April 2013 09:08, Gavin Williams wrote:

Gavin_Williams · April 25, 2013, 9:37am

Greg

No worries, glad I could help…

Guess it's probably easier if you submit a PR, as you have a better
understanding of the usage decisions taken on the Foreman modules to not
specify a target

Cheers
Gav

···

On 25 April 2013 10:31, Greg Sutcliffe wrote:

On 25 April 2013 09:08, Gavin Williams fatmcgav@gmail.com wrote:

Ah, think I’ve found it…

Mocked up a test case which gave me the same error:

Awesome!

Happy to submit a PR if it helps

It’s not our module, we use OnyxPoint’s concat module. Submit it there, or
I will if you like.

Thanks for the debug

Greg

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/7MsbECLOXrc/unsubscribe?hl=en
.
To unsubscribe from this group and all its topics, send an email to
foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Gwmngilfen · April 25, 2013, 3:35pm

Before I do, can I ask where your sourced the installer from? Was it the
1.1 package by any chance? I think this bug is actually already fixed in
the develop branch of our installer

Greg

···

On 25 April 2013 10:37, fatmcgav wrote:

Greg

No worries, glad I could help…

Guess it’s probably easier if you submit a PR, as you have a better
understanding of the usage decisions taken on the Foreman modules to not
specify a target

Cheers
Gav

Gavin_Williams · April 25, 2013, 3:41pm

Nope, was all pulled down from Git master…

···

On 25 April 2013 16:35, Greg Sutcliffe wrote:

On 25 April 2013 10:37, fatmcgav fatmcgav@gmail.com wrote:

Greg

No worries, glad I could help…

Guess it’s probably easier if you submit a PR, as you have a better
understanding of the usage decisions taken on the Foreman modules to not
specify a target

Cheers
Gav

Before I do, can I ask where your sourced the installer from? Was it the
1.1 package by any chance? I think this bug is actually already fixed in
the develop branch of our installer

Greg

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/7MsbECLOXrc/unsubscribe?hl=en
.
To unsubscribe from this group and all its topics, send an email to
foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Gavin_Williams · April 25, 2013, 3:45pm

Ahh, now I look at the Foreman-installer develop branch, can see that
concat is now pulled from onyxpoint repo rather than the foreman fork…
And looks like the issue with a missing target HAS been fixed in the
onyxpoint module

···

On Thursday, 25 April 2013 16:41:25 UTC+1, Gavin Williams wrote: > > Nope, was all pulled down from Git master... > > > On 25 April 2013 16:35, Greg Sutcliffe wrote: > >> On 25 April 2013 10:37, fatmcgav wrote: >> >>> Greg >>> >>> No worries, glad I could help... :) >>> >>> Guess it's probably easier if you submit a PR, as you have a better >>> understanding of the usage decisions taken on the Foreman modules to not >>> specify a target :) >>> >>> Cheers >>> Gav >>> >> >> Before I do, can I ask where your sourced the installer from? Was it the >> 1.1 package by any chance? I think this bug is actually already fixed in >> the develop branch of our installer ;) >> >> Greg >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Foreman users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/foreman-users/7MsbECLOXrc/unsubscribe?hl=en >> . >> To unsubscribe from this group and all its topics, send an email to >> foreman-users+unsubscribe@googlegroups.com. >> To post to this group, send email to foreman-users@googlegroups.com. >> Visit this group at http://groups.google.com/group/foreman-users?hl=en. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > >

Gwmngilfen · April 25, 2013, 3:46pm

Indeed, master != develop - we merge develop into master just before
cutting a new package, so essentially master and the package are the same
thing. Thanks for confirming though

Greg

···

On 25 April 2013 16:45, Gavin Williams wrote:

Ahh, now I look at the Foreman-installer develop branch, can see that
concat is now pulled from onyxpoint repo rather than the foreman fork…
And looks like the issue with a missing target HAS been fixed in the
onyxpoint module

Dominic_Cleal1 · May 1, 2013, 8:01am

We switched over to the upstream Onyxpoint module in the
foreman-installer repo (which has puppet-puppet etc as submodules), so
you'll want to switch over to that too if you've updated puppet-puppet.

···

-- Dominic Cleal Red Hat Engineering

On 30/04/13 22:05, Michael Robbert wrote:

Greg,
I’m hitting this same bug while using the most up to date branches of
the puppet-concat and puppet-puppet repositories from theforeman’s
github. It looks like that fork of the concat module hasn’t merged with
Onyxpoint in awhile. Should I be using it at all or will it be updated soon?

Thanks,
Mike

On Thursday, April 25, 2013 9:46:39 AM UTC-6, Greg Sutcliffe wrote:
On 25 April 2013 16:45, Gavin Williams <fatm...@gmail.com >     <javascript:>> wrote:

    Ahh, now I look at the Foreman-installer develop branch, can see
    that concat is now pulled from onyxpoint repo rather than the
    foreman fork... And looks like the issue with a missing target
    HAS been fixed in the onyxpoint module :) 


Indeed, master != develop :) - we merge develop into master just
before cutting a new package, so essentially master and the package
are the same thing. Thanks for confirming though :)

Greg
–
You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Gwmngilfen · May 1, 2013, 8:55am

Is that the "master" branch - that does have the old concat module, yes.
The "develop" branch of the installer uses the new version from OnyxPoint,
as Dominic says.

···

On 30 April 2013 22:05, Michael Robbert wrote:

Greg,
I’m hitting this same bug while using the most up to date branches of the
puppet-concat and puppet-puppet repositories from theforeman’s github. It
looks like that fork of the concat module hasn’t merged with Onyxpoint in
awhile. Should I be using it at all or will it be updated soon?

Jan_Vansteenkiste · May 1, 2013, 9:03am

Don't forget to run

git submodules sync; git submodules update

after checking out the develop branch of the foreman-installer.

···

On Wed 01 May 2013 10:55:56 AM CEST, Greg Sutcliffe wrote: > On 30 April 2013 22:05, Michael Robbert wrote: > >> Greg, >> I'm hitting this same bug while using the most up to date branches of the >> puppet-concat and puppet-puppet repositories from theforeman's github. It >> looks like that fork of the concat module hasn't merged with Onyxpoint in >> awhile. Should I be using it at all or will it be updated soon? >> > > Is that the "master" branch - that does have the old concat module, yes. > The "develop" branch of the installer uses the new version from OnyxPoint, > as Dominic says. >

Michael_Robbert · May 1, 2013, 3:30pm

So, to be clear the repo at https://github.com/theforeman/puppet-concat is
now obsolete and should not be used? Can we then add a dependency in the
documentation or Modulefile for puppet-puppet.

Mike

···

On Wed, May 1, 2013 at 3:03 AM, Jan Vansteenkiste wrote:

On Wed 01 May 2013 10:55:56 AM CEST, Greg Sutcliffe wrote:

On 30 April 2013 22:05, Michael Robbert mrobbert@gmail.com wrote:

Greg,
I’m hitting this same bug while using the most up to date branches of
the

puppet-concat and puppet-puppet repositories from theforeman’s github.
It

looks like that fork of the concat module hasn’t merged with Onyxpoint
in

awhile. Should I be using it at all or will it be updated soon?

Is that the “master” branch - that does have the old concat module, yes.
The “develop” branch of the installer uses the new version from
OnyxPoint,
as Dominic says.

Don’t forget to run

git submodules sync; git submodules update

after checking out the develop branch of the foreman-installer.

–
You received this message because you are subscribed to a topic in the
Google Groups “Foreman users” group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/foreman-users/7MsbECLOXrc/unsubscribe?hl=en
.
To unsubscribe from this group and all its topics, send an email to
foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Gwmngilfen · May 1, 2013, 4:06pm

For develop, yes, we use Onyxpoint's module directly (although we may have
to fork it again yet :P). For Master, that change has not yet been merged,
so our puppet-concat module is still in use.

···

On 1 May 2013 16:30, Michael Robbert wrote:

So, to be clear the repo at https://github.com/theforeman/puppet-concatis now obsolete and should not be used? Can we then add a dependency in the
documentation or Modulefile for puppet-puppet.

Mike