Foreman finish template is not getting resolve when user-data of the image is enabled

Hi,

I am using Foreman 1.6 and using AWS EC2 as compute resource.
Problem is, Foreman could not able to resolve the finish template when the
user-data of image is enabled. And, I could not able to provision the VM.

When user-date of image is* disabled, *foreman able to resolve the
finish-template and able to provision the vm (Without reading the template,
i.e puppet client installation).
Could you guide me where I am going wrong? Its been two week I am
struggling with this issue.

Thanks,
Sekhar

··· -- The information in this message may be proprietary and/or confidential, and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify ATMECS and delete it from your computer.

Balasekhar,

If your image has user-data enabled, you need to associate the
Operating system for that image with a Provisioning template of the
kind 'user-data'. Finish templates are only used for images that don't
have user-data enabled. You can alternatively turn off the user-data
flag for the image and it'll resolve the finish templates instead of
user-data (remember this won't use cloud-init or similar mechanisms).

I actually submitted a pull request a few days ago with this warning,
as I'm afraid it's not very clear:

Thanks for using Foreman!

··· On Mon, Mar 2, 2015 at 3:23 PM, Balasekhar Nelli wrote: > Hi, > > I am using Foreman 1.6 and using AWS EC2 as compute resource. > Problem is, Foreman could not able to resolve the finish template when the > user-data of image is enabled. And, I could not able to provision the VM. > > When user-date of image is disabled, foreman able to resolve the > finish-template and able to provision the vm (Without reading the template, > i.e puppet client installation). > Could you guide me where I am going wrong? Its been two week I am struggling > with this issue. > > Thanks, > Sekhar > > The information in this message may be proprietary and/or confidential, and > protected from disclosure. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. If you have received > this communication in error, please notify ATMECS and delete it from your > computer. > > -- > You received this message because you are subscribed to the Google Groups > "Foreman users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-users+unsubscribe@googlegroups.com. > To post to this group, send email to foreman-users@googlegroups.com. > Visit this group at http://groups.google.com/group/foreman-users. > For more options, visit https://groups.google.com/d/optout.


Daniel Lobato

@elobatoss

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30

Hi Daniel,
Thanks for your reply.
For some reason foreman not resolving any templates other than finish
template.
I have the following scenarios:

  1. If user-data is disabled then foreman is resolving finish template and
    could able to provision VM ( But, without the packages on the finish
    template. Since I have puppet module to get installed on the provisioning
    VM, it requires puppet client ). So, this is not useful.
  2. If user-data is enabled then foreman is not resolving any template ( Not
    even finish template )

For some reason in scenario 1, foreman is not pulling the finish template.

··· On Monday, March 2, 2015 at 6:53:16 PM UTC+5:30, Balasekhar Nelli wrote: > > Hi, > > I am using Foreman 1.6 and using AWS EC2 as compute resource. > Problem is, Foreman could not able to resolve the finish template when the > user-data of image is enabled. And, I could not able to provision the VM. > > When user-date of image is* disabled, *foreman able to resolve the > finish-template and able to provision the vm (Without reading the template, > i.e puppet client installation). > Could you guide me where I am going wrong? Its been two week I am > struggling with this issue. > > Thanks, > Sekhar > > The information in this message may be proprietary and/or confidential, > and protected from disclosure. If the reader of this message is not the > intended recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. If > you have received this communication in error, please notify ATMECS and > delete it from your computer.


The information in this message may be proprietary and/or confidential, and
protected from disclosure. If the reader of this message is not the
intended recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited. If
you have received this communication in error, please notify ATMECS and
delete it from your computer.

<%#
kind: user_data
name: atmecs-aws
oses:

  • CentOS 4
  • CentOS 5
  • CentOS 6
  • CentOS 6.4
  • CentOS 7
  • Fedora 16
  • Fedora 17
  • Fedora 18
  • Fedora 19
  • Fedora 20
    %>

echo "updating system time"
/usr/sbin/ntpdate -sub <%= @host.params["ntp-server"] %>

<%# we only change hostnames on image based provisioning %>
<% if @host.capabilities.include?(:image) -%>
echo "HOSTNAME=<%= @host %>" >> /etc/sysconfig/network
hostname <%= @host %>

cat > /etc/hosts << EOF
<%= snippets "etc-hosts" %>
EOF
<% end -%>

<%# set ssh root key for debugging, it never works on the first time :slight_smile: -%>
<% if @host.params["ssh_debug_key"] -%>
echo <%= @host.params["ssh_debug_key"] %> > /root/.ssh/authorized_keys
<% end -%>

install EPEL

rpm -Uvh
http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

and add the puppet and ruby-shadow package

yum -t -y -e 0 install puppet

echo "Configuring puppet"
cat > /etc/puppet/puppet.conf << EOF
<%= snippets "puppet.conf" %>
EOF

Setup puppet to run on system reboot

/sbin/chkconfig --level 345 puppet on

Disable most things. Puppet will activate these if required.

echo "Disabling various system services"
<% %w{autofs gpm sendmail cups iptables ip6tables auditd arptables_jf xfs
pcmcia isdn rawdevices hpoj bluetooth openibd avahi-daemon avahi-dnsconfd
hidd hplip pcscd restorecond mcstrans rhnsd yum-updatesd}.each do |service|
-%>
/sbin/chkconfig --level 345 <%= service %> off 2>/dev/null
<% end -%>

<% if @host.info["classes"].include? "puppet::server" -%>
<%= snippets "master_bootstrap" %>
<% else -%>
/usr/sbin/puppetd #–config /etc/puppet/puppet.conf -o --tags no_such_tag
–no-daemonize
<% end -%>

consider rebooting here instead, as we changed the hostname

exit 0

··· On Monday, March 2, 2015 at 6:53:16 PM UTC+5:30, Balasekhar Nelli wrote: > > Hi, > > I am using Foreman 1.6 and using AWS EC2 as compute resource. > Problem is, Foreman could not able to resolve the finish template when the > user-data of image is enabled. And, I could not able to provision the VM. > > When user-date of image is* disabled, *foreman able to resolve the > finish-template and able to provision the vm (Without reading the template, > i.e puppet client installation). > Could you guide me where I am going wrong? Its been two week I am > struggling with this issue. > > Thanks, > Sekhar > > The information in this message may be proprietary and/or confidential, > and protected from disclosure. If the reader of this message is not the > intended recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. If > you have received this communication in error, please notify ATMECS and > delete it from your computer.


The information in this message may be proprietary and/or confidential, and
protected from disclosure. If the reader of this message is not the
intended recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited. If
you have received this communication in error, please notify ATMECS and
delete it from your computer.

<https://lh3.googleusercontent.com/-9yc33vJcjq4/VPVhf4e2nSI/AAAAAAAAAEo/XU_B1V5HWio/s1600/foreman1.png>
Hi Daniel,
I did as you mentioned above. YeS!! could able to resolve user_data
template with user-data enabled.
But I still struck with few issues. VM got provisioned and it seems like it
did not pull the user_data template again ( Since puppet client package did
not install ).

I have few questions:

  1. Can't I install a puppet module with finish-template ?
  2. Is user_data is compulsory to pull the template ( i.e reading the
    template from foreman )?
  3. Do you think I went wrong in my configuration?
  4. Is it possible to provision VM (With template) with user-data disabled.
··· On Monday, March 2, 2015 at 6:53:16 PM UTC+5:30, Balasekhar Nelli wrote: > > Hi, > > I am using Foreman 1.6 and using AWS EC2 as compute resource. > Problem is, Foreman could not able to resolve the finish template when the > user-data of image is enabled. And, I could not able to provision the VM. > > When user-date of image is* disabled, *foreman able to resolve the > finish-template and able to provision the vm (Without reading the template, > i.e puppet client installation). > Could you guide me where I am going wrong? Its been two week I am > struggling with this issue. > > Thanks, > Sekhar > > The information in this message may be proprietary and/or confidential, > and protected from disclosure. If the reader of this message is not the > intended recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. If > you have received this communication in error, please notify ATMECS and > delete it from your computer.


The information in this message may be proprietary and/or confidential, and
protected from disclosure. If the reader of this message is not the
intended recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited. If
you have received this communication in error, please notify ATMECS and
delete it from your computer.

This the user_data template that I am using:

<%#
kind: user_data
name: Kickstart default user data
oses:
- CentOS 4
- CentOS 5
- CentOS 6
- CentOS 7
- Fedora 16
- Fedora 17
- Fedora 18
- Fedora 19
- Fedora 20
-%>
#!/bin/bash
rpm -ivh
http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

<%# Cloud instances frequently have incorrect hosts data %>
<%= snippet 'fix_hosts' %>

<%

  • safemode renderer does not support unary negation*

  • pm_set = @host.puppetmaster.empty? ? false : true*
  • puppet_enabled = pm_set || @host.params['force-puppet']*
    %>
    <% if puppet_enabled %>
    yum install -y puppet
    cat > /etc/puppet/puppet.conf << EOF
    <%= snippet 'puppet.conf' %>
    EOF

# Setup puppet to run on system reboot
/sbin/chkconfig --level 345 puppet on

/usr/bin/puppet agent --config /etc/puppet/puppet.conf --onetime --tags
no_such_tag <%= @host.puppetmaster.blank? ? '' : "–server
#{@host.puppetmaster}" %> --no-daemonize

<% end -%>

# UserData still needs wget to mark as finished
/usr/bin/wget --quiet --output-document=/dev/null --no-check-certificate
<%= foreman_url('built') %>

··· On Monday, March 2, 2015 at 6:53:16 PM UTC+5:30, Balasekhar Nelli wrote: > > Hi, > > I am using Foreman 1.6 and using AWS EC2 as compute resource. > Problem is, Foreman could not able to resolve the finish template when the > user-data of image is enabled. And, I could not able to provision the VM. > > When user-date of image is* disabled, *foreman able to resolve the > finish-template and able to provision the vm (Without reading the template, > i.e puppet client installation). > Could you guide me where I am going wrong? Its been two week I am > struggling with this issue. > > Thanks, > Sekhar > > The information in this message may be proprietary and/or confidential, > and protected from disclosure. If the reader of this message is not the > intended recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. If > you have received this communication in error, please notify ATMECS and > delete it from your computer.


The information in this message may be proprietary and/or confidential, and
protected from disclosure. If the reader of this message is not the
intended recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited. If
you have received this communication in error, please notify ATMECS and
delete it from your computer.

Is AMI causing a issue? I am using CentOS 6.5 Operating System. One more
thing is, I could not able to ssh to foreman server to provisioned VM and
vice versa.

··· On Monday, March 2, 2015 at 6:53:16 PM UTC+5:30, Balasekhar Nelli wrote: > > Hi, > > I am using Foreman 1.6 and using AWS EC2 as compute resource. > Problem is, Foreman could not able to resolve the finish template when the > user-data of image is enabled. And, I could not able to provision the VM. > > When user-date of image is* disabled, *foreman able to resolve the > finish-template and able to provision the vm (Without reading the template, > i.e puppet client installation). > Could you guide me where I am going wrong? Its been two week I am > struggling with this issue. > > Thanks, > Sekhar > > The information in this message may be proprietary and/or confidential, > and protected from disclosure. If the reader of this message is not the > intended recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. If > you have received this communication in error, please notify ATMECS and > delete it from your computer.


The information in this message may be proprietary and/or confidential, and
protected from disclosure. If the reader of this message is not the
intended recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited. If
you have received this communication in error, please notify ATMECS and
delete it from your computer.

Hi Daniel,
Thanks for your support.
I figured out that Foreman is provisioning RHEL 6.6 AMI ( Earlier I tried
with CentOS6.6, CentOS6.5 ). After going through the console output of the
Provisioned VM, I could say foreman has taken user-data template.
My new issue is:
The VM that got provisioned is unable to ssh to foreman machine ( Which has
also puppet-master, puppetmasterd is not running ). And also it has not
installed the puppet module ( Drupal Module ), it is throwing 400 Error.
My Assumptions:

Foreman hosted on CentOS 6.6 and user as 'root'. The VM got provisioned is
RHEL 6.6 and user as ec2-user.
*FOREMAN# ssh -i ***.pem ec2-user@<Private IP>
Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

Even vice-versa has same problem.

··· On Monday, March 2, 2015 at 6:53:16 PM UTC+5:30, Balasekhar Nelli wrote: > > Hi, > > I am using Foreman 1.6 and using AWS EC2 as compute resource. > Problem is, Foreman could not able to resolve the finish template when the > user-data of image is enabled. And, I could not able to provision the VM. > > When user-date of image is* disabled, *foreman able to resolve the > finish-template and able to provision the vm (Without reading the template, > i.e puppet client installation). > Could you guide me where I am going wrong? Its been two week I am > struggling with this issue. > > Thanks, > Sekhar > > The information in this message may be proprietary and/or confidential, > and protected from disclosure. If the reader of this message is not the > intended recipient, you are hereby notified that any dissemination, > distribution or copying of this communication is strictly prohibited. If > you have received this communication in error, please notify ATMECS and > delete it from your computer.


The information in this message may be proprietary and/or confidential, and
protected from disclosure. If the reader of this message is not the
intended recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited. If
you have received this communication in error, please notify ATMECS and
delete it from your computer.

Balakasehar,

That template looks right. What I'm afraid is your provisioning
template kind (where what you wrote above is stored) is not user_data,
or the OS is not associated with any user_data template.
Go to your provisioning template, when you click on the 'Type' tab, is
the type set to 'user_data'?
Go to your operating system, when you click on the 'Templates' tab, do
you see any 'user_data' template there? If not, you should first make
available your template to the OS and then go to that tab and set it
as the user_data template.

Hope that helps

··· On Tue, Mar 3, 2015 at 8:41 AM, Balasekhar Nelli wrote: > <%# > kind: user_data > name: atmecs-aws > oses: > - CentOS 4 > - CentOS 5 > - CentOS 6 > - CentOS 6.4 > - CentOS 7 > - Fedora 16 > - Fedora 17 > - Fedora 18 > - Fedora 19 > - Fedora 20 > %> > > echo "updating system time" > /usr/sbin/ntpdate -sub <%= @host.params["ntp-server"] %> > > <%# we only change hostnames on image based provisioning %> > <% if @host.capabilities.include?(:image) -%> > echo "HOSTNAME=<%= @host %>" >> /etc/sysconfig/network > hostname <%= @host %> > > cat > /etc/hosts < <%= snippets "etc-hosts" %> > EOF > <% end -%> > > <%# set ssh root key for debugging, it never works on the first time :-) -%> > <% if @host.params["ssh_debug_key"] -%> > echo <%= @host.params["ssh_debug_key"] %> > /root/.ssh/authorized_keys > <% end -%> > > # install EPEL > rpm -Uvh > http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm > > # and add the puppet and ruby-shadow package > yum -t -y -e 0 install puppet > > echo "Configuring puppet" > cat > /etc/puppet/puppet.conf < <%= snippets "puppet.conf" %> > EOF > > # Setup puppet to run on system reboot > /sbin/chkconfig --level 345 puppet on > > # Disable most things. Puppet will activate these if required. > echo "Disabling various system services" > <% %w{autofs gpm sendmail cups iptables ip6tables auditd arptables_jf xfs > pcmcia isdn rawdevices hpoj bluetooth openibd avahi-daemon avahi-dnsconfd > hidd hplip pcscd restorecond mcstrans rhnsd yum-updatesd}.each do |service| > -%> > /sbin/chkconfig --level 345 <%= service %> off 2>/dev/null > <% end -%> > > <% if @host.info["classes"].include? "puppet::server" -%> > <%= snippets "master_bootstrap" %> > <% else -%> > /usr/sbin/puppetd #--config /etc/puppet/puppet.conf -o --tags no_such_tag > --no-daemonize > <% end -%> > > # consider rebooting here instead, as we changed the hostname > exit 0 > > On Monday, March 2, 2015 at 6:53:16 PM UTC+5:30, Balasekhar Nelli wrote: >> >> Hi, >> >> I am using Foreman 1.6 and using AWS EC2 as compute resource. >> Problem is, Foreman could not able to resolve the finish template when the >> user-data of image is enabled. And, I could not able to provision the VM. >> >> When user-date of image is disabled, foreman able to resolve the >> finish-template and able to provision the vm (Without reading the template, >> i.e puppet client installation). >> Could you guide me where I am going wrong? Its been two week I am >> struggling with this issue. >> >> Thanks, >> Sekhar >> >> The information in this message may be proprietary and/or confidential, >> and protected from disclosure. If the reader of this message is not the >> intended recipient, you are hereby notified that any dissemination, >> distribution or copying of this communication is strictly prohibited. If you >> have received this communication in error, please notify ATMECS and delete >> it from your computer. > > > The information in this message may be proprietary and/or confidential, and > protected from disclosure. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. If you have received > this communication in error, please notify ATMECS and delete it from your > computer. > > -- > You received this message because you are subscribed to the Google Groups > "Foreman users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-users+unsubscribe@googlegroups.com. > To post to this group, send email to foreman-users@googlegroups.com. > Visit this group at http://groups.google.com/group/foreman-users. > For more options, visit https://groups.google.com/d/optout.


Daniel Lobato

@elobatoss
blog.daniellobato.me
daniellobato.me

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30

  1. Can't I install a puppet module with finish-template?
    Installing packages through Puppet modules (I think this is what
    you mean) is something that will happen after your finish-template
    goes in.

  2. Is user_data is compulsory to pull the template ( i.e reading the
    template from foreman )?
    I don't understand the question, user_data is pushed from Foreman to
    the Compute Resource.

  3. Do you think I went wrong in my configuration?
    No, it's just fine. I haven't read your user_data template in detail
    though. I'm not sure why your user_data didn't get in EC2, but it's
    worth che

  4. Is it possible to provision VM (With template) with user-data disabled.
    Yes, but you need an external IP to which Foreman can connect to,
    and a Smart Proxy with TFTP enabled. The advantage of user-data is
    that you get to skip that and there is no need for the external IP.

There are some diagrams that Lukas Zapletal is going to add to the
docs soon, https://github.com/theforeman/theforeman.org/pull/333/files
, take a look at the 2 diagrams regarding Cloud-based provisioning if
you want to understand what is going on behind the scenes.

··· On Wed, Mar 4, 2015 at 6:20 AM, Balasekhar Nelli wrote: > Is AMI causing a issue? I am using CentOS 6.5 Operating System. One more > thing is, I could not able to ssh to foreman server to provisioned VM and > vice versa. > > On Monday, March 2, 2015 at 6:53:16 PM UTC+5:30, Balasekhar Nelli wrote: >> >> Hi, >> >> I am using Foreman 1.6 and using AWS EC2 as compute resource. >> Problem is, Foreman could not able to resolve the finish template when the >> user-data of image is enabled. And, I could not able to provision the VM. >> >> When user-date of image is disabled, foreman able to resolve the >> finish-template and able to provision the vm (Without reading the template, >> i.e puppet client installation). >> Could you guide me where I am going wrong? Its been two week I am >> struggling with this issue. >> >> Thanks, >> Sekhar >> >> The information in this message may be proprietary and/or confidential, >> and protected from disclosure. If the reader of this message is not the >> intended recipient, you are hereby notified that any dissemination, >> distribution or copying of this communication is strictly prohibited. If you >> have received this communication in error, please notify ATMECS and delete >> it from your computer. > > > The information in this message may be proprietary and/or confidential, and > protected from disclosure. If the reader of this message is not the intended > recipient, you are hereby notified that any dissemination, distribution or > copying of this communication is strictly prohibited. If you have received > this communication in error, please notify ATMECS and delete it from your > computer. > > -- > You received this message because you are subscribed to the Google Groups > "Foreman users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-users+unsubscribe@googlegroups.com. > To post to this group, send email to foreman-users@googlegroups.com. > Visit this group at http://groups.google.com/group/foreman-users. > For more options, visit https://groups.google.com/d/optout.


Daniel Lobato

@elobatoss
blog.daniellobato.me
daniellobato.me

GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30