I'm new to Foreman, but am currently evaluating it (and am impressed) and
making rapid progress. I am little stuck on provisioning in EC2, however.
I have a compute resource set up appropriately and working. It lists a
CentOS 6.4 AMI among its images. I have OS (CentOS), architecture, etc.
all defined- all my ducks in a row, basically. I AM able to launch a
CentOS 6.4 instance into my VPC. Cool…
Where things fall apart… I am using what I think is an adequate finish
template (got it here<https://github.com/skottler/foreman-finish-templates/blob/master/finish.sh>)
because I didn't see any that came with Foreman that looked suitable for
CentOS (or RHEL). If I am understanding the Foreman documentation
correctly, it is the successful completion of the finish template that
tells Foreman that the host is done building. In my case, my host seems
stuck in the building state, leading me to believe that the finish template
either was not run or has errored out. To rule out the possibility that
something is wrong with that template itself, I've tried the same with a
"no-op" template that just exits 0. I get the same result.
Again, referring to the documentation, I believe there is a chain of events
that occurs WRT to these templates. It seems the PXE template kicks off
the provisioning template, which, in turn, kicks off the finish template.
As I understand it, these first two don't come into play when provisioning
into EC2. Does that, perhaps, play a role in the finish template not being
triggered? I'm grasping at straws here because try as I might, I also
can't find any logs that would shed any light on this. I'm looking with
the rest of the foreman-proxy logs in /var/log/foreman-proxy/proxy.log.
Any insight at all would be tremendously appreciated!
you mentioned EC2 here, in this scenario there is no PXE booting or
kickstart templating involved. When provisioning in the cloud (e.g.
EC2), the workflow is little bit different.
Foreman kicks off image on EC2, gets a public IP address and it is
trying to connect to ssh. Once connection is successful, the finish
script is executed via ssh on there.
So I bet something must be wrong with either ssh server (firewall, ssh
running) or ssh keys (server is not allowing Foreman in). Apparently
correct ssh key must be configured for the foreman user (which is
running foreman) on the EC2 account you are trying to connect to.
To generate keys follow this procedure:
su foreman; cd ~; ssh-keygen
LZ
···
On Wed, Nov 13, 2013 at 04:23:47PM -0800, Kent Rancourt wrote:
> Hey all,
>
> I'm new to Foreman, but am currently evaluating it (and am impressed) and
> making rapid progress. I am little stuck on provisioning in EC2, however.
>
> I have a compute resource set up appropriately and working. It lists a
> CentOS 6.4 AMI among its images. I have OS (CentOS), architecture, etc.
> all defined- all my ducks in a row, basically. I AM able to launch a
> CentOS 6.4 instance into my VPC. Cool...
>
> Where things fall apart... I am using what I think is an adequate finish
> template (got it here)
> because I didn't see any that came with Foreman that looked suitable for
> CentOS (or RHEL). If I am understanding the Foreman documentation
> correctly, it is the successful completion of the finish template that
> tells Foreman that the host is done building. In my case, my host seems
> stuck in the building state, leading me to believe that the finish template
> either was not run or has errored out. To rule out the possibility that
> something is wrong with that template itself, I've tried the same with a
> "no-op" template that just exits 0. I get the same result.
>
> Again, referring to the documentation, I believe there is a chain of events
> that occurs WRT to these templates. It seems the PXE template kicks off
> the provisioning template, which, in turn, kicks off the finish template.
> As I understand it, these first two don't come into play when provisioning
> into EC2. Does that, perhaps, play a role in the finish template not being
> triggered? I'm grasping at straws here because try as I might, I also
> can't find any logs that would shed any light on this. I'm looking with
> the rest of the foreman-proxy logs in /var/log/foreman-proxy/proxy.log.
>
> Any insight at all would be tremendously appreciated!
>
> - Kent
>
>
> --
> You received this message because you are subscribed to the Google Groups "Foreman users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-users+unsubscribe@googlegroups.com.
> To post to this group, send email to foreman-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/foreman-users.
> For more options, visit https://groups.google.com/groups/opt_out.
That's not true for EC2-like resources (EC2, Openstack, etc) - Foreman
generates it's own keypair automatically, stores it in the DB (key_pairs
table), and uploads the public key to EC2/Openstack for automatic injection
into the image. Keys on the filesystem will have no effect on EC2.
The best thing to do is probably to set up debug Foreman logs and check if
the Finish script is run at all. If it is, extract the private key from the
database and log in to the new instane to see what broke.
Greg
···
On 14 November 2013 09:13, Lukas Zapletal wrote:
So I bet something must be wrong with either ssh server (firewall, ssh
running) or ssh keys (server is not allowing Foreman in). Apparently
correct ssh key must be configured for the foreman user (which is
running foreman) on the EC2 account you are trying to connect to.
Lukas and Greg, thank you both for your assistance. Greg, you've validated
my understanding of how it all works (or is supposed to). I'm not sure
(yet) how to enable debug level logging in the Foreman proxy, but that'll
be my next step.
You also mentioned extracting the private key from the database and
attempting to ssh to the newly provisioned node. I tried that already, in
fact, but got stuck because it seems the private key is password protected.
Do you happen to have any sense of how I might go about unlocking that?
···
On Thursday, November 14, 2013 6:03:49 AM UTC-5, Greg Sutcliffe wrote:
>
> On 14 November 2013 09:13, Lukas Zapletal <lz...@redhat.com >wrote:
>
>> So I bet something must be wrong with either ssh server (firewall, ssh
>> running) or ssh keys (server is not allowing Foreman in). Apparently
>> correct ssh key must be configured for the foreman user (which is
>> running foreman) on the EC2 account you are trying to connect to.
>>
>> To generate keys follow this procedure:
>>
>> su foreman; cd ~; ssh-keygen
>>
>
> That's not true for EC2-like resources (EC2, Openstack, etc) - Foreman
> generates it's own keypair automatically, stores it in the DB (key_pairs
> table), and uploads the public key to EC2/Openstack for automatic injection
> into the image. Keys on the filesystem will have no effect on EC2.
>
> The best thing to do is probably to set up debug Foreman logs and check if
> the Finish script is run at all. If it is, extract the private key from the
> database and log in to the new instane to see what broke.
>
> Greg
>
> Lukas and Greg, thank you both for your assistance. Greg, you've
> validated my understanding of how it all works (or is supposed to). I'm
> not sure (yet) how to enable debug level logging in the Foreman proxy, but
> that'll be my next step.
>
> You also mentioned extracting the private key from the database and
> attempting to ssh to the newly provisioned node. I tried that already, in
> fact, but got stuck because it seems the private key is password protected.
> Do you happen to have any sense of how I might go about unlocking that?
>
They shouldn't be, Foreman has no need to password protect those keys.
Certainly last time I checked I could use the private key from the DB
directly (via ssh -i <path to where I saved the text>). That was a while
ago, admittedly, though.
···
On 14 November 2013 19:50, Kent Rancourt wrote:
As I understand it, debug logging should actually be enabled on the foreman
proxy and not upon foreman itself, but I've gone ahead and enabled it on
both. There is still nothing telling in the logs. :-/
I'm wondering if anyone has any further insight.
···
On Thursday, November 14, 2013 3:32:59 PM UTC-5, Greg Sutcliffe wrote:
>
> On 14 November 2013 19:50, Kent Rancourt <kent.r...@gmail.com > > wrote:
>
>> Lukas and Greg, thank you both for your assistance. Greg, you've
>> validated my understanding of how it all works (or is supposed to). I'm
>> not sure (yet) how to enable debug level logging in the Foreman proxy, but
>> that'll be my next step.
>>
>
>
> http://projects.theforeman.org/projects/foreman/wiki/Troubleshooting#How-do-I-enable-debugging
>
>
>> You also mentioned extracting the private key from the database and
>> attempting to ssh to the newly provisioned node. I tried that already, in
>> fact, but got stuck because it seems the private key is password protected.
>> Do you happen to have any sense of how I might go about unlocking that?
>>
>
> They shouldn't be, Foreman has no need to password protect those keys.
> Certainly last time I checked I could use the private key from the DB
> directly (via ssh -i ). That was a while
> ago, admittedly, though.
>
I may have found something. Idk how I missed it earlier:
Failed to launch script on ref-lab.domainredacted.com: closed
stream/opt/rh/ruby193/root/usr/share/gems/gems/net-ssh-2.6.7/lib/net/ssh/ruby_compat.rb:30:in
`select'
···
On Thursday, November 14, 2013 4:13:32 PM UTC-5, Kent Rancourt wrote:
>
> As I understand it, debug logging should actually be enabled on the
> foreman proxy and not upon foreman itself, but I've gone ahead and enabled
> it on both. There is still nothing telling in the logs. :-/
>
> I'm wondering if anyone has any further insight.
>
>
>
>
> On Thursday, November 14, 2013 3:32:59 PM UTC-5, Greg Sutcliffe wrote:
>>
>> On 14 November 2013 19:50, Kent Rancourt wrote:
>>
>>> Lukas and Greg, thank you both for your assistance. Greg, you've
>>> validated my understanding of how it all works (or is supposed to). I'm
>>> not sure (yet) how to enable debug level logging in the Foreman proxy, but
>>> that'll be my next step.
>>>
>>
>>
>> http://projects.theforeman.org/projects/foreman/wiki/Troubleshooting#How-do-I-enable-debugging
>>
>>
>>> You also mentioned extracting the private key from the database and
>>> attempting to ssh to the newly provisioned node. I tried that already, in
>>> fact, but got stuck because it seems the private key is password protected.
>>> Do you happen to have any sense of how I might go about unlocking that?
>>>
>>
>> They shouldn't be, Foreman has no need to password protect those keys.
>> Certainly last time I checked I could use the private key from the DB
>> directly (via ssh -i ). That was a while
>> ago, admittedly, though.
>>
>
Ok… after some further experimentation and tail -f'ing my logs during
provisioning, I can see what's going on. The script is successfully
getting uploaded and kicked off, but does not seem to be completing. The
script installs EPEL and the puppet agent. Maybe there's a network
connectivity problem on that instance (which I will troubleshoot). I went
back to using my no-op finish script that just exits 0 and that worked just
fine. (Idk why that didn't work the first time I tried it.) The end
result is that I can provision hosts in EC2, but I can't get puppet
installed. So now the host is no longer stuck in the "building" state, but
shows up as not having any reports. I can live with that momentarily.
This is progress. Guess I will have to figure out why that other script
is bombing out on me. Thanks to everyone who helped!
···
On Thursday, November 14, 2013 4:18:03 PM UTC-5, Kent Rancourt wrote:
>
> I may have found something. Idk how I missed it earlier:
>
> Failed to launch script on ref-lab.domainredacted.com: closed
> stream/opt/rh/ruby193/root/usr/share/gems/gems/net-ssh-2.6.7/lib/net/ssh/ruby_compat.rb:30:in
> `select'
>
>
>
>
> On Thursday, November 14, 2013 4:13:32 PM UTC-5, Kent Rancourt wrote:
>>
>> As I understand it, debug logging should actually be enabled on the
>> foreman proxy and not upon foreman itself, but I've gone ahead and enabled
>> it on both. There is still nothing telling in the logs. :-/
>>
>> I'm wondering if anyone has any further insight.
>>
>>
>>
>>
>> On Thursday, November 14, 2013 3:32:59 PM UTC-5, Greg Sutcliffe wrote:
>>>
>>> On 14 November 2013 19:50, Kent Rancourt wrote:
>>>
>>>> Lukas and Greg, thank you both for your assistance. Greg, you've
>>>> validated my understanding of how it all works (or is supposed to). I'm
>>>> not sure (yet) how to enable debug level logging in the Foreman proxy, but
>>>> that'll be my next step.
>>>>
>>>
>>>
>>> http://projects.theforeman.org/projects/foreman/wiki/Troubleshooting#How-do-I-enable-debugging
>>>
>>>
>>>> You also mentioned extracting the private key from the database and
>>>> attempting to ssh to the newly provisioned node. I tried that already, in
>>>> fact, but got stuck because it seems the private key is password protected.
>>>> Do you happen to have any sense of how I might go about unlocking that?
>>>>
>>>
>>> They shouldn't be, Foreman has no need to password protect those keys.
>>> Certainly last time I checked I could use the private key from the DB
>>> directly (via ssh -i ). That was a while
>>> ago, admittedly, though.
>>>
>>
