DNS update fails since update to 1.5.2

Hi,

I'm running a complete Foreman infrastructure including PUPPET, DNS, DHCP,
TFTP and REALM (freeIPA) smart proxies.
Everything was working perfectly until I upgraded Foreman and the smart
proxies to 1.5.2.

A deletion operation on a host for example now fails with "Remove Reverse
DNS record for testdns.example.com task failed with the following error:
ERF12-1261 [ProxyAPI::ProxyException]: Unable to delete DNS entry
([RestClient::BadRequest]: 400 Bad Request) for proxy
http://ipa.example.com:8443/dns"

Impossible also to create a new host.

I've got nothing in the logs for the freeIPA/DNS smartproxy concerning this
error. It seems the smartproxy doesn't even run nsupdate anymore.

I was able to roll back to a previous snapshot of my VM and confirm this
was caused by the upgrade of the foreman packages specifically and not by
other packages.

My servers run on Centos 6.5.

I would like to provide more info but didn't find much until now…

Best regards,

Jeremie Sebban

Rise Visual Effects Studios

> Hi,
>
> I'm running a complete Foreman infrastructure including PUPPET, DNS, DHCP,
> TFTP and REALM (freeIPA) smart proxies.
> Everything was working perfectly until I upgraded Foreman and the smart
> proxies to 1.5.2.
>
> A deletion operation on a host for example now fails with "Remove Reverse
> DNS record for testdns.example.com task failed with the following error:
> ERF12-1261 [ProxyAPI::ProxyException]: Unable to delete DNS entry
> ([RestClient::BadRequest]: 400 Bad Request) for proxy
> http://ipa.example.com:8443/dns"
>
> Impossible also to create a new host.
>
> I've got nothing in the logs for the freeIPA/DNS smartproxy concerning
> this error. It seems the smartproxy doesn't even run nsupdate anymore.
>
> I was able to roll back to a previous snapshot of my VM and confirm this
> was caused by the upgrade of the foreman packages specifically and not by
> other packages.
>
> My servers run on Centos 6.5.
>
> I would like to provide more info but didn't find much until now…
>

can you turn on logging on the foreman-proxy to debug and provide the logs?

thanks,
Ohad

··· On Wed, Jul 30, 2014 at 4:26 PM, Jem wrote:

Best regards,

Jeremie Sebban

Rise Visual Effects Studios


You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

Hi Ohad,

First line of the logs is an error:

"E, [2014-07-30T16:01:24.990850 #2087] ERROR – : can't convert false into
String"

Maybe related to the last line of my DNS smartproxy setting ?

:dns_provider: nsupdate_gss
:dns_tsig_keytab: /etc/foreman-proxy/dns.keytab
:dns_tsig_principal: foremanproxy/ipa.rise@RISE
:dns_key: false

Thanks,

Jeremie

··· Le mercredi 30 juillet 2014 15:40:53 UTC+2, ohad a écrit : > > > > > On Wed, Jul 30, 2014 at 4:26 PM, Jem <pil...@gmail.com > > wrote: > >> Hi, >> >> I'm running a complete Foreman infrastructure including PUPPET, DNS, >> DHCP, TFTP and REALM (freeIPA) smart proxies. >> Everything was working perfectly until I upgraded Foreman and the smart >> proxies to 1.5.2. >> >> A deletion operation on a host for example now fails with "Remove >> Reverse DNS record for testdns.example.com task failed with the >> following error: ERF12-1261 [ProxyAPI::ProxyException]: Unable to delete >> DNS entry ([RestClient::BadRequest]: 400 Bad Request) for proxy >> http://ipa.example.com:8443/dns" >> >> Impossible also to create a new host. >> >> I've got nothing in the logs for the freeIPA/DNS smartproxy concerning >> this error. It seems the smartproxy doesn't even run nsupdate anymore. >> >> I was able to roll back to a previous snapshot of my VM and confirm this >> was caused by the upgrade of the foreman packages specifically and not by >> other packages. >> >> My servers run on Centos 6.5. >> >> I would like to provide more info but didn't find much until now... >> > > can you turn on logging on the foreman-proxy to debug and provide the logs? > > thanks, > Ohad > >> >> Best regards, >> >> Jeremie Sebban >> >> Rise Visual Effects Studios >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Foreman users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to foreman-user...@googlegroups.com . >> To post to this group, send email to forema...@googlegroups.com >> . >> Visit this group at http://groups.google.com/group/foreman-users. >> For more options, visit https://groups.google.com/d/optout. >> > >

Hi again,

I removed the ":dns_key: false" statement from the DNS proxy settings.yml
and it's now working perfectly again.
It may be worth to update the doc at
http://theforeman.org/manuals/1.5/index.html#4.3.6GSS-TSIGDNS

Thanks for this amazing tool :slight_smile:

Jeremie

··· Le mercredi 30 juillet 2014 16:06:29 UTC+2, Jem a écrit : > > Hi Ohad, > > First line of the logs is an error: > > "E, [2014-07-30T16:01:24.990850 #2087] ERROR -- : can't convert false into > String" > > Maybe related to the last line of my DNS smartproxy setting ? > > :dns_provider: nsupdate_gss > :dns_tsig_keytab: /etc/foreman-proxy/dns.keytab > :dns_tsig_principal: foremanproxy/ipa.rise@RISE > :dns_key: false > > Thanks, > > Jeremie > > Le mercredi 30 juillet 2014 15:40:53 UTC+2, ohad a écrit : >> >> >> >> >> On Wed, Jul 30, 2014 at 4:26 PM, Jem wrote: >> >>> Hi, >>> >>> I'm running a complete Foreman infrastructure including PUPPET, DNS, >>> DHCP, TFTP and REALM (freeIPA) smart proxies. >>> Everything was working perfectly until I upgraded Foreman and the smart >>> proxies to 1.5.2. >>> >>> A deletion operation on a host for example now fails with "Remove >>> Reverse DNS record for testdns.example.com task failed with the >>> following error: ERF12-1261 [ProxyAPI::ProxyException]: Unable to delete >>> DNS entry ([RestClient::BadRequest]: 400 Bad Request) for proxy >>> http://ipa.example.com:8443/dns" >>> >>> Impossible also to create a new host. >>> >>> I've got nothing in the logs for the freeIPA/DNS smartproxy concerning >>> this error. It seems the smartproxy doesn't even run nsupdate anymore. >>> >>> I was able to roll back to a previous snapshot of my VM and confirm this >>> was caused by the upgrade of the foreman packages specifically and not by >>> other packages. >>> >>> My servers run on Centos 6.5. >>> >>> I would like to provide more info but didn't find much until now... >>> >> >> can you turn on logging on the foreman-proxy to debug and provide the >> logs? >> >> thanks, >> Ohad >> >>> >>> Best regards, >>> >>> Jeremie Sebban >>> >>> Rise Visual Effects Studios >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Foreman users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to foreman-user...@googlegroups.com. >>> To post to this group, send email to forema...@googlegroups.com. >>> Visit this group at http://groups.google.com/group/foreman-users. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >>

Yes, positive, foreman-proxy version is 1.5.2.1.el6, restarted the service
and the server.
Is there a way to verify the fix was correctly included in the package I
got ?

··· Le mercredi 30 juillet 2014 16:40:46 UTC+2, Dominic Cleal a écrit : > > Are you sure the foreman-proxy is version 1.5.2? We pulled a fix into > 1.5.2 which should parse this correctly. > > Glad it's sorted though. > > -- > Dominic Cleal > Red Hat Engineering > > On 30/07/14 15:31, Jem wrote: > > Hi again, > > > > I removed the ":dns_key: false" statement from the DNS proxy > > settings.yml and it's now working perfectly again. > > It may be worth to update the doc > > at http://theforeman.org/manuals/1.5/index.html#4.3.6GSS-TSIGDNS > > > > Thanks for this amazing tool :) > > > > Jeremie > > > > Le mercredi 30 juillet 2014 16:06:29 UTC+2, Jem a écrit : > > > > Hi Ohad, > > > > First line of the logs is an error: > > > > "E, [2014-07-30T16:01:24.990850 #2087] ERROR -- : can't convert > > false into String" > > > > Maybe related to the last line of my DNS smartproxy setting ? > > > > :dns_provider: nsupdate_gss > > :dns_tsig_keytab: /etc/foreman-proxy/dns.keytab > > :dns_tsig_principal: foremanproxy/ipa.rise@RISE > > :dns_key: false > > > > Thanks, > > > > Jeremie > > > > Le mercredi 30 juillet 2014 15:40:53 UTC+2, ohad a écrit : > > > > > > > > > > On Wed, Jul 30, 2014 at 4:26 PM, Jem wrote: > > > > Hi, > > > > I'm running a complete Foreman infrastructure including > > PUPPET, DNS, DHCP, TFTP and REALM (freeIPA) smart proxies. > > Everything was working perfectly until I upgraded Foreman > > and the smart proxies to 1.5.2. > > > > A deletion operation on a host for example now fails with > > "Remove Reverse DNS record for testdns.example.com > > task failed with the following > > error: ERF12-1261 [ProxyAPI::ProxyException]: Unable to > > delete DNS entry ([RestClient::BadRequest]: 400 Bad Request) > > for proxy http://ipa.example.com:8443/dns > > " > > > > Impossible also to create a new host. > > > > I've got nothing in the logs for the freeIPA/DNS smartproxy > > concerning this error. It seems the smartproxy doesn't even > > run nsupdate anymore. > > > > I was able to roll back to a previous snapshot of my VM and > > confirm this was caused by the upgrade of the foreman > > packages specifically and not by other packages. > > > > My servers run on Centos 6.5. > > > > I would like to provide more info but didn't find much until > > now... > > > > > > can you turn on logging on the foreman-proxy to debug and > > provide the logs? > > > > thanks, > > Ohad > > > > > > Best regards, > > > > Jeremie Sebban > > > > Rise Visual Effects Studios > > > > -- > > You received this message because you are subscribed to the > > Google Groups "Foreman users" group. > > To unsubscribe from this group and stop receiving emails > > from it, send an email to foreman-user...@googlegroups.com. > > To post to this group, send email to > forema...@googlegroups.com. > > Visit this group at > > http://groups.google.com/group/foreman-users > > . > > For more options, visit https://groups.google.com/d/optout > > . > > > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Foreman users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to foreman-user...@googlegroups.com > > <mailto:foreman-users+unsubscribe@googlegroups.com >. > > To post to this group, send email to forema...@googlegroups.com > > > <mailto:forema...@googlegroups.com >. > > Visit this group at http://groups.google.com/group/foreman-users. > > For more options, visit https://groups.google.com/d/optout. > >

No problem, thank you again for the amazing work :slight_smile:

··· Le mercredi 30 juillet 2014 16:55:49 UTC+2, Dominic Cleal a écrit : > > Ah sorry, just read into it a bit more and it's actually a consequence > of the workaround we recommended for the same issue. > > http://projects.theforeman.org/issues/5675#note-4 > > Apologies, we should have handled that a bit more smoothly and ensured > the workaround still worked when the real fix was applied. > > Having the dns_key line removed is the correct layout for the config > file, so I think you're fine now. > > -- > Dominic Cleal > Red Hat Engineering > > On 30/07/14 15:45, Jem wrote: > > Yes, positive, foreman-proxy version is 1.5.2.1.el6, restarted the > > service and the server. > > Is there a way to verify the fix was correctly included in the package I > > got ? > > > > > > Le mercredi 30 juillet 2014 16:40:46 UTC+2, Dominic Cleal a écrit : > > > > Are you sure the foreman-proxy is version 1.5.2? We pulled a fix > into > > 1.5.2 which should parse this correctly. > > > > Glad it's sorted though. > > > > -- > > Dominic Cleal > > Red Hat Engineering > > > > On 30/07/14 15:31, Jem wrote: > > > Hi again, > > > > > > I removed the ":dns_key: false" statement from the DNS proxy > > > settings.yml and it's now working perfectly again. > > > It may be worth to update the doc > > > at http://theforeman.org/manuals/1.5/index.html#4.3.6GSS-TSIGDNS > > > > > > > > Thanks for this amazing tool :) > > > > > > Jeremie > > > > > > Le mercredi 30 juillet 2014 16:06:29 UTC+2, Jem a écrit : > > > > > > Hi Ohad, > > > > > > First line of the logs is an error: > > > > > > "E, [2014-07-30T16:01:24.990850 #2087] ERROR -- : can't > convert > > > false into String" > > > > > > Maybe related to the last line of my DNS smartproxy setting ? > > > > > > :dns_provider: nsupdate_gss > > > :dns_tsig_keytab: /etc/foreman-proxy/dns.keytab > > > :dns_tsig_principal: foremanproxy/ipa.rise@RISE > > > :dns_key: false > > > > > > Thanks, > > > > > > Jeremie > > > > > > Le mercredi 30 juillet 2014 15:40:53 UTC+2, ohad a écrit : > > > > > > > > > > > > > > > On Wed, Jul 30, 2014 at 4:26 PM, Jem > > wrote: > > > > > > Hi, > > > > > > I'm running a complete Foreman infrastructure > including > > > PUPPET, DNS, DHCP, TFTP and REALM (freeIPA) smart > > proxies. > > > Everything was working perfectly until I upgraded > Foreman > > > and the smart proxies to 1.5.2. > > > > > > A deletion operation on a host for example now fails > > with > > > "Remove Reverse DNS record for testdns.example.com > > > > > task failed with the > > following > > > error: ERF12-1261 [ProxyAPI::ProxyException]: Unable > to > > > delete DNS entry ([RestClient::BadRequest]: 400 Bad > > Request) > > > for proxy http://ipa.example.com:8443/dns > > > > > > >" > > > > > > Impossible also to create a new host. > > > > > > I've got nothing in the logs for the freeIPA/DNS > > smartproxy > > > concerning this error. It seems the smartproxy doesn't > > even > > > run nsupdate anymore. > > > > > > I was able to roll back to a previous snapshot of my > > VM and > > > confirm this was caused by the upgrade of the foreman > > > packages specifically and not by other packages. > > > > > > My servers run on Centos 6.5. > > > > > > I would like to provide more info but didn't find much > > until > > > now... > > > > > > > > > can you turn on logging on the foreman-proxy to debug and > > > provide the logs? > > > > > > thanks, > > > Ohad > > > > > > > > > Best regards, > > > > > > Jeremie Sebban > > > > > > Rise Visual Effects Studios > > > > > > -- > > > You received this message because you are subscribed > > to the > > > Google Groups "Foreman users" group. > > > To unsubscribe from this group and stop receiving > emails > > > from it, send an email to > > foreman-user...@googlegroups.com. > > > To post to this group, send email to > > forema...@googlegroups.com. > > > Visit this group at > > > http://groups.google.com/group/foreman-users > > > > > > >. > > > For more options, visit > > https://groups.google.com/d/optout < > https://groups.google.com/d/optout> > > > > >. > > > > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "Foreman users" group. > > > To unsubscribe from this group and stop receiving emails from it, > > send > > > an email to foreman-user...@googlegroups.com > > > <mailto:foreman-users+unsubscribe@googlegroups.com > >. > > > To post to this group, send email to forema...@googlegroups.com > > > > > <mailto:forema...@googlegroups.com >. > > > Visit this group at http://groups.google.com/group/foreman-users > > . > > > For more options, visit https://groups.google.com/d/optout > > . > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Foreman users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to foreman-user...@googlegroups.com > > <mailto:foreman-users+unsubscribe@googlegroups.com >. > > To post to this group, send email to forema...@googlegroups.com > > > <mailto:forema...@googlegroups.com >. > > Visit this group at http://groups.google.com/group/foreman-users. > > For more options, visit https://groups.google.com/d/optout. > >

Are you sure the foreman-proxy is version 1.5.2? We pulled a fix into
1.5.2 which should parse this correctly.

Glad it's sorted though.

··· -- Dominic Cleal Red Hat Engineering

On 30/07/14 15:31, Jem wrote:

Hi again,

I removed the “:dns_key: false” statement from the DNS proxy
settings.yml and it’s now working perfectly again.
It may be worth to update the doc
at Foreman :: Manual

Thanks for this amazing tool :slight_smile:

Jeremie

Le mercredi 30 juillet 2014 16:06:29 UTC+2, Jem a écrit :

Hi Ohad,

First line of the logs is an error:

"E, [2014-07-30T16:01:24.990850 #2087] ERROR -- : can't convert
false into String"

Maybe related to the last line of my DNS smartproxy setting ?

:dns_provider: nsupdate_gss
:dns_tsig_keytab: /etc/foreman-proxy/dns.keytab
:dns_tsig_principal: foremanproxy/ipa.rise@RISE
:dns_key: false

Thanks,

Jeremie

Le mercredi 30 juillet 2014 15:40:53 UTC+2, ohad a écrit :




    On Wed, Jul 30, 2014 at 4:26 PM, Jem <pil...@gmail.com> wrote:

        Hi,

        I'm running a complete Foreman infrastructure including
        PUPPET, DNS, DHCP, TFTP and REALM (freeIPA) smart proxies.
        Everything was working perfectly until I upgraded Foreman
        and the smart proxies to 1.5.2.

        A deletion operation on a host  for example now fails with
        "Remove Reverse DNS record for testdns.example.com
        <http://testdns.example.com> task failed with the following
        error: ERF12-1261 [ProxyAPI::ProxyException]: Unable to
        delete DNS entry ([RestClient::BadRequest]: 400 Bad Request)
        for proxy http://ipa.example.com:8443/dns
        <http://ipa.example.com:8443/dns>"

        Impossible also to create a new host.

        I've got nothing in the logs for the freeIPA/DNS smartproxy
        concerning this error. It seems the smartproxy doesn't even
        run nsupdate anymore.

        I was able to roll back to a previous snapshot of my VM and
        confirm this was caused by the upgrade of the foreman
        packages specifically and not by other packages.

        My servers run on Centos 6.5.

        I would like to provide more info but didn't find much until
        now...


    can you turn on logging on the foreman-proxy to debug and
    provide the logs?

    thanks,
    Ohad 


        Best regards,

        Jeremie Sebban

        Rise Visual Effects Studios

        -- 
        You received this message because you are subscribed to the
        Google Groups "Foreman users" group.
        To unsubscribe from this group and stop receiving emails
        from it, send an email to foreman-user...@googlegroups.com.
        To post to this group, send email to forema...@googlegroups.com.
        Visit this group at
        http://groups.google.com/group/foreman-users
        <http://groups.google.com/group/foreman-users>.
        For more options, visit https://groups.google.com/d/optout
        <https://groups.google.com/d/optout>.


You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-users+unsubscribe@googlegroups.com
mailto:foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com
mailto:foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

Ah sorry, just read into it a bit more and it's actually a consequence
of the workaround we recommended for the same issue.

http://projects.theforeman.org/issues/5675#note-4

Apologies, we should have handled that a bit more smoothly and ensured
the workaround still worked when the real fix was applied.

Having the dns_key line removed is the correct layout for the config
file, so I think you're fine now.

··· -- Dominic Cleal Red Hat Engineering

On 30/07/14 15:45, Jem wrote:

Yes, positive, foreman-proxy version is 1.5.2.1.el6, restarted the
service and the server.
Is there a way to verify the fix was correctly included in the package I
got ?

Le mercredi 30 juillet 2014 16:40:46 UTC+2, Dominic Cleal a écrit :

Are you sure the foreman-proxy is version 1.5.2?  We pulled a fix into
1.5.2 which should parse this correctly.

Glad it's sorted though.

-- 
Dominic Cleal
Red Hat Engineering

On 30/07/14 15:31, Jem wrote:
> Hi again,
>
> I removed the ":dns_key: false" statement from the DNS proxy
> settings.yml and it's now working perfectly again.
> It may be worth to update the doc
> at http://theforeman.org/manuals/1.5/index.html#4.3.6GSS-TSIGDNS
<http://theforeman.org/manuals/1.5/index.html#4.3.6GSS-TSIGDNS>
>
> Thanks for this amazing tool :)
>
> Jeremie
>
> Le mercredi 30 juillet 2014 16:06:29 UTC+2, Jem a écrit :
>
>     Hi Ohad,
>
>     First line of the logs is an error:
>
>     "E, [2014-07-30T16:01:24.990850 #2087] ERROR -- : can't convert
>     false into String"
>
>     Maybe related to the last line of my DNS smartproxy setting ?
>
>     :dns_provider: nsupdate_gss
>     :dns_tsig_keytab: /etc/foreman-proxy/dns.keytab
>     :dns_tsig_principal: foremanproxy/ipa.rise@RISE
>     :dns_key: false
>
>     Thanks,
>
>     Jeremie
>
>     Le mercredi 30 juillet 2014 15:40:53 UTC+2, ohad a écrit :
>
>
>
>
>         On Wed, Jul 30, 2014 at 4:26 PM, Jem <pil...@gmail.com> >     wrote:
>
>             Hi,
>
>             I'm running a complete Foreman infrastructure including
>             PUPPET, DNS, DHCP, TFTP and REALM (freeIPA) smart
proxies.
>             Everything was working perfectly until I upgraded Foreman
>             and the smart proxies to 1.5.2.
>
>             A deletion operation on a host  for example now fails
with
>             "Remove Reverse DNS record for testdns.example.com
<http://testdns.example.com>
>             <http://testdns.example.com> task failed with the
following
>             error: ERF12-1261 [ProxyAPI::ProxyException]: Unable to
>             delete DNS entry ([RestClient::BadRequest]: 400 Bad
Request)
>             for proxy http://ipa.example.com:8443/dns
<http://ipa.example.com:8443/dns>
>             <http://ipa.example.com:8443/dns
<http://ipa.example.com:8443/dns>>"
>
>             Impossible also to create a new host.
>
>             I've got nothing in the logs for the freeIPA/DNS
smartproxy
>             concerning this error. It seems the smartproxy doesn't
even
>             run nsupdate anymore.
>
>             I was able to roll back to a previous snapshot of my
VM and
>             confirm this was caused by the upgrade of the foreman
>             packages specifically and not by other packages.
>
>             My servers run on Centos 6.5.
>
>             I would like to provide more info but didn't find much
until
>             now...
>
>
>         can you turn on logging on the foreman-proxy to debug and
>         provide the logs?
>
>         thanks,
>         Ohad
>
>
>             Best regards,
>
>             Jeremie Sebban
>
>             Rise Visual Effects Studios
>
>             --
>             You received this message because you are subscribed
to the
>             Google Groups "Foreman users" group.
>             To unsubscribe from this group and stop receiving emails
>             from it, send an email to
foreman-user...@googlegroups.com.
>             To post to this group, send email to
forema...@googlegroups.com.
>             Visit this group at
>             http://groups.google.com/group/foreman-users
<http://groups.google.com/group/foreman-users>
>             <http://groups.google.com/group/foreman-users
<http://groups.google.com/group/foreman-users>>.
>             For more options, visit
https://groups.google.com/d/optout <https://groups.google.com/d/optout>
>             <https://groups.google.com/d/optout
<https://groups.google.com/d/optout>>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Foreman users" group.
> To unsubscribe from this group and stop receiving emails from it,
send
> an email to foreman-user...@googlegroups.com <javascript:>
> <mailto:foreman-users+unsubscribe@googlegroups.com <javascript:>>.
> To post to this group, send email to forema...@googlegroups.com
<javascript:>
> <mailto:forema...@googlegroups.com <javascript:>>.
> Visit this group at http://groups.google.com/group/foreman-users
<http://groups.google.com/group/foreman-users>.
> For more options, visit https://groups.google.com/d/optout
<https://groups.google.com/d/optout>.


You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-users+unsubscribe@googlegroups.com
mailto:foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com
mailto:foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.