Varying number of facts over time

Hi,

I have set up Foreman and foreman-proxy on the same host as my
Puppetmaster. I have integrated reports, set up ENC and enabled facts.
Reports are fed back and facts are coming in.

My 8 hosts show up in the list and everything looks good, but when i go to
facts to query I rarely get the full 8 results. I do a query for "name =
memoryfree" . Some times I get 8, but most of the time I only get 4,5 or 6
results. And this varies throughout the day. I don't see any particular
pattern as to which hosts show up either, it seems random.

Any idea where to start looking?

My production log shows this:

Started POST "/reports/create?format=yml" for 127.0.0.1 at 2013-08-16
11:54:41 +0200
Processing by ReportsController#create as YML
Parameters: {"report"=>"[FILTERED]"}
processing report for host3.mydomain.com
Rendered text template (0.0ms)
Completed 200 OK in 38ms (Views: 0.9ms | ActiveRecord: 8.3ms)

But still there is not fact from the node in the log statement…

Cheers,
Anders,

No one? This is actually happening across two different instances of
Foreman in two different zones. Of course they are configured in the same
way, so it might be something with the way I have done it. From what I can
gather the ENC script is posting the facts. The facts also exist in the
yaml I think the script is posting. Could it be a timing issue? Are the
facts written as the client is trying to post them?

Thanks,
Anders,

··· On Friday, 16 August 2013 14:23:21 UTC+2, Anders Sveen wrote: > > Hi, > > I have set up Foreman and foreman-proxy on the same host as my > Puppetmaster. I have integrated reports, set up ENC and enabled facts. > Reports are fed back and facts are coming in. > > My 8 hosts show up in the list and everything looks good, but when i go to > facts to query I rarely get the full 8 results. I do a query for "name = > memoryfree" . Some times I get 8, but most of the time I only get 4,5 or 6 > results. And this varies throughout the day. I don't see any particular > pattern as to which hosts show up either, it seems random. > > Any idea where to start looking? > > My production log shows this: > > Started POST "/reports/create?format=yml" for 127.0.0.1 at 2013-08-16 > 11:54:41 +0200 > Processing by ReportsController#create as YML > Parameters: {"report"=>"[FILTERED]"} > processing report for host3.mydomain.com > Rendered text template (0.0ms) > Completed 200 OK in 38ms (Views: 0.9ms | ActiveRecord: 8.3ms) > > But still there is not fact from the node in the log statement... > > > > > Cheers, > Anders, > >

I've recently just upgraded to foreman 1.2 and am seeing the same issue and
came searching the list to see if anyone else has seen it. I'm seeings
hosts (in foreman) randomly dropping (i.e. not having) facts. I've verified
that the hosts yaml file on the puppet master does have all the facts so it
appears to be an issue in the passing of the facts to foreman with the
node.rb script or something in the foreman itself that is not properly
writing the facts to the DB. I've verified that my DB (postgres 8.4.13)
does NOT have the dropped facts, so it's not a problem with displaying them.

I tried turning the foremans log level up to debug but am not finding
anything useful.

Anyone have any ideas why facts are getting dropped? or how we can
troubleshoot this?

This is a big issue for us.

Any help/pointers is much appreciated.

Romeo

··· On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen wrote:

No one? This is actually happening across two different instances of
Foreman in two different zones. Of course they are configured in the same
way, so it might be something with the way I have done it. From what I can
gather the ENC script is posting the facts. The facts also exist in the
yaml I think the script is posting. Could it be a timing issue? Are the
facts written as the client is trying to post them?

Thanks,
Anders,

On Friday, 16 August 2013 14:23:21 UTC+2, Anders Sveen wrote:

Hi,

I have set up Foreman and foreman-proxy on the same host as my
Puppetmaster. I have integrated reports, set up ENC and enabled facts.
Reports are fed back and facts are coming in.

My 8 hosts show up in the list and everything looks good, but when i go
to facts to query I rarely get the full 8 results. I do a query for “name =
memoryfree” . Some times I get 8, but most of the time I only get 4,5 or 6
results. And this varies throughout the day. I don’t see any particular
pattern as to which hosts show up either, it seems random.

Any idea where to start looking?

My production log shows this:

Started POST “/reports/create?format=yml” for 127.0.0.1 at 2013-08-16
11:54:41 +0200
Processing by ReportsController#create as YML
Parameters: {“report”=>"[FILTERED]"}
processing report for host3.mydomain.com
Rendered text template (0.0ms)
Completed 200 OK in 38ms (Views: 0.9ms | ActiveRecord: 8.3ms)

But still there is not fact from the node in the log statement…

Cheers,
Anders,


You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.


Romeo

Can you (& Anders) look for "POST /fact_values/create" log entries in
/var/log/foreman/production.log?

The one posted at the start of the thread was a report coming in rather
than facts.

It might be that some of these are failing (part way through?), so if
you can check as many instances of this as you can, it might shed some
light on it.

··· -- Dominic Cleal Red Hat Engineering

On 20/08/13 01:19, Romeo Theriault wrote:

I’ve recently just upgraded to foreman 1.2 and am seeing the same issue
and came searching the list to see if anyone else has seen it. I’m
seeings hosts (in foreman) randomly dropping (i.e. not having) facts.
I’ve verified that the hosts yaml file on the puppet master does have
all the facts so it appears to be an issue in the passing of the facts
to foreman with the node.rb script or something in the foreman itself
that is not properly writing the facts to the DB. I’ve verified that my
DB (postgres 8.4.13) does NOT have the dropped facts, so it’s not a
problem with displaying them.

I tried turning the foremans log level up to debug but am not finding
anything useful.

Anyone have any ideas why facts are getting dropped? or how we can
troubleshoot this?

This is a big issue for us.

Any help/pointers is much appreciated.

Romeo

On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen <anders.r.sveen@gmail.com > mailto:anders.r.sveen@gmail.com> wrote:

No one? This is actually happening across two different instances of
Foreman in two different zones. Of course they are configured in the
same way, so it might be something with the way I have done it. From
what I can gather the ENC script is posting the facts. The facts
also exist in the yaml I think the script is posting. Could it be a
timing issue? Are the facts written as the client is trying to post
them?



Thanks,
Anders,


On Friday, 16 August 2013 14:23:21 UTC+2, Anders Sveen wrote:

    Hi,

    I have set up Foreman and foreman-proxy on the same host as my
    Puppetmaster. I have integrated reports, set up ENC and enabled
    facts. Reports are fed back and facts are coming in.

    My 8 hosts show up in the list and everything looks good, but
    when i go to facts to query I rarely get the full 8 results. I
    do a query for "name = memoryfree" . Some times I get 8, but
    most of the time I only get 4,5 or 6 results. And this varies
    throughout the day. I don't see any particular pattern as to
    which hosts show up either, it seems random.

    Any idea where to start looking?

    My production log shows this:

    Started POST "/reports/create?format=yml" for 127.0.0.1 at
    2013-08-16 11:54:41 +0200
    Processing by ReportsController#create as YML
      Parameters: {"report"=>"[FILTERED]"}
    processing report for host3.mydomain.com <http://host3.mydomain.com>
      Rendered text template (0.0ms)
    Completed 200 OK in 38ms (Views: 0.9ms | ActiveRecord: 8.3ms)

    But still there is not fact from the node in the log statement...




    Cheers,
    Anders,

-- 
You received this message because you are subscribed to the Google
Groups "Foreman users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to foreman-users+unsubscribe@googlegroups.com
<mailto:foreman-users%2Bunsubscribe@googlegroups.com>.
To post to this group, send email to foreman-users@googlegroups.com
<mailto:foreman-users@googlegroups.com>.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.


Romeo


You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Dominic, thanks for responding.

> Can you (& Anders) look for "POST /fact_values/create" log entries in
> /var/log/foreman/production.log?
>

There are lots of these in my logs:

$ grep -A 5 fact_values/create production.log

Started POST "/fact_values/create?format=yml" for 131.171.224.199 at
2013-08-20 15:38:51 -1000
Processing by FactValuesController#create as YML
Parameters: {"facts"=>"[FILTERED]"}
Rendered text template (0.0ms)
Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 7.7ms)

··· On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal wrote:


Started POST “/fact_values/create?format=yml” for 131.171.224.199 at
2013-08-20 15:38:54 -1000
Processing by FactValuesController#create as YML
Parameters: {“facts”=>"[FILTERED]"}
Rendered text template (0.0ms)
Completed 200 OK in 50ms (Views: 0.4ms | ActiveRecord: 8.5ms)


Started POST “/fact_values/create?format=yml” for 131.171.224.199 at
2013-08-20 15:38:54 -1000
Processing by FactValuesController#create as YML
Parameters: {“facts”=>"[FILTERED]"}

Started GET “/node/dbm99.pvt.hawaii.edu?format=yml” for 131.171.224.199 at
2013-08-20 15:38:54 -1000

Started POST “/fact_values/create?format=yml” for 131.171.224.199 at
2013-08-20 15:38:56 -1000
Processing by FactValuesController#create as YML
Parameters: {“facts”=>"[FILTERED]"}
Rendered text template (0.0ms)
Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 8.1ms)

I do notice that occasionally there are some log entries that don’t have
the:

Rendered text template (0.0ms)
Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 8.1ms)

lines. See the third entry above. I’m not sure if this is related or not.

Is there anything else I can provide? or do to troubleshoot?

One thing I’ll note is that one of the facts we see consistently being
dropped is a custom fact we created that facter/puppet pushes up. I’m not
sure this is at all relevant or not since I can see the fact in the server
side yaml file so I know it’s there.

Thanks,
Romeo

The one posted at the start of the thread was a report coming in rather
than facts.

It might be that some of these are failing (part way through?), so if
you can check as many instances of this as you can, it might shed some
light on it.


Dominic Cleal
Red Hat Engineering

On 20/08/13 01:19, Romeo Theriault wrote:

I’ve recently just upgraded to foreman 1.2 and am seeing the same issue
and came searching the list to see if anyone else has seen it. I’m
seeings hosts (in foreman) randomly dropping (i.e. not having) facts.
I’ve verified that the hosts yaml file on the puppet master does have
all the facts so it appears to be an issue in the passing of the facts
to foreman with the node.rb script or something in the foreman itself
that is not properly writing the facts to the DB. I’ve verified that my
DB (postgres 8.4.13) does NOT have the dropped facts, so it’s not a
problem with displaying them.

I tried turning the foremans log level up to debug but am not finding
anything useful.

Anyone have any ideas why facts are getting dropped? or how we can
troubleshoot this?

This is a big issue for us.

Any help/pointers is much appreciated.

Romeo

On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen <anders.r.sveen@gmail.com > > mailto:anders.r.sveen@gmail.com> wrote:

No one? This is actually happening across two different instances of
Foreman in two different zones. Of course they are configured in the
same way, so it might be something with the way I have done it. From
what I can gather the ENC script is posting the facts. The facts
also exist in the yaml I think the script is posting. Could it be a
timing issue? Are the facts written as the client is trying to post
them?



Thanks,
Anders,


On Friday, 16 August 2013 14:23:21 UTC+2, Anders Sveen wrote:

    Hi,

    I have set up Foreman and foreman-proxy on the same host as my
    Puppetmaster. I have integrated reports, set up ENC and enabled
    facts. Reports are fed back and facts are coming in.

    My 8 hosts show up in the list and everything looks good, but
    when i go to facts to query I rarely get the full 8 results. I
    do a query for "name = memoryfree" . Some times I get 8, but
    most of the time I only get 4,5 or 6 results. And this varies
    throughout the day. I don't see any particular pattern as to
    which hosts show up either, it seems random.

    Any idea where to start looking?

    My production log shows this:

    Started POST "/reports/create?format=yml" for 127.0.0.1 at
    2013-08-16 11:54:41 +0200
    Processing by ReportsController#create as YML
      Parameters: {"report"=>"[FILTERED]"}
    processing report for host3.mydomain.com <

http://host3.mydomain.com>

      Rendered text template (0.0ms)
    Completed 200 OK in 38ms (Views: 0.9ms | ActiveRecord: 8.3ms)

    But still there is not fact from the node in the log statement...




    Cheers,
    Anders,

--
You received this message because you are subscribed to the Google
Groups "Foreman users" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to foreman-users+unsubscribe@googlegroups.com
<mailto:foreman-users%2Bunsubscribe@googlegroups.com>.
To post to this group, send email to foreman-users@googlegroups.com
<mailto:foreman-users@googlegroups.com>.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.


Romeo


You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.


You received this message because you are subscribed to the Google Groups
"Foreman users" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.


Romeo

I can confirm that I see the same logs too with 200 OK. Still varying
number of facts though.

A,

··· On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo Theriault wrote: > > Hi Dominic, thanks for responding. > > On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal <dcl...@redhat.com > > wrote: > >> Can you (& Anders) look for "POST /fact_values/create" log entries in >> /var/log/foreman/production.log? >> > > There are lots of these in my logs: > > $ grep -A 5 fact_values/create production.log > > Started POST "/fact_values/create?format=yml" for 131.171.224.199 at > 2013-08-20 15:38:51 -1000 > Processing by FactValuesController#create as YML > Parameters: {"facts"=>"[FILTERED]"} > Rendered text template (0.0ms) > Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 7.7ms) > > -- > Started POST "/fact_values/create?format=yml" for 131.171.224.199 at > 2013-08-20 15:38:54 -1000 > Processing by FactValuesController#create as YML > Parameters: {"facts"=>"[FILTERED]"} > Rendered text template (0.0ms) > Completed 200 OK in 50ms (Views: 0.4ms | ActiveRecord: 8.5ms) > > -- > Started POST "/fact_values/create?format=yml" for 131.171.224.199 at > 2013-08-20 15:38:54 -1000 > Processing by FactValuesController#create as YML > Parameters: {"facts"=>"[FILTERED]"} > > > Started GET "/node/dbm99.pvt.hawaii.edu?format=yml" for 131.171.224.199 > at 2013-08-20 15:38:54 -1000 > -- > Started POST "/fact_values/create?format=yml" for 131.171.224.199 at > 2013-08-20 15:38:56 -1000 > Processing by FactValuesController#create as YML > Parameters: {"facts"=>"[FILTERED]"} > Rendered text template (0.0ms) > Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 8.1ms) > > > I do notice that occasionally there are some log entries that don't have > the: > > Rendered text template (0.0ms) > Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 8.1ms) > > lines. See the third entry above. I'm not sure if this is related or not. > > Is there anything else I can provide? or do to troubleshoot? > > One thing I'll note is that one of the facts we see consistently being > dropped is a custom fact we created that facter/puppet pushes up. I'm not > sure this is at all relevant or not since I can see the fact in the server > side yaml file so I know it's there. > > Thanks, > Romeo > > > > >> The one posted at the start of the thread was a report coming in rather >> than facts. >> >> It might be that some of these are failing (part way through?), so if >> you can check as many instances of this as you can, it might shed some >> light on it. >> >> -- >> Dominic Cleal >> Red Hat Engineering >> >> On 20/08/13 01:19, Romeo Theriault wrote: >> > I've recently just upgraded to foreman 1.2 and am seeing the same issue >> > and came searching the list to see if anyone else has seen it. I'm >> > seeings hosts (in foreman) randomly dropping (i.e. not having) facts. >> > I've verified that the hosts yaml file on the puppet master does have >> > all the facts so it appears to be an issue in the passing of the facts >> > to foreman with the node.rb script or something in the foreman itself >> > that is not properly writing the facts to the DB. I've verified that my >> > DB (postgres 8.4.13) does NOT have the dropped facts, so it's not a >> > problem with displaying them. >> > >> > I tried turning the foremans log level up to debug but am not finding >> > anything useful. >> > >> > Anyone have any ideas why facts are getting dropped? or how we can >> > troubleshoot this? >> > >> > This is a big issue for us. >> > >> > Any help/pointers is much appreciated. >> > >> > Romeo >> > >> > >> > On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen <anders....@gmail.com >> > <mailto:anders....@gmail.com >> wrote: >> > >> > No one? This is actually happening across two different instances of >> > Foreman in two different zones. Of course they are configured in the >> > same way, so it might be something with the way I have done it. From >> > what I can gather the ENC script is posting the facts. The facts >> > also exist in the yaml I think the script is posting. Could it be a >> > timing issue? Are the facts written as the client is trying to post >> > them? >> > >> > >> > >> > Thanks, >> > Anders, >> > >> > >> > On Friday, 16 August 2013 14:23:21 UTC+2, Anders Sveen wrote: >> > >> > Hi, >> > >> > I have set up Foreman and foreman-proxy on the same host as my >> > Puppetmaster. I have integrated reports, set up ENC and enabled >> > facts. Reports are fed back and facts are coming in. >> > >> > My 8 hosts show up in the list and everything looks good, but >> > when i go to facts to query I rarely get the full 8 results. I >> > do a query for "name = memoryfree" . Some times I get 8, but >> > most of the time I only get 4,5 or 6 results. And this varies >> > throughout the day. I don't see any particular pattern as to >> > which hosts show up either, it seems random. >> > >> > Any idea where to start looking? >> > >> > My production log shows this: >> > >> > Started POST "/reports/create?format=yml" for 127.0.0.1 at >> > 2013-08-16 11:54:41 +0200 >> > Processing by ReportsController#create as YML >> > Parameters: {"report"=>"[FILTERED]"} >> > processing report for host3.mydomain.com < >> http://host3.mydomain.com> >> > Rendered text template (0.0ms) >> > Completed 200 OK in 38ms (Views: 0.9ms | ActiveRecord: 8.3ms) >> > >> > But still there is not fact from the node in the log >> statement... >> > >> > >> > >> > >> > Cheers, >> > Anders, >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Foreman users" group. >> > To unsubscribe from this group and stop receiving emails from it, >> > send an email to foreman-user...@googlegroups.com >> > <mailto:foreman-users%2Bunsubscribe@googlegroups.com >> >. >> > To post to this group, send email to forema...@googlegroups.com >> > <mailto:forema...@googlegroups.com >. >> > Visit this group at http://groups.google.com/group/foreman-users. >> > For more options, visit https://groups.google.com/groups/opt_out. >> > >> > >> > >> > >> > -- >> > Romeo >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Foreman users" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> > an email to foreman-user...@googlegroups.com . >> > To post to this group, send email to forema...@googlegroups.com >> . >> > Visit this group at http://groups.google.com/group/foreman-users. >> > For more options, visit https://groups.google.com/groups/opt_out. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Foreman users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to foreman-user...@googlegroups.com . >> To post to this group, send email to forema...@googlegroups.com >> . >> Visit this group at http://groups.google.com/group/foreman-users. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > > > -- > Romeo >

When querying for "name = puppetversion" I get 8 hits. Memoryfree still
show too few…

Anders,

··· On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen wrote: > > I can confirm that I see the same logs too with 200 OK. Still varying > number of facts though. > > > A, > > On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo Theriault wrote: >> >> Hi Dominic, thanks for responding. >> >> On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal wrote: >> >>> Can you (& Anders) look for "POST /fact_values/create" log entries in >>> /var/log/foreman/production.log? >>> >> >> There are lots of these in my logs: >> >> $ grep -A 5 fact_values/create production.log >> >> Started POST "/fact_values/create?format=yml" for 131.171.224.199 at >> 2013-08-20 15:38:51 -1000 >> Processing by FactValuesController#create as YML >> Parameters: {"facts"=>"[FILTERED]"} >> Rendered text template (0.0ms) >> Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 7.7ms) >> >> -- >> Started POST "/fact_values/create?format=yml" for 131.171.224.199 at >> 2013-08-20 15:38:54 -1000 >> Processing by FactValuesController#create as YML >> Parameters: {"facts"=>"[FILTERED]"} >> Rendered text template (0.0ms) >> Completed 200 OK in 50ms (Views: 0.4ms | ActiveRecord: 8.5ms) >> >> -- >> Started POST "/fact_values/create?format=yml" for 131.171.224.199 at >> 2013-08-20 15:38:54 -1000 >> Processing by FactValuesController#create as YML >> Parameters: {"facts"=>"[FILTERED]"} >> >> >> Started GET "/node/dbm99.pvt.hawaii.edu?format=yml" for 131.171.224.199 >> at 2013-08-20 15:38:54 -1000 >> -- >> Started POST "/fact_values/create?format=yml" for 131.171.224.199 at >> 2013-08-20 15:38:56 -1000 >> Processing by FactValuesController#create as YML >> Parameters: {"facts"=>"[FILTERED]"} >> Rendered text template (0.0ms) >> Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 8.1ms) >> >> >> I do notice that occasionally there are some log entries that don't have >> the: >> >> Rendered text template (0.0ms) >> Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 8.1ms) >> >> lines. See the third entry above. I'm not sure if this is related or not. >> >> Is there anything else I can provide? or do to troubleshoot? >> >> One thing I'll note is that one of the facts we see consistently being >> dropped is a custom fact we created that facter/puppet pushes up. I'm not >> sure this is at all relevant or not since I can see the fact in the server >> side yaml file so I know it's there. >> >> Thanks, >> Romeo >> >> >> >> >>> The one posted at the start of the thread was a report coming in rather >>> than facts. >>> >>> It might be that some of these are failing (part way through?), so if >>> you can check as many instances of this as you can, it might shed some >>> light on it. >>> >>> -- >>> Dominic Cleal >>> Red Hat Engineering >>> >>> On 20/08/13 01:19, Romeo Theriault wrote: >>> > I've recently just upgraded to foreman 1.2 and am seeing the same issue >>> > and came searching the list to see if anyone else has seen it. I'm >>> > seeings hosts (in foreman) randomly dropping (i.e. not having) facts. >>> > I've verified that the hosts yaml file on the puppet master does have >>> > all the facts so it appears to be an issue in the passing of the facts >>> > to foreman with the node.rb script or something in the foreman itself >>> > that is not properly writing the facts to the DB. I've verified that my >>> > DB (postgres 8.4.13) does NOT have the dropped facts, so it's not a >>> > problem with displaying them. >>> > >>> > I tried turning the foremans log level up to debug but am not finding >>> > anything useful. >>> > >>> > Anyone have any ideas why facts are getting dropped? or how we can >>> > troubleshoot this? >>> > >>> > This is a big issue for us. >>> > >>> > Any help/pointers is much appreciated. >>> > >>> > Romeo >>> > >>> > >>> > On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen >> > > wrote: >>> > >>> > No one? This is actually happening across two different instances >>> of >>> > Foreman in two different zones. Of course they are configured in >>> the >>> > same way, so it might be something with the way I have done it. >>> From >>> > what I can gather the ENC script is posting the facts. The facts >>> > also exist in the yaml I think the script is posting. Could it be a >>> > timing issue? Are the facts written as the client is trying to post >>> > them? >>> > >>> > >>> > >>> > Thanks, >>> > Anders, >>> > >>> > >>> > On Friday, 16 August 2013 14:23:21 UTC+2, Anders Sveen wrote: >>> > >>> > Hi, >>> > >>> > I have set up Foreman and foreman-proxy on the same host as my >>> > Puppetmaster. I have integrated reports, set up ENC and enabled >>> > facts. Reports are fed back and facts are coming in. >>> > >>> > My 8 hosts show up in the list and everything looks good, but >>> > when i go to facts to query I rarely get the full 8 results. I >>> > do a query for "name = memoryfree" . Some times I get 8, but >>> > most of the time I only get 4,5 or 6 results. And this varies >>> > throughout the day. I don't see any particular pattern as to >>> > which hosts show up either, it seems random. >>> > >>> > Any idea where to start looking? >>> > >>> > My production log shows this: >>> > >>> > Started POST "/reports/create?format=yml" for 127.0.0.1 at >>> > 2013-08-16 11:54:41 +0200 >>> > Processing by ReportsController#create as YML >>> > Parameters: {"report"=>"[FILTERED]"} >>> > processing report for host3.mydomain.com < >>> http://host3.mydomain.com> >>> > Rendered text template (0.0ms) >>> > Completed 200 OK in 38ms (Views: 0.9ms | ActiveRecord: 8.3ms) >>> > >>> > But still there is not fact from the node in the log >>> statement... >>> > >>> > >>> > >>> > >>> > Cheers, >>> > Anders, >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> > Groups "Foreman users" group. >>> > To unsubscribe from this group and stop receiving emails from it, >>> > send an email to foreman-user...@googlegroups.com >>> > . >>> > To post to this group, send email to forema...@googlegroups.com >>> > . >>> > Visit this group at http://groups.google.com/group/foreman-users. >>> > For more options, visit https://groups.google.com/groups/opt_out. >>> > >>> > >>> > >>> > >>> > -- >>> > Romeo >>> > >>> > -- >>> > You received this message because you are subscribed to the Google >>> > Groups "Foreman users" group. >>> > To unsubscribe from this group and stop receiving emails from it, send >>> > an email to foreman-user...@googlegroups.com. >>> > To post to this group, send email to forema...@googlegroups.com. >>> > Visit this group at http://groups.google.com/group/foreman-users. >>> > For more options, visit https://groups.google.com/groups/opt_out. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Foreman users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to foreman-user...@googlegroups.com. >>> To post to this group, send email to forema...@googlegroups.com. >>> Visit this group at http://groups.google.com/group/foreman-users. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> >> >> -- >> Romeo >> >

So I decided to register an issue… I really can't figure out the issue
tracker at Issues - Foreman . Is it
not possible?

Anders,

··· On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen wrote: > > When querying for "name = puppetversion" I get 8 hits. Memoryfree still > show too few... > > > Anders, > > On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen wrote: >> >> I can confirm that I see the same logs too with 200 OK. Still varying >> number of facts though. >> >> >> A, >> >> On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo Theriault wrote: >>> >>> Hi Dominic, thanks for responding. >>> >>> On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal wrote: >>> >>>> Can you (& Anders) look for "POST /fact_values/create" log entries in >>>> /var/log/foreman/production.log? >>>> >>> >>> There are lots of these in my logs: >>> >>> $ grep -A 5 fact_values/create production.log >>> >>> Started POST "/fact_values/create?format=yml" for 131.171.224.199 at >>> 2013-08-20 15:38:51 -1000 >>> Processing by FactValuesController#create as YML >>> Parameters: {"facts"=>"[FILTERED]"} >>> Rendered text template (0.0ms) >>> Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 7.7ms) >>> >>> -- >>> Started POST "/fact_values/create?format=yml" for 131.171.224.199 at >>> 2013-08-20 15:38:54 -1000 >>> Processing by FactValuesController#create as YML >>> Parameters: {"facts"=>"[FILTERED]"} >>> Rendered text template (0.0ms) >>> Completed 200 OK in 50ms (Views: 0.4ms | ActiveRecord: 8.5ms) >>> >>> -- >>> Started POST "/fact_values/create?format=yml" for 131.171.224.199 at >>> 2013-08-20 15:38:54 -1000 >>> Processing by FactValuesController#create as YML >>> Parameters: {"facts"=>"[FILTERED]"} >>> >>> >>> Started GET "/node/dbm99.pvt.hawaii.edu?format=yml" for 131.171.224.199 >>> at 2013-08-20 15:38:54 -1000 >>> -- >>> Started POST "/fact_values/create?format=yml" for 131.171.224.199 at >>> 2013-08-20 15:38:56 -1000 >>> Processing by FactValuesController#create as YML >>> Parameters: {"facts"=>"[FILTERED]"} >>> Rendered text template (0.0ms) >>> Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 8.1ms) >>> >>> >>> I do notice that occasionally there are some log entries that don't have >>> the: >>> >>> Rendered text template (0.0ms) >>> Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 8.1ms) >>> >>> lines. See the third entry above. I'm not sure if this is related or >>> not. >>> >>> Is there anything else I can provide? or do to troubleshoot? >>> >>> One thing I'll note is that one of the facts we see consistently being >>> dropped is a custom fact we created that facter/puppet pushes up. I'm not >>> sure this is at all relevant or not since I can see the fact in the server >>> side yaml file so I know it's there. >>> >>> Thanks, >>> Romeo >>> >>> >>> >>> >>>> The one posted at the start of the thread was a report coming in rather >>>> than facts. >>>> >>>> It might be that some of these are failing (part way through?), so if >>>> you can check as many instances of this as you can, it might shed some >>>> light on it. >>>> >>>> -- >>>> Dominic Cleal >>>> Red Hat Engineering >>>> >>>> On 20/08/13 01:19, Romeo Theriault wrote: >>>> > I've recently just upgraded to foreman 1.2 and am seeing the same >>>> issue >>>> > and came searching the list to see if anyone else has seen it. I'm >>>> > seeings hosts (in foreman) randomly dropping (i.e. not having) facts. >>>> > I've verified that the hosts yaml file on the puppet master does have >>>> > all the facts so it appears to be an issue in the passing of the facts >>>> > to foreman with the node.rb script or something in the foreman itself >>>> > that is not properly writing the facts to the DB. I've verified that >>>> my >>>> > DB (postgres 8.4.13) does NOT have the dropped facts, so it's not a >>>> > problem with displaying them. >>>> > >>>> > I tried turning the foremans log level up to debug but am not finding >>>> > anything useful. >>>> > >>>> > Anyone have any ideas why facts are getting dropped? or how we can >>>> > troubleshoot this? >>>> > >>>> > This is a big issue for us. >>>> > >>>> > Any help/pointers is much appreciated. >>>> > >>>> > Romeo >>>> > >>>> > >>>> > On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen >>> > > wrote: >>>> > >>>> > No one? This is actually happening across two different instances >>>> of >>>> > Foreman in two different zones. Of course they are configured in >>>> the >>>> > same way, so it might be something with the way I have done it. >>>> From >>>> > what I can gather the ENC script is posting the facts. The facts >>>> > also exist in the yaml I think the script is posting. Could it be >>>> a >>>> > timing issue? Are the facts written as the client is trying to >>>> post >>>> > them? >>>> > >>>> > >>>> > >>>> > Thanks, >>>> > Anders, >>>> > >>>> > >>>> > On Friday, 16 August 2013 14:23:21 UTC+2, Anders Sveen wrote: >>>> > >>>> > Hi, >>>> > >>>> > I have set up Foreman and foreman-proxy on the same host as my >>>> > Puppetmaster. I have integrated reports, set up ENC and >>>> enabled >>>> > facts. Reports are fed back and facts are coming in. >>>> > >>>> > My 8 hosts show up in the list and everything looks good, but >>>> > when i go to facts to query I rarely get the full 8 results. I >>>> > do a query for "name = memoryfree" . Some times I get 8, but >>>> > most of the time I only get 4,5 or 6 results. And this varies >>>> > throughout the day. I don't see any particular pattern as to >>>> > which hosts show up either, it seems random. >>>> > >>>> > Any idea where to start looking? >>>> > >>>> > My production log shows this: >>>> > >>>> > Started POST "/reports/create?format=yml" for 127.0.0.1 at >>>> > 2013-08-16 11:54:41 +0200 >>>> > Processing by ReportsController#create as YML >>>> > Parameters: {"report"=>"[FILTERED]"} >>>> > processing report for host3.mydomain.com < >>>> http://host3.mydomain.com> >>>> > Rendered text template (0.0ms) >>>> > Completed 200 OK in 38ms (Views: 0.9ms | ActiveRecord: 8.3ms) >>>> > >>>> > But still there is not fact from the node in the log >>>> statement... >>>> > >>>> > >>>> > >>>> > >>>> > Cheers, >>>> > Anders, >>>> > >>>> > -- >>>> > You received this message because you are subscribed to the Google >>>> > Groups "Foreman users" group. >>>> > To unsubscribe from this group and stop receiving emails from it, >>>> > send an email to foreman-user...@googlegroups.com >>>> > . >>>> > To post to this group, send email to forema...@googlegroups.com >>>> > . >>>> > Visit this group at http://groups.google.com/group/foreman-users. >>>> > For more options, visit https://groups.google.com/groups/opt_out. >>>> > >>>> > >>>> > >>>> > >>>> > -- >>>> > Romeo >>>> > >>>> > -- >>>> > You received this message because you are subscribed to the Google >>>> > Groups "Foreman users" group. >>>> > To unsubscribe from this group and stop receiving emails from it, send >>>> > an email to foreman-user...@googlegroups.com. >>>> > To post to this group, send email to forema...@googlegroups.com. >>>> > Visit this group at http://groups.google.com/group/foreman-users. >>>> > For more options, visit https://groups.google.com/groups/opt_out. >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Foreman users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to foreman-user...@googlegroups.com. >>>> To post to this group, send email to forema...@googlegroups.com. >>>> Visit this group at http://groups.google.com/group/foreman-users. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>> >>> >>> >>> -- >>> Romeo >>> >>

If you've registered and logged in, there should be a "New issue" tab at
the top which points to:

http://projects.theforeman.org/projects/foreman/issues/new

Sorry, I don't know what's causing this at the moment. My only
suspicion is the proximity of timestamps in your log file, one of the
truncated requests was at the same time as another. Which database are
you using?

··· -- Dominic Cleal Red Hat Engineering

On 23/08/13 11:31, Anders Sveen wrote:

So I decided to register an issue… I really can’t figure out the issue
tracker at Issues - Foreman . Is
it not possible?

Anders,

On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen wrote:

When querying for "name = puppetversion" I get 8 hits. Memoryfree
still show too few...


Anders,

On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen wrote:

    I can confirm that I see the same logs too with 200 OK. Still
    varying number of facts though.


    A,

    On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo Theriault wrote:

        Hi Dominic, thanks for responding. 

        On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal >             <dcl...@redhat.com> wrote:

            Can you (& Anders) look for "POST /fact_values/create"
            log entries in
            /var/log/foreman/production.log?


        There are lots of these in my logs:

        $ grep -A 5 fact_values/create production.log

         Started POST "/fact_values/create?format=yml" for
        131.171.224.199 at 2013-08-20 15:38:51 -1000
        Processing by FactValuesController#create as YML
          Parameters: {"facts"=>"[FILTERED]"}
          Rendered text template (0.0ms)
        Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 7.7ms)

        --
        Started POST "/fact_values/create?format=yml" for
        131.171.224.199 at 2013-08-20 15:38:54 -1000
        Processing by FactValuesController#create as YML
          Parameters: {"facts"=>"[FILTERED]"}
          Rendered text template (0.0ms)
        Completed 200 OK in 50ms (Views: 0.4ms | ActiveRecord: 8.5ms)

        --
        Started POST "/fact_values/create?format=yml" for
        131.171.224.199 at 2013-08-20 15:38:54 -1000
        Processing by FactValuesController#create as YML
          Parameters: {"facts"=>"[FILTERED]"}


        Started GET "/node/dbm99.pvt.hawaii.edu?format=yml
        <http://dbm99.pvt.hawaii.edu?format=yml>" for
        131.171.224.199 at 2013-08-20 15:38:54 -1000
        --
        Started POST "/fact_values/create?format=yml" for
        131.171.224.199 at 2013-08-20 15:38:56 -1000
        Processing by FactValuesController#create as YML
          Parameters: {"facts"=>"[FILTERED]"}
          Rendered text template (0.0ms)
        Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 8.1ms)


        I do notice that occasionally there are some log entries
        that don't have the:

         Rendered text template (0.0ms)
        Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: 8.1ms)

        lines. See the third entry above. I'm not sure if this is
        related or not. 

        Is there anything else I can provide? or do to troubleshoot?

        One thing I'll note is that one of the facts we see
        consistently being dropped is a custom fact we created that
        facter/puppet pushes up. I'm not sure this is at all
        relevant or not since I can see the fact in the server side
        yaml file so I know it's there.

        Thanks,
        Romeo




            The one posted at the start of the thread was a report
            coming in rather
            than facts.

            It might be that some of these are failing (part way
            through?), so if
            you can check as many instances of this as you can, it
            might shed some
            light on it.

            --
            Dominic Cleal
            Red Hat Engineering

            On 20/08/13 01:19, Romeo Theriault wrote:
            > I've recently just upgraded to foreman 1.2 and am
            seeing the same issue
            > and came searching the list to see if anyone else has
            seen it. I'm
            > seeings hosts (in foreman) randomly dropping (i.e. not
            having) facts.
            > I've verified that the hosts yaml file on the puppet
            master does have
            > all the facts so it appears to be an issue in the
            passing of the facts
            > to foreman with the node.rb script or something in the
            foreman itself
            > that is not properly writing the facts to the DB. I've
            verified that my
            > DB (postgres 8.4.13) does NOT have the dropped facts,
            so it's not a
            > problem with displaying them.
            >
            > I tried turning the foremans log level up to debug but
            am not finding
            > anything useful.
            >
            > Anyone have any ideas why facts are getting dropped?
            or how we can
            > troubleshoot this?
            >
            > This is a big issue for us.
            >
            > Any help/pointers is much appreciated.
            >
            > Romeo
            >
            >
            > On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen >                 <anders....@gmail.com >                 > <mailto:anders....@gmail.com>> wrote:
            >
            >     No one? This is actually happening across two
            different instances of
            >     Foreman in two different zones. Of course they are
            configured in the
            >     same way, so it might be something with the way I
            have done it. From
            >     what I can gather the ENC script is posting the
            facts. The facts
            >     also exist in the yaml I think the script is
            posting. Could it be a
            >     timing issue? Are the facts written as the client
            is trying to post
            >     them?
            >
            >
            >
            >     Thanks,
            >     Anders,
            >
            >
            >     On Friday, 16 August 2013 14:23:21 UTC+2, Anders >                 Sveen wrote:
            >
            >         Hi,
            >
            >         I have set up Foreman and foreman-proxy on the
            same host as my
            >         Puppetmaster. I have integrated reports, set
            up ENC and enabled
            >         facts. Reports are fed back and facts are
            coming in.
            >
            >         My 8 hosts show up in the list and everything
            looks good, but
            >         when i go to facts to query I rarely get the
            full 8 results. I
            >         do a query for "name = memoryfree" . Some
            times I get 8, but
            >         most of the time I only get 4,5 or 6 results.
            And this varies
            >         throughout the day. I don't see any particular
            pattern as to
            >         which hosts show up either, it seems random.
            >
            >         Any idea where to start looking?
            >
            >         My production log shows this:
            >
            >         Started POST "/reports/create?format=yml" for
            127.0.0.1 at
            >         2013-08-16 11:54:41 +0200
            >         Processing by ReportsController#create as YML
            >           Parameters: {"report"=>"[FILTERED]"}
            >         processing report for host3.mydomain.com
            <http://host3.mydomain.com> <http://host3.mydomain.com>
            >           Rendered text template (0.0ms)
            >         Completed 200 OK in 38ms (Views: 0.9ms |
            ActiveRecord: 8.3ms)
            >
            >         But still there is not fact from the node in
            the log statement...
            >
            >
            >
            >
            >         Cheers,
            >         Anders,
            >
            >     --
            >     You received this message because you are
            subscribed to the Google
            >     Groups "Foreman users" group.
            >     To unsubscribe from this group and stop receiving
            emails from it,
            >     send an email to foreman-user...@googlegroups.com
            >     <mailto:foreman-users%2Bunsubscribe@googlegroups.com>.
            >     To post to this group, send email to
            forema...@googlegroups.com
            >     <mailto:forema...@googlegroups.com>.
            >     Visit this group at
            http://groups.google.com/group/foreman-users
            <http://groups.google.com/group/foreman-users>.
            >     For more options, visit
            https://groups.google.com/groups/opt_out
            <https://groups.google.com/groups/opt_out>.
            >
            >
            >
            >
            > --
            > Romeo
            >
            > --
            > You received this message because you are subscribed
            to the Google
            > Groups "Foreman users" group.
            > To unsubscribe from this group and stop receiving
            emails from it, send
            > an email to foreman-user...@googlegroups.com.
            > To post to this group, send email to
            forema...@googlegroups.com.
            > Visit this group at
            http://groups.google.com/group/foreman-users
            <http://groups.google.com/group/foreman-users>.
            > For more options, visit
            https://groups.google.com/groups/opt_out
            <https://groups.google.com/groups/opt_out>.

            --
            You received this message because you are subscribed to
            the Google Groups "Foreman users" group.
            To unsubscribe from this group and stop receiving emails
            from it, send an email to foreman-user...@googlegroups.com.
            To post to this group, send email to
            forema...@googlegroups.com.
            Visit this group at
            http://groups.google.com/group/foreman-users
            <http://groups.google.com/group/foreman-users>.
            For more options, visit
            https://groups.google.com/groups/opt_out
            <https://groups.google.com/groups/opt_out>.




        -- 
        Romeo


You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.

Doh! Right in front of me, thanks. :slight_smile:

Alright, added issue with some additional details here:
http://projects.theforeman.org/issues/2967

The strange thing is that name = puppetversion always seem to return all
values, while name = memoryfree does not…

Anders,

··· On Friday, 23 August 2013 12:41:32 UTC+2, Dominic Cleal wrote: > > If you've registered and logged in, there should be a "New issue" tab at > the top which points to: > > http://projects.theforeman.org/projects/foreman/issues/new > > Sorry, I don't know what's causing this at the moment. My only > suspicion is the proximity of timestamps in your log file, one of the > truncated requests was at the same time as another. Which database are > you using? > > -- > Dominic Cleal > Red Hat Engineering > > On 23/08/13 11:31, Anders Sveen wrote: > > So I decided to register an issue... I really can't figure out the issue > > tracker at http://projects.theforeman.org/projects/foreman/issues . Is > > it not possible? > > > > > > Anders, > > > > On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen wrote: > > > > When querying for "name = puppetversion" I get 8 hits. Memoryfree > > still show too few... > > > > > > Anders, > > > > On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen wrote: > > > > I can confirm that I see the same logs too with 200 OK. Still > > varying number of facts though. > > > > > > A, > > > > On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo Theriault > wrote: > > > > Hi Dominic, thanks for responding. > > > > On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal > > wrote: > > > > Can you (& Anders) look for "POST /fact_values/create" > > log entries in > > /var/log/foreman/production.log? > > > > > > There are lots of these in my logs: > > > > $ grep -A 5 fact_values/create production.log > > > > Started POST "/fact_values/create?format=yml" for > > 131.171.224.199 at 2013-08-20 15:38:51 -1000 > > Processing by FactValuesController#create as YML > > Parameters: {"facts"=>"[FILTERED]"} > > Rendered text template (0.0ms) > > Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: > 7.7ms) > > > > -- > > Started POST "/fact_values/create?format=yml" for > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 > > Processing by FactValuesController#create as YML > > Parameters: {"facts"=>"[FILTERED]"} > > Rendered text template (0.0ms) > > Completed 200 OK in 50ms (Views: 0.4ms | ActiveRecord: > 8.5ms) > > > > -- > > Started POST "/fact_values/create?format=yml" for > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 > > Processing by FactValuesController#create as YML > > Parameters: {"facts"=>"[FILTERED]"} > > > > > > Started GET "/node/dbm99.pvt.hawaii.edu?format=yml > > " for > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 > > -- > > Started POST "/fact_values/create?format=yml" for > > 131.171.224.199 at 2013-08-20 15:38:56 -1000 > > Processing by FactValuesController#create as YML > > Parameters: {"facts"=>"[FILTERED]"} > > Rendered text template (0.0ms) > > Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: > 8.1ms) > > > > > > I do notice that occasionally there are some log entries > > that don't have the: > > > > Rendered text template (0.0ms) > > Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: > 8.1ms) > > > > lines. See the third entry above. I'm not sure if this is > > related or not. > > > > Is there anything else I can provide? or do to troubleshoot? > > > > One thing I'll note is that one of the facts we see > > consistently being dropped is a custom fact we created that > > facter/puppet pushes up. I'm not sure this is at all > > relevant or not since I can see the fact in the server side > > yaml file so I know it's there. > > > > Thanks, > > Romeo > > > > > > > > > > The one posted at the start of the thread was a report > > coming in rather > > than facts. > > > > It might be that some of these are failing (part way > > through?), so if > > you can check as many instances of this as you can, it > > might shed some > > light on it. > > > > -- > > Dominic Cleal > > Red Hat Engineering > > > > On 20/08/13 01:19, Romeo Theriault wrote: > > > I've recently just upgraded to foreman 1.2 and am > > seeing the same issue > > > and came searching the list to see if anyone else has > > seen it. I'm > > > seeings hosts (in foreman) randomly dropping (i.e. not > > having) facts. > > > I've verified that the hosts yaml file on the puppet > > master does have > > > all the facts so it appears to be an issue in the > > passing of the facts > > > to foreman with the node.rb script or something in the > > foreman itself > > > that is not properly writing the facts to the DB. I've > > verified that my > > > DB (postgres 8.4.13) does NOT have the dropped facts, > > so it's not a > > > problem with displaying them. > > > > > > I tried turning the foremans log level up to debug but > > am not finding > > > anything useful. > > > > > > Anyone have any ideas why facts are getting dropped? > > or how we can > > > troubleshoot this? > > > > > > This is a big issue for us. > > > > > > Any help/pointers is much appreciated. > > > > > > Romeo > > > > > > > > > On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen > > > > > wrote: > > > > > > No one? This is actually happening across two > > different instances of > > > Foreman in two different zones. Of course they are > > configured in the > > > same way, so it might be something with the way I > > have done it. From > > > what I can gather the ENC script is posting the > > facts. The facts > > > also exist in the yaml I think the script is > > posting. Could it be a > > > timing issue? Are the facts written as the client > > is trying to post > > > them? > > > > > > > > > > > > Thanks, > > > Anders, > > > > > > > > > On Friday, 16 August 2013 14:23:21 UTC+2, Anders > > Sveen wrote: > > > > > > Hi, > > > > > > I have set up Foreman and foreman-proxy on the > > same host as my > > > Puppetmaster. I have integrated reports, set > > up ENC and enabled > > > facts. Reports are fed back and facts are > > coming in. > > > > > > My 8 hosts show up in the list and everything > > looks good, but > > > when i go to facts to query I rarely get the > > full 8 results. I > > > do a query for "name = memoryfree" . Some > > times I get 8, but > > > most of the time I only get 4,5 or 6 results. > > And this varies > > > throughout the day. I don't see any particular > > pattern as to > > > which hosts show up either, it seems random. > > > > > > Any idea where to start looking? > > > > > > My production log shows this: > > > > > > Started POST "/reports/create?format=yml" for > > 127.0.0.1 at > > > 2013-08-16 11:54:41 +0200 > > > Processing by ReportsController#create as YML > > > Parameters: {"report"=>"[FILTERED]"} > > > processing report for host3.mydomain.com > > > > > Rendered text template (0.0ms) > > > Completed 200 OK in 38ms (Views: 0.9ms | > > ActiveRecord: 8.3ms) > > > > > > But still there is not fact from the node in > > the log statement... > > > > > > > > > > > > > > > Cheers, > > > Anders, > > > > > > -- > > > You received this message because you are > > subscribed to the Google > > > Groups "Foreman users" group. > > > To unsubscribe from this group and stop receiving > > emails from it, > > > send an email to foreman-user...@googlegroups.com > > > foreman-users%2Bunsubscribe@googlegroups.com >. > > > To post to this group, send email to > > forema...@googlegroups.com > > > . > > > Visit this group at > > http://groups.google.com/group/foreman-users > > . > > > For more options, visit > > https://groups.google.com/groups/opt_out > > . > > > > > > > > > > > > > > > -- > > > Romeo > > > > > > -- > > > You received this message because you are subscribed > > to the Google > > > Groups "Foreman users" group. > > > To unsubscribe from this group and stop receiving > > emails from it, send > > > an email to foreman-user...@googlegroups.com. > > > To post to this group, send email to > > forema...@googlegroups.com. > > > Visit this group at > > http://groups.google.com/group/foreman-users > > . > > > For more options, visit > > https://groups.google.com/groups/opt_out > > . > > > > -- > > You received this message because you are subscribed to > > the Google Groups "Foreman users" group. > > To unsubscribe from this group and stop receiving emails > > from it, send an email to > foreman-user...@googlegroups.com. > > To post to this group, send email to > > forema...@googlegroups.com. > > Visit this group at > > http://groups.google.com/group/foreman-users > > . > > For more options, visit > > https://groups.google.com/groups/opt_out > > . > > > > > > > > > > -- > > Romeo > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Foreman users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to foreman-user...@googlegroups.com . > > To post to this group, send email to forema...@googlegroups.com. > > > Visit this group at http://groups.google.com/group/foreman-users. > > For more options, visit https://groups.google.com/groups/opt_out. > >

Could it be a search issue? If you search for facts for one of the
hosts that has its memoryfree missing, is memoryfree listed there?

I know Romeo said that his facts aren't present in the database, but
it's possible we're looking at two separate issues.

I'm also more concerned about your database because it's SQLite rather
than PostgreSQL, but that's a vague assertion to make.

··· -- Dominic Cleal Red Hat Engineering

On 23/08/13 12:27, Anders Sveen wrote:

Doh! Right in front of me, thanks. :slight_smile:

Alright, added issue with some additional details
here: Bug #2967: Foreman is randomly missing certain facts - Foreman

The strange thing is that name = puppetversion always seem to return all
values, while name = memoryfree does not…

Anders,

On Friday, 23 August 2013 12:41:32 UTC+2, Dominic Cleal wrote:

If you've registered and logged in, there should be a "New issue"
tab at
the top which points to:

http://projects.theforeman.org/projects/foreman/issues/new
<http://projects.theforeman.org/projects/foreman/issues/new>

Sorry, I don't know what's causing this at the moment.  My only
suspicion is the proximity of timestamps in your log file, one of the
truncated requests was at the same time as another.  Which database are
you using?

-- 
Dominic Cleal
Red Hat Engineering

On 23/08/13 11:31, Anders Sveen wrote:
> So I decided to register an issue... I really can't figure out the
issue
> tracker at http://projects.theforeman.org/projects/foreman/issues
<http://projects.theforeman.org/projects/foreman/issues> . Is
> it not possible?
>
>
> Anders,
>
> On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen wrote:
>
>     When querying for "name = puppetversion" I get 8 hits. Memoryfree
>     still show too few...
>
>
>     Anders,
>
>     On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen wrote:
>
>         I can confirm that I see the same logs too with 200 OK. Still
>         varying number of facts though.
>
>
>         A,
>
>         On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo >     Theriault wrote:
>
>             Hi Dominic, thanks for responding.
>
>             On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal >     >             <dcl...@redhat.com> wrote:
>
>                 Can you (& Anders) look for "POST
/fact_values/create"
>                 log entries in
>                 /var/log/foreman/production.log?
>
>
>             There are lots of these in my logs:
>
>             $ grep -A 5 fact_values/create production.log
>
>              Started POST "/fact_values/create?format=yml" for
>             131.171.224.199 at 2013-08-20 15:38:51 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord:
7.7ms)
>
>             --
>             Started POST "/fact_values/create?format=yml" for
>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 50ms (Views: 0.4ms | ActiveRecord:
8.5ms)
>
>             --
>             Started POST "/fact_values/create?format=yml" for
>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>
>
>             Started GET "/node/dbm99.pvt.hawaii.edu?format=yml
<http://dbm99.pvt.hawaii.edu?format=yml>
>             <http://dbm99.pvt.hawaii.edu?format=yml
<http://dbm99.pvt.hawaii.edu?format=yml>>" for
>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             --
>             Started POST "/fact_values/create?format=yml" for
>             131.171.224.199 at 2013-08-20 15:38:56 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord:
8.1ms)
>
>
>             I do notice that occasionally there are some log entries
>             that don't have the:
>
>              Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord:
8.1ms)
>
>             lines. See the third entry above. I'm not sure if this is
>             related or not.
>
>             Is there anything else I can provide? or do to
troubleshoot?
>
>             One thing I'll note is that one of the facts we see
>             consistently being dropped is a custom fact we created
that
>             facter/puppet pushes up. I'm not sure this is at all
>             relevant or not since I can see the fact in the server
side
>             yaml file so I know it's there.
>
>             Thanks,
>             Romeo
>
>
>
>
>                 The one posted at the start of the thread was a
report
>                 coming in rather
>                 than facts.
>
>                 It might be that some of these are failing (part way
>                 through?), so if
>                 you can check as many instances of this as you
can, it
>                 might shed some
>                 light on it.
>
>                 --
>                 Dominic Cleal
>                 Red Hat Engineering
>
>                 On 20/08/13 01:19, Romeo Theriault wrote:
>                 > I've recently just upgraded to foreman 1.2 and am
>                 seeing the same issue
>                 > and came searching the list to see if anyone
else has
>                 seen it. I'm
>                 > seeings hosts (in foreman) randomly dropping
(i.e. not
>                 having) facts.
>                 > I've verified that the hosts yaml file on the
puppet
>                 master does have
>                 > all the facts so it appears to be an issue in the
>                 passing of the facts
>                 > to foreman with the node.rb script or something
in the
>                 foreman itself
>                 > that is not properly writing the facts to the
DB. I've
>                 verified that my
>                 > DB (postgres 8.4.13) does NOT have the dropped
facts,
>                 so it's not a
>                 > problem with displaying them.
>                 >
>                 > I tried turning the foremans log level up to
debug but
>                 am not finding
>                 > anything useful.
>                 >
>                 > Anyone have any ideas why facts are getting
dropped?
>                 or how we can
>                 > troubleshoot this?
>                 >
>                 > This is a big issue for us.
>                 >
>                 > Any help/pointers is much appreciated.
>                 >
>                 > Romeo
>                 >
>                 >
>                 > On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen >     >                 <anders....@gmail.com >     >                 > <mailto:anders....@gmail.com>> wrote:
>                 >
>                 >     No one? This is actually happening across two
>                 different instances of
>                 >     Foreman in two different zones. Of course
they are
>                 configured in the
>                 >     same way, so it might be something with the
way I
>                 have done it. From
>                 >     what I can gather the ENC script is posting the
>                 facts. The facts
>                 >     also exist in the yaml I think the script is
>                 posting. Could it be a
>                 >     timing issue? Are the facts written as the
client
>                 is trying to post
>                 >     them?
>                 >
>                 >
>                 >
>                 >     Thanks,
>                 >     Anders,
>                 >
>                 >
>                 >     On Friday, 16 August 2013 14:23:21 UTC+2, >     Anders >     >                 Sveen wrote:
>                 >
>                 >         Hi,
>                 >
>                 >         I have set up Foreman and foreman-proxy
on the
>                 same host as my
>                 >         Puppetmaster. I have integrated reports,
set
>                 up ENC and enabled
>                 >         facts. Reports are fed back and facts are
>                 coming in.
>                 >
>                 >         My 8 hosts show up in the list and
everything
>                 looks good, but
>                 >         when i go to facts to query I rarely get
the
>                 full 8 results. I
>                 >         do a query for "name = memoryfree" . Some
>                 times I get 8, but
>                 >         most of the time I only get 4,5 or 6
results.
>                 And this varies
>                 >         throughout the day. I don't see any
particular
>                 pattern as to
>                 >         which hosts show up either, it seems
random.
>                 >
>                 >         Any idea where to start looking?
>                 >
>                 >         My production log shows this:
>                 >
>                 >         Started POST
"/reports/create?format=yml" for
>                 127.0.0.1 at
>                 >         2013-08-16 11:54:41 +0200
>                 >         Processing by ReportsController#create
as YML
>                 >           Parameters: {"report"=>"[FILTERED]"}
>                 >         processing report for host3.mydomain.com
<http://host3.mydomain.com>
>                 <http://host3.mydomain.com>
<http://host3.mydomain.com>
>                 >           Rendered text template (0.0ms)
>                 >         Completed 200 OK in 38ms (Views: 0.9ms |
>                 ActiveRecord: 8.3ms)
>                 >
>                 >         But still there is not fact from the
node in
>                 the log statement...
>                 >
>                 >
>                 >
>                 >
>                 >         Cheers,
>                 >         Anders,
>                 >
>                 >     --
>                 >     You received this message because you are
>                 subscribed to the Google
>                 >     Groups "Foreman users" group.
>                 >     To unsubscribe from this group and stop
receiving
>                 emails from it,
>                 >     send an email to
foreman-user...@googlegroups.com
>                 >    
<mailto:foreman-users%2Bunsubscribe@googlegroups.com <javascript:>>.
>                 >     To post to this group, send email to
>                 forema...@googlegroups.com
>                 >     <mailto:forema...@googlegroups.com>.
>                 >     Visit this group at
>                 http://groups.google.com/group/foreman-users
<http://groups.google.com/group/foreman-users>
>                 <http://groups.google.com/group/foreman-users
<http://groups.google.com/group/foreman-users>>.
>                 >     For more options, visit
>                 https://groups.google.com/groups/opt_out
<https://groups.google.com/groups/opt_out>
>                 <https://groups.google.com/groups/opt_out
<https://groups.google.com/groups/opt_out>>.
>                 >
>                 >
>                 >
>                 >
>                 > --
>                 > Romeo
>                 >
>                 > --
>                 > You received this message because you are
subscribed
>                 to the Google
>                 > Groups "Foreman users" group.
>                 > To unsubscribe from this group and stop receiving
>                 emails from it, send
>                 > an email to foreman-user...@googlegroups.com.
>                 > To post to this group, send email to
>                 forema...@googlegroups.com.
>                 > Visit this group at
>                 http://groups.google.com/group/foreman-users
<http://groups.google.com/group/foreman-users>
>                 <http://groups.google.com/group/foreman-users
<http://groups.google.com/group/foreman-users>>.
>                 > For more options, visit
>                 https://groups.google.com/groups/opt_out
<https://groups.google.com/groups/opt_out>
>                 <https://groups.google.com/groups/opt_out
<https://groups.google.com/groups/opt_out>>.
>
>                 --
>                 You received this message because you are
subscribed to
>                 the Google Groups "Foreman users" group.
>                 To unsubscribe from this group and stop receiving
emails
>                 from it, send an email to
foreman-user...@googlegroups.com.
>                 To post to this group, send email to
>                 forema...@googlegroups.com.
>                 Visit this group at
>                 http://groups.google.com/group/foreman-users
<http://groups.google.com/group/foreman-users>
>                 <http://groups.google.com/group/foreman-users
<http://groups.google.com/group/foreman-users>>.
>                 For more options, visit
>                 https://groups.google.com/groups/opt_out
<https://groups.google.com/groups/opt_out>
>                 <https://groups.google.com/groups/opt_out
<https://groups.google.com/groups/opt_out>>.
>
>
>
>
>             --
>             Romeo
>
> --
> You received this message because you are subscribed to the Google
> Groups "Foreman users" group.
> To unsubscribe from this group and stop receiving emails from it,
send
> an email to foreman-user...@googlegroups.com <javascript:>.
> To post to this group, send email to forema...@googlegroups.com
<javascript:>.
> Visit this group at http://groups.google.com/group/foreman-users
<http://groups.google.com/group/foreman-users>.
> For more options, visit https://groups.google.com/groups/opt_out
<https://groups.google.com/groups/opt_out>.


You received this message because you are subscribed to the Google
Groups “Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send
an email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at http://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/groups/opt_out.

Which database are
>
> you using?

I'm using postgres. 8.4.13…

I'm tempted to either try upgrading it or switching to mysql and seeing if
it fixes the issue. How would one go about changing databases engines
(postgres–>mysql), while keeping current foreman data/parameters/etc…?

Thanks,

··· On Fri, Aug 23, 2013 at 12:41 AM, Dominic Cleal wrote: -- Romeo

Is there a query I can try to issue to check facts directly in the db?
Maybe dump everything for a certain host to a file and grep?

Anders,

··· On Friday, 23 August 2013 13:47:59 UTC+2, Dominic Cleal wrote: > > Could it be a search issue? If you search for facts for one of the > hosts that has its memoryfree missing, is memoryfree listed there? > > I know Romeo said that his facts aren't present in the database, but > it's possible we're looking at two separate issues. > > I'm also more concerned about your database because it's SQLite rather > than PostgreSQL, but that's a vague assertion to make. > > -- > Dominic Cleal > Red Hat Engineering > > On 23/08/13 12:27, Anders Sveen wrote: > > Doh! Right in front of me, thanks. :) > > > > Alright, added issue with some additional details > > here: http://projects.theforeman.org/issues/2967 > > > > The strange thing is that name = puppetversion always seem to return all > > values, while name = memoryfree does not... > > > > > > > > Anders, > > > > On Friday, 23 August 2013 12:41:32 UTC+2, Dominic Cleal wrote: > > > > If you've registered and logged in, there should be a "New issue" > > tab at > > the top which points to: > > > > http://projects.theforeman.org/projects/foreman/issues/new > > > > > > Sorry, I don't know what's causing this at the moment. My only > > suspicion is the proximity of timestamps in your log file, one of > the > > truncated requests was at the same time as another. Which database > are > > you using? > > > > -- > > Dominic Cleal > > Red Hat Engineering > > > > On 23/08/13 11:31, Anders Sveen wrote: > > > So I decided to register an issue... I really can't figure out the > > issue > > > tracker at http://projects.theforeman.org/projects/foreman/issues > > . Is > > > it not possible? > > > > > > > > > Anders, > > > > > > On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen wrote: > > > > > > When querying for "name = puppetversion" I get 8 hits. > Memoryfree > > > still show too few... > > > > > > > > > Anders, > > > > > > On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen > wrote: > > > > > > I can confirm that I see the same logs too with 200 OK. > Still > > > varying number of facts though. > > > > > > > > > A, > > > > > > On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo > > Theriault wrote: > > > > > > Hi Dominic, thanks for responding. > > > > > > On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal > > > wrote: > > > > > > Can you (& Anders) look for "POST > > /fact_values/create" > > > log entries in > > > /var/log/foreman/production.log? > > > > > > > > > There are lots of these in my logs: > > > > > > $ grep -A 5 fact_values/create production.log > > > > > > Started POST "/fact_values/create?format=yml" for > > > 131.171.224.199 at 2013-08-20 15:38:51 -1000 > > > Processing by FactValuesController#create as YML > > > Parameters: {"facts"=>"[FILTERED]"} > > > Rendered text template (0.0ms) > > > Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: > > 7.7ms) > > > > > > -- > > > Started POST "/fact_values/create?format=yml" for > > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 > > > Processing by FactValuesController#create as YML > > > Parameters: {"facts"=>"[FILTERED]"} > > > Rendered text template (0.0ms) > > > Completed 200 OK in 50ms (Views: 0.4ms | ActiveRecord: > > 8.5ms) > > > > > > -- > > > Started POST "/fact_values/create?format=yml" for > > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 > > > Processing by FactValuesController#create as YML > > > Parameters: {"facts"=>"[FILTERED]"} > > > > > > > > > Started GET "/node/dbm99.pvt.hawaii.edu?format=yml > > > > > > >" for > > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 > > > -- > > > Started POST "/fact_values/create?format=yml" for > > > 131.171.224.199 at 2013-08-20 15:38:56 -1000 > > > Processing by FactValuesController#create as YML > > > Parameters: {"facts"=>"[FILTERED]"} > > > Rendered text template (0.0ms) > > > Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: > > 8.1ms) > > > > > > > > > I do notice that occasionally there are some log > entries > > > that don't have the: > > > > > > Rendered text template (0.0ms) > > > Completed 200 OK in 41ms (Views: 0.4ms | ActiveRecord: > > 8.1ms) > > > > > > lines. See the third entry above. I'm not sure if this > is > > > related or not. > > > > > > Is there anything else I can provide? or do to > > troubleshoot? > > > > > > One thing I'll note is that one of the facts we see > > > consistently being dropped is a custom fact we created > > that > > > facter/puppet pushes up. I'm not sure this is at all > > > relevant or not since I can see the fact in the server > > side > > > yaml file so I know it's there. > > > > > > Thanks, > > > Romeo > > > > > > > > > > > > > > > The one posted at the start of the thread was a > > report > > > coming in rather > > > than facts. > > > > > > It might be that some of these are failing (part > way > > > through?), so if > > > you can check as many instances of this as you > > can, it > > > might shed some > > > light on it. > > > > > > -- > > > Dominic Cleal > > > Red Hat Engineering > > > > > > On 20/08/13 01:19, Romeo Theriault wrote: > > > > I've recently just upgraded to foreman 1.2 and > am > > > seeing the same issue > > > > and came searching the list to see if anyone > > else has > > > seen it. I'm > > > > seeings hosts (in foreman) randomly dropping > > (i.e. not > > > having) facts. > > > > I've verified that the hosts yaml file on the > > puppet > > > master does have > > > > all the facts so it appears to be an issue in > the > > > passing of the facts > > > > to foreman with the node.rb script or something > > in the > > > foreman itself > > > > that is not properly writing the facts to the > > DB. I've > > > verified that my > > > > DB (postgres 8.4.13) does NOT have the dropped > > facts, > > > so it's not a > > > > problem with displaying them. > > > > > > > > I tried turning the foremans log level up to > > debug but > > > am not finding > > > > anything useful. > > > > > > > > Anyone have any ideas why facts are getting > > dropped? > > > or how we can > > > > troubleshoot this? > > > > > > > > This is a big issue for us. > > > > > > > > Any help/pointers is much appreciated. > > > > > > > > Romeo > > > > > > > > > > > > On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen > > > > > > > wrote: > > > > > > > > No one? This is actually happening across > two > > > different instances of > > > > Foreman in two different zones. Of course > > they are > > > configured in the > > > > same way, so it might be something with the > > way I > > > have done it. From > > > > what I can gather the ENC script is posting > the > > > facts. The facts > > > > also exist in the yaml I think the script is > > > posting. Could it be a > > > > timing issue? Are the facts written as the > > client > > > is trying to post > > > > them? > > > > > > > > > > > > > > > > Thanks, > > > > Anders, > > > > > > > > > > > > On Friday, 16 August 2013 14:23:21 UTC+2, > > Anders > > > Sveen wrote: > > > > > > > > Hi, > > > > > > > > I have set up Foreman and foreman-proxy > > on the > > > same host as my > > > > Puppetmaster. I have integrated reports, > > set > > > up ENC and enabled > > > > facts. Reports are fed back and facts > are > > > coming in. > > > > > > > > My 8 hosts show up in the list and > > everything > > > looks good, but > > > > when i go to facts to query I rarely get > > the > > > full 8 results. I > > > > do a query for "name = memoryfree" . > Some > > > times I get 8, but > > > > most of the time I only get 4,5 or 6 > > results. > > > And this varies > > > > throughout the day. I don't see any > > particular > > > pattern as to > > > > which hosts show up either, it seems > > random. > > > > > > > > Any idea where to start looking? > > > > > > > > My production log shows this: > > > > > > > > Started POST > > "/reports/create?format=yml" for > > > 127.0.0.1 at > > > > 2013-08-16 11:54:41 +0200 > > > > Processing by ReportsController#create > > as YML > > > > Parameters: {"report"=>"[FILTERED]"} > > > > processing report for host3.mydomain.com > > > > > > > > > > > Rendered text template (0.0ms) > > > > Completed 200 OK in 38ms (Views: 0.9ms | > > > ActiveRecord: 8.3ms) > > > > > > > > But still there is not fact from the > > node in > > > the log statement... > > > > > > > > > > > > > > > > > > > > Cheers, > > > > Anders, > > > > > > > > -- > > > > You received this message because you are > > > subscribed to the Google > > > > Groups "Foreman users" group. > > > > To unsubscribe from this group and stop > > receiving > > > emails from it, > > > > send an email to > > foreman-user...@googlegroups.com > > > > > > <mailto:foreman-users%2Bunsubscribe@googlegroups.com >. > > > > To post to this group, send email to > > > forema...@googlegroups.com > > > > . > > > > Visit this group at > > > http://groups.google.com/group/foreman-users > > > > > > >. > > > > For more options, visit > > > https://groups.google.com/groups/opt_out > > > > > > >. > > > > > > > > > > > > > > > > > > > > -- > > > > Romeo > > > > > > > > -- > > > > You received this message because you are > > subscribed > > > to the Google > > > > Groups "Foreman users" group. > > > > To unsubscribe from this group and stop > receiving > > > emails from it, send > > > > an email to foreman-user...@googlegroups.com. > > > > To post to this group, send email to > > > forema...@googlegroups.com. > > > > Visit this group at > > > http://groups.google.com/group/foreman-users > > > > > > >. > > > > For more options, visit > > > https://groups.google.com/groups/opt_out > > > > > > >. > > > > > > -- > > > You received this message because you are > > subscribed to > > > the Google Groups "Foreman users" group. > > > To unsubscribe from this group and stop receiving > > emails > > > from it, send an email to > > foreman-user...@googlegroups.com. > > > To post to this group, send email to > > > forema...@googlegroups.com. > > > Visit this group at > > > http://groups.google.com/group/foreman-users > > > > > > >. > > > For more options, visit > > > https://groups.google.com/groups/opt_out > > > > > > >. > > > > > > > > > > > > > > > -- > > > Romeo > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "Foreman users" group. > > > To unsubscribe from this group and stop receiving emails from it, > > send > > > an email to foreman-user...@googlegroups.com . > > > To post to this group, send email to forema...@googlegroups.com > > . > > > Visit this group at http://groups.google.com/group/foreman-users > > . > > > For more options, visit https://groups.google.com/groups/opt_out > > . > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Foreman users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to foreman-user...@googlegroups.com . > > To post to this group, send email to forema...@googlegroups.com. > > > Visit this group at http://groups.google.com/group/foreman-users. > > For more options, visit https://groups.google.com/groups/opt_out. > >

Finally got round to querying the database. Seems like entries are missing
in the DB too, so storing them seems to be the issue:

> select fact_values.* from fact_values inner join fact_names on
fact_names.id=fact_values.fact_name_id where fact_names.name='memoryfree';
16612|33.11 GB|17|8|2013-08-27 11:29:43.404051|2013-08-27 11:29:43.404051
16690|29.20 GB|17|7|2013-08-27 13:05:25.528203|2013-08-27 13:05:25.528203

Anders,

··· On Friday, 23 August 2013 13:59:01 UTC+2, Anders Sveen wrote: > > Is there a query I can try to issue to check facts directly in the db? > Maybe dump everything for a certain host to a file and grep? > > > > Anders, > > On Friday, 23 August 2013 13:47:59 UTC+2, Dominic Cleal wrote: >> >> Could it be a search issue? If you search for facts for one of the >> hosts that has its memoryfree missing, is memoryfree listed there? >> >> I know Romeo said that his facts aren't present in the database, but >> it's possible we're looking at two separate issues. >> >> I'm also more concerned about your database because it's SQLite rather >> than PostgreSQL, but that's a vague assertion to make. >> >> -- >> Dominic Cleal >> Red Hat Engineering >> >> On 23/08/13 12:27, Anders Sveen wrote: >> > Doh! Right in front of me, thanks. :) >> > >> > Alright, added issue with some additional details >> > here: http://projects.theforeman.org/issues/2967 >> > >> > The strange thing is that name = puppetversion always seem to return >> all >> > values, while name = memoryfree does not... >> > >> > >> > >> > Anders, >> > >> > On Friday, 23 August 2013 12:41:32 UTC+2, Dominic Cleal wrote: >> > >> > If you've registered and logged in, there should be a "New issue" >> > tab at >> > the top which points to: >> > >> > http://projects.theforeman.org/projects/foreman/issues/new >> > >> > >> > Sorry, I don't know what's causing this at the moment. My only >> > suspicion is the proximity of timestamps in your log file, one of >> the >> > truncated requests was at the same time as another. Which database >> are >> > you using? >> > >> > -- >> > Dominic Cleal >> > Red Hat Engineering >> > >> > On 23/08/13 11:31, Anders Sveen wrote: >> > > So I decided to register an issue... I really can't figure out >> the >> > issue >> > > tracker at http://projects.theforeman.org/projects/foreman/issues >> > . Is >> > > it not possible? >> > > >> > > >> > > Anders, >> > > >> > > On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen wrote: >> > > >> > > When querying for "name = puppetversion" I get 8 hits. >> Memoryfree >> > > still show too few... >> > > >> > > >> > > Anders, >> > > >> > > On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen >> wrote: >> > > >> > > I can confirm that I see the same logs too with 200 OK. >> Still >> > > varying number of facts though. >> > > >> > > >> > > A, >> > > >> > > On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo >> > Theriault wrote: >> > > >> > > Hi Dominic, thanks for responding. >> > > >> > > On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal >> > > wrote: >> > > >> > > Can you (& Anders) look for "POST >> > /fact_values/create" >> > > log entries in >> > > /var/log/foreman/production.log? >> > > >> > > >> > > There are lots of these in my logs: >> > > >> > > $ grep -A 5 fact_values/create production.log >> > > >> > > Started POST "/fact_values/create?format=yml" for >> > > 131.171.224.199 at 2013-08-20 15:38:51 -1000 >> > > Processing by FactValuesController#create as YML >> > > Parameters: {"facts"=>"[FILTERED]"} >> > > Rendered text template (0.0ms) >> > > Completed 200 OK in 41ms (Views: 0.4ms | >> ActiveRecord: >> > 7.7ms) >> > > >> > > -- >> > > Started POST "/fact_values/create?format=yml" for >> > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 >> > > Processing by FactValuesController#create as YML >> > > Parameters: {"facts"=>"[FILTERED]"} >> > > Rendered text template (0.0ms) >> > > Completed 200 OK in 50ms (Views: 0.4ms | >> ActiveRecord: >> > 8.5ms) >> > > >> > > -- >> > > Started POST "/fact_values/create?format=yml" for >> > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 >> > > Processing by FactValuesController#create as YML >> > > Parameters: {"facts"=>"[FILTERED]"} >> > > >> > > >> > > Started GET "/node/dbm99.pvt.hawaii.edu?format=yml >> > >> > > > > >" for >> > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 >> > > -- >> > > Started POST "/fact_values/create?format=yml" for >> > > 131.171.224.199 at 2013-08-20 15:38:56 -1000 >> > > Processing by FactValuesController#create as YML >> > > Parameters: {"facts"=>"[FILTERED]"} >> > > Rendered text template (0.0ms) >> > > Completed 200 OK in 41ms (Views: 0.4ms | >> ActiveRecord: >> > 8.1ms) >> > > >> > > >> > > I do notice that occasionally there are some log >> entries >> > > that don't have the: >> > > >> > > Rendered text template (0.0ms) >> > > Completed 200 OK in 41ms (Views: 0.4ms | >> ActiveRecord: >> > 8.1ms) >> > > >> > > lines. See the third entry above. I'm not sure if >> this is >> > > related or not. >> > > >> > > Is there anything else I can provide? or do to >> > troubleshoot? >> > > >> > > One thing I'll note is that one of the facts we see >> > > consistently being dropped is a custom fact we >> created >> > that >> > > facter/puppet pushes up. I'm not sure this is at all >> > > relevant or not since I can see the fact in the >> server >> > side >> > > yaml file so I know it's there. >> > > >> > > Thanks, >> > > Romeo >> > > >> > > >> > > >> > > >> > > The one posted at the start of the thread was a >> > report >> > > coming in rather >> > > than facts. >> > > >> > > It might be that some of these are failing (part >> way >> > > through?), so if >> > > you can check as many instances of this as you >> > can, it >> > > might shed some >> > > light on it. >> > > >> > > -- >> > > Dominic Cleal >> > > Red Hat Engineering >> > > >> > > On 20/08/13 01:19, Romeo Theriault wrote: >> > > > I've recently just upgraded to foreman 1.2 and >> am >> > > seeing the same issue >> > > > and came searching the list to see if anyone >> > else has >> > > seen it. I'm >> > > > seeings hosts (in foreman) randomly dropping >> > (i.e. not >> > > having) facts. >> > > > I've verified that the hosts yaml file on the >> > puppet >> > > master does have >> > > > all the facts so it appears to be an issue in >> the >> > > passing of the facts >> > > > to foreman with the node.rb script or something >> > in the >> > > foreman itself >> > > > that is not properly writing the facts to the >> > DB. I've >> > > verified that my >> > > > DB (postgres 8.4.13) does NOT have the dropped >> > facts, >> > > so it's not a >> > > > problem with displaying them. >> > > > >> > > > I tried turning the foremans log level up to >> > debug but >> > > am not finding >> > > > anything useful. >> > > > >> > > > Anyone have any ideas why facts are getting >> > dropped? >> > > or how we can >> > > > troubleshoot this? >> > > > >> > > > This is a big issue for us. >> > > > >> > > > Any help/pointers is much appreciated. >> > > > >> > > > Romeo >> > > > >> > > > >> > > > On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen >> > > > > > > > wrote: >> > > > >> > > > No one? This is actually happening across >> two >> > > different instances of >> > > > Foreman in two different zones. Of course >> > they are >> > > configured in the >> > > > same way, so it might be something with the >> > way I >> > > have done it. From >> > > > what I can gather the ENC script is posting >> the >> > > facts. The facts >> > > > also exist in the yaml I think the script >> is >> > > posting. Could it be a >> > > > timing issue? Are the facts written as the >> > client >> > > is trying to post >> > > > them? >> > > > >> > > > >> > > > >> > > > Thanks, >> > > > Anders, >> > > > >> > > > >> > > > On Friday, 16 August 2013 14:23:21 UTC+2, >> > Anders >> > > Sveen wrote: >> > > > >> > > > Hi, >> > > > >> > > > I have set up Foreman and foreman-proxy >> > on the >> > > same host as my >> > > > Puppetmaster. I have integrated >> reports, >> > set >> > > up ENC and enabled >> > > > facts. Reports are fed back and facts >> are >> > > coming in. >> > > > >> > > > My 8 hosts show up in the list and >> > everything >> > > looks good, but >> > > > when i go to facts to query I rarely >> get >> > the >> > > full 8 results. I >> > > > do a query for "name = memoryfree" . >> Some >> > > times I get 8, but >> > > > most of the time I only get 4,5 or 6 >> > results. >> > > And this varies >> > > > throughout the day. I don't see any >> > particular >> > > pattern as to >> > > > which hosts show up either, it seems >> > random. >> > > > >> > > > Any idea where to start looking? >> > > > >> > > > My production log shows this: >> > > > >> > > > Started POST >> > "/reports/create?format=yml" for >> > > 127.0.0.1 at >> > > > 2013-08-16 11:54:41 +0200 >> > > > Processing by ReportsController#create >> > as YML >> > > > Parameters: {"report"=>"[FILTERED]"} >> > > > processing report for >> host3.mydomain.com >> > >> > > >> > >> > > > Rendered text template (0.0ms) >> > > > Completed 200 OK in 38ms (Views: 0.9ms >> > >> > > ActiveRecord: 8.3ms) >> > > > >> > > > But still there is not fact from the >> > node in >> > > the log statement... >> > > > >> > > > >> > > > >> > > > >> > > > Cheers, >> > > > Anders, >> > > > >> > > > -- >> > > > You received this message because you are >> > > subscribed to the Google >> > > > Groups "Foreman users" group. >> > > > To unsubscribe from this group and stop >> > receiving >> > > emails from it, >> > > > send an email to >> > foreman-user...@googlegroups.com >> > > > >> > <mailto:foreman-users%2Bunsubscribe@googlegroups.com>. >> > > > To post to this group, send email to >> > > forema...@googlegroups.com >> > > > . >> > > > Visit this group at >> > > http://groups.google.com/group/foreman-users >> > >> > > > > >. >> > > > For more options, visit >> > > https://groups.google.com/groups/opt_out >> > >> > > > > >. >> > > > >> > > > >> > > > >> > > > >> > > > -- >> > > > Romeo >> > > > >> > > > -- >> > > > You received this message because you are >> > subscribed >> > > to the Google >> > > > Groups "Foreman users" group. >> > > > To unsubscribe from this group and stop >> receiving >> > > emails from it, send >> > > > an email to foreman-user...@googlegroups.com. >> > > > To post to this group, send email to >> > > forema...@googlegroups.com. >> > > > Visit this group at >> > > http://groups.google.com/group/foreman-users >> > >> > > > > >. >> > > > For more options, visit >> > > https://groups.google.com/groups/opt_out >> > >> > > > > >. >> > > >> > > -- >> > > You received this message because you are >> > subscribed to >> > > the Google Groups "Foreman users" group. >> > > To unsubscribe from this group and stop receiving >> > emails >> > > from it, send an email to >> > foreman-user...@googlegroups.com. >> > > To post to this group, send email to >> > > forema...@googlegroups.com. >> > > Visit this group at >> > > http://groups.google.com/group/foreman-users >> > >> > > > > >. >> > > For more options, visit >> > > https://groups.google.com/groups/opt_out >> > >> > > > > >. >> > > >> > > >> > > >> > > >> > > -- >> > > Romeo >> > > >> > > -- >> > > You received this message because you are subscribed to the >> Google >> > > Groups "Foreman users" group. >> > > To unsubscribe from this group and stop receiving emails from it, >> > send >> > > an email to foreman-user...@googlegroups.com . >> > > To post to this group, send email to forema...@googlegroups.com >> > . >> > > Visit this group at http://groups.google.com/group/foreman-users >> > . >> > > For more options, visit https://groups.google.com/groups/opt_out >> > . >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Foreman users" group. >> > To unsubscribe from this gro

This is the exact same issue I'm seeing, with postgres though, and
different facts.

··· On Tue, Aug 27, 2013 at 4:26 AM, Anders Sveen wrote:

Finally got round to querying the database. Seems like entries are missing
in the DB too, so storing them seems to be the issue:

select fact_values.* from fact_values inner join fact_names on
fact_names.id=fact_values.fact_name_id where fact_names.name=‘memoryfree’;
16612|33.11 GB|17|8|2013-08-27 11:29:43.404051|2013-08-27 11:29:43.404051
16690|29.20 GB|17|7|2013-08-27 13:05:25.528203|2013-08-27 13:05:25.528203

Anders,

On Friday, 23 August 2013 13:59:01 UTC+2, Anders Sveen wrote:

Is there a query I can try to issue to check facts directly in the db?
Maybe dump everything for a certain host to a file and grep?

Anders,

On Friday, 23 August 2013 13:47:59 UTC+2, Dominic Cleal wrote:

Could it be a search issue? If you search for facts for one of the
hosts that has its memoryfree missing, is memoryfree listed there?

I know Romeo said that his facts aren’t present in the database, but
it’s possible we’re looking at two separate issues.

I’m also more concerned about your database because it’s SQLite rather
than PostgreSQL, but that’s a vague assertion to make.


Dominic Cleal
Red Hat Engineering

On 23/08/13 12:27, Anders Sveen wrote:

Doh! Right in front of me, thanks. :slight_smile:

Alright, added issue with some additional details
here: http://projects.theforeman.**org/issues/2967http://projects.theforeman.org/issues/2967

The strange thing is that name = puppetversion always seem to return
all
values, while name = memoryfree does not…

Anders,

On Friday, 23 August 2013 12:41:32 UTC+2, Dominic Cleal wrote:

If you've registered and logged in, there should be a "New issue"
tab at
the top which points to:

http://projects.theforeman.**org/projects/foreman/issues/**new<http://projects.theforeman.org/projects/foreman/issues/new>
<http://projects.theforeman.**org/projects/foreman/issues/**new<http://projects.theforeman.org/projects/foreman/issues/new>>
Sorry, I don't know what's causing this at the moment.  My only
suspicion is the proximity of timestamps in your log file, one of

the

truncated requests was at the same time as another.  Which

database are

you using?

--
Dominic Cleal
Red Hat Engineering

On 23/08/13 11:31, Anders Sveen wrote:
> So I decided to register an issue... I really can't figure out

the

issue
> tracker at http://projects.theforeman.**

org/projects/foreman/issueshttp://projects.theforeman.org/projects/foreman/issues

<http://projects.theforeman.**org/projects/foreman/issues<http://projects.theforeman.org/projects/foreman/issues>>

. Is

> it not possible?
>
>
> Anders,
>
> On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen wrote:
>
>     When querying for "name = puppetversion" I get 8 hits.

Memoryfree

>     still show too few...
>
>
>     Anders,
>
>     On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen >>> wrote:
>
>         I can confirm that I see the same logs too with 200 OK.

Still

>         varying number of facts though.
>
>
>         A,
>
>         On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo >>> >     Theriault wrote:
>
>             Hi Dominic, thanks for responding.
>
>             On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal >>> >     >             <dcl...@redhat.com> wrote:
>
>                 Can you (& Anders) look for "POST
/fact_values/create"
>                 log entries in
>                 /var/log/foreman/production.**log?
>
>
>             There are lots of these in my logs:
>
>             $ grep -A 5 fact_values/create production.log
>
>              Started POST "/fact_values/create?format=**yml"

for

>             131.171.224.199 at 2013-08-20 15:38:51 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

7.7ms)
>
>             --
>             Started POST "/fact_values/create?format=**yml" for
>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 50ms (Views: 0.4ms |

ActiveRecord:

8.5ms)
>
>             --
>             Started POST "/fact_values/create?format=**yml" for
>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>
>
>             Started GET "/node/dbm99.pvt.hawaii.edu?**format=yml<http://dbm99.pvt.hawaii.edu?format=yml>
<*MailScanner has detected a possible fraud attempt from

dbm99.pvt.hawaii.edu” claiming to be* http://dbm99.pvt.hawaii.edu?**
format=yml http://dbm99.pvt.hawaii.edu?format=yml>

>             <*MailScanner has detected a possible fraud attempt

from “dbm99.pvt.hawaii.edu” claiming to be* http://dbm99.pvt.hawaii.edu?
**format=yml http://dbm99.pvt.hawaii.edu?format=yml

<*MailScanner has detected a possible fraud attempt from

dbm99.pvt.hawaii.edu” claiming to be* http://dbm99.pvt.hawaii.edu?**
format=yml http://dbm99.pvt.hawaii.edu?format=yml>>" for

>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             --
>             Started POST "/fact_values/create?format=**yml" for
>             131.171.224.199 at 2013-08-20 15:38:56 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

8.1ms)
>
>
>             I do notice that occasionally there are some log

entries

>             that don't have the:
>
>              Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

8.1ms)
>
>             lines. See the third entry above. I'm not sure if

this is

>             related or not.
>
>             Is there anything else I can provide? or do to
troubleshoot?
>
>             One thing I'll note is that one of the facts we see
>             consistently being dropped is a custom fact we

created

that
>             facter/puppet pushes up. I'm not sure this is at all
>             relevant or not since I can see the fact in the

server

side
>             yaml file so I know it's there.
>
>             Thanks,
>             Romeo
>
>
>
>
>                 The one posted at the start of the thread was a
report
>                 coming in rather
>                 than facts.
>
>                 It might be that some of these are failing (part

way

>                 through?), so if
>                 you can check as many instances of this as you
can, it
>                 might shed some
>                 light on it.
>
>                 --
>                 Dominic Cleal
>                 Red Hat Engineering
>
>                 On 20/08/13 01:19, Romeo Theriault wrote:
>                 > I've recently just upgraded to foreman 1.2 and

am

>                 seeing the same issue
>                 > and came searching the list to see if anyone
else has
>                 seen it. I'm
>                 > seeings hosts (in foreman) randomly dropping
(i.e. not
>                 having) facts.
>                 > I've verified that the hosts yaml file on the
puppet
>                 master does have
>                 > all the facts so it appears to be an issue in

the

>                 passing of the facts
>                 > to foreman with the node.rb script or

something

in the
>                 foreman itself
>                 > that is not properly writing the facts to the
DB. I've
>                 verified that my
>                 > DB (postgres 8.4.13) does NOT have the dropped
facts,
>                 so it's not a
>                 > problem with displaying them.
>                 >
>                 > I tried turning the foremans log level up to
debug but
>                 am not finding
>                 > anything useful.
>                 >
>                 > Anyone have any ideas why facts are getting
dropped?
>                 or how we can
>                 > troubleshoot this?
>                 >
>                 > This is a big issue for us.
>                 >
>                 > Any help/pointers is much appreciated.
>                 >
>                 > Romeo
>                 >
>                 >
>                 > On Sun, Aug 18, 2013 at 10:12 PM, Anders Sveen >>> >     >                 <anders....@gmail.com >>> >     >                 > <mailto:anders....@gmail.com>> wrote:
>                 >
>                 >     No one? This is actually happening across

two

>                 different instances of
>                 >     Foreman in two different zones. Of course
they are
>                 configured in the
>                 >     same way, so it might be something with

the

way I
>                 have done it. From
>                 >     what I can gather the ENC script is

posting the

>                 facts. The facts
>                 >     also exist in the yaml I think the script

is

>                 posting. Could it be a
>                 >     timing issue? Are the facts written as the
client
>                 is trying to post
>                 >     them?
>                 >
>                 >
>                 >
>                 >     Thanks,
>                 >     Anders,
>                 >
>                 >
>                 >     On Friday, 16 August 2013 14:23:21 UTC+2, >>> >     Anders >>> >     >                 Sveen wrote:
>                 >
>                 >         Hi,
>                 >
>                 >         I have set up Foreman and

foreman-proxy

on the
>                 same host as my
>                 >         Puppetmaster. I have integrated

reports,

set
>                 up ENC and enabled
>                 >         facts. Reports are fed back and facts

are

>                 coming in.
>                 >
>                 >         My 8 hosts show up in the list and
everything
>                 looks good, but
>                 >         when i go to facts to query I rarely

get

the
>                 full 8 results. I
>                 >         do a query for "name = memoryfree" .

Some

>                 times I get 8, but
>                 >         most of the time I only get 4,5 or 6
results.
>                 And this varies
>                 >         throughout the day. I don't see any
particular
>                 pattern as to
>                 >         which hosts show up either, it seems
random.
>                 >
>                 >         Any idea where to start looking?
>                 >
>                 >         My production log shows this:
>                 >
>                 >         Started POST
"/reports/create?format=yml" for
>                 127.0.0.1 at
>                 >         2013-08-16 11:54:41 +0200
>                 >         Processing by ReportsController#create
as YML
>                 >           Parameters: {"report"=>"[FILTERED]"}
>                 >         processing report for

host3.mydomain.com

<http://host3.mydomain.com>
>                 <http://host3.mydomain.com>
<http://host3.mydomain.com>
>                 >           Rendered text template (0.0ms)
>                 >         Completed 200 OK in 38ms (Views: 0.9ms

>                 ActiveRecord: 8.3ms)
>                 >
>                 >         But still there is not fact from the
node in
>                 the log statement...
>                 >
>                 >
>                 >
>                 >
>                 >         Cheers,
>                 >         Anders,
>                 >
>                 >     --
>                 >     You received this message because you are
>                 subscribed to the Google
>                 >     Groups "Foreman users" group.
>                 >     To unsubscribe from this group and stop
receiving
>                 emails from it,
>                 >     send an email to
foreman-user...@googlegroups.**com
>                 >
<mailto:foreman-users%**2Bunsubscribe@googlegroups.com<javascript:>>.
>                 >     To post to this group, send email to
>                 forema...@googlegroups.com
>                 >     <mailto:forema...@**googlegroups.com>.
>                 >     Visit this group at
>                 http://groups.google.com/**group/foreman-users<http://groups.google.com/group/foreman-users>
<http://groups.google.com/**group/foreman-users<http://groups.google.com/group/foreman-users>>
>                 <http://groups.google.com/**group/foreman-users<http://groups.google.com/group/foreman-users>
<http://groups.google.com/**group/foreman-users<http://groups.google.com/group/foreman-users>>>.
>                 >     For more options, visit
>                 https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
<https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>>
>                 <https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
<https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>>>.
>                 >
>                 >
>                 >
>                 >
>                 > --
>                 > Romeo
>                 >
>                 > --
>                 > You received this message because you are
subscribed
>                 to the Google
>                 > Groups "Foreman users" group.
>                 > To unsubscribe from this group and stop

receiving

>                 emails from it, send
>                 > an email to foreman-user

Is there a place I can add some print statements, logging or debug to
analyze this further? Maybe where the inserts are made to check if it
completes correctly?

Cheers,
Anders

··· On Tuesday, 27 August 2013 18:22:33 UTC+2, Romeo Theriault wrote: > > This is the exact same issue I'm seeing, with postgres though, and > different facts. > > > On Tue, Aug 27, 2013 at 4:26 AM, Anders Sveen <anders....@gmail.com > > wrote: > >> Finally got round to querying the database. Seems like entries are >> missing in the DB too, so storing them seems to be the issue: >> >> > select fact_values.* from fact_values inner join fact_names on >> fact_names.id=fact_values.fact_name_id where fact_names.name >> ='memoryfree'; >> 16612|33.11 GB|17|8|2013-08-27 11:29:43.404051|2013-08-27 11:29:43.404051 >> 16690|29.20 GB|17|7|2013-08-27 13:05:25.528203|2013-08-27 13:05:25.528203 >> >> >> >> Anders, >> >> On Friday, 23 August 2013 13:59:01 UTC+2, Anders Sveen wrote: >>> >>> Is there a query I can try to issue to check facts directly in the db? >>> Maybe dump everything for a certain host to a file and grep? >>> >>> >>> >>> Anders, >>> >>> On Friday, 23 August 2013 13:47:59 UTC+2, Dominic Cleal wrote: >>>> >>>> Could it be a search issue? If you search for facts for one of the >>>> hosts that has its memoryfree missing, is memoryfree listed there? >>>> >>>> I know Romeo said that his facts aren't present in the database, but >>>> it's possible we're looking at two separate issues. >>>> >>>> I'm also more concerned about your database because it's SQLite rather >>>> than PostgreSQL, but that's a vague assertion to make. >>>> >>>> -- >>>> Dominic Cleal >>>> Red Hat Engineering >>>> >>>> On 23/08/13 12:27, Anders Sveen wrote: >>>> > Doh! Right in front of me, thanks. :) >>>> > >>>> > Alright, added issue with some additional details >>>> > here: http://projects.theforeman.**org/issues/2967 >>>> > >>>> > The strange thing is that name = puppetversion always seem to return >>>> all >>>> > values, while name = memoryfree does not... >>>> > >>>> > >>>> > >>>> > Anders, >>>> > >>>> > On Friday, 23 August 2013 12:41:32 UTC+2, Dominic Cleal wrote: >>>> > >>>> > If you've registered and logged in, there should be a "New issue" >>>> > tab at >>>> > the top which points to: >>>> > >>>> > http://projects.theforeman.**org/projects/foreman/issues/**new >>>> > <http://projects.theforeman.**org/projects/foreman/issues/**new> >>>> >>>> > >>>> > Sorry, I don't know what's causing this at the moment. My only >>>> > suspicion is the proximity of timestamps in your log file, one of >>>> the >>>> > truncated requests was at the same time as another. Which >>>> database are >>>> > you using? >>>> > >>>> > -- >>>> > Dominic Cleal >>>> > Red Hat Engineering >>>> > >>>> > On 23/08/13 11:31, Anders Sveen wrote: >>>> > > So I decided to register an issue... I really can't figure out >>>> the >>>> > issue >>>> > > tracker at http://projects.theforeman.** >>>> org/projects/foreman/issues >>>> > <http://projects.theforeman.**org/projects/foreman/issues> >>>> . Is >>>> > > it not possible? >>>> > > >>>> > > >>>> > > Anders, >>>> > > >>>> > > On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen >>>> wrote: >>>> > > >>>> > > When querying for "name = puppetversion" I get 8 hits. >>>> Memoryfree >>>> > > still show too few... >>>> > > >>>> > > >>>> > > Anders, >>>> > > >>>> > > On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen >>>> wrote: >>>> > > >>>> > > I can confirm that I see the same logs too with 200 OK. >>>> Still >>>> > > varying number of facts though. >>>> > > >>>> > > >>>> > > A, >>>> > > >>>> > > On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo >>>> > Theriault wrote: >>>> > > >>>> > > Hi Dominic, thanks for responding. >>>> > > >>>> > > On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal >>>> > > wrote: >>>> > > >>>> > > Can you (& Anders) look for "POST >>>> > /fact_values/create" >>>> > > log entries in >>>> > > /var/log/foreman/production.**log? >>>> > > >>>> > > >>>> > > There are lots of these in my logs: >>>> > > >>>> > > $ grep -A 5 fact_values/create production.log >>>> > > >>>> > > Started POST "/fact_values/create?format=**yml" >>>> for >>>> > > 131.171.224.199 at 2013-08-20 15:38:51 -1000 >>>> > > Processing by FactValuesController#create as YML >>>> > > Parameters: {"facts"=>"[FILTERED]"} >>>> > > Rendered text template (0.0ms) >>>> > > Completed 200 OK in 41ms (Views: 0.4ms | >>>> ActiveRecord: >>>> > 7.7ms) >>>> > > >>>> > > -- >>>> > > Started POST "/fact_values/create?format=**yml" >>>> for >>>> > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 >>>> > > Processing by FactValuesController#create as YML >>>> > > Parameters: {"facts"=>"[FILTERED]"} >>>> > > Rendered text template (0.0ms) >>>> > > Completed 200 OK in 50ms (Views: 0.4ms | >>>> ActiveRecord: >>>> > 8.5ms) >>>> > > >>>> > > -- >>>> > > Started POST "/fact_values/create?format=**yml" >>>> for >>>> > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 >>>> > > Processing by FactValuesController#create as YML >>>> > > Parameters: {"facts"=>"[FILTERED]"} >>>> > > >>>> > > >>>> > > Started GET "/node/dbm99.pvt.hawaii.edu?** >>>> format=yml >>>> > <*MailScanner has detected a possible fraud attempt from >>>> "dbm99.pvt.hawaii.edu" claiming to be* http://dbm99.pvt.hawaii.edu?** >>>> format=yml > >>>> > > <*MailScanner has detected a possible fraud >>>> attempt from "dbm99.pvt.hawaii.edu" claiming to be*http://dbm99.pvt.hawaii.edu? >>>> **format=yml >>>> > <*MailScanner has detected a possible fraud attempt from >>>> "dbm99.pvt.hawaii.edu" claiming to be* http://dbm99.pvt.hawaii.edu?** >>>> format=yml >>" for >>>> > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 >>>> > > -- >>>> > > Started POST "/fact_values/create?format=**yml" >>>> for >>>> > > 131.171.224.199 at 2013-08-20 15:38:56 -1000 >>>> > > Processing by FactValuesController#create as YML >>>> > > Parameters: {"facts"=>"[FILTERED]"} >>>> > > Rendered text template (0.0ms) >>>> > > Completed 200 OK in 41ms (Views: 0.4ms | >>>> ActiveRecord: >>>> > 8.1ms) >>>> > > >>>> > > >>>> > > I do notice that occasionally there are some log >>>> entries >>>> > > that don't have the: >>>> > > >>>> > > Rendered text template (0.0ms) >>>> > > Completed 200 OK in 41ms (Views: 0.4ms | >>>> ActiveRecord: >>>> > 8.1ms) >>>> > > >>>> > > lines. See the third entry above. I'm not sure if >>>> this is >>>> > > related or not. >>>> > > >>>> > > Is there anything else I can provide? or do to >>>> > troubleshoot? >>>> > > >>>> > > One thing I'll note is that one of the facts we see >>>> > > consistently being dropped is a custom fact we >>>> created >>>> > that >>>> > > facter/puppet pushes up. I'm not sure this is at >>>> all >>>> > > relevant or not since I can see the fact in the >>>> server >>>> > side >>>> > > yaml file so I know it's there. >>>> > > >>>> > > Thanks, >>>> > > Romeo >>>> > > >>>> > > >>>> > > >>>> > > >>>> > > The one posted at the start of the thread was a >>>> > report >>>> > > coming in rather >>>> > > than facts. >>>> > > >>>> > > It might be that some of these are failing >>>> (part way >>>> > > through?), so if >>>> > > you can check as many instances of this as you >>>> > can, it >>>> > > might shed some >>>> > > light on it. >>>> > > >>>> > > -- >>>> > > Dominic Cleal >>>> > > Red Hat Engineering >>>> > > >>>> > > On 20/08/13 01:19, Romeo Theriault wrote: >>>> > > > I've recently just upgraded to foreman 1.2 >>>> and am >>>> > > seeing the same issue >>>> > > > and came searching the list to see if anyone >>>> > else has >>>> > > seen it. I'm >>>> > > > seeings hosts (in foreman) randomly dropping >>>> > (i.e. not >>>> > > having) facts. >>>> > > > I've verified that the hosts yaml file on the >>>> > puppet >>>> > > master does have >>>> > > > all the facts so it appears to be an issue in >>>> the >>>> > > passing of the facts >>>> > > > to foreman with the node.rb script or >>>> something >>>> > in the >>>> > > foreman itself >>>> > > > that is not properly writing the facts to the >>>> > DB. I've >>>> > > verified that my >>>> > > > DB (postgres 8.4.13) does NOT have the >>>> dropped >>>> > facts, >>>> > > so it's not a >>>> > > > problem with displaying them. >>>> > > > >>>> > > > I tried turning the foremans log level up to >>>> > debug but >>>> > > am not finding >>>> > > > anything useful. >>>> > > > >>>> > > > Anyone have any ideas why facts are getting >>>> > dropped? >>>> > > or how we can >>>> > > > troubleshoot this? >>>> > > > >>>> > > > This is a big issue for us. >>>> > > > >>>> > > > Any help/pointers is much appreciated. >>>> > > > >>>> > > > Romeo >>>> > > > >>>> > > > >>>> > > > On Sun, Aug 18, 2013 at 10:12 PM, Anders >>>> Sveen >>>> > > >>> > > > > wrote: >>>> > > > >>>> > > > No one? This is actually happening across >>>> two >>>> > > different instances of >>>> > > > Foreman in two different zones. Of course >>>> > they are >>>> > > configured in the >>>> > > > same way, so it might be something with >>>> the >>>> > way I >>>> > > have done it. From >>>> > > > what I can gather the ENC script is >>>> posting the >>>> > > facts. The facts >>>> > > > also exist in the yaml I think the script >>>> is >>>> > > posting. Could it be a >>>> > > > timing issue? Are the facts written as >>>> the >>>> > client >>>> > > is trying to post >>>> > > > them? >>>> > > > >>>> > > > >>>> > > > >>>> > > > Thanks, >>>> > > > Anders, >>>> > > > >>>> > > > >>>> > > > On Friday, 16 August 2013 14:23:21 UTC+2, >>>> > Anders >>>> > > Sveen wrote: >>>> > > > >>>> > > > Hi, >>>> > > > >>>> > > > I have set up Foreman and >>>> foreman-proxy >>>> > on the >>>> > > same host as my >>>> > > > Puppetmaster. I have integrated >>>> reports, >>>> > set >>>> > > up ENC and enabled >>>> > > > facts. Reports are fed back and facts >>>> are >>>> > > coming in. >>>> > > > >>>> > > > My 8 hosts show up in the list and >>>> > everything >>>> > > looks good, but >>>> > > > when i go to facts to query I rarely >>>> get >>>> > the >>>> > > full 8 results. I >>>> > > > do a query for "name = memoryfree" . >>>> Some >>>> > > times I get 8, but >>>> > > > most of the time I only get 4,5 or 6 >>>> > results. >>>> > > And this varies >>>> > > > throughout the day. I don't see any >>>> > particular >>>> > > pattern as to >>>> > > > which hosts show up either, it seems >>>> > random. >>>> > > > >>>> > > > Any idea where to start looking? >>>> > > > >>>> > > > My production log shows this: >>>> > > > >>>> > > > Started POST >>>> > "/reports/create?format=yml" for >>>> > > 127.0.0.1 at >>>> > > > 2013-08-16 11:54:41 +0200 >>>> > > > Processing by >>>> ReportsController#create >>>> > as YML >>>> > > > Parameters: >>>> {"report"=>"[FILTERED]"} >>>> > > > processing report for >>>> host3.mydomain.com >>>> > >>>> > > >>>> > >>>> > > > Rendered text template (0.0ms) >>>> > > > Completed 200 OK in 38ms (Views: >>>> 0.9ms | >>>> > > ActiveRecord: 8.3ms) >>>> > > > >>>> > > > But still there is not fact from the >>>> > node in >>>> > > the log statement... >>>> > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > Cheers, >>>> > > > Anders, >>>> > > > >>>> > > > -- >>>> > > > You received this message because you are >>>> > > subscribed to the Google >>>> > > > Groups "Foreman users" group. >>>> > > > To unsubscribe from this group and stop >>>> > receiving >>>> > > emails from it, >>>> > > > send an email to >>>> > foreman-user...@googlegroups.**com >>>> > > > >>>> > <mailto:foreman-users%**2Bunsubscribe@googlegroups.com>. >>>> > > > To post to this group, send email to >>>> > > forema...@googlegroups.com >>>> > > > . >>>> > > > Visit this group at >>>> > > http://groups.google.com/**group/foreman-users >>>> > <http://groups.google.com/**group/foreman-users> >>>> >>>> > > <http://groups.google.com/**group/foreman-users >>>> > <http://groups.google.com/**group/foreman-users>>. >>>> >>>> > > > For more options, visit >>>> > > https://groups.google.com/**groups/opt_out >>>> > <https://groups.google.com/**groups/opt_out> >>>> >>>> > > <https://groups.google.com/**groups/opt_out >>>> > <https://groups.google.com/**groups/opt_out>>. >>>> >>>> > > > >>>> > > > >>>> > > > >>>> > > > >>>> > > > -- >>>> > > > Romeo >>>> > > > >>>> > > > -- >>>> > > > You received this message because you are >>>> > subscribed >>>> > > to the Google >>>> > > > Groups "Foreman users" group. >>>> > > > To unsubscribe from this group and stop >>>> receiving >>>> > > emails from it, send >>>> > > > an email to foreman-user...@googlegroups.** >>>> com. >>>> > > > To post to this group, send email to >>>> > > forema..

> This is the exact same issue I'm seeing, with postgres though, and
> different facts.
>

the only thing that comes to mind, is that we 1. delete the changed facts,
and 2. create new ones.
this can happen if #1 happens, and #2 fails (not sure why).

I would suggest we try to update the facts instead of delete and create new
ones.

hopefully we can get it right for 1.3

Ohad

··· On Tue, Aug 27, 2013 at 7:22 PM, Romeo Theriault wrote:

On Tue, Aug 27, 2013 at 4:26 AM, Anders Sveen anders.r.sveen@gmail.comwrote:

Finally got round to querying the database. Seems like entries are
missing in the DB too, so storing them seems to be the issue:

select fact_values.* from fact_values inner join fact_names on
fact_names.id=fact_values.fact_name_id where fact_names.name
=‘memoryfree’;
16612|33.11 GB|17|8|2013-08-27 11:29:43.404051|2013-08-27 11:29:43.404051
16690|29.20 GB|17|7|2013-08-27 13:05:25.528203|2013-08-27 13:05:25.528203

Anders,

On Friday, 23 August 2013 13:59:01 UTC+2, Anders Sveen wrote:

Is there a query I can try to issue to check facts directly in the db?
Maybe dump everything for a certain host to a file and grep?

Anders,

On Friday, 23 August 2013 13:47:59 UTC+2, Dominic Cleal wrote:

Could it be a search issue? If you search for facts for one of the
hosts that has its memoryfree missing, is memoryfree listed there?

I know Romeo said that his facts aren’t present in the database, but
it’s possible we’re looking at two separate issues.

I’m also more concerned about your database because it’s SQLite rather
than PostgreSQL, but that’s a vague assertion to make.


Dominic Cleal
Red Hat Engineering

On 23/08/13 12:27, Anders Sveen wrote:

Doh! Right in front of me, thanks. :slight_smile:

Alright, added issue with some additional details
here: http://projects.theforeman.**org/issues/2967http://projects.theforeman.org/issues/2967

The strange thing is that name = puppetversion always seem to return
all
values, while name = memoryfree does not…

Anders,

On Friday, 23 August 2013 12:41:32 UTC+2, Dominic Cleal wrote:

If you've registered and logged in, there should be a "New issue"
tab at
the top which points to:

http://projects.theforeman.**org/projects/foreman/issues/**new<http://projects.theforeman.org/projects/foreman/issues/new>
<http://projects.theforeman.**org/projects/foreman/issues/**new<http://projects.theforeman.org/projects/foreman/issues/new>>
Sorry, I don't know what's causing this at the moment.  My only
suspicion is the proximity of timestamps in your log file, one of

the

truncated requests was at the same time as another.  Which

database are

you using?

--
Dominic Cleal
Red Hat Engineering

On 23/08/13 11:31, Anders Sveen wrote:
> So I decided to register an issue... I really can't figure out

the

issue
> tracker at http://projects.theforeman.**

org/projects/foreman/issueshttp://projects.theforeman.org/projects/foreman/issues

<http://projects.theforeman.**org/projects/foreman/issues<http://projects.theforeman.org/projects/foreman/issues>>

. Is

> it not possible?
>
>
> Anders,
>
> On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen >>>> wrote:
>
>     When querying for "name = puppetversion" I get 8 hits.

Memoryfree

>     still show too few...
>
>
>     Anders,
>
>     On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen >>>> wrote:
>
>         I can confirm that I see the same logs too with 200 OK.

Still

>         varying number of facts though.
>
>
>         A,
>
>         On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo >>>> >     Theriault wrote:
>
>             Hi Dominic, thanks for responding.
>
>             On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal >>>> >     >             <dcl...@redhat.com> wrote:
>
>                 Can you (& Anders) look for "POST
/fact_values/create"
>                 log entries in
>                 /var/log/foreman/production.**log?
>
>
>             There are lots of these in my logs:
>
>             $ grep -A 5 fact_values/create production.log
>
>              Started POST "/fact_values/create?format=**yml"

for

>             131.171.224.199 at 2013-08-20 15:38:51 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

7.7ms)
>
>             --
>             Started POST "/fact_values/create?format=**yml"

for

>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 50ms (Views: 0.4ms |

ActiveRecord:

8.5ms)
>
>             --
>             Started POST "/fact_values/create?format=**yml"

for

>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>
>
>             Started GET "/node/dbm99.pvt.hawaii.edu?**

format=yml http://dbm99.pvt.hawaii.edu?format=yml

<*MailScanner has detected a possible fraud attempt from

dbm99.pvt.hawaii.edu” claiming to be* http://dbm99.pvt.hawaii.edu?**
format=yml http://dbm99.pvt.hawaii.edu?format=yml>

>             <*MailScanner has detected a possible fraud

attempt from “dbm99.pvt.hawaii.edu” claiming to be*http://dbm99.pvt.hawaii.edu?
**format=yml http://dbm99.pvt.hawaii.edu?format=yml

<*MailScanner has detected a possible fraud attempt from

dbm99.pvt.hawaii.edu” claiming to be* http://dbm99.pvt.hawaii.edu?**
format=yml http://dbm99.pvt.hawaii.edu?format=yml>>" for

>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             --
>             Started POST "/fact_values/create?format=**yml"

for

>             131.171.224.199 at 2013-08-20 15:38:56 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

8.1ms)
>
>
>             I do notice that occasionally there are some log

entries

>             that don't have the:
>
>              Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

8.1ms)
>
>             lines. See the third entry above. I'm not sure if

this is

>             related or not.
>
>             Is there anything else I can provide? or do to
troubleshoot?
>
>             One thing I'll note is that one of the facts we see
>             consistently being dropped is a custom fact we

created

that
>             facter/puppet pushes up. I'm not sure this is at

all

>             relevant or not since I can see the fact in the

server

side
>             yaml file so I know it's there.
>
>             Thanks,
>             Romeo
>
>
>
>
>                 The one posted at the start of the thread was a
report
>                 coming in rather
>                 than facts.
>
>                 It might be that some of these are failing

(part way

>                 through?), so if
>                 you can check as many instances of this as you
can, it
>                 might shed some
>                 light on it.
>
>                 --
>                 Dominic Cleal
>                 Red Hat Engineering
>
>                 On 20/08/13 01:19, Romeo Theriault wrote:
>                 > I've recently just upgraded to foreman 1.2

and am

>                 seeing the same issue
>                 > and came searching the list to see if anyone
else has
>                 seen it. I'm
>                 > seeings hosts (in foreman) randomly dropping
(i.e. not
>                 having) facts.
>                 > I've verified that the hosts yaml file on the
puppet
>                 master does have
>                 > all the facts so it appears to be an issue in

the

>                 passing of the facts
>                 > to foreman with the node.rb script or

something

in the
>                 foreman itself
>                 > that is not properly writing the facts to the
DB. I've
>                 verified that my
>                 > DB (postgres 8.4.13) does NOT have the

dropped

facts,
>                 so it's not a
>                 > problem with displaying them.
>                 >
>                 > I tried turning the foremans log level up to
debug but
>                 am not finding
>                 > anything useful.
>                 >
>                 > Anyone have any ideas why facts are getting
dropped?
>                 or how we can
>                 > troubleshoot this?
>                 >
>                 > This is a big issue for us.
>                 >
>                 > Any help/pointers is much appreciated.
>                 >
>                 > Romeo
>                 >
>                 >
>                 > On Sun, Aug 18, 2013 at 10:12 PM, Anders >>>> Sveen >>>> >     >                 <anders....@gmail.com >>>> >     >                 > <mailto:anders....@gmail.com>> wrote:
>                 >
>                 >     No one? This is actually happening across

two

>                 different instances of
>                 >     Foreman in two different zones. Of course
they are
>                 configured in the
>                 >     same way, so it might be something with

the

way I
>                 have done it. From
>                 >     what I can gather the ENC script is

posting the

>                 facts. The facts
>                 >     also exist in the yaml I think the script

is

>                 posting. Could it be a
>                 >     timing issue? Are the facts written as

the

client
>                 is trying to post
>                 >     them?
>                 >
>                 >
>                 >
>                 >     Thanks,
>                 >     Anders,
>                 >
>                 >
>                 >     On Friday, 16 August 2013 14:23:21 >>>> UTC+2, >>>> >     Anders >>>> >     >                 Sveen wrote:
>                 >
>                 >         Hi,
>                 >
>                 >         I have set up Foreman and

foreman-proxy

on the
>                 same host as my
>                 >         Puppetmaster. I have integrated

reports,

set
>                 up ENC and enabled
>                 >         facts. Reports are fed back and facts

are

>                 coming in.
>                 >
>                 >         My 8 hosts show up in the list and
everything
>                 looks good, but
>                 >         when i go to facts to query I rarely

get

the
>                 full 8 results. I
>                 >         do a query for "name = memoryfree" .

Some

>                 times I get 8, but
>                 >         most of the time I only get 4,5 or 6
results.
>                 And this varies
>                 >         throughout the day. I don't see any
particular
>                 pattern as to
>                 >         which hosts show up either, it seems
random.
>                 >
>                 >         Any idea where to start looking?
>                 >
>                 >         My production log shows this:
>                 >
>                 >         Started POST
"/reports/create?format=yml" for
>                 127.0.0.1 at
>                 >         2013-08-16 11:54:41 +0200
>                 >         Processing by

ReportsController#create

as YML
>                 >           Parameters:

{“report”=>"[FILTERED]"}

>                 >         processing report for

host3.mydomain.com

<http://host3.mydomain.com>
>                 <http://host3.mydomain.com>
<http://host3.mydomain.com>
>                 >           Rendered text template (0.0ms)
>                 >         Completed 200 OK in 38ms (Views:

0.9ms |

>                 ActiveRecord: 8.3ms)
>                 >
>                 >         But still there is not fact from the
node in
>                 the log statement...
>                 >
>                 >
>                 >
>                 >
>                 >         Cheers,
>                 >         Anders,
>                 >
>                 >     --
>                 >     You received this message because you are
>                 subscribed to the Google
>                 >     Groups "Foreman users" group.
>                 >     To unsubscribe from this group and stop
receiving
>                 emails from it,
>                 >     send an email to
foreman-user...@googlegroups.**com
>                 >
<mailto:foreman-users%**2Bunsubscribe@googlegroups.com<javascript:>>.
>                 >     To post to this group, send email to
>                 forema...@googlegroups.com
>                 >     <mailto:forema...@**googlegroups.com>.
>                 >     Visit this group at
>                 http://group

Gah, this is really getting to me.

Weird: query for "name = memorysize" always returns 8 nodes. "name =
memoryfree" still returns varying number of nodes.

Romeo: Do you see the same behaviour? Did you find a solution on your end?

Anders,

··· On Wednesday, 4 September 2013 10:46:51 UTC+2, Anders Sveen wrote: > > Is there a place I can add some print statements, logging or debug to > analyze this further? Maybe where the inserts are made to check if it > completes correctly? > > > Cheers, > Anders > > On Tuesday, 27 August 2013 18:22:33 UTC+2, Romeo Theriault wrote: >> >> This is the exact same issue I'm seeing, with postgres though, and >> different facts. >> >> >> On Tue, Aug 27, 2013 at 4:26 AM, Anders Sveen wrote: >> >>> Finally got round to querying the database. Seems like entries are >>> missing in the DB too, so storing them seems to be the issue: >>> >>> > select fact_values.* from fact_values inner join fact_names on >>> fact_names.id=fact_values.fact_name_id where fact_names.name >>> ='memoryfree'; >>> 16612|33.11 GB|17|8|2013-08-27 11:29:43.404051|2013-08-27 11:29:43.404051 >>> 16690|29.20 GB|17|7|2013-08-27 13:05:25.528203|2013-08-27 13:05:25.528203 >>> >>> >>> >>> Anders, >>> >>> On Friday, 23 August 2013 13:59:01 UTC+2, Anders Sveen wrote: >>>> >>>> Is there a query I can try to issue to check facts directly in the db? >>>> Maybe dump everything for a certain host to a file and grep? >>>> >>>> >>>> >>>> Anders, >>>> >>>> On Friday, 23 August 2013 13:47:59 UTC+2, Dominic Cleal wrote: >>>>> >>>>> Could it be a search issue? If you search for facts for one of the >>>>> hosts that has its memoryfree missing, is memoryfree listed there? >>>>> >>>>> I know Romeo said that his facts aren't present in the database, but >>>>> it's possible we're looking at two separate issues. >>>>> >>>>> I'm also more concerned about your database because it's SQLite rather >>>>> than PostgreSQL, but that's a vague assertion to make. >>>>> >>>>> -- >>>>> Dominic Cleal >>>>> Red Hat Engineering >>>>> >>>>> On 23/08/13 12:27, Anders Sveen wrote: >>>>> > Doh! Right in front of me, thanks. :) >>>>> > >>>>> > Alright, added issue with some additional details >>>>> > here: http://projects.theforeman.**org/issues/2967 >>>>> > >>>>> > The strange thing is that name = puppetversion always seem to return >>>>> all >>>>> > values, while name = memoryfree does not... >>>>> > >>>>> > >>>>> > >>>>> > Anders, >>>>> > >>>>> > On Friday, 23 August 2013 12:41:32 UTC+2, Dominic Cleal wrote: >>>>> > >>>>> > If you've registered and logged in, there should be a "New >>>>> issue" >>>>> > tab at >>>>> > the top which points to: >>>>> > >>>>> > http://projects.theforeman.**org/projects/foreman/issues/**new >>>>> > <http://projects.theforeman.**org/projects/foreman/issues/**new> >>>>> >>>>> > >>>>> > Sorry, I don't know what's causing this at the moment. My only >>>>> > suspicion is the proximity of timestamps in your log file, one >>>>> of the >>>>> > truncated requests was at the same time as another. Which >>>>> database are >>>>> > you using? >>>>> > >>>>> > -- >>>>> > Dominic Cleal >>>>> > Red Hat Engineering >>>>> > >>>>> > On 23/08/13 11:31, Anders Sveen wrote: >>>>> > > So I decided to register an issue... I really can't figure out >>>>> the >>>>> > issue >>>>> > > tracker at http://projects.theforeman.** >>>>> org/projects/foreman/issues >>>>> > <http://projects.theforeman.**org/projects/foreman/issues> >>>>> . Is >>>>> > > it not possible? >>>>> > > >>>>> > > >>>>> > > Anders, >>>>> > > >>>>> > > On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen >>>>> wrote: >>>>> > > >>>>> > > When querying for "name = puppetversion" I get 8 hits. >>>>> Memoryfree >>>>> > > still show too few... >>>>> > > >>>>> > > >>>>> > > Anders, >>>>> > > >>>>> > > On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen >>>>> wrote: >>>>> > > >>>>> > > I can confirm that I see the same logs too with 200 >>>>> OK. Still >>>>> > > varying number of facts though. >>>>> > > >>>>> > > >>>>> > > A, >>>>> > > >>>>> > > On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo >>>>> > Theriault wrote: >>>>> > > >>>>> > > Hi Dominic, thanks for responding. >>>>> > > >>>>> > > On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal >>>>> > > wrote: >>>>> > > >>>>> > > Can you (& Anders) look for "POST >>>>> > /fact_values/create" >>>>> > > log entries in >>>>> > > /var/log/foreman/production.**log? >>>>> > > >>>>> > > >>>>> > > There are lots of these in my logs: >>>>> > > >>>>> > > $ grep -A 5 fact_values/create production.log >>>>> > > >>>>> > > Started POST "/fact_values/create?format=**yml" >>>>> for >>>>> > > 131.171.224.199 at 2013-08-20 15:38:51 -1000 >>>>> > > Processing by FactValuesController#create as YML >>>>> > > Parameters: {"facts"=>"[FILTERED]"} >>>>> > > Rendered text template (0.0ms) >>>>> > > Completed 200 OK in 41ms (Views: 0.4ms | >>>>> ActiveRecord: >>>>> > 7.7ms) >>>>> > > >>>>> > > -- >>>>> > > Started POST "/fact_values/create?format=**yml" >>>>> for >>>>> > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 >>>>> > > Processing by FactValuesController#create as YML >>>>> > > Parameters: {"facts"=>"[FILTERED]"} >>>>> > > Rendered text template (0.0ms) >>>>> > > Completed 200 OK in 50ms (Views: 0.4ms | >>>>> ActiveRecord: >>>>> > 8.5ms) >>>>> > > >>>>> > > -- >>>>> > > Started POST "/fact_values/create?format=**yml" >>>>> for >>>>> > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 >>>>> > > Processing by FactValuesController#create as YML >>>>> > > Parameters: {"facts"=>"[FILTERED]"} >>>>> > > >>>>> > > >>>>> > > Started GET "/node/dbm99.pvt.hawaii.edu?** >>>>> format=yml >>>>> > <*MailScanner has detected a possible fraud attempt from >>>>> "dbm99.pvt.hawaii.edu" claiming to be* http://dbm99.pvt.hawaii.edu?** >>>>> format=yml > >>>>> > > <*MailScanner has detected a possible fraud >>>>> attempt from "dbm99.pvt.hawaii.edu" claiming to be*http://dbm99.pvt.hawaii.edu? >>>>> **format=yml >>>>> > <*MailScanner has detected a possible fraud attempt from >>>>> "dbm99.pvt.hawaii.edu" claiming to be* http://dbm99.pvt.hawaii.edu?** >>>>> format=yml >>" for >>>>> > > 131.171.224.199 at 2013-08-20 15:38:54 -1000 >>>>> > > -- >>>>> > > Started POST "/fact_values/create?format=**yml" >>>>> for >>>>> > > 131.171.224.199 at 2013-08-20 15:38:56 -1000 >>>>> > > Processing by FactValuesController#create as YML >>>>> > > Parameters: {"facts"=>"[FILTERED]"} >>>>> > > Rendered text template (0.0ms) >>>>> > > Completed 200 OK in 41ms (Views: 0.4ms | >>>>> ActiveRecord: >>>>> > 8.1ms) >>>>> > > >>>>> > > >>>>> > > I do notice that occasionally there are some log >>>>> entries >>>>> > > that don't have the: >>>>> > > >>>>> > > Rendered text template (0.0ms) >>>>> > > Completed 200 OK in 41ms (Views: 0.4ms | >>>>> ActiveRecord: >>>>> > 8.1ms) >>>>> > > >>>>> > > lines. See the third entry above. I'm not sure if >>>>> this is >>>>> > > related or not. >>>>> > > >>>>> > > Is there anything else I can provide? or do to >>>>> > troubleshoot? >>>>> > > >>>>> > > One thing I'll note is that one of the facts we >>>>> see >>>>> > > consistently being dropped is a custom fact we >>>>> created >>>>> > that >>>>> > > facter/puppet pushes up. I'm not sure this is at >>>>> all >>>>> > > relevant or not since I can see the fact in the >>>>> server >>>>> > side >>>>> > > yaml file so I know it's there. >>>>> > > >>>>> > > Thanks, >>>>> > > Romeo >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > The one posted at the start of the thread was >>>>> a >>>>> > report >>>>> > > coming in rather >>>>> > > than facts. >>>>> > > >>>>> > > It might be that some of these are failing >>>>> (part way >>>>> > > through?), so if >>>>> > > you can check as many instances of this as you >>>>> > can, it >>>>> > > might shed some >>>>> > > light on it. >>>>> > > >>>>> > > -- >>>>> > > Dominic Cleal >>>>> > > Red Hat Engineering >>>>> > > >>>>> > > On 20/08/13 01:19, Romeo Theriault wrote: >>>>> > > > I've recently just upgraded to foreman 1.2 >>>>> and am >>>>> > > seeing the same issue >>>>> > > > and came searching the list to see if anyone >>>>> > else has >>>>> > > seen it. I'm >>>>> > > > seeings hosts (in foreman) randomly dropping >>>>> > (i.e. not >>>>> > > having) facts. >>>>> > > > I've verified that the hosts yaml file on >>>>> the >>>>> > puppet >>>>> > > master does have >>>>> > > > all the facts so it appears to be an issue >>>>> in the >>>>> > > passing of the facts >>>>> > > > to foreman with the node.rb script or >>>>> something >>>>> > in the >>>>> > > foreman itself >>>>> > > > that is not properly writing the facts to >>>>> the >>>>> > DB. I've >>>>> > > verified that my >>>>> > > > DB (postgres 8.4.13) does NOT have the >>>>> dropped >>>>> > facts, >>>>> > > so it's not a >>>>> > > > problem with displaying them. >>>>> > > > >>>>> > > > I tried turning the foremans log level up to >>>>> > debug but >>>>> > > am not finding >>>>> > > > anything useful. >>>>> > > > >>>>> > > > Anyone have any ideas why facts are getting >>>>> > dropped? >>>>> > > or how we can >>>>> > > > troubleshoot this? >>>>> > > > >>>>> > > > This is a big issue for us. >>>>> > > > >>>>> > > > Any help/pointers is much appreciated. >>>>> > > > >>>>> > > > Romeo >>>>> > > > >>>>> > > > >>>>> > > > On Sun, Aug 18, 2013 at 10:12 PM, Anders >>>>> Sveen >>>>> > > >>>> > > > > wrote: >>>>> > > > >>>>> > > > No one? This is actually happening >>>>> across two >>>>> > > different instances of >>>>> > > > Foreman in two different zones. Of >>>>> course >>>>> > they are >>>>> > > configured in the >>>>> > > > same way, so it might be something with >>>>> the >>>>> > way I >>>>> > > have done it. From >>>>> > > > what I can gather the ENC script is >>>>> posting the >>>>> > > facts. The facts >>>>> > > > also exist in the yaml I think the >>>>> script is >>>>> > > posting. Could it be a >>>>> > > > timing issue? Are the facts written as >>>>> the >>>>> > client >>>>> > > is trying to post >>>>> > > > them? >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > Thanks, >>>>> > > > Anders, >>>>> > > > >>>>> > > > >>>>> > > > On Friday, 16 August 2013 14:23:21 >>>>> UTC+2, >>>>> > Anders >>>>> > > Sveen wrote: >>>>> > > > >>>>> > > > Hi, >>>>> > > > >>>>> > > > I have set up Foreman and >>>>> foreman-proxy >>>>> > on the >>>>> > > same host as my >>>>> > > > Puppetmaster. I have integrated >>>>> reports, >>>>> > set >>>>> > > up ENC and enabled >>>>> > > > facts. Reports are fed back and >>>>> facts are >>>>> > > coming in. >>>>> > > > >>>>> > > > My 8 hosts show up in the list and >>>>> > everything >>>>> > > looks good, but >>>>> > > > when i go to facts to query I rarely >>>>> get >>>>> > the >>>>> > > full 8 results. I >>>>> > > > do a query for "name = memoryfree" . >>>>> Some >>>>> > > times I get 8, but >>>>> > > > most of the time I only get 4,5 or 6 >>>>> > results. >>>>> > > And this varies >>>>> > > > throughout the day. I don't see any >>>>> > particular >>>>> > > pattern as to >>>>> > > > which hosts show up either, it seems >>>>> > random. >>>>> > > > >>>>> > > > Any idea where to start looking? >>>>> > > > >>>>> > > > My production log shows this: >>>>> > > > >>>>> > > > Started POST >>>>> > "/reports/create?format=yml" for >>>>> > > 127.0.0.1 at >>>>> > > > 2013-08-16 11:54:41 +0200 >>>>> > > > Processing by >>>>> ReportsController#create >>>>> > as YML >>>>> > > > Parameters: >>>>> {"report"=>"[FILTERED]"} >>>>> > > > processing report for >>>>> host3.mydomain.com >>>>> > >>>>> > > >>>>> > >>>>> > > > Rendered text template (0.0ms) >>>>> > > > Completed 200 OK in 38ms (Views: >>>>> 0.9ms | >>>>> > > ActiveRecord: 8.3ms) >>>>> > > > >>>>> > > > But still there is not fact from the >>>>> > node in >>>>> > > the log statement... >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > Cheers, >>>>> > > > Anders, >>>>> > > > >>>>> > > > -- >>>>> > > > You received this message because you >>>>> are >>>>> > > subscribed to the Google >>>>> > > > Groups "Foreman users" group. >>>>> > > > To unsubscribe from this group and stop >>>>> > receiving >>>>> > > emails from it, >>>>> > > > send an email to >>>>> > foreman-user...@googlegroups.**com >>>>> > > > >>>>> > <mailto:foreman-users%**2Bunsubscribe@googlegroups.com>. >>>>> > > > To post to this group, send email to >>>>> > > forema...@googlegroups.com >>>>> > > > . >>>>> > > > Visit this group at >>>>> > > http://groups.google.com/**group/foreman-users >>>>> > <http://groups.google.com/**group/foreman-users> >>>>> >>>>> > > >>>> group/foreman-users >>>>> > <http://groups.google.com/**group/foreman-users>>. >>>>> >>>>> > > > For more options, visit >>>>> > > https://groups.google.com/**groups/opt_out >>>>> > <https://groups.google.com/**groups/opt_out> >>>>> >>>>> > > <https://groups.google.com/**groups/opt_out >>>>> > <https://groups.google.com/**groups/opt_out>>. >>>>> >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > -- >>>>> > > > Romeo >>>>> > > > >>>>> > > > -- >>>>> > > > You received this message because you are >>>>> >

Hi Anders, no I haven't found a work-around. I'm thinking of switching to
using puppetdb for our facts backend since we use the facts for different
external systems that require the facts to be consistently set and
available. I unfortunately don't have the time to wrap my head around how
foreman is written to track this down and offer up a patch, though I would
really like to. Hopefully this can be tracked down though.

··· On Mon, Sep 9, 2013 at 1:16 AM, Anders Sveen wrote:

Gah, this is really getting to me.

Weird: query for “name = memorysize” always returns 8 nodes. “name =
memoryfree” still returns varying number of nodes.

Romeo: Do you see the same behaviour? Did you find a solution on your end?

Anders,

On Wednesday, 4 September 2013 10:46:51 UTC+2, Anders Sveen wrote:

Is there a place I can add some print statements, logging or debug to
analyze this further? Maybe where the inserts are made to check if it
completes correctly?

Cheers,
Anders

On Tuesday, 27 August 2013 18:22:33 UTC+2, Romeo Theriault wrote:

This is the exact same issue I’m seeing, with postgres though, and
different facts.

On Tue, Aug 27, 2013 at 4:26 AM, Anders Sveen anders....@gmail.comwrote:

Finally got round to querying the database. Seems like entries are
missing in the DB too, so storing them seems to be the issue:

select fact_values.* from fact_values inner join fact_names on
fact_names.id=fact_values.**fact_name_id where fact_names.name
=‘memoryfree’;
16612|33.11 GB|17|8|2013-08-27 11:29:43.404051|2013-08-27
11:29:43.404051
16690|29.20 GB|17|7|2013-08-27 13:05:25.528203|2013-08-27
13:05:25.528203

Anders,

On Friday, 23 August 2013 13:59:01 UTC+2, Anders Sveen wrote:

Is there a query I can try to issue to check facts directly in the db?
Maybe dump everything for a certain host to a file and grep?

Anders,

On Friday, 23 August 2013 13:47:59 UTC+2, Dominic Cleal wrote:

Could it be a search issue? If you search for facts for one of the
hosts that has its memoryfree missing, is memoryfree listed there?

I know Romeo said that his facts aren’t present in the database, but
it’s possible we’re looking at two separate issues.

I’m also more concerned about your database because it’s SQLite
rather
than PostgreSQL, but that’s a vague assertion to make.


Dominic Cleal
Red Hat Engineering

On 23/08/13 12:27, Anders Sveen wrote:

Doh! Right in front of me, thanks. :slight_smile:

Alright, added issue with some additional details
here: http://projects.theforeman.org/issues/2967http://projects.theforeman.org/issues/2967

The strange thing is that name = puppetversion always seem to
return all
values, while name = memoryfree does not…

Anders,

On Friday, 23 August 2013 12:41:32 UTC+2, Dominic Cleal wrote:

If you've registered and logged in, there should be a "New

issue"

tab at
the top which points to:

http://projects.theforeman.**org**/projects/foreman/issues/**

new http://projects.theforeman.org/projects/foreman/issues/new

<http://projects.theforeman.**or**g/projects/foreman/issues/**

new http://projects.theforeman.org/projects/foreman/issues/new>

Sorry, I don't know what's causing this at the moment.  My only
suspicion is the proximity of timestamps in your log file, one

of the

truncated requests was at the same time as another.  Which

database are

you using?

--
Dominic Cleal
Red Hat Engineering

On 23/08/13 11:31, Anders Sveen wrote:
> So I decided to register an issue... I really can't figure

out the

issue
> tracker at http://projects.theforeman.**org**

/projects/foreman/issueshttp://projects.theforeman.org/projects/foreman/issues

<http://projects.theforeman.**or**g/projects/foreman/issues<http://projects.theforeman.org/projects/foreman/issues>>

. Is

> it not possible?
>
>
> Anders,
>
> On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen >>>>>> wrote:
>
>     When querying for "name = puppetversion" I get 8 hits.

Memoryfree

>     still show too few...
>
>
>     Anders,
>
>     On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders Sveen >>>>>> wrote:
>
>         I can confirm that I see the same logs too with 200

OK. Still

>         varying number of facts though.
>
>
>         A,
>
>         On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo >>>>>> >     Theriault wrote:
>
>             Hi Dominic, thanks for responding.
>
>             On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal >>>>>> >     >             <dcl...@redhat.com> wrote:
>
>                 Can you (& Anders) look for "POST
/fact_values/create"
>                 log entries in
>                 /var/log/foreman/production.**lo**g?
>
>
>             There are lots of these in my logs:
>
>             $ grep -A 5 fact_values/create production.log
>
>              Started POST "/fact_values/create?format=**ym**l"

for

>             131.171.224.199 at 2013-08-20 15:38:51 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

7.7ms)
>
>             --
>             Started POST "/fact_values/create?format=**ym**l"

for

>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 50ms (Views: 0.4ms |

ActiveRecord:

8.5ms)
>
>             --
>             Started POST "/fact_values/create?format=**ym**l"

for

>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>
>
>             Started GET "/node/dbm99.pvt.hawaii.edu?**fo**

rmat=yml http://dbm99.pvt.hawaii.edu?format=yml

<*MailScanner has detected a possible fraud attempt from

dbm99.pvt.hawaii.edu” claiming to be* http://dbm99.pvt.hawaii.edu?**
fo**rmat=yml http://dbm99.pvt.hawaii.edu?format=yml>

>             <*MailScanner has detected a possible fraud

attempt from “dbm99.pvt.hawaii.edu” claiming to be*http://dbm99.pvt.hawaii.edu?
format=yml http://dbm99.pvt.hawaii.edu?format=yml

<*MailScanner has detected a possible fraud attempt from

dbm99.pvt.hawaii.edu” claiming to be* http://dbm99.pvt.hawaii.edu?**
fo**rmat=yml http://dbm99.pvt.hawaii.edu?format=yml>>" for

>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             --
>             Started POST "/fact_values/create?format=**ym**l"

for

>             131.171.224.199 at 2013-08-20 15:38:56 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

8.1ms)
>
>
>             I do notice that occasionally there are some log

entries

>             that don't have the:
>
>              Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

8.1ms)
>
>             lines. See the third entry above. I'm not sure if

this is

>             related or not.
>
>             Is there anything else I can provide? or do to
troubleshoot?
>
>             One thing I'll note is that one of the facts we

see

>             consistently being dropped is a custom fact we

created

that
>             facter/puppet pushes up. I'm not sure this is at

all

>             relevant or not since I can see the fact in the

server

side
>             yaml file so I know it's there.
>
>             Thanks,
>             Romeo
>
>
>
>
>                 The one posted at the start of the thread was

a

report
>                 coming in rather
>                 than facts.
>
>                 It might be that some of these are failing

(part way

>                 through?), so if
>                 you can check as many instances of this as

you

can, it
>                 might shed some
>                 light on it.
>
>                 --
>                 Dominic Cleal
>                 Red Hat Engineering
>
>                 On 20/08/13 01:19, Romeo Theriault wrote:
>                 > I've recently just upgraded to foreman 1.2

and am

>                 seeing the same issue
>                 > and came searching the list to see if

anyone

else has
>                 seen it. I'm
>                 > seeings hosts (in foreman) randomly

dropping

(i.e. not
>                 having) facts.
>                 > I've verified that the hosts yaml file on

the

puppet
>                 master does have
>                 > all the facts so it appears to be an issue

in the

>                 passing of the facts
>                 > to foreman with the node.rb script or

something

in the
>                 foreman itself
>                 > that is not properly writing the facts to

the

DB. I've
>                 verified that my
>                 > DB (postgres 8.4.13) does NOT have the

dropped

facts,
>                 so it's not a
>                 > problem with displaying them.
>                 >
>                 > I tried turning the foremans log level up

to

debug but
>                 am not finding
>                 > anything useful.
>                 >
>                 > Anyone have any ideas why facts are getting
dropped?
>                 or how we can
>                 > troubleshoot this?
>                 >
>                 > This is a big issue for us.
>                 >
>                 > Any help/pointers is much appreciated.
>                 >
>                 > Romeo
>                 >
>                 >
>                 > On Sun, Aug 18, 2013 at 10:12 PM, Anders >>>>>> Sveen >>>>>> >     >                 <anders....@gmail.com >>>>>> >     >                 > <mailto:anders....@gmail.com>> wrote:
>                 >
>                 >     No one? This is actually happening

across two

>                 different instances of
>                 >     Foreman in two different zones. Of

course

they are
>                 configured in the
>                 >     same way, so it might be something with

the

way I
>                 have done it. From
>                 >     what I can gather the ENC script is

posting the

>                 facts. The facts
>                 >     also exist in the yaml I think the

script is

>                 posting. Could it be a
>                 >     timing issue? Are the facts written as

the

client
>                 is trying to post
>                 >     them?
>                 >
>                 >
>                 >
>                 >     Thanks,
>                 >     Anders,
>                 >
>                 >
>                 >     On Friday, 16 August 2013 14:23:21 >>>>>> UTC+2, >>>>>> >     Anders >>>>>> >     >                 Sveen wrote:
>                 >
>                 >         Hi,
>                 >
>                 >         I have set up Foreman and

foreman-proxy

on the
>                 same host as my
>                 >         Puppetmaster. I have integrated

reports,

set
>                 up ENC and enabled
>                 >         facts. Reports are fed back and

facts are

>                 coming in.
>                 >
>                 >         My 8 hosts show up in the list and
everything
>                 looks good, but
>                 >         when i go to facts to query I

rarely get

the
>                 full 8 results. I
>                 >         do a query for "name = memoryfree"

. Some

>                 times I get 8, but
>                 >         most of the time I only get 4,5 or

6

results.
>                 And this varies
>                 >         throughout the day. I don't see any
particular
>                 pattern as to
>                 >         which hosts show up either, it

seems

random.
>                 >
>                 >         Any idea where to start looking?
>                 >
>                 >         My production log shows this:
>                 >
>                 >         Started POST
"/reports/create?format=yml" for
>                 127.0.0.1 at
>                 >         2013-08-16 11:54:41 +0200
>                 >         Processing by

ReportsController#create

as YML
>                 >           Parameters:

{“report”=>"[FILTERED]"}

>                 >         processing report for

host3.mydomain.com

<http://host3.mydomain.com>
>                 <http://host3.mydomain.com>
<http://host3.mydomain.com>
>                 >           Rendered text template (0.0ms)
>                 >         Completed 200 OK in 38ms (Views:

0.9ms |

>                 ActiveRecord: 8.3ms)
>                 >
>                 >         But still there is not fact from

the

node in
>                 the log statement...
>

there is a semi patch that 'might' work, could you try to apply
https://github.com/theforeman/foreman/pull/646 ?

thanks!
Ohad

··· On Mon, Sep 9, 2013 at 10:47 PM, Romeo Theriault wrote:

Hi Anders, no I haven’t found a work-around. I’m thinking of switching to
using puppetdb for our facts backend since we use the facts for different
external systems that require the facts to be consistently set and
available. I unfortunately don’t have the time to wrap my head around how
foreman is written to track this down and offer up a patch, though I would
really like to. Hopefully this can be tracked down though.

On Mon, Sep 9, 2013 at 1:16 AM, Anders Sveen anders.r.sveen@gmail.comwrote:

Gah, this is really getting to me.

Weird: query for “name = memorysize” always returns 8 nodes. “name =
memoryfree” still returns varying number of nodes.

Romeo: Do you see the same behaviour? Did you find a solution on your end?

Anders,

On Wednesday, 4 September 2013 10:46:51 UTC+2, Anders Sveen wrote:

Is there a place I can add some print statements, logging or debug to
analyze this further? Maybe where the inserts are made to check if it
completes correctly?

Cheers,
Anders

On Tuesday, 27 August 2013 18:22:33 UTC+2, Romeo Theriault wrote:

This is the exact same issue I’m seeing, with postgres though, and
different facts.

On Tue, Aug 27, 2013 at 4:26 AM, Anders Sveen anders....@gmail.comwrote:

Finally got round to querying the database. Seems like entries are
missing in the DB too, so storing them seems to be the issue:

select fact_values.* from fact_values inner join fact_names on
fact_names.id=fact_values.**fact_name_id where fact_names.name
=‘memoryfree’;
16612|33.11 GB|17|8|2013-08-27 11:29:43.404051|2013-08-27
11:29:43.404051
16690|29.20 GB|17|7|2013-08-27 13:05:25.528203|2013-08-27
13:05:25.528203

Anders,

On Friday, 23 August 2013 13:59:01 UTC+2, Anders Sveen wrote:

Is there a query I can try to issue to check facts directly in the
db? Maybe dump everything for a certain host to a file and grep?

Anders,

On Friday, 23 August 2013 13:47:59 UTC+2, Dominic Cleal wrote:

Could it be a search issue? If you search for facts for one of the
hosts that has its memoryfree missing, is memoryfree listed there?

I know Romeo said that his facts aren’t present in the database, but
it’s possible we’re looking at two separate issues.

I’m also more concerned about your database because it’s SQLite
rather
than PostgreSQL, but that’s a vague assertion to make.


Dominic Cleal
Red Hat Engineering

On 23/08/13 12:27, Anders Sveen wrote:

Doh! Right in front of me, thanks. :slight_smile:

Alright, added issue with some additional details
here: http://projects.theforeman.org/issues/2967http://projects.theforeman.org/issues/2967

The strange thing is that name = puppetversion always seem to
return all
values, while name = memoryfree does not…

Anders,

On Friday, 23 August 2013 12:41:32 UTC+2, Dominic Cleal wrote:

If you've registered and logged in, there should be a "New

issue"

tab at
the top which points to:

http://projects.theforeman.**org**/projects/foreman/issues/**

new http://projects.theforeman.org/projects/foreman/issues/new

<http://projects.theforeman.**or**g/projects/foreman/issues/**

new http://projects.theforeman.org/projects/foreman/issues/new>

Sorry, I don't know what's causing this at the moment.  My

only

suspicion is the proximity of timestamps in your log file, one

of the

truncated requests was at the same time as another.  Which

database are

you using?

--
Dominic Cleal
Red Hat Engineering

On 23/08/13 11:31, Anders Sveen wrote:
> So I decided to register an issue... I really can't figure

out the

issue
> tracker at http://projects.theforeman.**org**

/projects/foreman/issueshttp://projects.theforeman.org/projects/foreman/issues

<http://projects.theforeman.**or**g/projects/foreman/issues<http://projects.theforeman.org/projects/foreman/issues>>

. Is

> it not possible?
>
>
> Anders,
>
> On Wednesday, 21 August 2013 14:26:50 UTC+2, Anders Sveen >>>>>>> wrote:
>
>     When querying for "name = puppetversion" I get 8 hits.

Memoryfree

>     still show too few...
>
>
>     Anders,
>
>     On Wednesday, 21 August 2013 14:01:37 UTC+2, Anders >>>>>>> Sveen wrote:
>
>         I can confirm that I see the same logs too with 200

OK. Still

>         varying number of facts though.
>
>
>         A,
>
>         On Wednesday, 21 August 2013 03:48:18 UTC+2, Romeo >>>>>>> >     Theriault wrote:
>
>             Hi Dominic, thanks for responding.
>
>             On Mon, Aug 19, 2013 at 9:09 PM, Dominic Cleal >>>>>>> >     >             <dcl...@redhat.com> wrote:
>
>                 Can you (& Anders) look for "POST
/fact_values/create"
>                 log entries in
>                 /var/log/foreman/production.**lo**g?
>
>
>             There are lots of these in my logs:
>
>             $ grep -A 5 fact_values/create production.log
>
>              Started POST "/fact_values/create?format=**ym**l"

for

>             131.171.224.199 at 2013-08-20 15:38:51 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

7.7ms)
>
>             --
>             Started POST "/fact_values/create?format=**ym**l"

for

>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 50ms (Views: 0.4ms |

ActiveRecord:

8.5ms)
>
>             --
>             Started POST "/fact_values/create?format=**ym**l"

for

>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>
>
>             Started GET "/node/dbm99.pvt.hawaii.edu?**fo**

rmat=yml http://dbm99.pvt.hawaii.edu?format=yml

<*MailScanner has detected a possible fraud attempt from

dbm99.pvt.hawaii.edu” claiming to be* http://dbm99.pvt.hawaii.edu?*
*fo**rmat=yml http://dbm99.pvt.hawaii.edu?format=yml>

>             <*MailScanner has detected a possible fraud

attempt from “dbm99.pvt.hawaii.edu” claiming to be*http://dbm99.pvt.hawaii.edu?
format=yml http://dbm99.pvt.hawaii.edu?format=yml

<*MailScanner has detected a possible fraud attempt from

dbm99.pvt.hawaii.edu” claiming to be* http://dbm99.pvt.hawaii.edu?*
*fo**rmat=yml http://dbm99.pvt.hawaii.edu?format=yml>>" for

>             131.171.224.199 at 2013-08-20 15:38:54 -1000
>             --
>             Started POST "/fact_values/create?format=**ym**l"

for

>             131.171.224.199 at 2013-08-20 15:38:56 -1000
>             Processing by FactValuesController#create as YML
>               Parameters: {"facts"=>"[FILTERED]"}
>               Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

8.1ms)
>
>
>             I do notice that occasionally there are some log

entries

>             that don't have the:
>
>              Rendered text template (0.0ms)
>             Completed 200 OK in 41ms (Views: 0.4ms |

ActiveRecord:

8.1ms)
>
>             lines. See the third entry above. I'm not sure

if this is

>             related or not.
>
>             Is there anything else I can provide? or do to
troubleshoot?
>
>             One thing I'll note is that one of the facts we

see

>             consistently being dropped is a custom fact we

created

that
>             facter/puppet pushes up. I'm not sure this is at

all

>             relevant or not since I can see the fact in the

server

side
>             yaml file so I know it's there.
>
>             Thanks,
>             Romeo
>
>
>
>
>                 The one posted at the start of the thread

was a

report
>                 coming in rather
>                 than facts.
>
>                 It might be that some of these are failing

(part way

>                 through?), so if
>                 you can check as many instances of this as

you

can, it
>                 might shed some
>                 light on it.
>
>                 --
>                 Dominic Cleal
>                 Red Hat Engineering
>
>                 On 20/08/13 01:19, Romeo Theriault wrote:
>                 > I've recently just upgraded to foreman 1.2

and am

>                 seeing the same issue
>                 > and came searching the list to see if

anyone

else has
>                 seen it. I'm
>                 > seeings hosts (in foreman) randomly

dropping

(i.e. not
>                 having) facts.
>                 > I've verified that the hosts yaml file on

the

puppet
>                 master does have
>                 > all the facts so it appears to be an issue

in the

>                 passing of the facts
>                 > to foreman with the node.rb script or

something

in the
>                 foreman itself
>                 > that is not properly writing the facts to

the

DB. I've
>                 verified that my
>                 > DB (postgres 8.4.13) does NOT have the

dropped

facts,
>                 so it's not a
>                 > problem with displaying them.
>                 >
>                 > I tried turning the foremans log level up

to

debug but
>                 am not finding
>                 > anything useful.
>                 >
>                 > Anyone have any ideas why facts are

getting

dropped?
>                 or how we can
>                 > troubleshoot this?
>                 >
>                 > This is a big issue for us.
>                 >
>                 > Any help/pointers is much appreciated.
>                 >
>                 > Romeo
>                 >
>                 >
>                 > On Sun, Aug 18, 2013 at 10:12 PM, Anders >>>>>>> Sveen >>>>>>> >     >                 <anders....@gmail.com >>>>>>> >     >                 > <mailto:anders....@gmail.com>> wrote:
>                 >
>                 >     No one? This is actually happening

across two

>                 different instances of
>                 >     Foreman in two different zones. Of

course

they are
>                 configured in the
>                 >     same way, so it might be something

with the

way I
>                 have done it. From
>                 >     what I can gather the ENC script is

posting the

>                 facts. The facts
>                 >     also exist in the yaml I think the

script is

>                 posting. Could it be a
>                 >     timing issue? Are the facts written as

the

client
>                 is trying to post
>                 >     them?
>                 >
>                 >
>                 >
>                 >     Thanks,
>                 >     Anders,
>                 >
>                 >
>                 >     On Friday, 16 August 2013 14:23:21 >>>>>>> UTC+2, >>>>>>> >     Anders >>>>>>> >     >                 Sveen wrote:
>                 >
>                 >         Hi,
>                 >
>                 >         I have set up Foreman and

foreman-proxy

on the
>                 same host as my
>                 >         Puppetmaster. I have integrated

reports,

set
>                 up ENC and enabled
>                 >         facts. Reports are fed back and

facts are

>                 coming in.
>                 >
>                 >         My 8 hosts show up in the list and
everything
>                 looks good, but
>                 >         when i go to facts to query I

rarely get

the
>                 full 8 results. I
>                 >         do a query for "name = memoryfree"

. Some

>                 times I get 8, but
>                 >         most of the time I only get 4,5 or

6

results.
>                 And this varies
>                 >         throughout the day. I don't see

any

particular
>                 pattern as to
>                 >         which hosts show up either