Massive externalNode performance degradation after upgrade to 1.16

Support
21

I’m trying to find an anonymiser for the database - can you suggest one that can be used?
I’ve tried https://github.com/DivanteLtd/anonymizer - didn’t work.

I’m currently trying https://github.com/realrolfje/anonimatron

22

if @bryder can find an anonymizer that works - i can also take a stab. My database is many many GB though - so i wouldn’t even know where to put it should I be able to anonymize it

23

I spent a lot of time trying to make anonimatron work which in the end required me to run it in a debugger to find out what was wrong with my configuration.

Once I got past that I discovered that it only supports a tiny subset of the postgres types and sadly I don’t have the time to implement the extra types we need. It doesn’t help that I’m not the fastest java programmer!

24

Unfortunately we can not share our db, but you can easily reproduce this behavior:

  • Install a foreman (reproducible with Foreman 1.16 and 1.17)
  • Be sure the setting “Enable smart variables in ENC” is true
  • Add some more puppet classes with the possibility to set smart class parameters
  • Add these classes to one host and overwrite the puppet class parameters.

-> The more parameters are overwritten, the higher the response time will be.

25

So, perhaps we have the solution:
https://github.com/theforeman/foreman/pull/5583

The “new” Code fixes the time problem on our testsystems, but still needs to be tested and merged. It also works in foreman 1.16 :slight_smile:

2 Likes
26

Good to hear there’s probably a fix for the problem now!

I figured out how to use anonimatron in the end but there’s still a lot of work to configure it properly for the foreman database (working through which columns should and should not be anonymised basically).

I’ll stop doing that now since it looks like it’s not needed.

27

I finished off what I can of this without looking into anonimatron bugs.

The final status is:

It can not current anonimise settings, subnets, and notification_blueprints due to bugs in the engine. nics and operating_systems have field sizes too small for the default hash, and notification_blueprints has no primary key so anonimatron will not try to anonymise it.

I put what I have done up here https://github.com/bryder/anonimatron_anonymizers in case I ever need to come back to it or someone wants the unfinished work.

1 Like
28

The PR has been merged and we are working on backporting. Nice catch!

1 Like