Thanks.
In regard to counts, well unit tests work fine, if you will be able to identify what’s wrong I’d be very interested 
Have a nice rest of the weekend.
Wow, this is great! Thanks @lzap for jumping on this so quickly! I am looking forward to testing this patch out - but we are entering a CHG freeze on my end so im “stuck” until July-ish.
Wanted to add for reference the scripts we worked up with my DBA to run nightly in response to this on my end. These have been running without major issue via CRON with a batch size of 50,000 deleting “about” 3500 rows/second. This keeps puppet from backing up and gives enough delay that our queued up reports can process in the time between batches. As we generate at minimum 26,000 reports/hour (More with Ansible Integrations) a batch size of 200 or so would probably never even keep up with the new reports being entered, so having that tweakable is probably super important! We also use a 3 node MySQL galera cluster, so I’m only running these on 1 of 3 nodes, something the foreman rake task would “handle” natively by design.
The one other item worth noting these cleanup scripts do that the current rake task(s) do not is the “Optimize Table” query, which helps to recover the disk space. In our case due to the old rake task failing, our logs/sources/reports table(s) had grown to over 100GB all by themselves. By optimizing we were able to get our overall DB size back to ~50 GB
(Apologies as these are .sh files I cannot upload directly in discourse)
Messages: https://pastebin.com/LpuscawZ
Reports: https://pastebin.com/jxrPfJcc
Sources: https://pastebin.com/t27SH86S
Logs: https://pastebin.com/ad2SNsRX
We also whipped one up for audits as that table was growing “forever” and had no known cleanup method:
Audits: https://pastebin.com/ab1kd6y7
I definitely look forward to testing this out as soon as I am able!
1 Like
one quick note is that audits table will have a cleanup task in 1.18:
1 Like
Are we going to drop a cron job to call it tho? I think the consensus was not to do it. 
Thank you for your feedback, appreciated!
I just pushed update to my PR, now it’s configurable in the rake task and defaults to 1k records in a batch.
no, we decided not to automatically create a cron job since different users have different requirements regarding how long they need to keep audits.
I agree no “default” cron-job should be created. However - as each admin has the option to “create” one manually as needed…that works just fine!