Smart proxy log rotate not working

Support
1

Problem:

Smart proxy log file is not being rotated correctly.

Example:

utils.rb: 40936 40986 foreman-proxy 9w REG 252,0 290101 1324582 /var/log/foreman-proxy/proxy.log.1 (deleted)

Expected outcome:

Smart proxy log file is rotated correctly.

Foreman and Proxy versions:

1.16.1

Foreman and Proxy plugin versions:

1.16.1

Other relevant data:

Ubuntu. May not be the case for other OSes.

In Feature #19480: Support systemd's notify service type, call sd_notify on startup - Smart Proxy - Foreman and the resulting https://github.com/theforeman/smart-proxy/pull/525 the smart proxy was switched to type=notify and daemonization was disabled. In this configuration no pid file is written.

/etc/logrotate.d/foreman-proxy still calls the sysv init script:

[ -e /etc/init.d/foreman-proxy ] && /etc/init.d/foreman-proxy logrotate >/dev/null 2>&1 || true

which relies on the pid file:

start-stop-daemon --stop --signal USR1 --quiet --pidfile $PIDFILE

2

Thanks for report, please go ahead and fix this on github:

It seems to be fixed for red hats:

  /var/log/foreman-proxy/migrate_settings.log
  /var/log/foreman-proxy/proxy.log
  {
    missingok
    notifempty
    create 0644 foreman-proxy foreman-proxy
    sharedscripts
    rotate 5
    compress
          daily
    postrotate
      /bin/systemctl kill --signal=SIGUSR1 foreman-proxy >/dev/null 2>&1 || true
    endscript
  }
4

I’ve created pull request #2622 in theforeman/foreman-packaging on github. All my comments have been marked as spam. Hopefully you will see this.

2 Likes
5

Sorry about that, looks like discourse thought that the multiple links to github from a new user is spam. I changed the setting a bit so hopefully now it will work.

6

Thank you, I guess @mmoll will do the review. Red Hats look good.

7

Log rotate doesn’t seem to be working on EL7 for versions;
foreman-proxy-1.16.1-1.el7.noarch
foreman-proxy-1.16.2-1.el7.noarch

I tried running /bin/systemctl kill --signal=SIGUSR1 foreman-proxy >/dev/null 2>&1 manually first while the service was running and log size was zero but the log file was not written to.

If I systemctl restart foreman-proxy it will start logging out to /var/log/foreman-proxy/proxy.log. Once log rotate occurs the log remains empty.

As you can see below I have logging set to debug.

settings.yaml contains:

:log_file: /var/log/foreman-proxy/proxy.log
:log_level: DEBUG
:log_buffer: 2000
:log_buffer_errors: 1000
8

When proxy receives this signal, it does not rotate immediately but it marks a flag and it closes the logging file once first log message arrives. Can you verify this behavior?

9

Yes that is the behaviour, after a puppet cert is set for autosign the log is populated on disk.

At a loss then why the log size is 0 every time I go to check debug logs.

10

To me this is the correct behavior. Ideally it would write some extra info message after logs are rotated something like “logs were rotated” but due to synchronization that would be challenging.

I am not sure what you mean with the zero-sized logs. When you see this? Is this some bug in the debug script? Can you compare files from the tarball and files on disk?