Foreman Instance has lost all data

Hi All,

Over the weekend, my Foreman instance basically lost all data: hosts, lifecycle environment, libraries, repos, etc…

I’m not sure where to begin troubleshooting such an issue? I noticed the ‘Foreman’ service was dead, so I attempted to restart it and it came up but there’s still not data. I noticed on a server reboot my Foreman service also no longer runs (Even though we’re enabled in systemctl). Any ideas where to begin to look or what may have happened? This server was untouched.

Thanks.

EDIT: after a few minutes of waiting after restarting foreman service, the information is back. Any ideas where to look and what logs to look into to troubleshoot what caused this to break and why foreman wont start on boot anymore? Thanks.

that is odd, any chance you had another foreman process started with wrong configuration data (such as db server?) alternatively, any chance you were in a org / location context so you didn’t see your hosts?

Are you able to view any data using hammer? (e.g. hammer host list )

So are you saying data is back?

In regard to finding out what brought httpd service down (assuming you are using our standard configuration via Passenger), this is linux admin 101: start with system journal/logs and find the httpd service restarts. By default systemd attemts to bring the service up few times before it gives up. Or something killed the process (OOM)? The truth is in the logs there.