Just now Jenkins had a full disk. Turns out there was a Katello PR test job build (number 9791) that had a 47GB log file. I’ve removed that log file which freed up more than enough disk space. After that I restarted the Jenkins process.
Given the excessive size of a single build (the
/var/lib/jenkins partition is 100G), I’m not sure we could (or should) have planned ahead for this. I’m not sure if Jenkins has some built in limit for this to keep the size small. For now I’d suggest to keep this in mind and continue. If it pops up more often, we can look into technical solutions.