[RFE] Option to disable caching of initrd & vmlinuz

benipeled · August 4, 2019, 9:34am

Situation:
The initrd and vmlinuz files from installation media are stored in Foreman as cache (or something like that) and used by the build process (provision).

Problem:
Foreman stores the initrd and vmlinuz based on the path specified in the installation media and doesn’t update them as long as the path points to the same location,

In case the initrd and vmlinuz files change in the repo specified in the installation media path, the files stored on Foreman will not be changed and the build will continue to use their old version.

Suggested solution:
Add option to mark installation media source with “Don’t sync/cache” flag (by checkbox) on the installation media level, which will cause initrd and vmlinuz to download in each build.

lzap · August 5, 2019, 12:16pm

Hello,

for the PXE process, this is not a cache. We need to expose those files via TFTP protocol in order to achieve PXE. Thus this will never be optional.

For bootdisk process, Foreman really downloads kernel/initramdisk into Rails cache. But there is already a setting to turn this off in Administer - Settings - Bootdisk.

ekohl · August 5, 2019, 12:29pm

My understanding is that they’re redownloaded if needed. Perhaps tmpwatch could be useful in your environment to clean old files?

It would also be interesting to know what you distro you use that actually changes files in place.

benipeled · August 5, 2019, 2:23pm

This is an option, but I want to avoid setting this option on all mediums,
It will be better If there is such an option per medium.

benipeled · August 5, 2019, 2:28pm

tmpwatch won’t help here because the files remain in the same location and with the same name and continue to be used, they are just a different version of the files.

benipeled · August 5, 2019, 2:29pm

It’s actually not about the distro (RHEL), it’s about the source - the source is dev-release of RHEL which is updated on every dev build.

ohadlevy · August 5, 2019, 2:45pm

in theory, its a different OS, can you configure it as a new os
/url/version ?

benipeled · August 5, 2019, 6:59pm

No, take a look (screenshot attached) at what we have today
I don’t want to update manually the Path/Medium every time the initrd/vmlinuz is updated in the source.

ohadlevy · August 6, 2019, 6:22am

I do wonder if the minor should be 7pre or something instead of just 7, this way you could have multiple versions at similar time (e.g. rc1, rc2 etc) and not to conflict with each other.
you would probably want to script it, so you dont create the os records by yourself all the time?

lzap · August 6, 2019, 8:39am

Beni, I still don’t understand if your workflow is PXE or Bootdisk. Let’s assume you meant “this is the option” which implies the latter.

Bootdisk caches the files under the key of the URL of the downloaded file, therefore if it changes this can cause problems. The download code is very simple and it does not respect ETAG or LAST MODIFIED HTTP headers (and yes this is a bug):

github.com

theforeman/foreman_bootdisk/blob/c50793ad79f2b3253934146eaa18291319f12a30/app/services/foreman_bootdisk/iso_generator.rb#L127-L164


def self.fetch(path, uri, limit = 10)
  dir = File.dirname(path)
  FileUtils.mkdir_p(dir) unless File.exist?(dir)


  use_cache = !!Setting[:bootdisk_cache_media]
  write_cache = false
  File.open(path, 'w') do |file|
    file.binmode


    if use_cache && !(contents = Rails.cache.fetch(uri, raw: true)).nil?
      ForemanBootdisk.logger.info("Retrieved #{uri} from local cache (use foreman-rake tmp:cache:clear to empty)")
      file.write(contents)
    else
      ForemanBootdisk.logger.info("Fetching #{uri}")
      write_cache = use_cache
      uri = URI(uri)
      Net::HTTP.start(uri.host, uri.port) do |http|
        request = Net::HTTP::Get.new(uri.request_uri, 'Accept-Encoding' => 'plain')


        http.request(request) do |response|

This file has been truncated. show original

If this is causing issues, just turn off the “bootdisk_cache_media” setting and you are good to go.

lzap · August 6, 2019, 9:01am

I wish we could use down library [1] recommended by @TimoGoebel but it only supports Ruby 2.2-2.4 and we are on newer. Would be also nice to use it on smart proxy but that’s Ruby 2.0.

[1] https://github.com/janko/down

Shimon_Shtein · August 11, 2019, 11:11am

Since the scenario is testing prerelease OS builds, I am not sure it won’t be too much trouble to create an OS per test scenario. Plus it could cause too much OS records.

@lzap, I think Benny is referring to PXE deployment, and he wants to disable the “do not download, if not changed” optimization that we have. It should be possible to add this switch as an option on InstallationMedium record.

@benipeled, for the record, this option can cause trouble, if multiple hosts are provisioned in close succession.

bhawksfan · August 12, 2019, 5:16am

I haven’t tried either of these, but they do seem technically possible:
If you can switch your boot loader to ipxe, it would pull the kernel/initrd directly over http.

Recent versions of syslinux/pxelinux also support http. This would probably require building from source, since redhat’s packages are old. You would need to modify the the kickstart default template to refer to the kernel/initrd via http url (@host.url_for_boot(:kernel)).

lzap · August 12, 2019, 6:40am

Yes, this will work.

benipeled · August 14, 2019, 5:07am

Yep, the iPXE sounds like a helpful solution,
Although the possibility of controlling it per medium in pxelinux can also be helpful, but I understand that there is no demand for such, so it will not happen soon.

Thank you all!