Do on-demand download policies work with content-views? I added a product and a repo to Foreman using “on-demand” policy, added it to my content-view, removed CV filter from that repo and published new version of the CV. The client attached to the product. The client can see the repo but could not install or get info on any package from that repo. After 20 mins, I finally forced a sync on the repo, published new version and it worked. Is this working as expected?
On-demand means it syncs the metadata during sync and the packages only when requested by a client.
So when I correctly understood you, yes it should work like described and you were misunderstanding the intended behaviour.
I’d have similar question to “background” policy. When it actually syncs the data and what happens if client asks for content, which is not synced yet? Does it behave like on demand? I didn’t find this info in katello manual nor form inline help.
Background works similar by downloading metadata and then enabling you to work with the data while doing the download of the packages in the background. I am not sure what happens when a client requests a package while not being downloaded, perhaps it gets prioritized but I expect the client has to wait.
Thanks for the information. I have one more question on how these modes (on-demand, background) interact with Content Views, as we use them extensively. I read the blurb at http://www.outsidaz.org/2017/02/12/adventures-in-katello-part-1/ on download policies but they don’t seem to work quite as “seamlessly” as documented with CVs.
Adding a repo that has not been sync’ed yet to a CV and publishing a new version results in the client not seeing any packages, which makes sense if you think of it as a point-in-time snapshot of the repo. If an on-demand repo is sync’ed, which just provides the metadata, and is then added to a CV and a new version published, will the clients still have access to the packages even though they weren’t present on the Foreman/Katello server when the CV was published?
Just trying to understand the situations for when a CV publish is required in relation to on-demand repos.
Be careful with on demand and repos that change quickly, like EPEL. You can get into a situation where you have the metadata for a package synced and published but the actual package has never been pulled down. A host may request the package some time later and find it is no longer available upstream so it can’t be downloaded. This can lead to some fun dependency issues.
In answer to your question, I think the behaviour you are describing is what is intended. You set a repo to on-demand, add it to a CV and publish it. If a host requests a package from the CV that has never been pulled down then Katello has a mechanism to retrieve it from upstream, if it still exists.
I can’t say for certain if that’s how it actually works as we stopped using on-demand pretty early on due to the issues I described.
1 Like
Yes, this would be exactly what I would have answered, because this is a possible problem some customers already have run into. But I would say in most environments with normal update intervals it is not so problematic, but if you have higher intervals keep this in mind.
We have just recently run into this issue with the on-demand download policy using Katello 3.9.
We added a whole new OS Release of SLES using the on-demand policy and found katello just not downloading some of the required packages.
After fiddling around a lot with this (using on demand was an accident we did not realize untile quite some time later), we found that the on-demand policy was what caused all the inconsistent repos in the content fews. Due to that, we dicided not use that feature anymore. I personally tend to advise against using it at all;aybe there have been improvements since then or the SLES repos were just to rapidly changing, I don’t know.
Thanks for doing the digging! Using ‘on-demand’ with fast changing repos does seem like a valid concern. If you have the space, I could see it being better to use the immediate policy for those repos.
We can think about a better way to handle this in Katello or at least make this point more clear. @Justin_Sherrill do you have any thoughts or pointers around using on-demand with fast-changing repos?
Yes, i would not recommend on_demand for repos who remove rpms from their upstream sources. In fact I think we should change the default behavior to ‘immediate’, i’ve opened an issue here:
I’ve also opened an upstream PR to change the default value to prevent new users from hitting this.
An inline help for this field explaining all modes would be great.
2 Likes
Thanks for all the input here. I will probably switch to “immediate” to be safe, as capacity is not an issue for us. Cheers.
So do you also need to disable the “Mirror on sync” setting on these repo’s? Otherwise they still get removed I think.
I can not confirm this 100%, but mirror on sync should not remove packages that are already in CVs. They should only be dropped out in future versions.
Correct, mirror on sync just removes them from the Library repo, but if
you are using immediate, they’ve already been downloaded and stored on
disk. If they exist in a content view they will not ever be deleted and
you shouldn’t ever get a 404.
Justin
Ok thanks for clarifying!