Squid/pulp migration

Support
1

Hello,

First let me thank you for the great support I received in in this forum until now.
I read the documentation here Foreman :: Plugin Manuals and it says:

The path /var/spool/squid/ is used as a temporary location for some types of repository syncs and may grow to consume 10s of GB of space before the files are migrated to /var/lib/pulp.

I’m wondering how/when this “migration” from squid to pulp happens. Is there something that can/should be tuned for performance?

2

Hello, what migration are you talking about?

Katello uses both Pulp and Squid combined to deliver both ondemand and immediate sync workflows. When you use ondemand, Pulp redirects requests through Squid which caches the RPM files and other artifacts. This works all transparently, you don’t need to care.

Amount of space is defined in squid configuration (therefore katello-installer) and I think by default is 10 GB. You can increase this at will, it’s probably too low by default (fine for few OS installation versions I guess).

3

[msommer] msommer https://community.theforeman.org/u/msommer
July 10

Hello,

First let me thank you for the great support I received in in this
forum until now.
I read the documentation here Foreman :: Plugin Manuals
https://theforeman.org/plugins/katello/3.12/installation/index.html
and it says:

The path /var/spool/squid/ is used as a temporary location for
some types of repository syncs and may grow to consume 10s of GB
of space before the files are migrated to /var/lib/pulp.

I’m wondering how/when this “migration” from squid to pulp happens. Is
there something that can/should be tuned for performance?

A task gets run by pulp every 10 minutes which causes anything that
squid has downloaded to be fetched and placed in /var/lib/pulp/.

You can adjust this via the ‘download_interval: 10’ option in
/etc/pulp/server.conf, although i don’t think its exposed in the foreman
installer at the moment, as we’ve never really seen much use in
customizing it.

The 10s of gigs scenario is unlikely to happen very often, but is a
worse case scenario.

4

We should probably get Satellite docs fixed, I remember somebody from docs team asked me and I replied in a sense of 10 GiB is probably too low, let’s do like 100 GiB to be on a safe side.

5

I don’t know how it transfers files, but if it’s a move then ensuring that /var/spool/squid and /var/lib/pulp are on the same filesystem would make those operations very cheap. Can save a lot of IO.

6

It actually fetches the files through squid directly and writes them to
/var/lib/pulp, so sharing a file system may not be beneficial in this case.

Justin

7

This probably renders my issue invalid, can you confirm and close it? Feature #21594: Katello squid cache_dir defaults are too small - Installer - Foreman