How to sync content from our main server to all other proxies?

Support
,
1

Problem: Cannot get content to sync from main foreman server to proxies

Expected outcome: All proxies should have same content and repos as main foreman server

Foreman and Proxy versions: 3.3.1/3.3.1 - Katello 4.5.1

Foreman and Proxy plugin versions:

Distribution and version:

Other relevant data:

Im not sure if we have something misconfigured or are just not understanding the process. We have main foreman/katello server that houses all our subscriptions/content/repos. We do not want server all over the world hitting this server for updates, so we spun up 6 more proxy servers throughout different regions. Now, when we attempt to go to those proxies in the main foreman server, and synchronize, we see no pulp content from the main server syncing into the proxies.
Are we doing wrong and just not understanding how it works?

Thanks for any help.

1 Like
2

Just a guess, but is it possible you have not assigned any lifecycle environments to your proxies?

Proxies do not just sync all content from the server, but rather specific lifecycle environments. To select what lifecycle environments to sync, navigate to your proxy and then edit it as follows:

Infrastructure > Smart Proxies > select a proxy from the list > Edit (button top right) > Lifecycle Environments (tab) > Move desired lifecycle environments to the right > Submit (button)

You may also want to take a moment to consider the available “Download Policies” on the “Smart Proxy” tab.

1 Like
3

Hey thanks. I have lifecycle environments set, and actually when I click on the content tab of a proxy, I see content. However, it just doesnt seem right. On the main foreman server, the repos are taking up almost 2TB. On the foreman proxy, like 9GB. It just doesnt seem to match up.

1 Like
4

What download policy did you choose on the smart proxy? Default download policy is “On Demand”, i.e. the proxy will only download and cache packages requested through the proxy. With on demand the proxy requires much less space then the main server…

1 Like
5

Sounds like download policy! I will just add a quick explanation of the three available policies:

  • Immediate: Sync everything in the selected lifecycle environments to the proxy immediately.
  • On Demand: Sync metadata immediately, but only sync the packages when they are first requested by a client, then keep those packages in case they are requested again.
  • Streamed: Like “On Demand”, but never keep the packages on the proxy. Always re-fetch them from the main server every time they are requested by a client.

What you get is a trade off between disk usage and network traffic (and by extension speed).

Note: That I am not sure what version of Katello “Streamed” was added. Maybe it has not even been released yet.

1 Like
6

Thank you both very much. I read docs and various posts for a good 4 hours and still didnt quite understand how it all worked. I’ll take this newly provided information youve given and see what things do over the next couple days. Ive got a test host Im going to attempt to update through one of the new proxies and see/watch how it pulls down content from the main server.
Appreciate the responses! I havent used Foreman/Katello in over 5 years, and its really nice to come back and see the same kind of community support!

1 Like
7

So I dont think its working. I pointed a test server at the new proxy, and its like nothing has synced at all:

[root@10-222-13-33 jsparrow]# yum update
Updating Subscription Management repositories.
8 Modular x86_64                                                                                                                                  337  B/s |  55  B     00:00
Errors during downloading metadata for repository 'redacted_epel-8_8_Modular_x86_64':
  - Status code: 500 for https://redacted/pulp/content/redacted/cloud-instances/centos-8-stream/custom/epel-8/8_Modular_x86_64/repodata/repomd.xml (IP: 10.234.88.146)
Error: Failed to download metadata for repo 'redacted_epel-8_8_Modular_x86_64': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried
1 Like
8

Some general things to check/try (mostly to gather more inofrmation): Which download policy is the proxy in question using? How exactly did you “point a test server” at the new proxy? Did the task synchronizing to the smart proxy report anything suspicious? Are the Pulp services on the proxy running? Are the Pulp services on the main Foreman instance running? Can you download the repomd.xml in question on the main Foreman instance? Can you download it from the main Foreman instance to the proxy?

Perhaps somebody else knows more targeted questions with regards to that error message…

1 Like
9

lol… One of my colleagues was rebooting the content server at the time. It appears to be working now. We had thought that we needed to sync all the content from the upstream server, down to the proxy. But now that I see this on demand option, it makes sense. This is really cool thanks.

1 Like