Slow downloads from deb.theforeman.org

Support
1

Problem: Foreman upgrades / installation on Debian take forever due to slow download speeds from deb.theforeman.org. This must have changed recently :thinking:

wget http://deb.theforeman.org/pool/buster/2.1/f/foreman/foreman_2.1.3-1_amd64.deb
--2020-10-01 10:52:59--  http://deb.theforeman.org/pool/buster/2.1/f/foreman/foreman_2.1.3-1_amd64.deb
Resolving deb.theforeman.org (deb.theforeman.org)... 140.211.167.4
Connecting to deb.theforeman.org (deb.theforeman.org)|140.211.167.4|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 73169228 (70M) [application/x-troff-man]
Saving to: ‘foreman_2.1.3-1_amd64.deb’

foreman_2.1.3-1_amd64.deb                   0%[                                                                                     ] 231.07K  15.6KB/s    eta 63m 0s

Expected outcome: Faster download speeds so that installation does not take 1 hour for a package of 60MB.

I’m downloading this from Germany, the server seems to be located in the US. Are there plans on providing Debian packages on aptly CDN as well?

Thanks in advance
Sebastian

2

I can confirm on my home connection I get similar speeds but from my server in an AMS DC I get:

2020-10-01 11:28:20 (6.69 MB/s) - ‘foreman_2.1.3-1_amd64.deb’ saved [73169228/73169228]

Traceroute for the home connection (trimmed for privacy)

 4  asd-tr0021-cr101-be150-10.core.as9143.net (213.51.158.22)  14.348 ms  18.582 ms  18.483 ms
 5  nl-ams04a-ri3-ae51-0.aorta.net (213.51.64.194)  17.153 ms  18.264 ms  18.147 ms
 6  nl-sar.nordu.net (109.105.98.186)  17.860 ms  12.556 ms  12.859 ms
 7  us-man.nordu.net (109.105.97.64)  99.474 ms  93.999 ms  97.607 ms
 8  99.83.66.169 (99.83.66.169)  102.496 ms  105.168 ms  106.619 ms
 9  ae-32.2603.rtsw2.ashb.net.internet2.edu (64.57.21.53)  239.457 ms  239.166 ms  238.913 ms
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  lo-0.8.rtsw.port.net.internet2.edu (64.57.20.224)  164.767 ms  166.418 ms  160.602 ms
17  * * *
18  ptck-p1-gw.nero.net (207.98.64.79)  193.764 ms  188.896 ms  175.763 ms
q19  207.98.64.45 (207.98.64.45)  175.354 ms  175.164 ms corv-p1-gw.nero.net (207.98.64.25)  175.565 ms
20  corv-car1-gw.nero.net (207.98.64.19)  176.011 ms  174.998 ms  174.818 ms
 3  213.51.193.197 (213.51.193.197)  11.934 ms  11.943 ms  12.074 ms
 4  asd-tr0021-cr101-be150-10.core.as9143.net (213.51.158.22)  14.348 ms  18.582 ms  18.483 ms
 5  nl-ams04a-ri3-ae51-0.aorta.net (213.51.64.194)  17.153 ms  18.264 ms  18.147 ms
 6  nl-sar.nordu.net (109.105.98.186)  17.860 ms  12.556 ms  12.859 ms
 7  us-man.nordu.net (109.105.97.64)  99.474 ms  93.999 ms  97.607 ms
 8  99.83.66.169 (99.83.66.169)  102.496 ms  105.168 ms  106.619 ms
 9  ae-32.2603.rtsw2.ashb.net.internet2.edu (64.57.21.53)  239.457 ms  239.166 ms  238.913 ms
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  lo-0.8.rtsw.port.net.internet2.edu (64.57.20.224)  164.767 ms  166.418 ms  160.602 ms
17  * * *
18  ptck-p1-gw.nero.net (207.98.64.79)  193.764 ms  188.896 ms  175.763 ms
19  207.98.64.45 (207.98.64.45)  175.354 ms  175.164 ms corv-p1-gw.nero.net (207.98.64.25)  175.565 ms
20  corv-car1-gw.nero.net (207.98.64.19)  176.011 ms  174.998 ms  174.818 ms

Traceroute from the DC:

 2  eth4-100.dcg-2.router.nl.coloclue.net (94.142.244.253)  0.718 ms  0.636 ms  0.608 ms
 3  a3837.coloclue.atom86.net (95.142.96.229)  0.967 ms  1.717 ms  1.279 ms
 4  available.atom86.net (95.142.96.6)  4.361 ms  4.290 ms  4.261 ms
 5  185.1.94.20 (185.1.94.20)  1.504 ms  1.614 ms  1.442 ms
 6  us-man.nordu.net (109.105.97.64)  87.040 ms  86.817 ms  86.763 ms
 7  99.83.66.169 (99.83.66.169)  92.911 ms  92.911 ms  92.847 ms
 8  ae-32.2603.rtsw2.ashb.net.internet2.edu (64.57.21.53)  153.410 ms  153.402 ms  153.321 ms
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  lo-0.8.rtsw.port.net.internet2.edu (64.57.20.224)  153.923 ms  153.931 ms  153.900 ms
16  198.71.47.137 (198.71.47.137)  170.031 ms  170.010 ms  170.117 ms
17  ptck-p1-gw.nero.net (207.98.64.79)  154.391 ms  154.355 ms  154.547 ms
18  207.98.64.45 (207.98.64.45)  160.710 ms  160.672 ms corv-p1-gw.nero.net (207.98.64.25)  161.614 ms
19  corv-car1-gw.nero.net (207.98.64.19)  157.407 ms  156.857 ms  156.838 ms
3

Hey,

thanks for the report!

We recently switched the location of the webserver (still in the US) and that might be due to that.

I personally can’t reproduce from Germany (getting 7-8MB/s, Unitymedia), but from US East Coast and @ekohl can from Europe.

I’ll see that we add the same CDN in front as we have for the RPM downloads.

Out of curiosity, what speed are you getting from http://stagingdeb.theforeman.org/pool/buster/theforeman-2.1/f/foreman/foreman_2.1.3-1_amd64.deb? (that one is on the CDN already)

4 
2020-10-01 11:36:36 (82.3 MB/s) - ‘foreman_2.1.3-1_amd64.deb.2’ saved [73169228/73169228]

Downloads from CDN are pretty fast. I am downloading from AS8560 (1&1) if this is important.

Thanks for confirming @ekohl, I will attach traceroutes as well.

 5  ae-9.bb-b.fr7.fra.de.oneandone.net (212.227.120.169)  3.078 ms  5.671 ms  3.093 ms
 6  ae-14-0.bb-a.fra3.fra.de.oneandone.net (212.227.120.148)  2.893 ms  2.933 ms  3.014 ms
 7  62.67.23.37 (62.67.23.37)  3.167 ms  3.405 ms  3.381 ms
 8  ae-1-3504.ear1.sanjose1.level3.net (4.69.214.146)  150.770 ms * *
 9  cenic.ear1.sanjose1.level3.net (4.15.122.46)  151.132 ms  151.104 ms  150.999 ms
10  dc-svl-agg8--svl-agg4-100ge-2.cenic.net (137.164.11.31)  151.421 ms
    svl-agg8--svl-agg4-100ge.cenic.net (137.164.11.65)  152.315 ms
    dc-svl-agg8--svl-agg4-100ge-2.cenic.net (137.164.11.31)  151.656 ms
11  dc-svl-agg10--svl-agg8-300g.cenic.net (137.164.11.80)  150.946 ms  151.470 ms  151.479 ms
12  lon--cenic-100ge.cenic.net (137.164.3.103)  162.317 ms  162.455 ms  162.692 ms
13  eugn-p1-gw.nero.net (207.98.64.196)  164.318 ms  164.535 ms  164.783 ms
14  corv-p2-gw.nero.net (207.98.64.13)  166.347 ms  166.417 ms
    207.98.64.47 (207.98.64.47)  165.870 ms
15  corv-car1-gw.nero.net (207.98.64.17)  163.836 ms  163.662 ms  163.623 ms

If you need more information just let me know.

Thanks
Sebastian

5

I’ve opened a PR to move deb.theforeman.org to the CDN

https://github.com/theforeman/foreman-infra/pull/1466

6

Merged and updated DNS. You should see improved speeds in ~10min (that’s the TTL on that host)

3 Likes
7

Awesome! Thanks for your quick help :slight_smile:

1 Like