[Katello] - Pulp replication - Resetting dropped connection

Hi all,

I'm running a katello 2.3 with a capsule in a remote datacenter.
Repository replication is quite long and fails quite often. The only
solution I found until now to restart replication is to reboot the capsule
itself :frowning:

Furthermore, in the capsule logs (/var/log/message), I've got tons of
message
Dec 11 19:42:50 pasinvnuxapp01 pulp:
requests.packages.urllib3.connectionpool:INFO: Resetting dropped
connection: pebinvnuxapp01.emea.cegedim.grp

Dec 11 19:42:51 pasinvnuxapp01 pulp:
requests.packages.urllib3.connectionpool:INFO: Resetting dropped
connection: pebinvnuxapp01.emea.cegedim.grp

Dec 11 19:42:51 pasinvnuxapp01 pulp:
requests.packages.urllib3.connectionpool:INFO: Resetting dropped
connection: pebinvnuxapp01.emea.cegedim.grp

Dec 11 19:42:51 pasinvnuxapp01 pulp:
requests.packages.urllib3.connectionpool:INFO: Resetting dropped
connection: pebinvnuxapp01.emea.cegedim.grp

Dec 11 19:42:51 pasinvnuxapp01 pulp:
requests.packages.urllib3.connectionpool:INFO: Resetting dropped
connection: pebinvnuxapp01.emea.cegedim.grp

Dec 11 19:42:51 pasinvnuxapp01 pulp:
requests.packages.urllib3.connectionpool:INFO: Resetting dropped
connection: pebinvnuxapp01.emea.cegedim.grp

Dec 11 19:42:51 pasinvnuxapp01 pulp:
requests.packages.urllib3.connectionpool:INFO: Resetting dropped
connection: pebinvnuxapp01.emea.cegedim.grp

Dec 11 19:42:51 pasinvnuxapp01 pulp:
requests.packages.urllib3.connectionpool:INFO: Resetting dropped
connection: pebinvnuxapp01.emea.cegedim.grp

Dec 11 19:42:51 pasinvnuxapp01 pulp:
requests.packages.urllib3.connectionpool:INFO: Resetting dropped
connection: pebinvnuxapp01.emea.cegedim.grp

Dec 11 19:42:52 pasinvnuxapp01 pulp:
requests.packages.urllib3.connectionpool:INFO: Resetting dropped
connection: pebinvnuxapp01.emea.cegedim.grp

I found a bug open for the logs
(Bug #11324: Pulp: endless stream of Resetting dropped connection in Logfiles when syncing repo - Katello - Foreman), but could it be the reason
for repository replication failure ?
If yes, does anybody knows if this bug do exist in version 2.2 or 2.4 ?
Is there a solution to restart replication without rebooting server ?
(restarting pulp services by command: for i in pulp_resource_manager
pulp_workers pulp_celerybeat; do service $i restart; done
seems to not be
sufficient)
Tasks launched on katello server while pulp is down (publish a view or sync
a repo) are not even taken into account, so I have to restart those tasks…

Best regards
Yannig

Hi all,

After some more test, I suspect my issue around replication between katello
and capsule is due to gofer not resetting the connection after any network
issue (timeout, packet loss…).
I found 2 bugzilla tickets (here
<https://bugzilla.redhat.com/show_bug.cgi?id=1169416>and there
<https://bugzilla.redhat.com/show_bug.cgi?id=1249890>) but the proposed
solution are only for satellite.

Is there a solution ported for katello / foreman ?

Regards
Yannig

··· Le vendredi 11 décembre 2015 18:19:46 UTC+1, yannig rousseau a écrit : > > Hi all, > > I'm running a katello 2.3 with a capsule in a remote datacenter. > Repository replication is quite long and fails quite often. The only > solution I found until now to restart replication is to reboot the capsule > itself :-( > > Furthermore, in the capsule logs (/var/log/message), I've got tons of > message > *Dec 11 19:42:50 pasinvnuxapp01 pulp: > requests.packages.urllib3.connectionpool:INFO: Resetting dropped > connection: pebinvnuxapp01.emea.cegedim.grp* > *Dec 11 19:42:51 pasinvnuxapp01 pulp: > requests.packages.urllib3.connectionpool:INFO: Resetting dropped > connection: pebinvnuxapp01.emea.cegedim.grp* > *Dec 11 19:42:51 pasinvnuxapp01 pulp: > requests.packages.urllib3.connectionpool:INFO: Resetting dropped > connection: pebinvnuxapp01.emea.cegedim.grp* > *Dec 11 19:42:51 pasinvnuxapp01 pulp: > requests.packages.urllib3.connectionpool:INFO: Resetting dropped > connection: pebinvnuxapp01.emea.cegedim.grp* > *Dec 11 19:42:51 pasinvnuxapp01 pulp: > requests.packages.urllib3.connectionpool:INFO: Resetting dropped > connection: pebinvnuxapp01.emea.cegedim.grp* > *Dec 11 19:42:51 pasinvnuxapp01 pulp: > requests.packages.urllib3.connectionpool:INFO: Resetting dropped > connection: pebinvnuxapp01.emea.cegedim.grp* > *Dec 11 19:42:51 pasinvnuxapp01 pulp: > requests.packages.urllib3.connectionpool:INFO: Resetting dropped > connection: pebinvnuxapp01.emea.cegedim.grp* > *Dec 11 19:42:51 pasinvnuxapp01 pulp: > requests.packages.urllib3.connectionpool:INFO: Resetting dropped > connection: pebinvnuxapp01.emea.cegedim.grp* > *Dec 11 19:42:51 pasinvnuxapp01 pulp: > requests.packages.urllib3.connectionpool:INFO: Resetting dropped > connection: pebinvnuxapp01.emea.cegedim.grp* > *Dec 11 19:42:52 pasinvnuxapp01 pulp: > requests.packages.urllib3.connectionpool:INFO: Resetting dropped > connection: pebinvnuxapp01.emea.cegedim.grp* > > I found a bug open for the logs ( > http://projects.theforeman.org/issues/11324), but could it be the reason > for repository replication failure ? > If yes, does anybody knows if this bug do exist in version 2.2 or 2.4 ? > Is there a solution to restart replication without rebooting server ? > (restarting pulp services by command: *for i in pulp_resource_manager > pulp_workers pulp_celerybeat; do service $i restart; done* seems to not > be sufficient) > Tasks launched on katello server while pulp is down (publish a view or > sync a repo) are not even taken into account, so I have to restart those > tasks... > > Best regards > Yannig >