Katello pulp_export_destination checks happen inside the application

Hello all,

While working on splitting the Katello installation from its services I
ran into the pulp_export_destination setting. That made me look at the
Repository Export1 which assumes the pulp server and application server
share the same filesystem.

That means I'm stuck in attempting to split services to different
machines where I don't want to assume a shared filesystem. Looking at
the containers it's not handled there either. I don't see a way to
support this feature without a major rewrite of the code but I'm not
familiar with the Katello code base.

Anyone has advice on how to deal with this?

> Hello all,
>
> While working on splitting the Katello installation from its services I
> ran into the pulp_export_destination setting. That made me look at the
> Repository Export1 which assumes the pulp server and application server
> share the same filesystem.
>
> That means I'm stuck in attempting to split services to different machines
> where I don't want to assume a shared filesystem. Looking at the containers
> it's not handled there either. I don't see a way to support this feature
> without a major rewrite of the code but I'm not familiar with the Katello
> code base.
>
> Anyone has advice on how to deal with this?
>

I think if Pulp took a parameter to publish to a non-default location, we
would not need to do the extra copy step that ties Katello and Pulp closely
together. Currently it is needed because we need the export to land in
pulp_export_destination.

··· On Sun, Aug 20, 2017 at 4:02 PM, Ewoud Kohl van Wijngaarden < ewoud@kohlvanwijngaarden.nl> wrote:

ns/katello/repository/export.rb


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

>
>> Hello all,
>>
>> While working on splitting the Katello installation from its services I
>> ran into the pulp_export_destination setting. That made me look at the
>> Repository Export1 which assumes the pulp server and application server
>> share the same filesystem.
>>
>> That means I'm stuck in attempting to split services to different machines
>> where I don't want to assume a shared filesystem. Looking at the containers
>> it's not handled there either. I don't see a way to support this feature
>> without a major rewrite of the code but I'm not familiar with the Katello
>> code base.
>>
>> Anyone has advice on how to deal with this?
>>
>
>I think if Pulp took a parameter to publish to a non-default location, we
>would not need to do the extra copy step that ties Katello and Pulp closely
>together. Currently it is needed because we need the export to land in
>pulp_export_destination.

That would be nice. To me it looks like it already sends the
export_directory to pulp and has an absolute path. Is there a reason
this can't be set to Setting['pulp_export_destination'] directly?

··· On Sun, Aug 20, 2017 at 04:11:13PM -0400, Christopher Duryee wrote: >On Sun, Aug 20, 2017 at 4:02 PM, Ewoud Kohl van Wijngaarden < >ewoud@kohlvanwijngaarden.nl> wrote:

What would you think, if we let smart_proxy_pulp handle the copy job and
checking if the directories exist? That would loosen the coupling
between Katello and Pulp?

Timo

··· Am 20.08.17 um 22:39 schrieb Ewoud Kohl van Wijngaarden: >> I think if Pulp took a parameter to publish to a non-default >> location, we >> would not need to do the extra copy step that ties Katello and Pulp >> closely >> together. Currently it is needed because we need the export to land in >> pulp_export_destination. > > That would be nice. To me it looks like it already sends the > export_directory to pulp and has an absolute path. Is there a reason > this can't be set to Setting['pulp_export_destination'] directly?

That was my first thought as well and it is the typical approach taken
within the (vanilla) Foreman architecture. This is where Katello really
differs and would require a major rewrite of certain integrations.

If we all agree Katello should never assume a service shares its
filesystem with another service I can create issues for everything I run
into.

··· On Mon, Aug 21, 2017 at 08:06:01AM +0200, Timo Goebel wrote: >Am 20.08.17 um 22:39 schrieb Ewoud Kohl van Wijngaarden: >>>I think if Pulp took a parameter to publish to a non-default >>>location, we >>>would not need to do the extra copy step that ties Katello and >>>Pulp closely >>>together. Currently it is needed because we need the export to land in >>>pulp_export_destination. >> >>That would be nice. To me it looks like it already sends the >>export_directory to pulp and has an absolute path. Is there a reason >>this can't be set to Setting['pulp_export_destination'] directly? >What would you think, if we let smart_proxy_pulp handle the copy job >and checking if the directories exist? That would loosen the coupling >between Katello and Pulp?

You are suggesting to apply the vanilla Foreman approach and I certainly
agree this is a nice architecture that I always liked as a user but it
does take effort to get there.

A fix in Pulp would also be nice because it's now doing copies which
can be a lot of IO.

For this issue I created Bug #20854: Katello repository export requires a shared filesystem with Pulp - Katello - Foreman and
for now set it to Blocks Tracker #20850: Allow split deployments using the installer - Katello - Foreman but
it can be downgraded to a known limitation.

··· On Mon, Aug 21, 2017 at 11:05:46AM +0200, Ewoud Kohl van Wijngaarden wrote: >On Mon, Aug 21, 2017 at 08:06:01AM +0200, Timo Goebel wrote: >>Am 20.08.17 um 22:39 schrieb Ewoud Kohl van Wijngaarden: >>>>I think if Pulp took a parameter to publish to a non-default >>>>location, we >>>>would not need to do the extra copy step that ties Katello and >>>>Pulp closely >>>>together. Currently it is needed because we need the export to land in >>>>pulp_export_destination. >>> >>>That would be nice. To me it looks like it already sends the >>>export_directory to pulp and has an absolute path. Is there a >>>reason this can't be set to Setting['pulp_export_destination'] >>>directly? >>What would you think, if we let smart_proxy_pulp handle the copy job >>and checking if the directories exist? That would loosen the >>coupling between Katello and Pulp? > >That was my first thought as well and it is the typical approach taken >within the (vanilla) Foreman architecture. This is where Katello >really differs and would require a major rewrite of certain >integrations. > >If we all agree Katello should never assume a service shares its >filesystem with another service I can create issues for everything I >run into.