Katello on Foreman: Handling Certificates

All,

As part of the work to make changes such that Katello can be added to an
existing Foreman, we have to deal with the certificate setup required by
Katello and that of existing Foremans. The goal of this thread is to get
feedback on possible design changes to the way we handle certificates
before I go too far down a particular path.

Background

··· -----------------

The strategy that Katello takes today is to use a single puppet module [1]
that contains classes for each service that requires certificates. Each of
these classes assumes knowledge of the init.pp that creates a root CA.
These certificate classes are then orchestrated within the puppet-katello
module [2]. For those curious, the diagram at [3] shows a near complete
list of SSL communication that has to occur today. Further, for a general
idea of all the certificates we generate and services that use them see [4].

Requirements

Looking ahead to scenarios we have to support, there are:

  1. Existing Foreman with Puppet certificates as the root CA
  2. Existing Katello+Foreman with puppet-certs custom self-signed
    certificates
  3. New installs that want to use Puppet CA as the root CA
  4. New installs that want to use custom certificates (similar to what
    Katello does today)
  5. Existing Foreman installs changed over to custom certificates
  6. User provided webserver certificates

Design Proposals

  1. puppet-certs approach
    pros:

    • centralized certificate management into a single module
    • allows orchestrating large certificate changes (such as user provided
      certificates or certificate regeneration)
      cons:
    • requires an orchestration/wrapper module to ensure certificates and
      other services get done in the right order
    • requires every module that needs certificates to have a dependency on
      puppet-certs
    • requires puppet-certs to have knowledge of every “service” requiring
      certs
  2. Modules control their own certificates
    pros:

    • modules remain independent
    • no orchestration module needed
    • can be in charge of generating certificates or being handed
      certificates to use
    • follows same model as other things like configuration files, service
      objects or datastores
      cons:
    • every module needs to know how to generate and/or sign certificates
    • orchestrating regeneration or addition of user certificates might be
      difficult
  3. ??? - there are other possible design ideas that we have not thought of

As I said at the beginning, I’d appreciate some feedback and debate in this
area from those who have had experience with Katello certificate management
and the installer (especially @inecas) and those who are outsiders to the
process that might see some fundamentals we are missing. Along with the
requirements above, my goals are:

  1. Enable the aforementioned requirement scenarios
  2. Simplify certificate management where possible
  3. Enable ease of adoption of Katello with respect to certificates
  4. Provide benefits to Foreman community where possible (such as using non
    Puppet CA certificates in an easy to use manner)

Thanks,
Eric

[1] https://github.com/Katello/puppet-certs
[2]
https://github.com/Katello/puppet-katello/blob/master/manifests/init.pp#L63
[3]
https://raw.githubusercontent.com/ehelms/connection_diagram/master/katello.png
[4]
https://docs.google.com/drawings/d/121tMce_neRowWG8nkDh6BVZCxxjRMGyzFjV1T0jmo2I/edit?usp=sharing


Eric D. Helms
Red Hat Engineering
Ph.D. Student - North Carolina State University

Disclaimer: I have not looked at puppet-certs so this is mostly from a
foreman perspective. This is sort an option 3 / variation on option 1.

I know the current foreman installer is tied to puppet certs, both
implicit or explicit causing odd failures if puppet is disabled that are
non-obvious for users. That means I'm leaning to a puppet-certs
approach. My suggestion would be to create a new module based on
puppet-certs with a more specific name (foreman-certs?) that can handle
both Puppet CA and custom certs. This would be a benefit for Foreman
users who want SSL but not puppet, but also align nicely with Katellos
needs.

I think the module could be written without a need for
orchestration/wrapper module if we have proper dependencies in all
modules. It would become a hard dependency in all modules and possibly
require major version bumps in modules, but I think it will be cleaner.
Of course the module would be optional if the user chooses insecure
setups.

The hardest part might be to decouple certificate generation from
puppet-puppet but puppet-foreman-certs relies on puppet-puppet to
install the tools. This screams circular dependencies if you're not
careful.

What I would want from the module is a way to get the certificates and
the result should be paths that the respective modules could use. So a
main class that configures Puppet CA / custom CA, then a class for
foreman itself. foreman::config would require that class and then use
$::foreman_certs::foreman::cert_path / …::key_path / …::ca_path.

··· On Wed, Sep 23, 2015 at 09:29:49PM -0400, Eric D Helms wrote: > All, > > As part of the work to make changes such that Katello can be added to an > existing Foreman, we have to deal with the certificate setup required by > Katello and that of existing Foremans. The goal of this thread is to get > feedback on possible design changes to the way we handle certificates > before I go too far down a particular path. > > Background > ----------------- > > The strategy that Katello takes today is to use a single puppet module [1] > that contains classes for each service that requires certificates. Each of > these classes assumes knowledge of the init.pp that creates a root CA. > These certificate classes are then orchestrated within the puppet-katello > module [2]. For those curious, the diagram at [3] shows a near complete > list of SSL communication that has to occur today. Further, for a general > idea of all the certificates we generate and services that use them see [4]. > > Requirements > -------------------- > > Looking ahead to scenarios we have to support, there are: > > 1) Existing Foreman with Puppet certificates as the root CA > 2) Existing Katello+Foreman with puppet-certs custom self-signed > certificates > 3) New installs that want to use Puppet CA as the root CA > 4) New installs that want to use custom certificates (similar to what > Katello does today) > 5) Existing Foreman installs changed over to custom certificates > 6) User provided webserver certificates > > Design Proposals > ------------------------- > > 1) puppet-certs approach > pros: > - centralized certificate management into a single module > - allows orchestrating large certificate changes (such as user provided > certificates or certificate regeneration) > cons: > - requires an orchestration/wrapper module to ensure certificates and > other services get done in the right order > - requires every module that needs certificates to have a dependency on > puppet-certs > - requires puppet-certs to have knowledge of every "service" requiring > certs > > 2) Modules control their own certificates > pros: > - modules remain independent > - no orchestration module needed > - can be in charge of generating certificates or being handed > certificates to use > - follows same model as other things like configuration files, service > objects or datastores > cons: > - every module needs to know how to generate and/or sign certificates > - orchestrating regeneration or addition of user certificates might be > difficult > > 3) ??? - there are other possible design ideas that we have not thought of > > > As I said at the beginning, I'd appreciate some feedback and debate in this > area from those who have had experience with Katello certificate management > and the installer (especially @inecas) and those who are outsiders to the > process that might see some fundamentals we are missing. Along with the > requirements above, my goals are: > > 1) Enable the aforementioned requirement scenarios > 2) Simplify certificate management where possible > 3) Enable ease of adoption of Katello with respect to certificates > 4) Provide benefits to Foreman community where possible (such as using non > Puppet CA certificates in an easy to use manner) > > > Thanks, > Eric > > > [1] https://github.com/Katello/puppet-certs > [2] > https://github.com/Katello/puppet-katello/blob/master/manifests/init.pp#L63 > [3] > https://raw.githubusercontent.com/ehelms/connection_diagram/master/katello.png > [4] > https://docs.google.com/drawings/d/121tMce_neRowWG8nkDh6BVZCxxjRMGyzFjV1T0jmo2I/edit?usp=sharing

What you are describing would be a spin of puppet-certs but as you say,
making each module dependent on it and using it's defined certificate setup
for that service instead of a single orchestrating module. If we take the
case of needing a root CA and server certificates, you can see that in [1]
we generate a CA (this would be adopted to generate or use Puppet's) and
then in [2] we have all the logic needed to generate certificates for the
server (i.e. Apache). You can see these being orchestrated at [3]. The
other thing our puppet-certs module does today is break this process down
into generation, deployment and regeneration of certificates.

So option 3/1 would be some changes, but a large re-use of the existing
puppet-certs paradigm and code we have today.

[1] https://github.com/Katello/puppet-certs/blob/master/manifests/init.pp
[2] https://github.com/Katello/puppet-certs/blob/master/manifests/apache.pp
[3]
https://github.com/Katello/puppet-katello/blob/master/manifests/init.pp#L63-L64

··· On Thu, Sep 24, 2015 at 8:48 AM, Ewoud Kohl van Wijngaarden < ewoud@kohlvanwijngaarden.nl> wrote:

On Wed, Sep 23, 2015 at 09:29:49PM -0400, Eric D Helms wrote:

All,

As part of the work to make changes such that Katello can be added to an
existing Foreman, we have to deal with the certificate setup required by
Katello and that of existing Foremans. The goal of this thread is to get
feedback on possible design changes to the way we handle certificates
before I go too far down a particular path.

Background

The strategy that Katello takes today is to use a single puppet module
[1]
that contains classes for each service that requires certificates. Each
of
these classes assumes knowledge of the init.pp that creates a root CA.
These certificate classes are then orchestrated within the puppet-katello
module [2]. For those curious, the diagram at [3] shows a near complete
list of SSL communication that has to occur today. Further, for a general
idea of all the certificates we generate and services that use them see
[4].

Requirements

Looking ahead to scenarios we have to support, there are:

  1. Existing Foreman with Puppet certificates as the root CA
  2. Existing Katello+Foreman with puppet-certs custom self-signed
    certificates
  3. New installs that want to use Puppet CA as the root CA
  4. New installs that want to use custom certificates (similar to what
    Katello does today)
  5. Existing Foreman installs changed over to custom certificates
  6. User provided webserver certificates

Design Proposals

  1. puppet-certs approach
    pros:

    • centralized certificate management into a single module
    • allows orchestrating large certificate changes (such as user
      provided
      certificates or certificate regeneration)
      cons:
    • requires an orchestration/wrapper module to ensure certificates and
      other services get done in the right order
    • requires every module that needs certificates to have a dependency
      on
      puppet-certs
    • requires puppet-certs to have knowledge of every "service"
      requiring
      certs
  2. Modules control their own certificates
    pros:

    • modules remain independent
    • no orchestration module needed
    • can be in charge of generating certificates or being handed
      certificates to use
    • follows same model as other things like configuration files,
      service
      objects or datastores
      cons:
    • every module needs to know how to generate and/or sign certificates
    • orchestrating regeneration or addition of user certificates might
      be
      difficult
  3. ??? - there are other possible design ideas that we have not thought
    of

As I said at the beginning, I’d appreciate some feedback and debate in
this
area from those who have had experience with Katello certificate
management
and the installer (especially @inecas) and those who are outsiders to the
process that might see some fundamentals we are missing. Along with the
requirements above, my goals are:

  1. Enable the aforementioned requirement scenarios
  2. Simplify certificate management where possible
  3. Enable ease of adoption of Katello with respect to certificates
  4. Provide benefits to Foreman community where possible (such as using
    non
    Puppet CA certificates in an easy to use manner)

Thanks,
Eric

[1] https://github.com/Katello/puppet-certs
[2]

https://github.com/Katello/puppet-katello/blob/master/manifests/init.pp#L63

[3]

https://raw.githubusercontent.com/ehelms/connection_diagram/master/katello.png

[4]

https://docs.google.com/drawings/d/121tMce_neRowWG8nkDh6BVZCxxjRMGyzFjV1T0jmo2I/edit?usp=sharing

Disclaimer: I have not looked at puppet-certs so this is mostly from a
foreman perspective. This is sort an option 3 / variation on option 1.

I know the current foreman installer is tied to puppet certs, both
implicit or explicit causing odd failures if puppet is disabled that are
non-obvious for users. That means I’m leaning to a puppet-certs
approach. My suggestion would be to create a new module based on
puppet-certs with a more specific name (foreman-certs?) that can handle
both Puppet CA and custom certs. This would be a benefit for Foreman
users who want SSL but not puppet, but also align nicely with Katellos
needs.

I think the module could be written without a need for
orchestration/wrapper module if we have proper dependencies in all
modules. It would become a hard dependency in all modules and possibly
require major version bumps in modules, but I think it will be cleaner.
Of course the module would be optional if the user chooses insecure
setups.

The hardest part might be to decouple certificate generation from
puppet-puppet but puppet-foreman-certs relies on puppet-puppet to
install the tools. This screams circular dependencies if you’re not
careful.

What I would want from the module is a way to get the certificates and
the result should be paths that the respective modules could use. So a
main class that configures Puppet CA / custom CA, then a class for
foreman itself. foreman::config would require that class and then use
$::foreman_certs::foreman::cert_path / …::key_path / …::ca_path.


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Eric D. Helms
Red Hat Engineering
Ph.D. Student - North Carolina State University

I don't think option 2) would be able to support all the cases
we need. Think about, for example custom (commercial) CA for
https. It's pretty complicated even now, and if every puppet
module handled it on its own, one would need to pass the arguments
about the custom CA for all the modules that are dealing with running
https or verifying the remote server.

I agree that puppet-certs is unnecessary complex (it has its own history)
although some of the issues are mostly usability problems. But the principle
works pretty well: the module is responsible for generating and deploying the
certificates.

There still needs however to be some logic on other modules part: I think
it makes sense here to use some defined types that the modules could use
to declare what certs they need. The certs module would generate the certs
and put it on right place, and the modules could count on the fact that the
certs are then there. The puppet-certs uses custom types, which is probably not
the best to do with Puppet 4. But I think we could get similar behavior with defined
types as well.

With this approach, all the cert implementation logic would be in the certs module
(including support for different CAs, revocation/renewal etc.) and all other foreman's
puppet modules would use the same approach to the certs.

Also, I would recommend to research how hard would then be to employ FreeIPA
to the certs generation/distribution process. Ideally, the only change that would
be needed would be on the certs module part, while all the other modules would
stay the same.

– Ivan

> What you are describing would be a spin of puppet-certs but as you say,
> making each module dependent on it and using it's defined certificate setup
> for that service instead of a single orchestrating module. If we take the
> case of needing a root CA and server certificates, you can see that in [1]
> we generate a CA (this would be adopted to generate or use Puppet's) and
> then in [2] we have all the logic needed to generate certificates for the
> server (i.e. Apache). You can see these being orchestrated at [3]. The
> other thing our puppet-certs module does today is break this process down
> into generation, deployment and regeneration of certificates.
>
> So option 3/1 would be some changes, but a large re-use of the existing
> puppet-certs paradigm and code we have today.
>
> [1] https://github.com/Katello/puppet-certs/blob/master/manifests/init.pp
> [2] https://github.com/Katello/puppet-certs/blob/master/manifests/apache.pp
> [3]
> https://github.com/Katello/puppet-katello/blob/master/manifests/init.pp#L63-L64
>
>
>
> > > All,
> > >
> > > As part of the work to make changes such that Katello can be added to an
> > > existing Foreman, we have to deal with the certificate setup required by
> > > Katello and that of existing Foremans. The goal of this thread is to get
> > > feedback on possible design changes to the way we handle certificates
> > > before I go too far down a particular path.
> > >
> > > Background
> > > -----------------
> > >
> > > The strategy that Katello takes today is to use a single puppet module
> > [1]
> > > that contains classes for each service that requires certificates. Each
> > of
> > > these classes assumes knowledge of the init.pp that creates a root CA.
> > > These certificate classes are then orchestrated within the puppet-katello
> > > module [2]. For those curious, the diagram at [3] shows a near complete
> > > list of SSL communication that has to occur today. Further, for a general
> > > idea of all the certificates we generate and services that use them see
> > [4].
> > >
> > > Requirements
> > > --------------------
> > >
> > > Looking ahead to scenarios we have to support, there are:
> > >
> > > 1) Existing Foreman with Puppet certificates as the root CA
> > > 2) Existing Katello+Foreman with puppet-certs custom self-signed
> > > certificates
> > > 3) New installs that want to use Puppet CA as the root CA
> > > 4) New installs that want to use custom certificates (similar to what
> > > Katello does today)
> > > 5) Existing Foreman installs changed over to custom certificates
> > > 6) User provided webserver certificates
> > >
> > > Design Proposals
> > > -------------------------
> > >
> > > 1) puppet-certs approach
> > > pros:
> > > - centralized certificate management into a single module
> > > - allows orchestrating large certificate changes (such as user
> > provided
> > > certificates or certificate regeneration)
> > > cons:
> > > - requires an orchestration/wrapper module to ensure certificates and
> > > other services get done in the right order
> > > - requires every module that needs certificates to have a dependency
> > on
> > > puppet-certs
> > > - requires puppet-certs to have knowledge of every "service"
> > requiring
> > > certs
> > >
> > > 2) Modules control their own certificates
> > > pros:
> > > - modules remain independent
> > > - no orchestration module needed
> > > - can be in charge of generating certificates or being handed
> > > certificates to use
> > > - follows same model as other things like configuration files,
> > service
> > > objects or datastores
> > > cons:
> > > - every module needs to know how to generate and/or sign certificates
> > > - orchestrating regeneration or addition of user certificates might
> > be
> > > difficult
> > >
> > > 3) ??? - there are other possible design ideas that we have not thought
> > of
> > >
> > >
> > > As I said at the beginning, I'd appreciate some feedback and debate in
> > this
> > > area from those who have had experience with Katello certificate
> > management
> > > and the installer (especially @inecas) and those who are outsiders to the
> > > process that might see some fundamentals we are missing. Along with the
> > > requirements above, my goals are:
> > >
> > > 1) Enable the aforementioned requirement scenarios
> > > 2) Simplify certificate management where possible
> > > 3) Enable ease of adoption of Katello with respect to certificates
> > > 4) Provide benefits to Foreman community where possible (such as using
> > non
> > > Puppet CA certificates in an easy to use manner)
> > >
> > >
> > > Thanks,
> > > Eric
> > >
> > >
> > > [1] https://github.com/Katello/puppet-certs
> > > [2]
> > >
> > https://github.com/Katello/puppet-katello/blob/master/manifests/init.pp#L63
> > > [3]
> > >
> > https://raw.githubusercontent.com/ehelms/connection_diagram/master/katello.png
> > > [4]
> > >
> > https://docs.google.com/drawings/d/121tMce_neRowWG8nkDh6BVZCxxjRMGyzFjV1T0jmo2I/edit?usp=sharing
> >
> > Disclaimer: I have not looked at puppet-certs so this is mostly from a
> > foreman perspective. This is sort an option 3 / variation on option 1.
> >
> > I know the current foreman installer is tied to puppet certs, both
> > implicit or explicit causing odd failures if puppet is disabled that are
> > non-obvious for users. That means I'm leaning to a puppet-certs
> > approach. My suggestion would be to create a new module based on
> > puppet-certs with a more specific name (foreman-certs?) that can handle
> > both Puppet CA and custom certs. This would be a benefit for Foreman
> > users who want SSL but not puppet, but also align nicely with Katellos
> > needs.
> >
> > I think the module could be written without a need for
> > orchestration/wrapper module if we have proper dependencies in all
> > modules. It would become a hard dependency in all modules and possibly
> > require major version bumps in modules, but I think it will be cleaner.
> > Of course the module would be optional if the user chooses insecure
> > setups.
> >
> > The hardest part might be to decouple certificate generation from
> > puppet-puppet but puppet-foreman-certs relies on puppet-puppet to
> > install the tools. This screams circular dependencies if you're not
> > careful.
> >
> > What I would want from the module is a way to get the certificates and
> > the result should be paths that the respective modules could use. So a
> > main class that configures Puppet CA / custom CA, then a class for
> > foreman itself. foreman::config would require that class and then use
> > $::foreman_certs::foreman::cert_path / …::key_path / …::ca_path.

The problem here is, the certs class needs to be initialized before we could
use the params. And if we want to have some class params on the certs module,
(such sa CA configuration), we would need to make sure we initialize the class first.
Kafo can handle this, but puppet-master environment AFAIK we can't influence what gets
loaded first.

– Ivan

··· ----- Original Message ----- > On Thu, Sep 24, 2015 at 8:48 AM, Ewoud Kohl van Wijngaarden < > ewoud@kohlvanwijngaarden.nl> wrote: > > On Wed, Sep 23, 2015 at 09:29:49PM -0400, Eric D Helms wrote:


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Eric D. Helms
Red Hat Engineering
Ph.D. Student - North Carolina State University


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

As I have dug into this more, I have realized that regardless of
implementation, there is a question of design that I need to lock down
before hand. I've been considering the following three scenarios:

  1. existing Foreman
  2. existing Katello
  3. new installs

The biggest issue I have had to design around is how to handle the CA that
we need to generate for Candlepin. For those that don't know, Candlepin is
responsible for generating the client certificates that are used by
subscription-manager based clients. As part of the client support, Katello
makes available a bootstrap RPM that contains the CA certificate that
signed both the server and client certificates. This is so that the client
can verify the identity of the server it is registering to. Thus, the
client needs a copy of the CA certificate that signed the server
certificate. With this in mind, I have been trying to design with the goal
of isolating the certificate setup that Katello pieces need from the
baseline certificate setup of a regular Foreman server. I have concluded
that the best way to accomplish this is the following:

  1. For existing Foreman:
  • generate a CA for Candlepin that is then signed by the CA that
    generated the server certificates
  • configure the bootstrap RPM to contain the CA that signed the server
    certificates, and the CA that was generated for Candlepin
  • generate all other certificates needed by Katello pieces (e.g. qpid)
    from the CA generated for Candlepin's use
  1. For existing Katello:
  • business as usual, to ensure upgrades and existing clients keep working
    without the need to update their certificate setups
  1. For new installs:
  • treat the same as #1

As far as I know, Foreman today supports Puppet CA and complete custom
specification of server certificates. Implementation wise, I believe I can
put a CSR in the right Puppet CA directory to have Puppet sign the
certificates to then use as the CA for Candlepin. As for entirely custom
certificates by users - that one is a little trickier and I have not
figured out how to handle at the moment.

Feedback on this approach would be greatly appreciated (and other ideas).

Thanks,
Eric

Hi All,

As a previous thread mentioned, a number of developers met up a week prior
to discuss a variety of topics. One of those was getting to discuss the
future of certificates in person which introduced a different approach than
previously mentioned. Below is my attempt to distill that approach for
discussion. For those that were part of the original discussion, if I have
missed anything please correct them.

The proposal is to separate "internal communication" from external
communication. That is, segment infrastructure communication (e.g. smart
proxies, clients) to their own certificate setup and ports from that of
external communication (e.g. API, UI). This would work by deploying two
ports through two apache vhosts. One port would be the externally facing
port (i.e. 443) configured with either Puppet, user supplied or
auto-generated CA and server certificates. The other port would be
configured with a second auto-generated CA and server certificates. This
second CA would also be responsible for issuing certificates for Smart
Proxies and clients (other than puppet). See the attached diagram for a
general picture of what this would look like.

This has the following advantages:

  1. Changing the user facing certificates (e.g. those a user imports into
    their browser or gets from their IT department) is simply a matter of
    changing out the Apache server certificates and CA
  2. Allowing multi-home setups (e.g. two hostnames pointing at the same box)
  3. Revoking internal certificates or CA doesn't break the externally facing
    configuration that a user may have customized
  4. Gives us more control over changes to internal infrastructure and
    certificates such as using FreeIPA to control certificates

What about upgrades?

To handle upgrades, for one release we could deploy the new port with the
internally generated infrastructure certificates and inform users to change
over their smart proxy certificates ones generated from the infrastructure
CA. This could be done by providing a script to help generate a set of
certificates. For Katello users, the current 'default CA' would be moved to
the infrastructure port and a new set of certificates (or user provided
certificates) would be used on the externally facing port. In all
scenarios, communication should be interrupted with a path towards the new
design.

Thoughts?

Eric

··· On Mon, Oct 5, 2015 at 8:58 PM, Eric D Helms wrote:

As I have dug into this more, I have realized that regardless of
implementation, there is a question of design that I need to lock down
before hand. I’ve been considering the following three scenarios:

  1. existing Foreman
  2. existing Katello
  3. new installs

The biggest issue I have had to design around is how to handle the CA that
we need to generate for Candlepin. For those that don’t know, Candlepin is
responsible for generating the client certificates that are used by
subscription-manager based clients. As part of the client support, Katello
makes available a bootstrap RPM that contains the CA certificate that
signed both the server and client certificates. This is so that the client
can verify the identity of the server it is registering to. Thus, the
client needs a copy of the CA certificate that signed the server
certificate. With this in mind, I have been trying to design with the goal
of isolating the certificate setup that Katello pieces need from the
baseline certificate setup of a regular Foreman server. I have concluded
that the best way to accomplish this is the following:

  1. For existing Foreman:
  • generate a CA for Candlepin that is then signed by the CA that
    generated the server certificates
  • configure the bootstrap RPM to contain the CA that signed the server
    certificates, and the CA that was generated for Candlepin
  • generate all other certificates needed by Katello pieces (e.g. qpid)
    from the CA generated for Candlepin’s use
  1. For existing Katello:
  • business as usual, to ensure upgrades and existing clients keep
    working without the need to update their certificate setups
  1. For new installs:
  • treat the same as #1

As far as I know, Foreman today supports Puppet CA and complete custom
specification of server certificates. Implementation wise, I believe I can
put a CSR in the right Puppet CA directory to have Puppet sign the
certificates to then use as the CA for Candlepin. As for entirely custom
certificates by users - that one is a little trickier and I have not
figured out how to handle at the moment.

Feedback on this approach would be greatly appreciated (and other ideas).

Thanks,
Eric


Eric D. Helms
Red Hat Engineering
Ph.D. Student - North Carolina State University

The main issues which I am seeing downstream are:

  1. Changing hostnames - The machine in installed in a test environment,
    and they want to rename it as part of moving the machine into prod
    environment.

  2. limiting access tot he webui to certain network segments

  3. limiting the number of ports which need to be opened in firewalls /
    networks.

I think you are addressing 2 which is great. I am concerned that we may
be sacrificing on (3) however. Is that an accurate concern?

– bk

··· On 10/20/2015 08:38 AM, Eric D Helms wrote: > Hi All, > > As a previous thread mentioned, a number of developers met up a week > prior to discuss a variety of topics. One of those was getting to > discuss the future of certificates in person which introduced a > different approach than previously mentioned. Below is my attempt to > distill that approach for discussion. For those that were part of the > original discussion, if I have missed anything please correct them. > > The proposal is to separate "internal communication" from external > communication. That is, segment infrastructure communication (e.g. smart > proxies, clients) to their own certificate setup and ports from that of > external communication (e.g. API, UI). This would work by deploying two > ports through two apache vhosts. One port would be the externally facing > port (i.e. 443) configured with either Puppet, user supplied or > auto-generated CA and server certificates. The other port would be > configured with a second auto-generated CA and server certificates. This > second CA would also be responsible for issuing certificates for Smart > Proxies and clients (other than puppet). See the attached diagram for a > general picture of what this would look like. > > This has the following advantages: > > 1. Changing the user facing certificates (e.g. those a user imports into > their browser or gets from their IT department) is simply a matter of > changing out the Apache server certificates and CA > 2. Allowing multi-home setups (e.g. two hostnames pointing at the same box) > 3. Revoking internal certificates or CA doesn't break the externally > facing configuration that a user may have customized > 4. Gives us more control over changes to internal infrastructure and > certificates such as using FreeIPA to control certificates > > What about upgrades? > > To handle upgrades, for one release we could deploy the new port with > the internally generated infrastructure certificates and inform users to > change over their smart proxy certificates ones generated from the > infrastructure CA. This could be done by providing a script to help > generate a set of certificates. For Katello users, the current 'default > CA' would be moved to the infrastructure port and a new set of > certificates (or user provided certificates) would be used on the > externally facing port. In all scenarios, communication should be > interrupted with a path towards the new design. > > Thoughts?

>
>
> > Hi All,
> >
> > As a previous thread mentioned, a number of developers met up a week
> > prior to discuss a variety of topics. One of those was getting to
> > discuss the future of certificates in person which introduced a
> > different approach than previously mentioned. Below is my attempt to
> > distill that approach for discussion. For those that were part of the
> > original discussion, if I have missed anything please correct them.
> >
> > The proposal is to separate "internal communication" from external
> > communication. That is, segment infrastructure communication (e.g. smart
> > proxies, clients) to their own certificate setup and ports from that of
> > external communication (e.g. API, UI). This would work by deploying two
> > ports through two apache vhosts. One port would be the externally facing
> > port (i.e. 443) configured with either Puppet, user supplied or
> > auto-generated CA and server certificates. The other port would be
> > configured with a second auto-generated CA and server certificates. This
> > second CA would also be responsible for issuing certificates for Smart
> > Proxies and clients (other than puppet). See the attached diagram for a
> > general picture of what this would look like.
> >
> > This has the following advantages:
> >
> > 1. Changing the user facing certificates (e.g. those a user imports into
> > their browser or gets from their IT department) is simply a matter of
> > changing out the Apache server certificates and CA
> > 2. Allowing multi-home setups (e.g. two hostnames pointing at the same box)
> > 3. Revoking internal certificates or CA doesn't break the externally
> > facing configuration that a user may have customized
> > 4. Gives us more control over changes to internal infrastructure and
> > certificates such as using FreeIPA to control certificates
> >
> > What about upgrades?
> >
> > To handle upgrades, for one release we could deploy the new port with
> > the internally generated infrastructure certificates and inform users to
> > change over their smart proxy certificates ones generated from the
> > infrastructure CA. This could be done by providing a script to help
> > generate a set of certificates. For Katello users, the current 'default
> > CA' would be moved to the infrastructure port and a new set of
> > certificates (or user provided certificates) would be used on the
> > externally facing port. In all scenarios, communication should be
> > interrupted with a path towards the new design.
> >
> > Thoughts?
>
>
> The main issues which I am seeing downstream are:
>
> 1) Changing hostnames - The machine in installed in a test environment,
> and they want to rename it as part of moving the machine into prod
> environment.
>
> 2) limiting access tot he webui to certain network segments
>
> 3) limiting the number of ports which need to be opened in firewalls /
> networks.
>
> I think you are addressing 2 which is great. I am concerned that we may
> be sacrificing on (3) however. Is that an accurate concern?

With proxy-isolation feature, we should be ok mainly to open the port between
the main server and proxies, the rest should stay pretty the same as it's right
now.

··· ----- Original Message ----- > On 10/20/2015 08:38 AM, Eric D Helms wrote:

– bk


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

I support this proposal. It would solve a lot of problems we see both in upstream and downstream.
Just a side-note - we still need a way how to better manage the internal CA, than just relying on
the Puppet CA.

– Ivan

··· ----- Original Message ----- > Hi All, > > As a previous thread mentioned, a number of developers met up a week prior > to discuss a variety of topics. One of those was getting to discuss the > future of certificates in person which introduced a different approach than > previously mentioned. Below is my attempt to distill that approach for > discussion. For those that were part of the original discussion, if I have > missed anything please correct them. > > The proposal is to separate "internal communication" from external > communication. That is, segment infrastructure communication (e.g. smart > proxies, clients) to their own certificate setup and ports from that of > external communication (e.g. API, UI). This would work by deploying two > ports through two apache vhosts. One port would be the externally facing > port (i.e. 443) configured with either Puppet, user supplied or > auto-generated CA and server certificates. The other port would be > configured with a second auto-generated CA and server certificates. This > second CA would also be responsible for issuing certificates for Smart > Proxies and clients (other than puppet). See the attached diagram for a > general picture of what this would look like. > > This has the following advantages: > > 1. Changing the user facing certificates (e.g. those a user imports into > their browser or gets from their IT department) is simply a matter of > changing out the Apache server certificates and CA > 2. Allowing multi-home setups (e.g. two hostnames pointing at the same box) > 3. Revoking internal certificates or CA doesn't break the externally facing > configuration that a user may have customized > 4. Gives us more control over changes to internal infrastructure and > certificates such as using FreeIPA to control certificates > > What about upgrades? > > To handle upgrades, for one release we could deploy the new port with the > internally generated infrastructure certificates and inform users to change > over their smart proxy certificates ones generated from the infrastructure > CA. This could be done by providing a script to help generate a set of > certificates. For Katello users, the current 'default CA' would be moved to > the infrastructure port and a new set of certificates (or user provided > certificates) would be used on the externally facing port. In all > scenarios, communication should be interrupted with a path towards the new > design. > > Thoughts? > > > Eric > > > On Mon, Oct 5, 2015 at 8:58 PM, Eric D Helms wrote: > > > As I have dug into this more, I have realized that regardless of > > implementation, there is a question of design that I need to lock down > > before hand. I've been considering the following three scenarios: > > > > 1) existing Foreman > > 2) existing Katello > > 3) new installs > > > > The biggest issue I have had to design around is how to handle the CA that > > we need to generate for Candlepin. For those that don't know, Candlepin is > > responsible for generating the client certificates that are used by > > subscription-manager based clients. As part of the client support, Katello > > makes available a bootstrap RPM that contains the CA certificate that > > signed both the server and client certificates. This is so that the client > > can verify the identity of the server it is registering to. Thus, the > > client needs a copy of the CA certificate that signed the server > > certificate. With this in mind, I have been trying to design with the goal > > of isolating the certificate setup that Katello pieces need from the > > baseline certificate setup of a regular Foreman server. I have concluded > > that the best way to accomplish this is the following: > > > > 1) For existing Foreman: > > - generate a CA for Candlepin that is then signed by the CA that > > generated the server certificates > > - configure the bootstrap RPM to contain the CA that signed the server > > certificates, and the CA that was generated for Candlepin > > - generate all other certificates needed by Katello pieces (e.g. qpid) > > from the CA generated for Candlepin's use > > > > 2) For existing Katello: > > - business as usual, to ensure upgrades and existing clients keep > > working without the need to update their certificate setups > > > > 3) For new installs: > > - treat the same as #1 > > > > > > As far as I know, Foreman today supports Puppet CA and complete custom > > specification of server certificates. Implementation wise, I believe I can > > put a CSR in the right Puppet CA directory to have Puppet sign the > > certificates to then use as the CA for Candlepin. As for entirely custom > > certificates by users - that one is a little trickier and I have not > > figured out how to handle at the moment. > > > > Feedback on this approach would be greatly appreciated (and other ideas). > > > > > > Thanks, > > Eric > > > > > > -- > Eric D. Helms > Red Hat Engineering > Ph.D. Student - North Carolina State University > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >

>
>
>
>> Hi All,
>>
>> As a previous thread mentioned, a number of developers met up a week
>> prior to discuss a variety of topics. One of those was getting to
>> discuss the future of certificates in person which introduced a
>> different approach than previously mentioned. Below is my attempt to
>> distill that approach for discussion. For those that were part of the
>> original discussion, if I have missed anything please correct them.
>>
>> The proposal is to separate "internal communication" from external
>> communication. That is, segment infrastructure communication (e.g. smart
>> proxies, clients) to their own certificate setup and ports from that of
>> external communication (e.g. API, UI). This would work by deploying two
>> ports through two apache vhosts. One port would be the externally facing
>> port (i.e. 443) configured with either Puppet, user supplied or
>> auto-generated CA and server certificates. The other port would be
>> configured with a second auto-generated CA and server certificates. This
>> second CA would also be responsible for issuing certificates for Smart
>> Proxies and clients (other than puppet). See the attached diagram for a
>> general picture of what this would look like.
>>
>> This has the following advantages:
>>
>> 1. Changing the user facing certificates (e.g. those a user imports into
>> their browser or gets from their IT department) is simply a matter of
>> changing out the Apache server certificates and CA
>> 2. Allowing multi-home setups (e.g. two hostnames pointing at the same
>> box)
>> 3. Revoking internal certificates or CA doesn't break the externally
>> facing configuration that a user may have customized
>> 4. Gives us more control over changes to internal infrastructure and
>> certificates such as using FreeIPA to control certificates
>>
>> What about upgrades?
>>
>> To handle upgrades, for one release we could deploy the new port with
>> the internally generated infrastructure certificates and inform users to
>> change over their smart proxy certificates ones generated from the
>> infrastructure CA. This could be done by providing a script to help
>> generate a set of certificates. For Katello users, the current 'default
>> CA' would be moved to the infrastructure port and a new set of
>> certificates (or user provided certificates) would be used on the
>> externally facing port. In all scenarios, communication should be
>> interrupted with a path towards the new design.
>>
>> Thoughts?
>>
>
>
> The main issues which I am seeing downstream are:
>
> 1) Changing hostnames - The machine in installed in a test environment,
> and they want to rename it as part of moving the machine into prod
> environment.
>

This is still a challenge – however if you are wanting to keep an internal
hostname and different external hostname changing the external is a lot
easier.

>
> 2) limiting access tot he webui to certain network segments
>
> 3) limiting the number of ports which need to be opened in firewalls /
> networks.

>
> I think you are addressing 2 which is great. I am concerned that we may be
> sacrificing on (3) however. Is that an accurate concern?

Potentially, but it would only be one well known port that we could
potentially bundle with other things (e.g. repository access on the capsule
which should already be open) and thus not really introducing more. See
Ivan's answer as well on this.

··· On Tue, Oct 20, 2015 at 8:57 AM, Bryan Kearney wrote: > On 10/20/2015 08:38 AM, Eric D Helms wrote:

– bk


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Eric D. Helms
Red Hat Engineering
Ph.D. Student - North Carolina State University

Based on some work and ensuing discussion from this PR (
https://github.com/Katello/puppet-katello/pull/90) I am replying to here
for a broader awareness and discussion. In order to achieve the
infrastructure port concept and with attempting to bring Katello users into
the standard Foreman deployment there is unfortunately issues with which
service is deployed where. I'll try to summarize:

The problem comes with all the various services we can deploy and their
ports, currently that is:

Katello
web server : 443
smart proxy : 9090
pulp : 443
candlepin : 8443
reverse proxy on capsule : 8443
smart proxy capsule : 9090
capsule content + gpg keys: 443

Foreman
web server : 443
smart proxy : 8443

Now if we introduce infrastructure port concept that needs to live at a
particular port as well. If we pick pre-configured ports, then we are
limiting ourselves and risking clashing with other services. Versus if we
picked some unused port range that we deployed all of our services to we
have better control and visibility (IMO). For example (keeping in mind this
is greenfield):

443 - Web Server
9070 - Infrastructure Port (would include reverse proxy on Capsules)
9071 - Smart Proxy
9072 - Pulp
9073 - Candlepin

This of course would raise upgrade concerns as smart proxies are natively
deployed on 8443 and Katello clients are attempting to get content/register
through either 443 on the server or 8443 via the reverse proxy when
isolating via a capsule. One option, if we like the proposed greenfield
deployment configuration, is to support the old and new for a release and
then drop the old style.

I grant you that Katello and it's systems bring in more complication than a
base level install at present. However, the previous arguments towards
having an infrastructure port and isolated certificates for internal
communications as well as standardizing on a port range would allow future
plugins/services a defined and confined setup.

Eric

··· On Wed, Oct 21, 2015 at 10:58 AM, Eric D Helms wrote:

On Tue, Oct 20, 2015 at 8:57 AM, Bryan Kearney bryan.kearney@gmail.com > wrote:

On 10/20/2015 08:38 AM, Eric D Helms wrote:

Hi All,

As a previous thread mentioned, a number of developers met up a week
prior to discuss a variety of topics. One of those was getting to
discuss the future of certificates in person which introduced a
different approach than previously mentioned. Below is my attempt to
distill that approach for discussion. For those that were part of the
original discussion, if I have missed anything please correct them.

The proposal is to separate “internal communication” from external
communication. That is, segment infrastructure communication (e.g. smart
proxies, clients) to their own certificate setup and ports from that of
external communication (e.g. API, UI). This would work by deploying two
ports through two apache vhosts. One port would be the externally facing
port (i.e. 443) configured with either Puppet, user supplied or
auto-generated CA and server certificates. The other port would be
configured with a second auto-generated CA and server certificates. This
second CA would also be responsible for issuing certificates for Smart
Proxies and clients (other than puppet). See the attached diagram for a
general picture of what this would look like.

This has the following advantages:

  1. Changing the user facing certificates (e.g. those a user imports into
    their browser or gets from their IT department) is simply a matter of
    changing out the Apache server certificates and CA
  2. Allowing multi-home setups (e.g. two hostnames pointing at the same
    box)
  3. Revoking internal certificates or CA doesn’t break the externally
    facing configuration that a user may have customized
  4. Gives us more control over changes to internal infrastructure and
    certificates such as using FreeIPA to control certificates

What about upgrades?

To handle upgrades, for one release we could deploy the new port with
the internally generated infrastructure certificates and inform users to
change over their smart proxy certificates ones generated from the
infrastructure CA. This could be done by providing a script to help
generate a set of certificates. For Katello users, the current ‘default
CA’ would be moved to the infrastructure port and a new set of
certificates (or user provided certificates) would be used on the
externally facing port. In all scenarios, communication should be
interrupted with a path towards the new design.

Thoughts?

The main issues which I am seeing downstream are:

  1. Changing hostnames - The machine in installed in a test environment,
    and they want to rename it as part of moving the machine into prod
    environment.

This is still a challenge – however if you are wanting to keep an
internal hostname and different external hostname changing the external is
a lot easier.

  1. limiting access tot he webui to certain network segments

  2. limiting the number of ports which need to be opened in firewalls /
    networks.

I think you are addressing 2 which is great. I am concerned that we may
be sacrificing on (3) however. Is that an accurate concern?

Potentially, but it would only be one well known port that we could
potentially bundle with other things (e.g. repository access on the capsule
which should already be open) and thus not really introducing more. See
Ivan’s answer as well on this.

– bk


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Eric D. Helms
Red Hat Engineering
Ph.D. Student - North Carolina State University


Eric D. Helms
Red Hat Engineering
Ph.D. Student - North Carolina State University