Exposing capabilities in the smart proxy


#1

This is still very early and it still needs to take shape.

The problem I want to solve is that various providers can should be able to implement only a subset of the functionality. A concrete example is DNS:

Some DNS backends can fully manage domains (Create, Read, Update & Delete) while others can’t. In the present situation we would implement it and providers that can’t should return an error. My proposal is to expand the features API endpoint to expose more information, such as a list of capabilities. If Foreman sees MANAGE_DOMAIN then it knows it can enable additional functionality in the Create Domain process, namely on which smart proxy to actually create it. If the proxy doesn’t have the capability, it can warn the user it needs to happen manually.

I have written a very small proof of concept:

https://github.com/theforeman/smart-proxy/pull/585

This will also need modifications in Foreman to actually take advantage of this and I don’t know how that would actually look. Perhaps this is a RFRFC: Request For Request For Comments. Please share your thoughts on this.


Setup of a smart proxy for a private network
Also-notify in custom-hiera?
#2

I had some more time to work on this to be a bit more concrete. This is about smart proxy registration in Foreman.

Current

The smart-proxy exposes a GET /features API endpoint returning a simple list of features:

[
  "dns",
  "logs"
]

On registration (or save actually) Foreman calls this API and stores the features (code). It does this in a has_and_belongs_to_many :features relation. This means we have table smart_proxies and a table features. These are joined via a table with 2 foreign keys.

Problem

It is impossible to know what a feature actually supports.

My concrete use case is that I want to enable domain CRUD management to the smart proxy DNS. I could implement a dnsv2 feature, but not every DNS backend can actually support it. That means we now need to choose between feature dnsv2 and dns.

Another is various DNS types. The libvirt DNS provider doesn’t support PTR records. Currently it’s implemented as a noop but lying like this isn’t very nice. As we want to add more records this could only get more complicated. If we are exposing a UI to create SRV or CNAME records, then it would be nice if we could statically determine that the field should be disabled with a message telling the user it’s unsupported.

Pulp is another application where content types (RPM, Deb, Puppet, OS tree) are pluggable. Capabilities could be used to tell Katello which UI options to enable.

Proposal

Smart proxy side

I’d propose we add a new GET /v2/features which changes to a mapping. Every feature is a mapping:

{
  "dns": {
    "capabilities": [
      "TYPE_A",
      "TYPE_AAAA",
      "TYPE_CNAME",
      "TYPE_PTR",
      "TYPE_SRV"
    ],
    "settings": {}
  },
  "tftp": {
    "capabilities": [],
    "settings": {
      "tftp_servername": null
    }
  },
  "logs": {
    "capabilities": [],
    "settings": {}
  }
}

capabilities

A simple list of strings that are flags. Intended module DSL:

module Proxy::Dns
  class Plugin < ::Proxy::Plugin
    # ...
    capability :TYPE_A
    capability :TYPE_AAAA
    capability :TYPE_CNAME
    capability :TYPE_PTR
    capability :TYPE_SRV
    # ...
  end
end

settings

A mapping of settings that every module can expose. Intended module DSL:

module Proxy::TFTP
  class Plugin < ::Proxy::Plugin
    expose_setting :tftp_servername
  end
end

The benefit of this was not in my initial idea but it turns out TFTP exposes the tftp_servername setting via GET /tftp/serverName. The pulp plugin exposes GET /status/puppet where it exposes the puppet_content_dir setting.

Formalizing this in the features API means we can create a unified API within Foreman to retrieve this. We also have fewer REST endpoints.

There is a difference that this is currently queried live while the features API is typically only called at registration. This can be a benefit in performance and reliability at the cost of some flexibility.

Foreman side

Registration

The registration wouldn’t be very different on a conceptual level.

First we get the v2 features API. If this returns a 404 we fall back to the v1 API. We’d assume no capabilities and settings are present.

To store capabilities and settings we change the join table to a real table. In rails terms a has_many :features, :through => :smart_proxy_features (I think).

The SmartProxyFeature model would have:

  • id Primary Key
  • smart_proxy Foreign Key
  • feature Foreign Key
  • capabilities Array (serializing to a string in sqlite/mysql, text[] in postgresql)
  • settings store/hash (serialized to a JSON string in sqlite/mysql, jsonb in postgresql)

Model functions

We’d introduce some helpers. Don’t pin me on these names:

  • feature_capability?(feature, capability)
  • feature_setting(feature, setting)

#3

With exposing settings we should thing about security considerations. Should we make it an authenticated end point?

In @sean797’s proxy HA proposal we also talked about the HA mode (apply operations to one or all proxies). We can either model this as a capability of expose it as another per-feature option so Foreman doesn’t have to hard code HA behavior per feature. This new features end point provides the flexibility. It can even enable per-provider behavior.


#4

Nicely done, I have few comments.

First, as I already said in the WIP POC PR, I don’t like mixed case for features (lower case) and capabilities (upper case). I would like to standardize on one or another, preferably lower case. These are all symbols, not constants and in Ruby world these has always been lower case.

Second, as we can see it was not good idea to return array as root element in JSON because it is not extendable. Your proposal is extendable, but it assumes the result is always hash of features. When we make decision to transform the endpoint into something more generic, it will be confusing output. Therefore I suggest to create wrapper - a root hash with key “features”.

If we make it authenticated, let’s add more info about proxy and rename it to something more generic like /status. It could return basic information about proxy - version, uptime (of smart-proxy process not OS) and one thing I was always wanting to show in foreman UI - current settings (the yaml tree).


#5

You are talking about the Ruby DSL, not the API? I could see them as lower case in the DSL but in the REST API I think uppercasing them is cleaner since there they are just strings instead of symbols.

I could see how that would make sense. We can bikeshed over status vs info vs something else. I have also thought about exposing which provider is used for debugging purposes.


#6

What kind of errors is it throwing now? If the advantage for the capabilities API is that Foreman can warn users that the proxy is unable to do something, would it be simpler to capture the errors the proxy throws and warn accordingly?


#7

One other idea - how about coming up with a simple API for plugins so they could add their own elements into the status/features call? For example puppet could return puppet master version.


#8

IMHO it should just represent the state of the proxy. I don’t like reaching out to backend services because that could make it slow or unreliable. We also already have a setting puppet_version in the puppet_proxy module and expose_setting will already do what you want.

The current status is that there are 2 WIP PRs:
https://github.com/theforeman/smart-proxy/pull/585
https://github.com/theforeman/foreman/pull/5954

There are some things that need to be done:

  • Foreman has data models to store the capabilities and settings, but the code currently doesn’t do this.
  • Tests
  • Documentation
  • Examples on how to use capabilities

Another thing @Marek_Hulan suggested was to report whether a module is enabled/disabled. The proxy will start up if any module fails to load. Currently that means if you register/refresh features you can silently disable features you intended to enable. Currently the installer has code to detect a mismatch between requested modules and actual registered modules but by detecting disabled modules we can also do this on manual registrations/refreshes. In the v1 features there was no way to report this, but the new datastructure does allow it.

Currently I don’t have a lot of time to work on this, but IMHO this should make it in for 1.20. If there’s anyone who would like to help me to complete the feature, I’d highly appreciate it.


#9

I’d like to raise awareness that I have taken over this effort from ewoud and opened two new prs here:


These build on his PRs, but add tests, and address various comments. In addition it allows for ‘dynamic’ capabilities that are calculated at run time instead of proxy boot time.


#10

It’s on my radar Justin. Will try hard to squeeze it into the last week of the year but don’t bet on it.


Foreman 1.22 schedule and planning