Introducing fp-curl


#1

If you’re like me, you may sometimes want to run a request to a foreman-proxy in production. When you often spin up fresh test instances, you don’t have a consistent hostname. This script wraps curl with the right connection parameters based on the settings.yml.

In theory it should work for Katello as well, even though it’s using different certificates and a different port since it parses the config file, but I haven’t tested it.

I’d be interested if other people think this is useful and if we should include it in the foreman-proxy package (where we already ship some libexec files).


#2

This is awesome, please include it.


#3

That even smells like foreman-maintain subcommand candidate!

I was using a snippet:

# SATELLITE:
# curl --cert /etc/foreman/client_cert.pem --key /etc/foreman/client_key.pem --cacert /etc/foreman/proxy_ca.pem https://       +$(hostname):9090/dhcp; echo

# FOREMAN:
# curl --cert /etc/puppetlabs/puppet/ssl/certs/foreman.nat.lan.pem --key /etc/puppetlabs/puppet/ssl/private_keys/foreman.nat.  +lan.pem --cacert  /etc/puppetlabs/puppet/ssl/certs/ca.pem https://$(hostname):8443/dhcp; echo

# DEVEL:
#PARAMS="-#k --cert $HOME/.puppet/ssl/certs/$(hostname).pem --key $HOME/.puppet/ssl/private_keys/$(hostname).pem --cacert      +$HOME/.puppet/ssl/certs/ca.pem https://$(hostname):8443/${1:-features}"
PARAMS="http://$(hostname):8448/${1:-features}"
(curl $PARAMS | tee | json_reformat) || curl $PARAMS

Love it.


#4

Maybe add json_reformat? But only when the output is a valid JSON, this fails hard if proxy returns a plaintext error… :slight_smile:


#5

Since foreman-maintain is not available on Debian (and I personally don’t use foreman-maintain) I’d prefer for it to live in the foreman-proxy package. That also ensures it’s kept in sync with the codebase. The settings.yml is not an interface and if we ever make changes, it’d be horrible to try and detect it in foreman-maintain. It’s also not aimed at end-users but rather developers who try to debug issues.

That’d be trying to add too much logic into it. There’s an example piping it to https://stedolan.github.io/jq/ which can also do filtering and in general is much better for that. The unix philosophy is the right one here :slight_smile:


#6

I disagree, it is the right place and folks are working on getting it to debian AFAIK. I shared your opinion last year however then I changed my mind and contributed to foreman-maintain for exactly same reason. Consider contributing to the central place for all maintaining tools, folks did a good job of collecting all of our “random” scripts floating everywhere into a single repository and it’s not worth doing exactly the opposite.

@iNecas any insights when we get debian packages?

In this case I think it’s reasonable because proxy sometimes returns JSON (most of the time) and sometimes not. So pipes won’t help much as they will generate output like “Unable to parse JSON…”

With all that said, I think it is useful. Good job.


#7

@mbacovsky is on mission to get foreman-maintain into debian world, so he might know more on when it could be realistic.


#8

For me the biggest reason I wouldn’t support added this to foreman-maintain is f-maintain is a user-facing tool, why would a user use this? How often are they likely to use this? I’ve never seen a user want to interact with the f-proxy API directly.

From the f-maintain readme “foreman_maintain aims to provide various features that helps keeps the Foreman/Satellite up and running.” I’m not saying f-maintains aims or goals cannot evolve over time, but i don’t see this tool as fitting for that purpose.


#9

@sean797 expressed my thoughts more eloquent. Looks like I didn’t even write that part down but was very much the core of my argument against including it in f-maintain.


#10

Getting foreman-maintain on Debian is now a priority for me. Some work towards that goal was already merged there are some existing patches waiting on review but some parts are still completely missing so I have no time estimate at the moment (expect weeks).

As for fp-curl in foreman-maintain for me the important question is what UX that would provide. I can see the benefits of having everything at one place with unified interface, possible checks the proxy is up etc. On the other hand fp-curl /features looks more convenient then foreman-maintain proxy api /features.

It is worth mentioning that the code parsing settings.yaml and extracting the connection details is already in foreman-maintain as we already talk to the proxy are reading features etc. I wanted to suggest to have fp-curl in proxy let it do the hard work and use it in foreman-maintain but as we have to support older Foreman and Satellite versions in foreman-maintain the code will have to remain.

As Sean mentioned foreman-maintain should provide more comfort for the user so I assume we should have the frequent proxy calls predefined and with human readable output e.g. foreman-maintain proxy features. I don’t have problem with foreman-maintain proxy api for arbitrary API call being part of them.


#11

I don’t care that much about where, just please add it to one or another. We can move it later. If I had to chose now, I vote for proxy for the reason @sean797 shared.


#12

Many users are asked by me or RH support to list all leases and reservations from the proxy DHCP API and then compare it to Foreman inventory. This is the only good solution when they run into DHCP conflicts.


#13

Isn’t that what we have the DHCP browser for?


#14

I need to test this plugin, could be a good candidate to put into core. This functionality is really useful, the only concern are customers with many subnets/reservations/leases with A-B class networks. Our DHCP API is not efficient.


#15

Turns out we already have something like this (but slightly different) in the codebase:


#16

Had no idea, this is where foreman-maintain is useful. One place to remember.


#17

Regardless what we end up with, please demo it on community demo, so people know it exists :slight_smile: