Hammer oauth login with Keycloak

I do have configured Foreman / httpd to use external auth against a Keycloak server.
The login in the WebUI works fine, however I can’t get Hammer to work with this.

I’ve tried both flows (auth code and password grant), however keycloak always complains about CODE_TO_TOKEN_ERROR or LOGIN_ERROR (see log lines below) and prevents the login.
Foreman itself receives an empty token afterwards (in /usr/share/foreman/app/services/jwt_token.rb:48).

Keycloak and httpd config was generated using keycloak-httpd-client-install and modified accordingly.

Expected outcome:

Login succeeds

Foreman and Proxy versions:

Foreman 2.3
Hammer 2.5.0
Hammer-Foreman 2.5.0

Foreman and Proxy plugin versions:

Distribution and version:
Server: RHEL 7.9
Client: MacOS 11.6.1 with Ruby 2.7.2

Other relevant data:

keycloak-0 keycloak 08:18:04,102 WARN  [org.keycloak.events] (default task-1) type=CODE_TO_TOKEN_ERROR, realmId=demo, clientId=foreman-release, userId=null, ipAddress=, error=invalid_client_credentials, grant_type=authorization_code
keycloak-0 keycloak 09:02:02,751 WARN  [org.keycloak.events] (default task-1) type=LOGIN_ERROR, realmId=demo, clientId=foreman-release, userId=null, ipAddress=, error=invalid_client_credentials, grant_type=password

Hi @laugmanuel ,
I have not run into similar problems, but have you tried using Hammer version 2.3.0 instead of 2.5.0? As far as I know, the hammer version corresponds to the Foreman version.

1 Like

Hi @nadjaheitmann,
I’ve also tried hammer 2.3.0 and 3.1.0 just to be sure - none of the versions work.
Do you have a working keycloak and foreman instance and can test this?

I will dig deeper into the Hamemr oauth code and see what requests are sent

Unfortunately, I don’t have a keycloak instance to test against :frowning:

I’ve looked at the code and found the following:

HammerCLIForeman::OpenidConnect connects to the IDP (Keycloak in my case) to exchange the authorization_code for a valid JWT token. This token would be passed to Foreman afterwards.
However, the request to the IDP fails here: https://github.com/theforeman/hammer-cli-foreman/blob/master/lib/hammer_cli_foreman/openid_connect.rb#L45-L53 with

[18] pry(#<HammerCLIForeman::OpenidConnect>)> response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |https|
=> #<Net::HTTPUnauthorized 401 Unauthorized readbody=true>

[19] pry(#<HammerCLIForeman::OpenidConnect>)> response.body
=> "{\"error\":\"unauthorized_client\",\"error_description\":\"Client secret not provided in request\"}"

which makes sense as the client_secret should be private and only present on the foreman server (mod_auth_openidc module configuration) if the access type is set to confidential (see screenshot):


This means:

  • If used with confidential a client would need to send a valid authorization_code, client_id AND client_secret in order to exchange the authorization_code for a valid token
  • If used with public a client would only need to send the authorization_code and the client_id to get the token

And I can confirm, that both ways work if I either set access type to public or provide the client_secret as part of the hammer request.

Based on that, I have two questions/proposals:

  • Hammer should display such failed requests to the IDP to the user. If the response body is show, it is quite obvious where the problem lays
  • if access type needs to be configured to public we should document this as keycloak-httpd-client-install does not provide the needed config options. However, I still need to look into the implications of setting the access type to public!

@mcorr do you know anyone who might be able to help with this?

There are a lot of other CLI tools which do the auth using OAuth differently and start a local callback-server and then let the user open the auth-url manually in the browser; e.g.:
Hashicorp Vault and Azure CLI

1 Like

@laugmanuel I’ll try to find someone.

1 Like

@ofedoren or @ezr-ondrej can you help here in any way? Not sure if you were involved in thias. If I’m not mistaken, our hammer keycloak support was designed the same way as Manuel describes, I think Rahul demoed it at some point.

Rahul was the one who used to answer and troubleshoot these types of questions, and he has been good to come back and answer others since his departure from the Foreman dev crew…

This seems to describe the whole thing - stackconf online 2020 | Securing Infrastructure with Keycloak by Rahul Bajaj - YouTube at the end you see the whole process. Hammer prints the auth url that user needs to manually open. There you get the code that must be manually put to hammer so it can obtain the token, if I got it right, there’s no callback server involved in the case of hammer. Around 22:50 he also describes the public and confidential access type.

1 Like

Thanks @Marek_Hulan for the link. I had a look and it confirms my findings in post#5..
However, I’m not sure if the general gist of this is correct, because we do have a server (Foreman/Httpd) which can keep the client_secret safe and support auth that way. That’s exactly what e.g. Hashicorp Vault does.
From my understanding, a public client would only be required if there were no such server side component (e.g. mobile app without persistent connection to some appserver, …).

The steps Vault uses are the following ones:

  • client connects to the app (Vault) to retrieve the auth url and modifies the redirect uri to point to the local callback server
  • client starts a local callback server (listening only on localhost)
  • client prompts user to open the auth url
  • user opens the browser and logs in using the configured credentials
  • automatic redirect to local callback server happens
  • client parses the response and sends the required data to the app (Vault)

So what I’m trying to say is: In my perspective hammer is a client but not a a client-side application in relation to OAuth/OpenID Connect.
Same goes for WebUI: The browser is just a client using the server side application; not a client-side application (even though it’s running on a client device).

Well, your findings are correct, sure. But login via hammer does work through Keycloack server, you just need to register hammer as a public client. That’s how support for this was implemented in hammer, since hammer can be installed on any machine (it shouldn’t be always “a part” of Foreman server) and we don’t have a decent/supported by hammer way of storing this secret.

Sorry if I missed what you’re asking help with: whether it’s more about documentation, “Confidential” access type support or a better error message?

1 Like