I have mixed feeling about the subject following some tests I’ved made, and took also the weekend to think of it a bit before answering.
A. In order to turn off the response, we need to entangle ourself more with Webrick.
I was unable to access the response settings at all with the current way we implement things.
B. Moving to Puma, means a lot of other work such as supporting scl, packaging will change, and I haven’t researched how to do it, but following a lot of reading about both Rack and Webrick, I think it’s more or less the same as Webrick in that case.
Being more server agnostic means that we need to inject code on runtime based on the server type.
But it can render new issues that I don’t really think we should have.
That’s why I have mixed feelings here.
I dislike the idea that just because of settings we are failing here, but making SP more entangled with a single server I’m not sure is the proper way to go as well.
BTW I didn’t test a standalone Webrick app that is fully configured, because of the above issue.
It’s a single gem without other dependencies [1]. It requires Ruby 2.2, which is why I guess you mention SCL. We could use 3.11.4 (just two months older version) that is compatible with 1.9.3+. According to [2], we wouldn’t lose much. If puma opens a potential to at least test proxy in multithreaded setup, I think it’s a big win already.
We are starting to use SCL right?
So now we can also add higher version of Ruby as well, because we are controlling now the installation.
And now we are at a dependency game that each decision can be taken based on packaging abilities, rather then what arrives from the system itself.
So it’s like a Pandora’s box here, if you start with even a single dependency management using SCL, it’s an opening for everything else.
Should we take that path?
Sorry for being the Devil’s advocate on this discussion
Why he have to move to SCL? If we take 3.11.4, it will run on EL7 without SCL as its Ruby is 2.0.
Let’s say we decide to SCL-ize the proxy, the majority of our packaging already uses SCL, proxy is just a small subset with limited dependencies. How would that be different from core? What problems this brought to core so far?
What’s the difference between packaging our dependencies in SCL and non-SCL? Most of gems don’t have RPMs and we need to build them anyway. See how many things we package today [1]. Most if not all specs has an SCL variant and it’s just a matter of switching configuration how we want to build them AFAIK. I think it would be best to ask @packaging team for their potential concerns.
You always want to be able to update things if there is any security issue, or there are known bugs.
I dislike the idea that the distro handles the Ruby support.
The reason for it is that for example ruby v2.0 is not supported anymore by Matz, so if there is any bug that requires update, we can’t count on the distro to solve it, because they will not have anywhere to take an update, and sometimes there are new implementations at the Ruby level (e.g. rewrite of the GC that happens many times).
That’s actually my point, or at least what I have tried to raise
We either control everything or nothing, we can’t just control Puma for example.
That’s the point I was trying to make.
But my question is about the energy and effort on the matter. What is better here?
M original question is, should we control webrick, because it is less of an effort then starting SCL, even though it will render us single server solution.
But controlling Puma open new issues, for example requests to control unicorn for example or other rack related server.
Why don’t we just open a PR that moves smart-proxy to puma. I think, there are a lot of advantages and this is completely independent from SCLing the proxy. That way we see how much effort this actually is and can test some modules. We can then return to the discussion once we have learned more and have actual experience.
From a packaging perspective I wouldn’t mind SCLing the proxy. We could solve the dynflow situation with _core packages could be solved much cleaner. On the packaging side we’d also simplify a lot of things when we only rely on an SCL for Ruby. For that we’d also need to SCL the installer but nowadays that could be done (why we couldn’t before is a bit off topic here). On the CI side it’d also simplify our test matrices when we can move to Ruby 2.3+ for everything. IMHO SCLing shouldn’t be a blocker for moving to puma.
It’s on me, I’m going to support Puma after the shutdown we are having.
My discussion is that prior of going and doing that, let’s talk about the cost of it, and if it is really needed, something that I still fill we are not talking about.
I feel there is a lot of noise about this RFC but the reality is not that complex. Eric wrote that SCLing is out of table now, but Ewoud is now telling us on the IRC that SCLing “should not be that of a big deal”.
Of course, if we are going to SCL proxy anytime soon we can easily bump to more recent version of webrick which is little bit better.
This RFC would be still valid tho, it’s worth exploring what Puma could give us. The good approach will be evolution and not revolution, therefore some kind of a “tech preview” would be appropriate.
Honestly, I currently don’t have time to work on Puma support, anyone is welcome to take a look on this. I thought it’s good idea.
It’s still a decently sized effort and I never denied that, but as I said: it’s not a blocker. It does need some planning and effort. With the 1.20 merge window coming up in less than a month I’d say it’s more 1.21+ thing.
The PR has reached a point where things are green and testing has passed in a few scenarios. I want to draw attention to @ekohlcomment
I want to ensure we are aligned to the why of the change and that there are no gotchas before we merge it. We’ve moved Foreman server to Puma by default so having both our web services running on the same stack has some advantages but we should ensure the Proxy’s use cases are met with how it spawns services.
Do all the previous issues in this thread still stand for why we should move to Puma?
Are there any other concerns?
Are there benefits we are missing in the move to Puma to bolster the need for the change?
I would like to re-address this RFC as there have been updates since the last post on this. In a way, I would like to smart the discussion fresh from this posting onward as a lot has changed and been discussed since the original post and the post prior to this one. There have been various async discussions through PRs and other communications without a centralized breakdown of the pros and cons of moving the smart-proxy to a Puma based deployment. I think it would benefit us all to have the discussion here and ask the various interested parties to lay out the technical concerns and advantages.
I am going to attempt start (an incomplete list) and maintain a list of the pros and cons and ask that @lzap, @ekohl and others present (even if it’s copy/paste from PRs to centralize the discussion) their view points. My hope is that through some focused discussion we can decide as a community what the right path forward is, and what set of changes are needed to get to the end of that path.
Factors that complicate matters
Some plugins run under HTTP, some under HTTPS and some both
Pros
Single Web Server for the Foreman project, knowledge and investment in Puma benefits Foreman and Smart Proxy
Puma scales better under heavy load than Webrick
Features that may impact this:
Global registration
Smart proxy container registry path
Katello on Foreman - RHSM traffic to the smart proxy
Cons
Performance testing between Puma and Webrick with Smart Proxy does not currently show a difference
This is a change to a main part of the Foreman software stack that has been stable for many years
We can modify the way we implement this by turning on both endpoints by default and having a middleware that enables or disables particular features according to configuration. In the end, HTTP 500 error “This feature is disabled for HTTP” is much better than HTTP 404 “Not found”. How many times did you hit this scratching your head “why the heck it does not find the endpoint”?
I expect it to show the difference once we change Puma to cluster mode, that was not planned in the first phase tho.
Note: Migration to Puma started two years ago when we were suffering from a bug in Webrick 1.3 which we shipped. Newer version required newer Ruby and we were stuck with an old one. This was actually resolved by SCLing proxy about a year ago.
