Even though I don’t see connection stacking issue in my environment, slow-header tests definitely show service impact for SmartProxy that may explain some instability of the latter in our production (all of the sudden I see that no responses coming back from foreman-proxy yet the process is running - I think I posted a question about that in a support forum at some point back).
Also, increasing performance 3x is a definite and a big “yes” factor, so I’d love to run Puma under our production load and see how things are. Are there any (even preliminary) docs describing a process of SmartProxy migration to Puma?
Well, yes, but I meant a doc that may have actual steps like those that you had to do to perform testing of smart-proxy under Puma vs Webrick. So unfamiliar with Puma like myself could try it before the deployment procedure is fully automated (under installer or whatever). Again, if there is such a doc already.
I have mixed feeling about the subject following some tests I’ved made, and took also the weekend to think of it a bit before answering.
A. In order to turn off the response, we need to entangle ourself more with Webrick.
I was unable to access the response settings at all with the current way we implement things.
B. Moving to Puma, means a lot of other work such as supporting scl, packaging will change, and I haven’t researched how to do it, but following a lot of reading about both Rack and Webrick, I think it’s more or less the same as Webrick in that case.
Being more server agnostic means that we need to inject code on runtime based on the server type.
But it can render new issues that I don’t really think we should have.
That’s why I have mixed feelings here.
I dislike the idea that just because of settings we are failing here, but making SP more entangled with a single server I’m not sure is the proper way to go as well.
BTW I didn’t test a standalone Webrick app that is fully configured, because of the above issue.
It’s a single gem without other dependencies . It requires Ruby 2.2, which is why I guess you mention SCL. We could use 3.11.4 (just two months older version) that is compatible with 1.9.3+. According to , we wouldn’t lose much. If puma opens a potential to at least test proxy in multithreaded setup, I think it’s a big win already.
We are starting to use SCL right?
So now we can also add higher version of Ruby as well, because we are controlling now the installation.
And now we are at a dependency game that each decision can be taken based on packaging abilities, rather then what arrives from the system itself.
So it’s like a Pandora’s box here, if you start with even a single dependency management using SCL, it’s an opening for everything else.
Should we take that path?
Sorry for being the Devil’s advocate on this discussion
Why he have to move to SCL? If we take 3.11.4, it will run on EL7 without SCL as its Ruby is 2.0.
Let’s say we decide to SCL-ize the proxy, the majority of our packaging already uses SCL, proxy is just a small subset with limited dependencies. How would that be different from core? What problems this brought to core so far?
What’s the difference between packaging our dependencies in SCL and non-SCL? Most of gems don’t have RPMs and we need to build them anyway. See how many things we package today . Most if not all specs has an SCL variant and it’s just a matter of switching configuration how we want to build them AFAIK. I think it would be best to ask @packaging team for their potential concerns.
You always want to be able to update things if there is any security issue, or there are known bugs.
I dislike the idea that the distro handles the Ruby support.
The reason for it is that for example ruby v2.0 is not supported anymore by Matz, so if there is any bug that requires update, we can’t count on the distro to solve it, because they will not have anywhere to take an update, and sometimes there are new implementations at the Ruby level (e.g. rewrite of the GC that happens many times).
That’s actually my point, or at least what I have tried to raise
We either control everything or nothing, we can’t just control Puma for example.
That’s the point I was trying to make.
But my question is about the energy and effort on the matter. What is better here?
M original question is, should we control webrick, because it is less of an effort then starting SCL, even though it will render us single server solution.
But controlling Puma open new issues, for example requests to control unicorn for example or other rack related server.
Why don’t we just open a PR that moves smart-proxy to puma. I think, there are a lot of advantages and this is completely independent from SCLing the proxy. That way we see how much effort this actually is and can test some modules. We can then return to the discussion once we have learned more and have actual experience.
From a packaging perspective I wouldn’t mind SCLing the proxy. We could solve the dynflow situation with _core packages could be solved much cleaner. On the packaging side we’d also simplify a lot of things when we only rely on an SCL for Ruby. For that we’d also need to SCL the installer but nowadays that could be done (why we couldn’t before is a bit off topic here). On the CI side it’d also simplify our test matrices when we can move to Ruby 2.3+ for everything. IMHO SCLing shouldn’t be a blocker for moving to puma.
I feel there is a lot of noise about this RFC but the reality is not that complex. Eric wrote that SCLing is out of table now, but Ewoud is now telling us on the IRC that SCLing “should not be that of a big deal”.
Of course, if we are going to SCL proxy anytime soon we can easily bump to more recent version of webrick which is little bit better.
This RFC would be still valid tho, it’s worth exploring what Puma could give us. The good approach will be evolution and not revolution, therefore some kind of a “tech preview” would be appropriate.
Honestly, I currently don’t have time to work on Puma support, anyone is welcome to take a look on this. I thought it’s good idea.
It’s still a decently sized effort and I never denied that, but as I said: it’s not a blocker. It does need some planning and effort. With the 1.20 merge window coming up in less than a month I’d say it’s more 1.21+ thing.