Sure, but I see those as an oddness in the main repo. If you go that route, you could argue you need all plugins in the same repo so you can fix any related issues. However, some plugins live outside our organization and might even be internal to a company. We can always change this, but I’d like to avoid designing the perfect solution holding us back from implementing a good solution (even though I’ve been probably guilty of doing the same in the past).
The way I’m thinking about it now is that we need it to be run continuously because that’s the only way it’s useful. Currently we have 3 pipelines in forklift that we run: Foreman (without plugins), Katello and luna.
At this point I’d say that we can also add additional pipelines with other plugins. The current limitation is that they need to be packaged. Ideally also part of the installer because it has all the logic for package names in various distros and versions.
Ideally we’d have tests for all plugins that are installable via the installer.
Currently the pipelines I mentioned report to https://community.theforeman.org/c/infra and there we would notice if tests break. Fixing is a harder question. Shared responsibility is always hard, but in this case it will be. Ideally speaking the maintainer would notice it, but perhaps we need a ping. From there on we can decide how to fix it (as with all bugs). If no solution can be found and it’s in a plugin, it can be a reason to drop the plugin altogether from the installer and packaging.
I was more thinking of a meat smoker than the person who does the smoking.