As part of the upgrade to Rails 6, we found out that the latest version of SQLite shipped in EL7 is not compatible with Rails 6. This means that if we want to continue supporting it, we would need to ship our own packages for SQLite in the Foreman SCL.
SQLite is only needed for build tasks (specifically, apipie cache and asset compilation) that require a functional Rails environment. Considering these tasks don’t actually need a working database, just the ability for the application to start, we started looking at possible alternatives.
The leading candidate is the activerecord-nulldb-adapter gem, which basically converts all DB actions into no-ops. This has proven sufficient for the build tasks, with the nice benefit of speeding them up a bit as migrations are not needed. @ehelms has opened several PRs introducing NullDB in Foreman to make sure this approach is working:
Once these are merged and we verify we are able to generate working builds with them, I would like to fully drop SQLite support from Foreman (which is currently only supported for build and development).
This will give us two advantages:
Drop SQLite testing from all the test matrices, freeing up jenkins worker capacity for other tasks (e.g. adding ruby 2.7 tests).
Drop any workarounds in our code needed for making sqlite functional, and allow us to fully take advantage of advanced PostgreSQL capabilities.
SQLite will still be used by the smart proxy for dynflow persistence, as there we don’t have the requirement of upgrading it to support Rails 6.
What does this mean for you?
If you are developing with sqlite, you will need to set up PG and migrate your development environment to it. This should be a fairly simple one-time effort, that will give you the added benefit of running closer to production environment in your development setup.
If you have a plugin with CI testing running outside Jenkins (e.g. on travis) that rely on sqlite, you will need to either convert them to use PG or migrate them to Jenkins (which already provides templates for running plugin PR and build tests).
Please let me know if you have any concerns with this proposed approach - the goal is to finish implementing it in time for 2.1 so that we can upgrade to Rails 6 in this release.
Testing has found there is one place where apipie does need a migrated database, which is for displaying field types for search fields. However, this isn’t used anywhere for validation and is only informational - and in any case the search parameters are sent in the url params as a string, so it does not really matter if the field is defined as integer, string or text.
This was introduced in Feature #17964: Extend apidoc with list of fields to use in a search - Foreman which only requested a list of searchable fields. I suggest removing the field type from the apipie docs, which would hopefully allow us to generate identical docs with nulldb. Alternatively, we could allow users to manually regenerate it with types if they want but only ship docs with empty types where we can’t determine the field type from the scoped search definition.
I see how this could be perceived as a breaking change since we published these in our docs. But given the server never validated these they were simply “incorrect” if a client was creating some sort of validation layer. That would further imply any change to a parameters type would be a breaking change as well but we are not versioning our API based upon this.
I think it’s reasonable to either drop this from the docs all together or to report them all as strings.
Please do, even if all you do is copy this particular post to a thread to continue discussion. I would love to better understand if its benefits of Github Actions due to their functionality or just gaps in our Jenkins CI; and discuss further.
I believe this falls in the category of breaking changes that are reasonable to do in a regular release with a note in the release notes. The alternative is to continue relying on sqlite for the build process, which means we can’t upgrade to Rails 6 until we package our own newer version of sqlite.