Using Travis-CI

lzap · February 15, 2018, 3:14pm

I am wondering why aren’t we using Travis-CI for Ruby codebase in Foreman core. I’ve successfully used it for few of my gems and they seem to be free for open-source projects. I could not find out any limits they enforce for projects in free tier, but so far all testing was blazing fast.

It looks like we started using it when Node.JS was introduced, have we considered moving more things in there? If Foreman Core turns to be problematic, maybe we can consider moving other projects there. Smart Proxy comes to my mind first.

ekohl · February 15, 2018, 3:30pm

One issue is that you can’t use two languages in Travis. This means the configuration would be very complex. It’d also heavily overload the Travis free tier. When we do a few PRs to our puppet modules we can already feel the rate limiting and that’s nothing compared to what we test in foreman core.

Our team is preparing a proposal to update our testing infrastructure. This should allow us to modernize our testing while also being able to use a sponsor that only provides a Kubernetes cluster.

https://ci.theforeman.org/blue/organizations/jenkins/katello-pr-test/ is an example of how improved jobs can look. This is IMHO more powerful than what Travis can offer us now.

ehelms · February 16, 2018, 12:49am

@lzap I am curious what the underlying reasons you are thinking about the use of Travis are. I read this as there are things about our current CI that are insufficient. For example, I see you note blazing speed which I translate to our CI is slow (and in some cases, it is!). As mentioned, a team of us are working on building our a set of improvements to make to CI. Any information you can share about issues you see, or wish list items would help us to build our list. I will be sending out an initial RFC once we’ve narrowed our set of items for futher feedback.

lzap · February 16, 2018, 1:29pm

Nothing in particular really, I pushed to a (different) project and it was working nice, was wondering what the reasons were. Two languages is really big issue for us.

ekohl · February 16, 2018, 1:59pm

https://github.com/travis-ci/travis-ci/issues/4090

ehelms · February 16, 2018, 6:23pm

You can do multiple languages in some cases, it’s not necessarily as pretty as it could be and you can’t easily do intricate matrices:

github.com

theforeman/forklift/blob/master/.travis.yml

sudo: false
language: python
python:
  - 2.7
install:
  - |
    if git diff origin/master --name-only | grep -q "lib\|test\|boxes.d"; then
      rvm install 2.2.4
    fi

  - |
    if git diff origin/master --name-only | grep -q "playbooks\|roles\|containers/roles"; then
      pip install ansible-lint
    fi
script:
  - |
    if git diff origin/master --name-only | grep -q "lib\|test\|boxes.d"; then
      bundle install
      bundle exec rake
    fi

This file has been truncated. show original

Dmitri_Dolguikh · February 16, 2018, 9:48pm

We can probably keep Jenkins for more complicated (including multi-language) environments, and use travis for simpler projects (plugins and extensions?). For example, pretty much all smart-proxy modules and providers can be tested via travis, but quite a few, even pretty trivial ones use jenkins.

Probably would reduce the load on jenkins a little bit too?