New repository for smoke tests

Development
14

So it’s been another few months so time for another status update. It took a while to find time and properly wire everything up, but it’s now integrated in the Foreman Nightly RPM Pipeline. If you click through to the actual job, then you’ll find the actual report.

Currently a failure can be found and this appears to be a legitimate failure on the logout page. Copying here since the output might be rotated later:

Error Message

assert not ['https://pipeline-foreman-server-nightly-centos7.n26.example.com/webpack/foreman-vendor.bundle-v4.0.2-production-ed65...6.example.com/webpack/bundle-2096cee89e99794e7d06.js 0:194011 Uncaught TypeError: Cannot read property 'icon' of null"]

Stacktrace

selenium = <selenium.webdriver.chrome.webdriver.WebDriver (session="77958fae29a66c039021105372fef04a")>
user = User(username='admin', password='changeme', name='Admin User')
url = 'https://pipeline-foreman-server-nightly-centos7.n26.example.com/users/logout'

    @pytest.mark.nondestructive
    @pytest.mark.selenium
    def test_menu_item(selenium, user, url):
        selenium.get(url)
        assert selenium.current_url.endswith('/users/login')
        login_field = selenium.find_element_by_name('login[login]')
        login_field.send_keys(user.username)
        password_field = selenium.find_element_by_name('login[password]')
        password_field.send_keys(user.password)
        password_field.submit()
    
        account_menu = WebDriverWait(selenium, 10).until(
            EC.presence_of_element_located((By.ID, 'account_menu'))
        )
        assert account_menu.text == user.name
        assert selenium.current_url == url
    
        if selenium.name == 'firefox':
            print("Firefox hasn't implemented webdriver logging")
            print("https://github.com/mozilla/geckodriver/issues/284")
    
        logs = selenium.get_log('browser')
>       assert not [x['message'] for x in logs if x.get('level') == 'SEVERE']
E       assert not ['https://pipeline-foreman-server-nightly-centos7.n26.example.com/webpack/foreman-vendor.bundle-v4.0.2-production-ed65...6.example.com/webpack/bundle-2096cee89e99794e7d06.js 0:194011 Uncaught TypeError: Cannot read property 'icon' of null"]

tests/test_pages.py:122: AssertionError

I’ve opened https://github.com/theforeman/smoker/pull/8 to improve the error reporting but this appears to be a legitimate failure.

Currently smoker test failures make a build UNSTABLE which is treated as a success. Once this proves itself, we can make it fail the build. The reporting also needs to be added to the other pipelines, but I already saw that there’s an error in every page for Katello in the current staging repositories.

1 Like