Foreman 1.18 test week

testing
testweek

#1

Foreman 1.18 RC1 has been announced! 1.18.0-RC1 has been released We would like to start a test week, organized manual testing of important scenarios in Foreman. It’s easy to be part of this:

  1. Pick a scenario from this post
  2. Test it with any RC version
  3. If you find an issue, file it in the RedMine and make a comment in this thread linking the issue
  4. Mark the scenario as checked in this OP (this is an editable wiki post), here is syntax of checked and unchecked lines (you can click on checkboxes directly with mouse):
    • Unchecked
    • Checked
    • Checked (alternative)

You can start right away, the ideal timing is from Sunday June 17th June until Friday June 22th when we evaluate how many scenarios were found and do some bug counts. If you test a scenario which is not on the list, please add it. Thank you!

Installation

  • Install Foreman using existing script/forklift/beaker
    • RHEL / CentOS latest stable version
    • Debian stable
    • Ubuntu stable LTS
  • Install Foreman manually by following our installation guide
    • RHEL / CentOS latest stable version
    • Debian stable
    • Ubuntu stable LTS
  • Upgrade existing Foreman deployment (advertise in RedMine it was an upgraded instance if you encounter bug)
    • RHEL / CentOS latest stable version
    • Debian stable
    • Ubuntu stable LTS

Provisioning

  • Bare-metal or virtualized PXE provisioning (host exits build mode and reboots)
    • BIOS host with CentOS
    • UEFI host with CentOS
    • BIOS host with Debian or Ubuntu
    • UEFI host with Debian or Ubuntu
    • BIOS host with Atomic OS
  • Compute Resources (VM is successfully created, finish or cloud-init is executed)
    • Create VMware host (Image Based/Network Based)
    • Create OpenStack host (Image Based)
    • Create Ovirt host (Image Based/Network Based)
    • Create Libvirt host (Image Based/Network Based)
    • Creare AWS host (Image Based)
    • Create GCE host
  • Puppet manifest import (classes are imported, parameters recognized)
  • Puppet configuration (class is assigned to a host, agent performs changes, reports and facts appears correctly)
  • Log in using user from LDAP (user account is created from LDAP)

Foreman Discovery

  • Bare-metal or virtualized provisioning via Provision - Customize Host (host exits build mode and reboots)
    • BIOS with discovery from PXE
    • UEFI with discovery from PXE
    • BIOS with discovery PXE-less
    • UEFI with discovery PXE-less
  • Provision a host via discovery rule
  • Provision a host via Customize UI button
  • Provision a host without hostgroup via Customize UI button
  • Provision a host via hammer via hostgroup
  • Provision a host via hammer via auto provisioning rule

Foreman Bootdisk

  • Bootdisk basic provisioning (host exits build mode and reboots)
    • Full host image
    • Host image
    • Generic image
    • Subnet image

Foreman Ansible

  • Import Roles
    • With/From Smart-Proxy
  • Assign Roles
    • Hostgroup
    • Hosts
  • Play Roles
    • Hostgroup
    • Hosts
  • Run shipped Ansible playbook (job), e.g. to install ansible role from galaxy

Foreman Remote Execution

  • Run some job, e.g. ‘ls /etc’ on a system that was provisioned from Foreman, it should work out of the box
  • Run some job against the Foreman host itself, only key configuration should be needed

Foreman Openscap

  • Create new content file, define a policy, assign it to a host and deploy the foreman_scap_client using puppet
  • Verify ARF report gets uploaded upon foreman_scap_client run and full version of it can be rendered
  • Create tailoring file, assign it to the policy and rerun client with the tailoring file

Foreman Virt Who Configure

  • Create a configuration definition and run it e.g. through REX on some provisioned host. It should succeed as long as it has access to sat tools repo on RHEL, epel (I think) on centos.

Foreman Templates

  • hammer-import templates --lock true # sync newest templates from community-templates repo, see audits
  • mkdir /repo; chown foreman /repo; hammer export-templates --repo /repo # may need setenforce 0

Increasing integration testing
Foreman Community Newsletter - June 2018
Foreman 1.18 test week event
Releases and nightlies stabilization effort
pinned #2

#3

Unable to associate organization with location (regression 1.18)
http://projects.theforeman.org/issues/23918


#4

Page headers not aligned properly, see screenshot in redmine
http://projects.theforeman.org/issues/23920


#5

Set template automatically if it’s the only one when associating with OS
http://projects.theforeman.org/issues/23923

Not necessary a bug, but a feature that bugs me every time I set up new Foreman. I am gonna fix this.


#6

Not sure if I should file a bug but Smart Proxy registered via installer was not in my Organization and Location, therefore DHCP free_ip call silently failed. I believe this is regression, any similar observations someone?


#7

reporting on behalf of Adam, I was also able to reproduce the REX issue that on job invocation page, the result chart is not displayed
http://projects.theforeman.org/issues/23902


#8

Foreman prefers PXEGrub2 UEFI loader for generic host
http://projects.theforeman.org/issues/23930


#9

Make -c (continue) wget flag optional when downloading kernel/initram
https://projects.theforeman.org/issues/20823

@iNecas I start to thing that we should better remove the flag completely. From wget man page: Continue getting a partially-downloaded file. This is useful when you want to finish up a download started by a previous instance of Wget, or by another program. - This only controls to continue file but it has nothing to do with “don’t download if you have up-to-date file” that’s a different option -N. The only limitation is you can’t use -O with -N so we need to rename file back to the original name prior redownloading.


#10

#11

We now have the checklist plugin installed, so you should be able to click on things directly. We (well, @lzap) did discover an issue with checklist items in backticks, so sorry for the mess of edits in the last few minutes, and don’t do that in your own checklists :slight_smile: Enjoy!


#12

It looks like the checklist plugin or discourse can overwrite the wiki with outdated version, be sure to refresh the page prior checking/unchecking. If this turns out to be a problem, I will mark the OP as readonly and moderate checking asking you to post comments with scenarios verified.


#13

Wiki pages should update in real time after another person has changed it (I’ve seen this myself), so old state in that direction shouldn’t be an issue. We can hop on a video chat and test it together if you think that’s not the case.


#14

While it makes sense to me, it probably deserves a separate thread: otherwise, this one will be a complete mess soon.


#15

It already exists: RFC: Drop -c flag from wget on TFTP proxy


#16

#17

Cannot Create OSP compute resource with v2 auth api

https://projects.theforeman.org/issues/23948


#18

Cannot Create OSP compute resource with v3 auth api, error on submit. It is fixed in 1.19 but would highly recommend to backport if possible

https://projects.theforeman.org/issues/23949


#19

the fix has been cherry-picked to 1.18-stable after RC1, therefore it should be fixed in RC2, we’d appreciate if you retest once RC2 is ready (cherry-pick link - https://github.com/theforeman/foreman/pull/5668)


#20

Thank you for participating so far, we are half-way though. Some notes:

  • Katello 3.7 RC1 was released and it’s still some time to test some scenarios
  • VMWare and oVirt CR haven’t been tested yet (these are important)
  • No upgrade test scenarios performed yet
  • Lots of plugins are not yet on the list