I've just set plugin maintainers as "team maintainers" within the
Foreman GitHub organisation for their plugin team (which grants commit
access to the plugin repos). This will let you add new committers as
you like to your own plugin team without an org admin needing to do it.
https://help.github.com/articles/repository-permission-levels-for-an-organization/#team-maintainers
has instructions on how to add people and more info about other
privileges this new feature grants.
In addition, Daniel, Marek and Lukas you've got the same flag on the
general "Members" team so you can invite regular(-ish) contributors to
Foreman itself to become org members for automatic testing of their PRs.