Is it sufficient to download the Ansible roles from the git repository to the selected Smart Proxy, or should we additionally trigger a task to import the roles from the Smart Proxy post-download?
If we opt to maintain the feature as it stands (only downloading the roles from git), what is the most suitable naming convention? Download / Install / Clone / Import?
What is the acceptable structure for the git repository that contains the Ansible roles? Does a repository contain one role, or multiple roles?
I already had an extensive look at the VCS integration to foreman_ansible by @thorbend, so I want to share my thoughts:
Automatically start the import: Yes, this makes sense to me. I think this can be a follow-up task.
Are you looking for an alternative wording to “Get” in the title? I suggest “Download”; “clone” is just git terminology; “import” (to me) means that you create entities in Foreman (such as products as repositories when importing SUSE content or RH content), and “install” could be associated with “Install X on my hosts”.
I have not tested with anything else but “one repo ↔ one Ansible role” which I consider best pratice. Did you test multiple Ansible roles in a git repo? And if so, do you have a use case for this?
Naming: I think VCS is fine in case someone wants to implement adding an alternative to git. I am also OK with “Git” though.
API & Hammer: Yes, IMHO necessary but again not for 1.0 because it’ll be useable and valuable without.
Documentation: We’re working on it; docs are primarily blocked by merging the PRs in foreman_ansible.
I like the UI a lot. Where do you see rough edges?
I think we should focus on getting in what’s ready and working and then give room for further improvements in follow-up tickets and PRs.
The UI is nice to import one git repository, but this is something I typically do not do. Have one repository for all Ansible code is not good practice in my eyes, clicking roles one by one is not how I like to work (but perhaps some customers). So I would still prefer something that uses requirements.yml to populate the Ansible directory as mentioned initially. If this format seems to technical, such a UI could help as a generator for the format.
I have a few loose thoughts about the whole topic, but first about the questions:
I think the most sense from the user perspective would be if the download gets triggered only once, and then you can use either a rake task, hammer command, button in the UI or being able to attach a schedule with recurring tasks (future versions stuff I guess)
As maximilian already says I too think from the terminology stand point Download makes the most sense
We are currently using ansible-galaxy collection install to get the roles to the Foreman server, so I would have expected this being very similar to that, galaxy repos have a defined repo structure, where the roles, plugins, and such have to be placed, but as well there are galaxy meta files, which can define more stuff like, which roles/collections this collection depends on, we use that to make sure the roles that are used inside the collection are actually installed on the system as well (i.e. role uses ansible.posix.selinux then we add ansible.posix with either a specific version, min version or no version)
Usage of requirements.yaml would also be a possibility, that would be the way how AWX/AAP does it, personally I like the very defined galaxy structure more, but using a more AWX structure, which is more flat would also work (/roles/requirements.yaml, /collections/requierements.yaml and the roles laying in the roles directory is how it’s done by default there, except for playbook local roles which are in the roles subdirectory when the playbook is not in the root)
On the UI side,
I tried to set it up in a forklift instance, it got stuck for a few seconds after entering an URI in the URL box, looks like it isn’t async (or it’s waiting for the missing proxy component)
that you have to add .git at the end of the repo name is not written somewhere up to now, at least git clone and ansible-galaxy collection install also work without that, so was a bit confusing in the first second
Beside that, pretty simple and straight forward UI
Naming, is it actually possible to use a different Source Control software beside Git or will one be supported in the future? Or if this is very reliant on a special repo structure is it good to name it something that doesn’t directly expose that?
Makes sense. We’ll use “Download” for fetching the role to the smart-proxy and “Import” for integrating the role into Foreman.
Currently, having multiple Ansible roles in a git repository won’t function correctly. The role will be downloaded to the smart-proxy and will be available for importing. However, attempting to run it on a host will fail due to differences in file structure (having 2 directories instead of 1).
Regarding the UI, I agree that it looks good. However, I want to ensure it’s intuitive and user-friendly.
Since this feature will be available for all users, we need to ensure it meets our standards. That means we can’t release it without proper documentation, API, and Hammer integration.
Thank you for your feedback! I’ll take your comments into consideration.
Leaving some improvements for the next release is a good idea (such as supporting multiple role structures or adding a subsequent task to also import the downloaded roles).
This current implementation only supports Git, and there are no current plans to support other version control systems. But maybe someday!
After discussing the proposed feature with my colleagues, it appears that utilizing the requirements.yml approach is widely accepted within the Ansible community.
I’d like to understand why the initial direction didn’t align with this approach. Using the requirements.yml file should reduce the amount of code that needs to be maintained, and also provide flexibility to install roles and collections from various SCM platforms and from Galaxy.
One step at a time. We have this feature also on our plate but what to have the basic git role functionality first. One of the next would be to add support of requirements.yaml.
Site note, we started the discussion regarding git roles support some time ago: RFC: Importing Ansible roles from VCS
The question is, why do both? Supporting requirements.yaml inherently involves cloning from Git. I fail to see the necessity of additionally maintaining code for Git integration.
I think the right approach here is to remove the Git integration and implement support for requirements.yaml. The current API implementation heavily relies on Git support. Transitioning to requirements.yaml would require redesigning the API. I believe we can use some of the existing code for this purpose. I appreciate all the hard work, and it seems there has been a misunderstanding here on both sides, but approving code that we know we won’t need in the future doesn’t seem prudent to me.
Furthermore, I wasn’t aware of the initial discussion when I began the PR review. The previous RFC lacked detail and I would expect from you to address Dirk’s comment about requirements.yaml support.
It seems that the acceptable process for contributing to the Foreman project could use some refinement. We’ll definitely work on making it smoother for the future.
One advantage of this approach is, ironically, that it doesn’t require the user to write a requirements.yaml every time they want to add a role to the Smart Proxy.
I agree that using the ansible-galaxy cli would have been the better choice, but as of now that has not been done. I might have implemented this in a few months and then, it’s a different story. Then it would indeed make sense to partially drop this but right now, the options are literally this or nothing.
Also, I must state again that this feature is not very “invasive”. It does not remove any functionality and does not add any new migrations.
While building new features is always nice, the merge needs to always be thoroughly considered.
Adding code that may be removed months from now comes with the risk, it won’t be removed as it’s typically lower priority. The maintainer of the codebase then need to keep it up to date with new version of Ruby, Rails, node.js, webpack etc. The other aspect that needs to be considered here is, once we add some flow that we plan to redo in the future, we’ll have to justify the change in something that users are already used to.
We’ve seen this many times before, so I’d rather be careful. I also think it’s never too late to discuss the desired solution through RFC. I remember we changed the new registration many times even after months of development and it was for the good.
My biggest worry is the long term supportability. The whole API (in the current PRs) is built around cloning, which is a git operation. If the API was designed on ensuring certain roles are present (and the cloning is just an implementation detail), then it may be possible to migrate to requirements.yml long term.
Another thing to keep in mind is that Ansible is moving to Execution Environments. Possibly with RHEL 10 they already will drop the guarantee that Ansible on the controller can talk to any OS. So you’ll need an execution environment for each target. Note that execution-environment.yml refers to a requirements.yml for its dependencies. What this means for Foreman is unclear to me at this point.
This is something I realized while reviewing. The current implementation is one where we’re pushing us to take on a maintenance burden that long term is going in the opposite direction of where Ansible is heading.
Was wondering if Execution Environments are just an Automation Plattform / AWX thing as the link points to this documentation, but it is just the link you choose. Documentation of Ansible points also to Getting started with Execution Environment, so thanks for the hint. I think I will have a good read on this as I was not aware of it yet!
