Many open source projects are adopting guidelines on how to use AI for the benefit of their communities. With this RFC, I’d like to start a conversation about whether the Foreman community could benefit from an AI usage policy, and if so, what that policy could look like.
Proposal
I’d like to center the conversation around two main areas:
If the Foreman community adopts an AI policy, what principles should it be based upon?
If the Foreman community adopts an AI policy, what form should it take?
Looking around and taking inspiration from other communities and conversations, there are many options.
When it comes to principles, people consider licensing, fair use, security aspects of AI-based contributions, environmental impact, transparency, and many others.
However, I’d like to hear from you who contribute to the Foreman/Katello project. What do you consider important? What is your priority? What do you think would be the best fit for Foreman?
Alternative Designs
N/A
Decision Outcome
TBD
Impacts
The expected (but still only potential) impact is to agree upon, develop, and then merge a document or statement summarizing the Foreman project’s approach to AI-assisted or AI-generated contributions.
NOTE: This post was not AI-generated or AI-assisted
I’ll start with my own motivation: My priority is to encourage responsible use of AI. On that note, I appreciated Fedora’s call to “Embrace your human side”, which suggests to avoid using AI for minor items like polishing one’s own messages (AI policy in Fedora - WIP - Fedora Discussion).
A significant part of responsible use of AI is also allowing each other to learn from our experiments and mistakes, which is where disclosing whether AI was used for a contribution comes in. This is because if I learn that another contributor has used AI and I can learn from his experience, there is a chance I’ll be able to build on that experience rather than run the same experiments and make the same mistakes on my own.
Also, I believe the goal is not and should not be to discourage AI usage for contributions. If anyone discloses that their PR was AI-generated or AI-assisted, that contribution should be fairly reviewed by the other members of the community, just like any other contribution.
Thanks for putting some structured thought into this.
As someone who is slow to adopt LLM assistance, I don’t have much to contribute on the details, but I do stand to benefit disproportionately from in your words: “allowing each other to learn from our experiments and mistakes”. So thanks!
I also want to second the “just like any other contribution.” point. If the contribution provides good work, who am I to complain that an LLM was involved. If there are issues with the contribution, then that is what reviewers should say in their review.
As someone that is/was skeptical about AI/LLMs usage, I’m starting to use more and more in the agentic approach.
For assisted tasks/PR I have feeling that we need to be explicit on requesting that contributors state in the commit message/signature that the code was written with help of one model/llm. I’m also in favor of us adding AI definitions in the projects, to ensure that IDEs that auto enable models understand that we have rules for contributions that are AI based.
This is excellent and relevant reading. I’d definitely keep the policy very open. It should rather focus on recommendations, one of them being to denote a contribution generated or semi-generated with AI. Code completion is IMHO not worth tagging in any way.
Thanks to everyone who has chimed in to the conversation so far!
There are a few major themes I’m seeing here:
General interest in facilitating knowledge sharing over how AI is used
Preference in not being prescriptive (or restrictive) about the use of AI, which should stay a matter of choice
Disclosing the usage of AI helps preserve clarity and community trust
However, when it comes to only requiring disclosure for contributions that use AI in a “non-trivial” way, that obviously raises an important question: Where to draw the line between “trivial” and “substantial”? Having AI generate code is surely substantial. Should AI auto-completion, on the other hand, be considered trivial? What about querying an LLM for ideas during the brainstorming phase of a PR, before you start writing the code?
If we don’t answer this question, we might also end up with basically every PR being marked as AI-assisted, and that would just generate unhelpful noise.
Where I’m going with this is that it might indeed be beneficial to agree on a formal policy document that will explain these things, and more. Which brings us back to the question of where to store that policy along with how to ensure contributors are aware of it and have interest in following it to the best of their ability.
In all of these, AI was used to summarize existing documentation to produce introductions to various sections in our documentation.
One useful thing that I’ve observed is that if as a reviewer I am aware of the fact that a PR is AI-assisted, and then during the review I notice a pattern, I can ask the author to adjust their prompt before performing a detailed line-by-line review.
Also, sharing the prompt (or its summary) can show a reviewer what the author considered and what the priorities for the PR were. Again, then the review can start with discussing the prompt and perhaps tweaking it, before a detailed line-by-line review.
In my mind, this further supports the benefits of disclosing AI usage.
The Red Hat AI policy suggests using the identifying the commit with Assisted-By: <tool> or Generated-By: <tool>. The easiest way is git commit --trailer 'Assisted-By: $tool. I think this is a nice way.
My personal interpretation is that it’s generated if no human was involved in writing while assisting means there was some manual modification.
I like that one as well. I think it would be nice to have some common policy across the Foreman eco system.
We recently saw some PRs where even the comments in the PR looked AI generated - I would also like to see something like that being marked as AI generated.
I don’t mind using AI at all, I think it is a useful tool but I also like to see the origin of the code/text I am working with.
At the risk of oversimplifying things, as a reviewer/maintainer I don’t really need to know how the patch was produced. I don’t need to know if it was written by the person or by a LLM as long as the person submitting that patch takes ownership of it. The author should still understand the patch and be able to explain every single thing in the patch when asked about it.
I’ve seen this a couple of times and it made me sad every time. In my book, if the person who submits the patch acts as a transparent proxy between the reviewer and a LLM, then that’s not ownership. In this case, I’d vote for much stricter measures.
I agree to that but I am not sure if it is that easy in terms of licensing though I am sure no expert in that area.
Personally, I also see transparency as an important issue here. It is not only the origin of the code but it also helps us to evaluate the tools we are using. E.g. some AI bot generates better results, more precise PR descriptions, etc.
As a maintainer I don’t just need people to take ownership. I also need to be able to tell that they have taken ownership before sinking scarce review resources into the PR. Asking contributors to proactively explain to what extent the PR is LLM generated or assisted, contains some information about the extent to which they have taken ownership. My 2 ct.
I’d like to share a practical perspective on this. Not all maintainers seem to have the same view on AI co-authoring, and in my experience, contributions that involve AI assistance can sometimes be met with skepticism or even dismissed, regardless of their technical merit.
One thing I’d like to highlight is a use case that often gets overlooked: as a non-native English speaker, I use AI to translate and refine PR descriptions and comments from my native language (Czech). This lets me write much more detailed and precise explanations than I could manage in English on my own, which ultimately benefits everyone involved in the review. Without this, my contributions would either be shorter and less clear, or I’d spend a disproportionate amount of time just on the language side rather than on the actual code.
For newer contributors, AI assistance can significantly lower the barrier to participating in a project as large and complex as Foreman. And I think it’s important to acknowledge that even with AI help, contributing is far from effortless - if someone is genuinely engaged and not just acting as a proxy for an LLM, the process still takes considerable time and effort that they’re dedicating to the community and the project. That time is just as valuable as anyone else’s. Maintainers play a crucial role here too - they help newcomers catch all the details, and through their expertise, a PR goes through many improvements and changes, even if it sometimes drifts from the original idea. That collaborative process is valuable regardless of how the initial draft came about.
Several of my PRs have already been approved and merged, which I appreciate, because some of them addressed bugs that were directly broken for me - blocking my work with a production Foreman deployment. Without AI assistance, I realistically wouldn’t have been able to contribute those fixes myself, and nobody else would have implemented them in that timeframe either - so the production system would have remained broken.
I think it’s worth considering that AI-assisted communication is not the same as AI-generated code, and a blanket negative attitude toward any AI involvement could unintentionally raise the barrier for non-native speakers and newcomers who are trying to contribute meaningfully.
Thanks for your perspective @jakduch I didn’t think of the language barrier this way. There’s a difference between polishing message and translating it from one language to another. We definitely want to avoid policies which would create barriers to contributors. Given your recent contribution experience, you may be interested in this thread too RFC: Agentic Workflow Practices for the Community - #4 by Marek_Hulan
