Does anyone have experience with full audit of an API and not just activerecord changes? I’m adding the docker v2 api and wish to audit all actions peformed through it, some of which don’t cause db changes.
Side note… Is anyone working on converting the audit UI to react?
A while ago we put together a definition of what Foreman audit should track, because we had a lot of different understandings. Audit tracks successfully finished user activity that has changed the state of the system. This was always covered by ActiveRecord changes so far. This gives audit reviewer brief information about who modified the system or what actions led to the current state. It allows filtering of the stream by things like organization, user etc. It also allows searching by using the search capibilities we have everywhere. We try to keep the audit page reasonably brief.
These are the main differences from logging. People tend to audit activities, that are already logged. But we don’t want to store logs in DB. There are many external systems, that do better job. With recent @lzap’s changes, it’s easy to let Foreman log through journal and configure it so that it passes logs to e.g. ELK stack. These logs contain superset of audit information. If we want to search in logs, we need to use grep/kibana/etc.
It would help to know, what exactly you would like to track, to help you picking the right tool.
For rewriting the audits UI to react part - Tereza works on wireframes for audits pages. If that will be feasible and reasonable effort, we’d like to do it soon. Whether that will be in react or not, that’s another story. If it will be simple table, we won’t most likely write components for tables as part of this.
Hope that helps
–
Marek
Some examples of what I’d like a user to have visibility into:
- The image associated with a tag changed (show digest of old and new image)
- A user pulled an image (show pull info like tag and digest)
Perhaps using logging is better, it does make more sense. However, this info needs to be visible in the UI/CLI so it is accessible to a regular user.
Could the existing audits functionality be replaced by a log view of some sort? Or could certain log messages be flagged as audit entries?
One question is, are the audits limited by RBAC? An user responsible for objects in one organization should only see audits of that org. Is this how it works today? /me will try it later
If this is served by Apache httpd, there won’t be any audit log and should not be there really. This is pure logging. I am working on common logging initiative, first steps were merged upstream and we will come up with a blogpost soon on how to send logs into ELK from main components. That would do it, but we don’t aim having this configured via installer - it is opt-in manual only.
I like the idea of logging (it seems that audits are just a view of logging) but again, how do I expose that to an average user scoped to their roles and permissions?
My 5 cents: Audits are meant to track changes in the system: who, what, when.
Logs are good to track access (reading) ,performance, statistics, debugging and end-users don’t necessary get access. But with ELK you can build a dashboard and expose some data there. Separate user accounts and permissions, but there are systems to help you with that - IdM.
Are audits for only changes? If we take login as an example, would an audit include “User Susan failed login”? Would an audit include “User Susan failed to provision on compute resource X”?
I’ll try adding auditing to the resource changes and go from there. If any devs are working in the audits area over the next couple months I’d enjoy using and evaluating the work.
Thanks!
These are good examples, “change” was not the best term, I see audits as a resource where you can go to when something happens (someone deletes something or change it). Failed loging or some error are also good candidates.
There has been discussion about who is subject of audits. Some opinions were that only people should be subject of audits, others prefer that people and managed clients should be included. And audit team took the latter approach, but we need to be aware that clients can generate thousands of records and audit is stored in the RDBM and by default records are not expired. That’s the concern I had.
AFAIU, the current understanding is that Audits should only be for state changes in the system that were initiated by a user, so that in case something breaks it would be possible to understand who did a breaking action.
Failed logins or other requests aren’t audited, as the internal state of foreman did not change. They are logged however.
There are several devs who worked on audits in the past few months - @Marek_Hulan, @kgaikwad and myself, feel free to reach out to us with questions.
This sounds like audit
This sounds more like a log (httpd access log even) as it didn’t change Foreman system state. For displaying events like this, we were thinking of adding a new system call activity stream. It would be simple string messages that could be linked to any object, the data would be unstructured. That was not implemented yet and is not on our roadmap on foreseeable future. We would like to avoid displaying existing logs in UI, there are systems focused on this (ELK stack) which users can already use for this purpose today.
No, I don’t think it makes sense to replace one with another, they are two different things. We can add more thing to audits as long as they fit the audit definition. See my previous post.
Yes, audits are now scoped by organization and you need to have view_audit permission to see them. It does not respect things like view_hosts or view_domains, if user can see audits, they can see audits for all hosts and domains. We can’t really enforce RBAC here since audits are there even for objects that no longer exist. So we can’t say whether this used to be a host/domain that user used to have permission for in past.
Yes, only for changes. Failed login attempts are not audited, they are explicitly logged though. Failed provisioning is also not audited, see the definition in first post. It must be successful operation that changed the state.
For the record, telemetry stack also has a failed login counter which give you rate of successful and unsuccessful attempts in your monitoring.
@Marek_Hulan - moving the “that’s not what audits are for” discussion here…
As a real world example, consider the “authorized oauth apps” on github[1]. You’ll see that each of the apps has a “last used” message. This is useful information to the user managing their own account.
Now consider the same concept for our own PersonalAccessToken. As a user managing my account, I would like to know when a token was last used to login. This is not something an average user should be sent to the logs to find out. (Average user should never see logs, imho.)
Consider a host that periodically communicates with foreman to check for updates (like via subscription-manager). It is useful information to know the “last checkin” time and this is in fact a stored value. If this was audited, a user managing that host could look back in that host’s audit history to see the period of checkins, gaps, etc.
In my case, the user is managing thousands of container images with many tags for each. There are many reasons to want to know if a particular image has been accessed recently and by whom.
I hope this makes more sense now.
Sure. I understand you want to show history of some activity in the UI. I
can’t find the old thread on discourse now, but I think we had several talks
on this topic and we always ended up with the conclusion, audits are not the
right implementation for this kind of information.
Today we have audit, which provides structure and can be searched through by
org, loc, resource, user, time etc. The UI is also supposed to display this
information since we know, it’s always there.
We also have mostly unstructured log which can be search using fulltext
searching. Now with 1.18, one could use external system, e.g. ELK stack to
visualize this. But this usually contains a lot of messages that are not that
important for users.
What you are asking for, in my opinion, does not exist in Foreman yet and
would be something in between. Back then called it activity stream. That would
be a new list of simple strings with timestamp and it would always be linked
to some resource. That way you could track all related activities for that
resource. Once you delete the resource, you also delete all activities stored
with it. It wouldn’t contain any structure and it would only be created when
developer explicitly says. User would find the stream in resource detail page.
If you compare this to audits, we want to audit every change to every resource
automatically.
I understand it’s tempting to use audits for this as it has half of the
functionality already, but to me, it would be mixing use cases just because
these objects looks similarly.
I hope that helps to understand why I’m trying to avoid cluttering audits,
with all we want to store and display later 
To be honest, I’d be curious if users find the existing audits useful. The UI page doesn’t return all the audits as it is, though hammer has a more functional search. Perhaps my using it to add value will lead to further suggestions and improvements.
Even for your suggestion for an activity stream basically exists with audits. When an object is removed, the associated audits could easily be removed from db. This could be configurable for resources, users, whatever. Certainly would be a fine starting point.
At this time audits seem like the most natural existing way to do this. If a new system comes along, I’d be happy to convert to that. I don’t see a new system being implemented in the next three months, though, which is the timeline I’m working at.
Thanks for all the discussion! Very helpful.
It’s more about the structure of information and how we present that in UI, than automatic deletion. Honestly I think adding new thing would be easier, than hacking around audits and potentially making it more confusing, than it’s today. Adding one model with polymorhic association, currently only used for repositories, but easily usable for any resource, seem as easy and good addition.
If this became popular, I can imagine further improvements, such as linking activities with audits or showing both in activity stream.
Please, let’s not make audits more complicated or cluttered than they are today.
Audits let you know who changed what resource how and when. Audits are usually used when you need to figure out what user made which change in the system that (usually) led to something breaking. This use case has been confirmed in multiple talks with users.
Having a “last check in” or similar attribute may very well be useful in certain cases (we have it for example for content hosts, or similarly for last report on host). There is no argument against that, and that question is not related to audits. However, having it audited will just create many useless records in the audit table every time it changes. In the use case you mentioned, of managing thousands of containers, this means every time any of these is touched a new record is created in the table. that will quickly lead the audit table to bloating and becoming unusable, as well as drown any significant user actions in the flow of check-ins.
Just to be clear, I’m not using audits in any way that goes against existing use cases. I won’t be taking advantage of the audit relations that exist in the gem or doing anything that adds an inundation of audits. The hosts last report, for example, would far outweigh the quantity of audits for container images.
The only PR I made is related to scoped search on an audit comment, which I hope is not controversial.
As I said, if there is a replacement for the general concepts of audit/logging/whatever that comes along, I’d be quite happy to make use of it.
Related: Is anyone working on exposing the personal access tokens via UI? Maybe a good project to try moving users page (w/ tokens) to react/redux? I would like to incorporate auditing on the tokens too.
I have been working on a page based on react/redux, see https://github.com/theforeman/foreman/pull/4597.
But as it’s pretty difficult to move forward there, I have stopped work for now. A user can already see the time a token was last used.
Would you mind elaborating a bit on that? I still don’t get what you want to do or what user story you are trying to implement.