Integration & Packaging Pulpcore Diagnostics

Development
,
1

Hello community,

@dralley recently merged Make TASK_DIAGNOSTICS more configurable, and add a memray diagnostic by dralley · Pull Request #6526 · pulp/pulpcore · GitHub, which adds optional instrumentation to Pulp to see which lines of code are causing problematic memory usage. This would enable users to send better information about memory issues they are having with Pulp, and it will make it easier for developers to identify issues in Pulp.

There is one requirement besides enabling a setting in the Pulp settings file - memray.

It’s not a strict dependency on Pulp, but it is required to use this new diagnostics feature.
Memray appears to be packaged into Fedora RPM resource memray

Would it be worth considering packaging it inside pulpcore-packaging? If we start packaging it, it’s possible that many other Pulp stakeholders might start using it, so that should be weighed as well. @Odilhao you might have some opinions there :slight_smile:

We do have memory consumption problems from time to time in Foreman, which makes me wonder if there is a profiler out there for Ruby / Rails that we could use as well. Perhaps it’s worth waiting until the next nasty memory issue comes up.

-Ian

1 Like
2

I just want to add that this is a feature for development & debugging, not something you would want to enable in production, at least not long-term. But it could be very useful for support, especially with disconnected users, or users that have private repos that can’t be shared with us. They could just turn it on temporarily, do their thing, and send us the profiles.

There’s another tool called pyinstrument for which the same story applies.

2 Likes
3

+1 to adding this to help with debugging memory issues.

2 Likes
4

Looking at Tree - rpms/python-memray - src.fedoraproject.org it appears to be a fairly standard package without too many complications. I don’t see why we shouldn’t package this.

2 Likes
5

Looking at the requirements we already have pkginfo packaged, as mentioned by @ekohl it’s regular package for us to maintain.

@dralley could you add this as a optional package inside of pulpcore so we can track it’s dependencies in our automation?

1 Like
6