Problem: rpms are seen and are hosted, but they do not exist in /var/lib/pulp/media/artifact/. Pulp is looking for the rpms and is expecting it there, but the content is missing when I try to find the matching hash. The content was pushed to a disconnected server and imported using a content transfer method.
Expected outcome: To download content.
Foreman and Proxy versions: foreman-3.5.1.24-1.el8 foreman-proxy-3.5.1.el8
Distribution and version: RHEL8.10
Other relevant data:
katello rpm: katello-4.7.0-1
pulp: pulpcore
pulp version: 3.21.18
The error is a generic 500 error stating the the server is misbehaving.
Hi @theredgreek ,
Can we have some more info about how you performed your content export and import?
We exported a content-view and then imported it into another.
I was able to fix it by running a pullp repair
import requests
import json
# Replace with the appropriate URL for your Pulp server
base_url = "https://foreman.lou.land"
# Paths to your client certificate and key
cert = "/etc/foreman/client_cert.pem"
key = "/etc/foreman/client_key.pem"
try:
# Start repair task
response = requests.post(
f"{base_url}/pulp/api/v3/repair/",
cert=(cert, key),
verify=False
)
response.raise_for_status()
task = response.json()['task']
# Print the task ID
print(f"Repair task ID: {task}")
# Check repair task status
task_status_response = requests.get(
f"{base_url}{task}",
cert=(cert, key),
verify=False
)
# Ensure we get a successful response
task_status_response.raise_for_status()
# Print task status
print(json.dumps(task_status_response.json(), indent=2))
except requests.exceptions.RequestException as e:
print(f"An error occurred: {e}")
curl -s --cert /etc/foreman/client_cert.pem --key /etc/foreman/client_key.pem -k https://foreman.lou.land/pulp/api/v3/tasks/$TASK_ID/ | jq```
Glad to hear you were able to fix it at least. Were there any failed tasks, either on the export side or the import side? If this happens again, I’d be curious if it’s happening consistently.
We are working with three satellites. Two are importing content without an issue. The third one started having issues when an admin switched from downloading content from a webserver, to importing content using content-views.
We tried everything to fix it. We even tried removing all content-views and repositories. Nothing worked until I started reading the code from the pulp repo and found that gem in the python code ( not pun intended).
Interesting, any back traces to share from /var/log/messages, perhaps from Pulp? Or /var/log/foreman/production.log, but the errors would also likely show up on the Dynflow console for the export or import tasks.
Also, I here is a link to where in the code this is failing.
Link to Code
The errors look something like this:
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: [2024-07-22 16:34:00 +0000] [1958] [ERROR] Error handling request
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: Traceback (most recent call last):
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: File "/usr/lib64/python3.9/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: resp = await request_handler(request)
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: File "/usr/lib64/python3.9/site-packages/aiohttp/web_app.py", line 504, in _handle
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: resp = await handler(request)
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: File "/usr/lib64/python3.9/site-packages/aiohttp/web_middlewares.py", line 117, in impl
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: return await handler(request)
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: File "/usr/lib/python3.9/site-packages/pulpcore/content/authentication.py", line 41, in authenticate
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: return await handler(request)
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: File "/usr/lib/python3.9/site-packages/pulpcore/cache/cache.py", line 339, in cached_function
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: response = await self.make_entry(
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: File "/usr/lib/python3.9/site-packages/pulpcore/cache/cache.py", line 378, in make_entry
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: response = await handler(*args, **kwargs)
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: File "/usr/lib/python3.9/site-packages/pulpcore/content/handler.py", line 237, in stream_content
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: return await self._match_and_stream(path, request)
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: File "/usr/lib/python3.9/site-packages/pulpcore/content/handler.py", line 542, in _match_and_stream
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: return await self._serve_content_artifact(ca, headers, request)
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: File "/usr/lib/python3.9/site-packages/pulpcore/content/handler.py", line 815, in _serve_content_artifact
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: raise Exception(_("Expected path '{}' is not found").format(path))
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: Exception: Expected path '/var/lib/pulp/media/artifact/e5/7d348e35b0f500b6188f775531da6284e92ed38b52258f523c7064840e166a' is not found
Jul 22 16:34:00 satellite.lou.land pulpcore-content[1958]: [22/Jul/2024:16:34:00 +0000] "GET /pulp/content/hallas/content_views/RHEL-KS/2.0/content/dist/rhel8/8.8/x86_64/appstream/kickstart/repodata/e57d348e35b0f500b6188f775531da6284e92ed38b52258f523c7>
Jul 22 17:53:39 satellite.lou.land pulpcore-content[1961]: [2024-07-22 17:53:39 +0000] [1961] [ERROR] Error handling request```
1 Like
Can you explain the cause of this error?
