General Troubleshooting- hammer content-import library

I am unable to import my library
Expected outcome:
Import of the data should complete without errors
Foreman and Proxy versions:
Foreman and Proxy plugin versions:
Katello 4.4
Distribution and version:
Rocky 8.5
Other relevant data:
I have been reading and following the content management guide here:

One of my systems is air gapped. The system is a new build and getting the repos set up for the 1st time. I export my content from the connected system by running a full export of the library. The data is then transferred to the disconnected system in the imports folder. I then run: hammer content-import library --organization"<name_here>" --path=/var/lib/pulp/imports/

After about 20 mins the process fails with: Error: 1 subtask(s) failed the task group /pulp/api/v3/task-groups/894b1e7c-xxx-xxx-xxx-xxx/.

By reviewing Montior>Tasks and finding the import task that was run, I can find more info about the failure. From there I can review the Errors tab and see the Action:, Input:, Output:, Exception:, and Backtrace:

Can anyone provide some general troubleshooting hints and tips? Due to the type of system I am unable to pull logs for upload easily. It requires approval by authorities above me. This is why I am asking a general question. Sorry I am not able to be more specific at this time.

If you go to the subtask which failed, you may find out what goes wrong by checking the reason of the error result from the subtask’s output.

1 Like

Yes, thanks. The subtask Output is generally the same as what I listed above, the last three lines say this-

          " /pulp/api/v3/workers/<some_alphanumeric_number>/"}]}],
"poll_attempts:=>("total"=>111, "failed=>1)}

The Exception section says this:
Katello::Errors::Pulp3Error: Error: 1 subtask(s) failed the task group /pulp/api/v3/task-groups/894b1e7c-xxx-xxx-xxx-xxx/.

From the Task tab of the failed task, click on Dynflow console button.
In the popup Dynflow window, click on Run tab. You should see a list of actions.
The failed action is in red. Click on it that will show what error you run into.

Ok, I don’t see anything in Red in the Dynflow console. All the items are in orange/blue but no red. Also, every action listed in the console has a status of (success). Now I am a little more confused.

My understanding is that importing a library will also import all the RPMs located in the repository. Is my understanding correct? Maybe I have it wrong…

I do see the Products and Repos getting created when running the content-import job.

@lfu I appreciate your support greatly! Are there any other suggestions you can provide or any further comments based on my last response?

I do have approval now to upload logs if needed.

This issue seems very similar to BZ2032602 (maybe even the same issue)

Let’s find out the status of the failed task-group. Please replace with your task-group id before running the command.
curl https://`hostname`/pulp/api/v3/task-groups/<894b1e7c-xxx-xxx-xxx-xxx>/ --cert /etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key | python -m json.tool

    "tasks": [
            "pulp_href": "/pulp/api/v3/tasks/f029f55e-01f7-4282-81ea-523c3c195abc/",

From its result, grab pulp_href and run:

curl https://`hostname`<pulp_href> --cert /etc/pki/katello/certs/pulp-client.crt --key /etc/pki/katello/private/pulp-client.key | python -m json.tool

Similar to this:

curl https://`hostname`/pulp/api/v3/tasks/f029f55e-01f7-4282-81ea-523c3c195abc/   --cert /etc/pki/katello/certs/pulp-client.crt  --key /etc/pki/katello/private/pulp-client.key | python -m json.tool

That should tell us the reason why the task failed.

This was perfect. It allowed me to see what the problem really was. My humble .02 worth - pin this. Getting to the bottom of a cryptic error message was invaluable in solving the problem. By seeing the actual error message that was buried two layers deep I was able to resolve my issue.