The Foreman + Katello nightly build to gather installed packages via REX

Problem:
I have added clients/servers to The Foreman without any agents and I am utilising REmote eXecution {REX later} function to collect data utilising facter in the background. BUT I am not able to send package info to The Foreman/Katello.
My Current way of ulpoading data is using REX JOB Template

#!/bin/bash


install_package_if_missing() {
    local pkg_name="$1"
    if ! command -v "$pkg_name" &> /dev/null
    then
        echo "${pkg_name} command not found. Attempting to install ${pkg_name}..."
        apt update && apt install -y "$pkg_name"
        if [ $? -ne 0 ]; then
            echo "ERROR: Failed to install ${pkg_name}. Cannot proceed."
            return 1 # Indicate failure
        fi
        echo "${pkg_name} installed successfully."
    fi
    return 0 # Indicate success
}

install_package_if_missing facter || exit 1
install_package_if_missing jq || exit 1

# Collect facts in JSON format
FACTS_JSON=$(facter --json 2>/dev/null)

# Get Foreman URL from template context (provided by Foreman)
FOREMAN_URL="<%= foreman_server_url %>"

# Get Host UUID from template context
HOST_FQDN="<%= @host.name %>"

# API endpoint for fact upload
API_URL="${FOREMAN_URL}/api/hosts/facts"

FOREMAN_API_USER="automation_user" # Create this user in Foreman with 'edit_facts' permission
FOREMAN_API_PASS="SimpleAPI" # Replace with actual API password


echo "DEBUG: FQDN determined for API call: '${HOST_FQDN}'"
# If HOST_FQDN is empty from template, try to get it from facter's JSON output
if [ -z "${HOST_FQDN}" ]; then
    echo "WARNING: @host.name is empty from template."
    exit 1
fi

FACTS_PAYLOAD=$(jq -n \
  --arg hostname "${HOST_FQDN}" \
  --argjson facts_data "${FACTS_JSON}" \
  '{name: $hostname, facts: $facts_data}')

# Check if jq command for payload construction was successful
if [ $? -ne 0 ]; then
    echo "ERROR: Failed to construct facts payload with jq. Ensure facter output is valid JSON."
    exit 1
fi

# Upload facts using curl
CURL_RESPONSE=$(curl -sS -k -f \
--header "Content-Type: application/json" \
--user "${FOREMAN_API_USER}:${FOREMAN_API_PASS}" \
--request POST \
--data "${FACTS_PAYLOAD}" \
"${API_URL}")

# Check curl's exit status for HTTP errors
if [ $? -eq 0 ]; then
    echo "Facts uploaded successfully for host ${HOSTNAME}."
    echo "API Response: ${CURL_RESPONSE}"
else
    echo "Failed to upload facts for host ${HOSTNAME}."
    echo "Curl command failed with exit status $?."
    echo "API Response (if any): ${CURL_RESPONSE}"
    echo "Curl command: curl -sS -k -f --header \"Content-Type: application/json\" --user \"${FOREMAN_API_USER}:${FOREMAN_API_PASS}\" --request POST --data \"${FACTS_PAYLOAD}\" \"${API_URL}\""
    exit 1
fi

I would like to add extra “line” that currently generates me output:

bash packages.sh | jq
[
  {
    "name": "adduser",
    "version": "3.134"
  },
  {
    "name": "zlib1g",
    "version": "1:1.2.13.dfsg-1"
  },
  {
    "name": "zstd",
    "version": "100:1.5.7-1"
  }
]

tho I did a list slightly shortter, but you get the idea.

Sorry for the script sanity, it is mainly generated by AI for now with small adjustments.

Expected outcome:
Package list is populated to the Foreamn and able to update the OS

Foreman and Proxy versions:

foreman.noarch                                             3.16.0-0.11.develop.20250721164311git8663870.el9    @foreman
foreman-cli.noarch                                         3.16.0-0.11.develop.20250721164311git8663870.el9    @foreman
foreman-dynflow-sidekiq.noarch                             3.16.0-0.11.develop.20250721164311git8663870.el9    @foreman
foreman-installer.noarch                                   1:3.16.0-0.1.develop.20250721031151git3b14d32.el9   @foreman
foreman-installer-katello.noarch                           1:3.16.0-0.1.develop.20250721031151git3b14d32.el9   @foreman
foreman-postgresql.noarch                                  3.16.0-0.11.develop.20250721164311git8663870.el9    @foreman
foreman-proxy.noarch                                       3.16.0-0.1.develop.20250721211632git9877f80.el9     @foreman
foreman-redis.noarch                                       3.16.0-0.11.develop.20250721164311git8663870.el9    @foreman
foreman-release.noarch                                     3.16.0-0.1.develop.el9                              @@commandline
foreman-selinux.noarch                                     3.16.0-0.1.develop.20250716113930git6f2746c.el9     @foreman
foreman-service.noarch                                     3.16.0-0.11.develop.20250721164311git8663870.el9    @foreman

dnf repolist
Updating Subscription Management repositories.
repo id                                                                    repo name
appstream                                                                  CentOS Stream 9 - AppStream
baseos                                                                     CentOS Stream 9 - BaseOS
candlepin                                                                  Candlepin: an open source entitlement management system.
extras-common                                                              CentOS Stream 9 - Extras packages
foreman                                                                    Foreman nightly
foreman-plugins                                                            Foreman plugins nightly
katello                                                                    Katello Nightly
pulpcore                                                                   pulpcore: Fetch, Upload, Organize, and Distribute Software Packages.
puppet8                                                                    Puppet 8 Repository el 9 - x86_64

Foreman and Proxy plugin versions:

Distribution and version:
CentOS Stream 9 {The Foreman host}
Debian BookWorm {ALL clients}

Other relevant data:
In general, my main concern, since I am utilising facts API POST, will it support such? If so, what would be the name of “section” for the package list?

It seems like you’re trying to rewrite existing functionality. Katello already receives installed package lists via package manager plugin (if rhsm config is set to package_profile_on_trans = 1), and also every 4 hours via rhsmcertd. (Not sure if the deb version of sub-man also does both, but I’m pretty sure it does at least one of these?) These both call the API via PUT to /consumers/:id/profiles. Once the installed package list is received, the packages should be viewable in the web UI in the host details > Content > Packages tab.

Thank you @jeremylenz

Yes, you are right, I am rewriting, cause I have some specific usage in mind, which I want to try out. I do not like agents on my systems, so thinking of removing them, and also understand how Foreman+Katello works for the future.

I will take a look at consumers api, thank you so much.