This thread might be helpful: Does Foreman/Katello support s3 storage for pulp content? - #4 by ekohl
I also know @m-bucher has a tracker in katello Tracker #36121: Support S3 cloud storage backend - Katello - Foreman to document/support this.
Wonder if updating foreman custom-hiera will do it for you. For the installer option, I suppose we need some changes to puppet-pulpcore.