O.K. Here is one way on how to get a 404 after changing paths.
Create a test (external) repository on a web server with rpms in a Packages subdirectory: Package/abc.rpm
Important: make sure those RPMs do not already exists in any form on the server. I think pulp can associate RPMs by hash and thus would use a local copy if hash fits regardless of where it moved in the external repository.
Now create the repository in katello with “mirror on sync” and “on demand” download policy and sync.
This creates entries in core_remoteartifact with Packages/ urls.
Now “restructure” your external repository, e.g. move all RPMS from the Packages subdirectory to the root and rerun “createrepo” to update metadata.
Sync again: nothing changes in the database. core_remoteartifact remains as it is. If I now try to access the rpm through pulp I get a 404 because it tries to download using the Packages subdirectory.
I guess, something similar happens between the main server and the content proxy (with on demand) if you regenerate metadata on the main server…