"Candlepin Consumer: 410 Gone" messages in Foreman 1.12.3

Hello,

We are running Foreman Version 1.12.3 and are encountering the following error message frequently in the logs:

2018-09-27 14:49:09 [app] [E] RestClient::Gone: Katello::Resources::Candlepin::Consumer: 410 Gone {"displayMessage":"Unit 94778303-5baa-4401-9da2-7d8d6ebcf44a has been deleted","requestUuid":"42bcf081-825c-43f6-828a-cdc6cfa2ce07","deletedId":"94778303-5baa-4401-9da2-7d8d6ebcf44a"} (GET /candlepin/consumers/94778303-5baa-4401-9da2-7d8d6ebcf44a)
 | Body: {"displayMessage":"Unit 94778303-5baa-4401-9da2-7d8d6ebcf44a has been deleted","requestUuid":"42bcf081-825c-43f6-828a-cdc6cfa2ce07","deletedId":"94778303-5baa-4401-9da2-7d8d6ebcf44a"}

The UI is also unstable as some operations are timing out due to the above error.

I have tried running:

foreman-rake katello:reindex

and then

foreman-rake katello:clean_backend_objects --trace COMMIT=true

But this fails with

# foreman-rake katello:clean_backend_objects --trace COMMIT=true
** Invoke katello:clean_backend_objects (first_time)
** Invoke environment (first_time)
** Execute environment
** Invoke katello:check_ping (first_time)
** Execute katello:check_ping
** Execute katello:clean_backend_objects
1 orphaned consumer id(s) found.
exception when destroying candlepin consumer d4191a36-9429-4df5-8721-305783588e12Katello::Resources::Candlepin::Consumer: 500 Internal Server Error {"displayMessage":"Runtime Error could not execute statement at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse:2,102","requestUuid":"1ef493a5-0c93-431f-aead-f95777babe51"} (DELETE /candlepin/consumers/d4191a36-9429-4df5-8721-305783588e12): {"displayMessage":"Runtime Error could not execute statement at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse:2,102","requestUuid":"1ef493a5-0c93-431f-aead-f95777babe51"}

Attempting to delete this UUID using the interactive shell shows that this object does not exist.

irb(main):014:0*  Katello::System.find_by_uuid("1ef493a5-0c93-431f-aead-f95777babe51")
=> nil

Each time I run the above steps, the UUID appears to be different

Hello,
Foreman 1.12 is long out of support, the last supported version is 1.18. I highly recommend upgrading to a more recent version.