Creating a Foreman HA cluster

Foreman HA Cluster
I am looking Forword to build a Foreman HA cluster. Right now, we have single nodes workable Foreman environment like this:
Single Main Foreman / Katello Server - Ricky Linux
Single Memcache Server - Oracle Linux
Single Database (PostgreSQL) Server - Oracle Linux
Single Smart Proxy (Ansible) Server - Ricky Linux

We would like to promote this environment to HA cluster, that spread across two Data Centers.
Basically, like this:
Two node cluster in DC1 and DC2, Main Foreman / Katello Server - Ricky Linux
Two node cluster in DC1 and DC2, Memcache Server - Oracle Linux
Two node cluster in DC1 and DC2, Database (PostgreSQL) Server - Oracle Linux
Two node cluster in DC1 and DC2, Smart Proxy (Ansible) Server - Ricky Linux

Is this possible?
If possible, how can it be done?
Obviously, we would like to promote our existing environment rather than rebuilding it as we already have lot of configurations are done and it has been heavily used.
Could someone help me with some experience and knowledge?

Thanks, in advance.

I have looked into this same topic in the past and afaik, HA setups are not possible right now, at least not with Katello.
IIRC, it might be possible without Katello, but with Katello there are some components involved that do not not support HA configurations.

With Katello, you can have load-balanced smart proxies. See the documentation

Note, however, the restrictions:

The main Foreman server cannot be set up for HA or load balancing.

Might be this could be a topic for foreman professional services?

NB: I am working for ATIX.

In principle you can setup any service on an HA cluster, but you have to make the whole stack HA. Means storage needs to be duplicated/replicated/HA available. Same for each component. Network layer may be an issue, but you can work with a virtual IP moving from one node to the other. Instead of multiple 2 node physical clusters, you can setup virtual clusters with failover and rules avoiding to have the same type of worker on the same physical host. This can be handled on the virtual layer or a cluster software manages the virtual hosts.
NB: I was responsible for SUSE HA for some time.

Thank you to all three of you. I am reading your messages and discussing them with my team. I will update the outcome, problems, and any further questions in this chat. I greatly appreciate your inputs.