Server failure in hub and spoke (smart-proxy) model

We are running Foreman/Katello in a hub and spoke model in AWS. I have one hub (the master) in one AWS region and 2 spokes each in different regions. We did this to try to limit the amount of cross region traffic that happens for updates which costs money.

First question is what happens if one of the spokes dies? For example. The hub is in US-East-1. The Spokes are in US-West-1 and US-West-2. If the spoke in US-West-2 were to die would the hosts attached to it (for content updates) still be able to update from the HUB??? Or would they just fail until the spoke comes back online?

Second question. I have the creation of spokes pretty much scripted (about 99% there). However, getting it to join the HUB is not scripted (but I am sure I can figure it out). Nor is assigning the host to spokes scripted (again I am sure I can figure it out). Would spinning up a new spoke probably be the best (fastest, easiest) solution assuming I can fully script it out. Or does some one have a better solution???

I fully get that if the HUB goes down I am screwed right now. But that is about 80% scripted.

If one of your smart proxies (spokes) dies, hosts will not automatically fall back to receiving content from the main Foreman server. If it’s going to be an extended outage, you could manually switch hosts back to the main Foreman server using the ‘Change Content Source’ feature.

If you require some sort of redundancy you may look into a load balanced setup, but this would be a bit more complex.