I suspect you have incorrectly installed your smart proxy.
With Katello 3.5 (freshly installed) I have JUST installed 2 smart proxies into my architecture (more to come). When I attempt to hit either of them through:
I get a 403 - Forbidden attempting to access / on the smart proxy, which I think is what I’m supposed to get.
Accessing 9090 on that port gives me a browser “import certificate” pop up (should be expecting a certificate signed by the Katello Root CA) which I haven’t tried to get past.
The correct installation of the smart proxy is seen by following this documentation: (For Katello 3.4 which is what you said you have installed).
I wouldn’t know why you have access to your web UI through your smart proxy, but unless the smart proxy is in a DMZ you’re no less safe…whether you type in webserver1.domain.com or webserver2.domain.com is no different.
Can you try https://<fqdn.system.name>:8443 as this is the port im able to access it by. It appears to be able to manipulate the central foreman database, which leads me to believe its not its own foreman instance.
Also below is my install command i used. Note I used the foreman-proxy-content scenario.