While I haven’t got around to setting up forman and proxmox like you are trying to achieve its on my bucket list. I have used image builder at work for a similer approach just using vmware and Satellite Server. I find image builder easier to setup and maintaining and updating the image or making small changes easier. The image is small around 1.8GB. Good luck.
