i could not find a similar problem report or ticket so maybe someone can
give me a hint:
When i enable the build for multiple host by selecting them on the Hosts
page and clicking on the Build button, it seems that the foreman-proxy
executes wget simultaneously for each host to copy the installation image
files to the tftp directory. Since the Operating System is the same for all
hosts, the same files are simultaneously copied to the same target files
which leads to corrupt files.
This domain (wgetting images and init ram disks) definitely needs more
and more love. There are couple of issues with this approach.
I can't tell you anything else that wait when you click on multiple
hosts right now.
I think as a temporary solution, we could tell wget only to download
files if size/lastest change time differs maybe. That could speed up
unnecessary downloads. This is one-liner that could help. Try it, the
code lives in: lib/proxy/tftp.rb
LZ
···
On Tue, Apr 08, 2014 at 01:32:31AM -0700, Peter Bauer wrote:
> hi,
>
> i could not find a similar problem report or ticket so maybe someone can
> give me a hint:
>
> When i enable the build for multiple host by selecting them on the Hosts
> page and clicking on the Build button, it seems that the foreman-proxy
> executes wget simultaneously for each host to copy the installation image
> files to the tftp directory. Since the Operating System is the same for all
> hosts, the same files are simultaneously copied to the same target files
> which leads to corrupt files.
>
> thx,
> Peter
>
> --
> You received this message because you are subscribed to the Google Groups "Foreman users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-users+unsubscribe@googlegroups.com.
> To post to this group, send email to foreman-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/foreman-users.
> For more options, visit https://groups.google.com/d/optout.
thx for the hint. The problem is not that the files are downloaded multiple
times but that the resulting file is corrupted since multiple instances of
wget are "continuing" the download at the same time due to the -c option
used for wget:
lib/proxy/tftp.rb
1
wget --timeout=10 --tries=3 --no-check-certificate -nv -c #{src} -O
"#{destination}"
With these parameters, wget will only download the files if they are not
already present WITH THE SAME SIZE in the target directory. This means if
the target file is smaller than the source file, the "missing" part of the
target file is downloaded and appended to the target file.
The problem is that the wget commands are executed in parallel so after the
first wget instance has started the download, the second one sees an
incomplete file and starts another download operation to append the
downloaded data to the existing file -> corrupt file with same size as
source file.
What i have tried is to use the --no-clobber option instead of the -c
option for wget. The behaviour is that the file is only downloaded once and
the resulting file is not corrupted by multiple writes since the subsequent
wget instances are not appending the downloaded file content to the
existing (partial) target file. The drawback of this solution is that
downloads will not be continued if they are interrupted and that the files
in the tftp directory need to be removed manually in these cases since wget
will not start another download as long as there are files in the target
directory named as the target files.
For my usage this behaviour is fine since the HTTP and TFTP server, Foreman
Proxy and Foreman are all running on the same machine, but i don't know if
you want to use this in future foreman-proxy versions.
hth,
Peter
···
Am Dienstag, 8. April 2014 22:49:16 UTC+2 schrieb Lukas Zapletal:
>
> This domain (wgetting images and init ram disks) definitely needs more
> and more love. There are couple of issues with this approach.
>
> I can't tell you anything else that wait when you click on multiple
> hosts right now.
>
> I think as a temporary solution, we could tell wget only to download
> files if size/lastest change time differs maybe. That could speed up
> unnecessary downloads. This is one-liner that could help. Try it, the
> code lives in: lib/proxy/tftp.rb
>
> LZ
>
> On Tue, Apr 08, 2014 at 01:32:31AM -0700, Peter Bauer wrote:
> > hi,
> >
> > i could not find a similar problem report or ticket so maybe someone can
> > give me a hint:
> >
> > When i enable the build for multiple host by selecting them on the Hosts
> > page and clicking on the Build button, it seems that the foreman-proxy
> > executes wget simultaneously for each host to copy the installation
> image
> > files to the tftp directory. Since the Operating System is the same for
> all
> > hosts, the same files are simultaneously copied to the same target files
> > which leads to corrupt files.
> >
> > thx,
> > Peter
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Foreman users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to foreman-user...@googlegroups.com .
> > To post to this group, send email to forema...@googlegroups.com.
>
> > Visit this group at http://groups.google.com/group/foreman-users.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> Later,
>
> Lukas "lzap" Zapletal
> irc: lzap #theforeman
>
> For my usage this behaviour is fine since the HTTP and TFTP server, Foreman
> Proxy and Foreman are all running on the same machine, but i don't know if
> you want to use this in future foreman-proxy versions.
This really needs some decent coding in the future:
download kernel/image only once and do not delete it from boot/ dir
(just leave it there and make the name unique for the whole OS life)
synchronous downloading - make Foreman UI to wait spawning VM until
image and kernel are finished downloading
show progress bar during download (nice one
solve all the race conditions
or maybe refactor the way it works today:
IMHO we should download image/kernel as soon as the distribution is
created. We should provide a button to re-download image/kernel (very
rarely distros fix issues in images). And we should keep those until
distributions are deleted. Database ID should be enough to make
image/kernel unique enough (while keeping the similar naming pattern).
With this approach, we can also add all distributions in the local boot
menu as optional menu entries for network "guests" that needs to
manually install a distribution (e.g. you have a contractor who booted a
new server on the network installing Fedora to do some testing).
Of course this has some drawback for example when media path is changed,
then we need to at least warn the user to re-download images and
kernels. But I see these as minor issues - the problems you face are
rather bad.
