Koji is back online

All,

since it's Friday afternoon here, I decided to delete Fedora 19 and
Fedora 20 i386 external repos which gives us some space to breathe :slight_smile:

rm -rf /mnt/tmp/external-repos/fedora{19,20}-i386/

I have edited /etc/mrepo.conf and removed it from there as well.
Currently mrepo for external repositories is commented out from cron, so
I believe it is not running at all.

Also I moved postgres backups from /mnt/koji to /mnt/tmp and created a
symlink which gave us 400 MB of free space. Fair enough until jblazek
returns back from his PTO.

Postgres was restarted and koji is operational.

And the most important thing - I am going to call it a day.

Good luck everyone :slight_smile:

In fact I have still some work to do on the discovery imageโ€ฆ

ยทยทยท -- Later, Lukas #lzap Zapletal

Btw we are now:

[root@koji ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvde1 5.7G 3.0G 2.7G 53% /
none 1.8G 0 1.8G 0% /dev/shm
tmpfs 4.0G 116M 3.9G 3% /tmp
/dev/xvdt1 50G 44G 2.8G 95% /mnt/koji
/dev/xvdj1 367G 286G 66G 82% /mnt/tmp

And if you have some recommendations for other delete candidates, let me
know:

[root@koji ~]# ls /mnt/tmp/external-repos/ -1
epel5-i386
epel5-x86_64
epel6-i386
epel6-x86_64
epel7-x86_64
fedora19-x86_64
fedora20-x86_64
jpackage-x86_64
rhel5-i386
rhel5-x86_64
rhel6-i386
rhel6-x86_64
rhel7-x86_64
rhscl-rhel6-x86_64
rhscl-rhel7-x86_64

ยทยทยท -- Later, Lukas #lzap Zapletal

I see few builds failed since then, mostly jenkins.pulp user, only
Fedora builds. Apparently it does not find packages for the buildroot
for i386.

Is this a problem?

http://koji.katello.org/koji/tasks?start=50&state=all&view=tree&method=all&order=-id

LZ

ยทยทยท -- Later, Lukas #lzap Zapletal

All,

koji went down this night again. The /mnt/koji folder was filled which
brought our pgsql database down.

I moved the /mnt/koji/pgsql directory to /mnt/tmp where we have plenty
of space now and changed the /var/lib/pgsql symlink. This should prevent
the situation when full disks brings it down again.

I am leaving /mnt/koji/pgsql-ORIG.tar.gz tarball with the old content
just for case something goes wrong. The database files are 1 GB in size
fyi.

Decommissioning our koji is our priority in the upcoming weeks. We need
to look at Copr and evaluate if it fits our needs. We are looking for
public VM hosting with enough disk space (1 TB data volume) if Copr does
not work for us - if you can help drop us a line.

ยทยทยท -- Later, Lukas #lzap Zapletal

FYI we only dropped Fedora i386 build roots, not the EPELs.

LZ

ยทยทยท On Mon, Sep 01, 2014 at 08:47:19PM +0200, Lukas Zapletal wrote: > I see few builds failed since then, mostly jenkins.pulp user, only > Fedora builds. Apparently it does not find packages for the buildroot > for i386. > > Is this a problem? > > http://koji.katello.org/koji/tasks?start=50&state=all&view=tree&method=all&order=-id > > LZ > > -- > Later, > Lukas #lzap Zapletal > > _______________________________________________ > Pulp-list mailing list > Pulp-list@redhat.com > https://www.redhat.com/mailman/listinfo/pulp-list

โ€“
Later,
Lukas #lzap Zapletal

I'd be OK with evaluating Copr, but wouldn't want to move until the
migration until a new Fedora Cloud instance has been made. Currently
the reliability's not great and the queues can get very long (hours, a
whole day). That's enough to make day to day work difficult, and would
be a real pain with releases.

My main concern apart from that is designing a release workflow with
Copr that makes it possible to put together releases from multiple Copr
repos and avoids rebuilding dependencies between releases. Nothing too
hard I shouldn't think, but clearly it won't be a 1-1 mapping between
Koji and Copr workflows.

We also need to keep the capability to build 1.6 releases in Koji over
the next few months, and probably 1.5 for a month or so.

ยทยทยท On 03/09/14 10:07, Lukas Zapletal wrote: > Decommissioning our koji is our priority in the upcoming weeks. We need > to look at Copr and evaluate if it fits our needs. We are looking for > public VM hosting with enough disk space (1 TB data volume) if Copr does > not work for us - if you can help drop us a line.

โ€“
Dominic Cleal
Red Hat Engineering