I've been meaning to migrate my foreman instance from the (old) default
sqlite3 db to postgresql. I was brave enough to give it a shot today, and
it was all going well until I went to actually migrate the data to postgres.
I'm running Foreman 1.2 on RHEL 6.
I configured /etc/foreman/database.yml as follows:
Maybe my google fu is not up to snuff this morning.
···
On Thursday, July 25, 2013 6:45:51 AM UTC-4, Lukas Zapletal wrote:
>
> > LINE 1: ...re_id", "operatingsystem_id") VALUES ($1, $2) RETURNING "id"
>
> There was a patch in regard to RETURNING statements somewhere, but I
> can't google it :(
>
>
> --
> Later,
>
> Lukas "lzap" Zapletal
> irc: lzap #theforeman
>
My knowledge about Ruby and Rails is quite limited, but there's a
boolean switch 'insert_returning' in
ActiveRecord::ConnectionAdapters::PostgreSQLAdapter which defaults true.
Maybe that is causing the trouble.
?
LZ
···
On Thu, Jul 25, 2013 at 06:55:09AM -0700, Andy Bohne wrote:
> I was only able to find this bug that was filed:
> http://projects.theforeman.org/issues/2783
>
> Maybe my google fu is not up to snuff this morning.
>
> On Thursday, July 25, 2013 6:45:51 AM UTC-4, Lukas Zapletal wrote:
> >
> > > LINE 1: ...re_id", "operatingsystem_id") VALUES ($1, $2) RETURNING "id"
> >
> > There was a patch in regard to RETURNING statements somewhere, but I
> > can't google it :(
> >
> >
> > --
> > Later,
> >
> > Lukas "lzap" Zapletal
> > irc: lzap #theforeman
> >
>
> --
> You received this message because you are subscribed to the Google Groups "Foreman users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-users+unsubscribe@googlegroups.com.
> To post to this group, send email to foreman-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/foreman-users.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
I'm wondering if using the taps gem to migrate this data is my best option
at this point.
···
On Friday, July 26, 2013 3:40:58 AM UTC-4, Lukas Zapletal wrote:
>
> And if you try this:
>
> My knowledge about Ruby and Rails is quite limited, but there's a
> boolean switch 'insert_returning' in
> ActiveRecord::ConnectionAdapters::PostgreSQLAdapter which defaults true.
> Maybe that is causing the trouble.
>
> ?
>
> LZ
>
> On Thu, Jul 25, 2013 at 06:55:09AM -0700, Andy Bohne wrote:
> > I was only able to find this bug that was filed:
> > http://projects.theforeman.org/issues/2783
> >
> > Maybe my google fu is not up to snuff this morning.
> >
> > On Thursday, July 25, 2013 6:45:51 AM UTC-4, Lukas Zapletal wrote:
> > >
> > > > LINE 1: ...re_id", "operatingsystem_id") VALUES ($1, $2) RETURNING
> "id"
> > >
> > > There was a patch in regard to RETURNING statements somewhere, but I
> > > can't google it :(
> > >
> > >
> > > --
> > > Later,
> > >
> > > Lukas "lzap" Zapletal
> > > irc: lzap #theforeman
> > >
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Foreman users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to foreman-user...@googlegroups.com .
> > To post to this group, send email to forema...@googlegroups.com.
>
> > Visit this group at http://groups.google.com/group/foreman-users.
> > For more options, visit https://groups.google.com/groups/opt_out.
> >
> >
>
> --
> Later,
>
> Lukas "lzap" Zapletal
> irc: lzap #theforeman
>
I'm still trying to figure out the best way to do this. Will migrating
with taps cause me any issues? Will I still need to run the sequence
number fix? The documentation seems a little hazy on this process and it
seems it hasn't been tested in some time.
Would I be better off switching to mysql? Or am I going to run into issues
there as well?
···
On Friday, July 26, 2013 9:18:21 AM UTC-4, Andy Bohne wrote:
>
> I see that option is supported in rails 4.0, per the documentation at
> http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/PostgreSQLAdapter.html
>
> I don't see it being supported in rails 3.2.8, which is what foreman 1.2
> includes,
> http://api.rubyonrails.org/v3.2.8/classes/ActiveRecord/ConnectionAdapters/PostgreSQLAdapter.html
>
> I'm wondering if using the taps gem to migrate this data is my best option
> at this point.
>
> On Friday, July 26, 2013 3:40:58 AM UTC-4, Lukas Zapletal wrote:
>>
>> And if you try this:
>>
>> My knowledge about Ruby and Rails is quite limited, but there's a
>> boolean switch 'insert_returning' in
>> ActiveRecord::ConnectionAdapters::PostgreSQLAdapter which defaults true.
>> Maybe that is causing the trouble.
>>
>> ?
>>
>> LZ
>>
>> On Thu, Jul 25, 2013 at 06:55:09AM -0700, Andy Bohne wrote:
>> > I was only able to find this bug that was filed:
>> > http://projects.theforeman.org/issues/2783
>> >
>> > Maybe my google fu is not up to snuff this morning.
>> >
>> > On Thursday, July 25, 2013 6:45:51 AM UTC-4, Lukas Zapletal wrote:
>> > >
>> > > > LINE 1: ...re_id", "operatingsystem_id") VALUES ($1, $2) RETURNING
>> "id"
>> > >
>> > > There was a patch in regard to RETURNING statements somewhere, but I
>> > > can't google it :(
>> > >
>> > >
>> > > --
>> > > Later,
>> > >
>> > > Lukas "lzap" Zapletal
>> > > irc: lzap #theforeman
>> > >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups "Foreman users" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an email to foreman-user...@googlegroups.com.
>> > To post to this group, send email to forema...@googlegroups.com.
>> > Visit this group at http://groups.google.com/group/foreman-users.
>> > For more options, visit https://groups.google.com/groups/opt_out.
>> >
>> >
>>
>> --
>> Later,
>>
>> Lukas "lzap" Zapletal
>> irc: lzap #theforeman
>>
>
To be honest, I don't know what you (or Joseph) mean by "taps" gem.
Maybe hacking the postgres driver to disable that feature?
LZ
···
On Mon, Jul 29, 2013 at 11:06:09AM -0700, Andy Bohne wrote:
> I'm still trying to figure out the best way to do this. Will migrating
> with taps cause me any issues? Will I still need to run the sequence
> number fix? The documentation seems a little hazy on this process and it
> seems it hasn't been tested in some time.
>
> Would I be better off switching to mysql? Or am I going to run into issues
> there as well?
>
> On Friday, July 26, 2013 9:18:21 AM UTC-4, Andy Bohne wrote:
> >
> > I see that option is supported in rails 4.0, per the documentation at
> > http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/PostgreSQLAdapter.html
> >
> > I don't see it being supported in rails 3.2.8, which is what foreman 1.2
> > includes,
> > http://api.rubyonrails.org/v3.2.8/classes/ActiveRecord/ConnectionAdapters/PostgreSQLAdapter.html
> >
> > I'm wondering if using the taps gem to migrate this data is my best option
> > at this point.
> >
> > On Friday, July 26, 2013 3:40:58 AM UTC-4, Lukas Zapletal wrote:
> >>
> >> And if you try this:
> >>
> >> My knowledge about Ruby and Rails is quite limited, but there's a
> >> boolean switch 'insert_returning' in
> >> ActiveRecord::ConnectionAdapters::PostgreSQLAdapter which defaults true.
> >> Maybe that is causing the trouble.
> >>
> >> ?
> >>
> >> LZ
> >>
> >> On Thu, Jul 25, 2013 at 06:55:09AM -0700, Andy Bohne wrote:
> >> > I was only able to find this bug that was filed:
> >> > http://projects.theforeman.org/issues/2783
> >> >
> >> > Maybe my google fu is not up to snuff this morning.
> >> >
> >> > On Thursday, July 25, 2013 6:45:51 AM UTC-4, Lukas Zapletal wrote:
> >> > >
> >> > > > LINE 1: ...re_id", "operatingsystem_id") VALUES ($1, $2) RETURNING
> >> "id"
> >> > >
> >> > > There was a patch in regard to RETURNING statements somewhere, but I
> >> > > can't google it :(
> >> > >
> >> > >
> >> > > --
> >> > > Later,
> >> > >
> >> > > Lukas "lzap" Zapletal
> >> > > irc: lzap #theforeman
> >> > >
> >> >
> >> > --
> >> > You received this message because you are subscribed to the Google
> >> Groups "Foreman users" group.
> >> > To unsubscribe from this group and stop receiving emails from it, send
> >> an email to foreman-user...@googlegroups.com.
> >> > To post to this group, send email to forema...@googlegroups.com.
> >> > Visit this group at http://groups.google.com/group/foreman-users.
> >> > For more options, visit https://groups.google.com/groups/opt_out.
> >> >
> >> >
> >>
> >> --
> >> Later,
> >>
> >> Lukas "lzap" Zapletal
> >> irc: lzap #theforeman
> >>
> >
>
> --
> You received this message because you are subscribed to the Google Groups "Foreman users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-users+unsubscribe@googlegroups.com.
> To post to this group, send email to foreman-users@googlegroups.com.
> Visit this group at http://groups.google.com/group/foreman-users.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
I'm leary of unintended consequences when hacking the postgres driver.
···
On Monday, July 29, 2013 2:25:11 PM UTC-4, Lukas Zapletal wrote:
>
> To be honest, I don't know what you (or Joseph) mean by "taps" gem.
>
> Maybe hacking the postgres driver to disable that feature?
>
> LZ
>
> On Mon, Jul 29, 2013 at 11:06:09AM -0700, Andy Bohne wrote:
> > I'm still trying to figure out the best way to do this. Will migrating
> > with taps cause me any issues? Will I still need to run the sequence
> > number fix? The documentation seems a little hazy on this process and
> it
> > seems it hasn't been tested in some time.
> >
> > Would I be better off switching to mysql? Or am I going to run into
> issues
> > there as well?
> >
> > On Friday, July 26, 2013 9:18:21 AM UTC-4, Andy Bohne wrote:
> > >
> > > I see that option is supported in rails 4.0, per the documentation at
> > >
> http://api.rubyonrails.org/classes/ActiveRecord/ConnectionAdapters/PostgreSQLAdapter.html
> > >
> > > I don't see it being supported in rails 3.2.8, which is what foreman
> 1.2
> > > includes,
> > >
> http://api.rubyonrails.org/v3.2.8/classes/ActiveRecord/ConnectionAdapters/PostgreSQLAdapter.html
> > >
> > > I'm wondering if using the taps gem to migrate this data is my best
> option
> > > at this point.
> > >
> > > On Friday, July 26, 2013 3:40:58 AM UTC-4, Lukas Zapletal wrote:
> > >>
> > >> And if you try this:
> > >>
> > >> My knowledge about Ruby and Rails is quite limited, but there's a
> > >> boolean switch 'insert_returning' in
> > >> ActiveRecord::ConnectionAdapters::PostgreSQLAdapter which defaults
> true.
> > >> Maybe that is causing the trouble.
> > >>
> > >> ?
> > >>
> > >> LZ
> > >>
> > >> On Thu, Jul 25, 2013 at 06:55:09AM -0700, Andy Bohne wrote:
> > >> > I was only able to find this bug that was filed:
> > >> > http://projects.theforeman.org/issues/2783
> > >> >
> > >> > Maybe my google fu is not up to snuff this morning.
> > >> >
> > >> > On Thursday, July 25, 2013 6:45:51 AM UTC-4, Lukas Zapletal wrote:
> > >> > >
> > >> > > > LINE 1: ...re_id", "operatingsystem_id") VALUES ($1, $2)
> RETURNING
> > >> "id"
> > >> > >
> > >> > > There was a patch in regard to RETURNING statements somewhere,
> but I
> > >> > > can't google it :(
> > >> > >
> > >> > >
> > >> > > --
> > >> > > Later,
> > >> > >
> > >> > > Lukas "lzap" Zapletal
> > >> > > irc: lzap #theforeman
> > >> > >
> > >> >
> > >> > --
> > >> > You received this message because you are subscribed to the Google
> > >> Groups "Foreman users" group.
> > >> > To unsubscribe from this group and stop receiving emails from it,
> send
> > >> an email to foreman-user...@googlegroups.com.
> > >> > To post to this group, send email to forema...@googlegroups.com.
> > >> > Visit this group at http://groups.google.com/group/foreman-users.
> > >> > For more options, visit https://groups.google.com/groups/opt_out.
> > >> >
> > >> >
> > >>
> > >> --
> > >> Later,
> > >>
> > >> Lukas "lzap" Zapletal
> > >> irc: lzap #theforeman
> > >>
> > >
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "Foreman users" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to foreman-user...@googlegroups.com .
> > To post to this group, send email to forema...@googlegroups.com.
>
> > Visit this group at http://groups.google.com/group/foreman-users.
> > For more options, visit https://groups.google.com/groups/opt_out.
> >
> >
>
> --
> Later,
>
> Lukas "lzap" Zapletal
> irc: lzap #theforeman
>
Should be fine, although I've no idea if Taps handles the sequnece_id
problem that our rake task has with postgres. One to watch out for.
In general, it's easy to test, since the end db has nothing to do with the
start db. If it works, great! If not, you go back to the old db while
finding a new solution
I tried hacking the postgres adapter and removing the RETURNING statement,
but I then ran into an issue where updated_at was not set on most of the
rows in lookup_keys.
In the end I migrated using taps, it seems to be working ok after
switching. I did not have the run the sequence_id fix.
···
On Monday, July 29, 2013 3:24:14 PM UTC-4, Greg Sutcliffe wrote:
>
> On 29 July 2013 19:38, Andy Bohne <andy....@gmail.com >wrote:
>
>> https://github.com/ricardochimal/taps
>>
>> I'm leary of unintended consequences when hacking the postgres driver.
>>
>
> Should be fine, although I've no idea if Taps handles the sequnece_id
> problem that our rake task has with postgres. One to watch out for.
>
> In general, it's easy to test, since the end db has nothing to do with the
> start db. If it works, great! If not, you go back to the old db while
> finding a new solution :)
>
> Greg
>
> I tried hacking the postgres adapter and removing the RETURNING statement,
> but I then ran into an issue where updated_at was not set on most of the
> rows in lookup_keys.
>
> In the end I migrated using taps, it seems to be working ok after
> switching. I did not have the run the sequence_id fix.
>
Hmm… can you document it then? maybe we should recommend people to be
using that instead?
thanks!
Ohad
···
On Wed, Jul 31, 2013 at 4:39 PM, Andy Bohne wrote:
On Monday, July 29, 2013 3:24:14 PM UTC-4, Greg Sutcliffe wrote:
I’m leary of unintended consequences when hacking the postgres driver.
Should be fine, although I’ve no idea if Taps handles the sequnece_id
problem that our rake task has with postgres. One to watch out for.
In general, it’s easy to test, since the end db has nothing to do with
the start db. If it works, great! If not, you go back to the old db while
finding a new solution
I also used taps to migrate to PostgreSQL but afiar you need to start a Sinatra server.
Joseph
···
----- Original Message -----
From: "Ohad Levy"
To: foreman-users@googlegroups.com
Sent: Wednesday, July 31, 2013 4:47:39 PM
Subject: Re: [foreman-users] Issue when migrating from sqlite3 db to postgresql
I tried hacking the postgres adapter and removing the RETURNING statement,
but I then ran into an issue where updated_at was not set on most of the
rows in lookup_keys.
In the end I migrated using taps, it seems to be working ok after
switching. I did not have the run the sequence_id fix.
Hmm… can you document it then? maybe we should recommend people to be
using that instead?
thanks!
Ohad
On Monday, July 29, 2013 3:24:14 PM UTC-4, Greg Sutcliffe wrote:
I’m leary of unintended consequences when hacking the postgres driver.
Should be fine, although I’ve no idea if Taps handles the sequnece_id
problem that our rake task has with postgres. One to watch out for.
In general, it’s easy to test, since the end db has nothing to do with
the start db. If it works, great! If not, you go back to the old db while
finding a new solution
I was able to successfully register some new nodes with puppet since the
migration. I haven't seen any issues with anything so far.
···
On Wednesday, July 31, 2013 1:20:48 PM UTC-4, Lukas Zapletal wrote:
>
> On Wed, Jul 31, 2013 at 06:39:33AM -0700, Andy Bohne wrote:
> > switching. I did not have the run the sequence_id fix.
>
> If you create a new resource (e.g. host) does it work? If this means
> sequences are set improperly, you can get conflicts.
>
> LZ
>
> --
> Later,
>
> Lukas "lzap" Zapletal
> irc: lzap #theforeman
>
