API index templates in Katello

I wish I had noticed this earlier but in Katello for our index templates, we are simply including our show templates. This is causing us a lot of headaches as there's data in our show templates that's really meant to only be there for a single object (associations and so forth). So when this data is shown for multiple objects, it causes API index calls to take upwards of minutes.

Foreman's API, on the other hand, uses a base template that is shared between the index and show templates. The base template (main) includes the attributes for the model (id, name, etc). Then in the show templates, they also have associations and other data that might slow down index calls. See operating systems as an example:

https://github.com/theforeman/foreman/tree/develop/app/views/api/v2/operatingsystems

I propose that we should use Foreman's model for any new APIs in V2 (and we should definitely use it for V3 as well). And also, if you're adding any association data to a show template, consider breaking out a base template first so it doesn't slow down index.

David

> I wish I had noticed this earlier but in Katello for our index templates,
> we are simply including our show templates. This is causing us a lot of
> headaches as there's data in our show templates that's really meant to only
> be there for a single object (associations and so forth). So when this data
> is shown for multiple objects, it causes API index calls to take upwards of
> minutes.
>

there is an open bug [1] which slows down api requests with SQL n+1
queries, or lack of eager loading. i wonder if this can mitigate some of
this pain.

>
> Foreman's API, on the other hand, uses a base template that is shared
> between the index and show templates. The base template (main) includes the
> attributes for the model (id, name, etc). Then in the show templates, they
> also have associations and other data that might slow down index calls. See
> operating systems as an example:
>
>
> https://github.com/theforeman/foreman/tree/develop/app/views/api/v2/operatingsystems
>
> I propose that we should use Foreman's model for any new APIs in V2 (and
> we should definitely use it for V3 as well). And also, if you're adding any
> association data to a show template, consider breaking out a base template
> first so it doesn't slow down index.
>

this is really a good question, and my gut feeling is that the answer wold
be: 'it depends' , let me try to explain.

in some cases, it really make sense to provide a simple index, without
associations.

I was looking at json api spec [1] and i kind of like their approach, where
instead of nesting objects under index, you always return their ID, and
potentially return additional objects (not nested) in the same request to
avoid multiple api requests. (can be a query param to toggle it)

e.g.:
}, "posts": [{ "id": "1", "title": "Rails is Omakase", "comments": [ "1",
"2", "3" ] }, { "id": "2", "title": "The Parley Letter", "comments": [ "4",
"5" ] }, { "id": "3", "title": "Dependency Injection is Not a Virtue",
"comments": [ "6" ] } }], "comments": [{ "id": "1", "body": "Mmmmmakase" },
{ "id": "2", "body": "I prefer unagi" }, { "id": "3", "body": "What's
Omakase?" }, { "id": "4", "body": "Parley is a discussion, especially one
between enemies" }, { "id": "5", "body": "The parsley letter" }, { "id": "6"
, "body": "Dependency Injection is Not a Vice" }] } }

[1] Bug #8343: API resource_scope ignores options - Foreman
[2] http://jsonapi.org/

··· On Wed, Nov 26, 2014 at 3:51 AM, David Davis wrote:

David


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

> From: "Ohad Levy" <ohadlevy@gmail.com>
> To: "foreman-dev" <foreman-dev@googlegroups.com>
> Sent: Wednesday, November 26, 2014 9:23:15 AM
> Subject: Re: [foreman-dev] API index templates in Katello

> > I wish I had noticed this earlier but in Katello for our index templates,
> > we
> > are simply including our show templates. This is causing us a lot of
> > headaches as there's data in our show templates that's really meant to only
> > be there for a single object (associations and so forth). So when this data
> > is shown for multiple objects, it causes API index calls to take upwards of
> > minutes.
>

> there is an open bug [1] which slows down api requests with SQL n+1 queries,
> or lack of eager loading. i wonder if this can mitigate some of this pain.

> > Foreman's API, on the other hand, uses a base template that is shared
> > between
> > the index and show templates. The base template (main) includes the
> > attributes for the model (id, name, etc). Then in the show templates, they
> > also have associations and other data that might slow down index calls. See
> > operating systems as an example:
>

> > https://github.com/theforeman/foreman/tree/develop/app/views/api/v2/operatingsystems
>

> > I propose that we should use Foreman's model for any new APIs in V2 (and we
> > should definitely use it for V3 as well). And also, if you're adding any
> > association data to a show template, consider breaking out a base template
> > first so it doesn't slow down index.
>

> this is really a good question, and my gut feeling is that the answer wold
> be: 'it depends' , let me try to explain.

> in some cases, it really make sense to provide a simple index, without
> associations.

> I was looking at json api spec [1] and i kind of like their approach, where
> instead of nesting objects under index, you always return their ID, and
> potentially return additional objects (not nested) in the same request to
> avoid multiple api requests. (can be a query param to toggle it)

I am also in favor of the jsonapi.org spec for side-loading child nodes instead of nesting objects.

··· > On Wed, Nov 26, 2014 at 3:51 AM, David Davis < daviddavis@redhat.com > wrote:

e.g.:
}, “posts” : [{ “id” : “1” , “title” : “Rails is Omakase” , “comments” : [
“1” , “2” , “3” ] }, { “id” : “2” , “title” : “The Parley Letter” ,
“comments” : [ “4” , “5” ] }, { “id” : “3” , “title” : “Dependency Injection
is Not a Virtue” , “comments” : [ “6” ] } }], “comments” : [{ “id” : “1” ,
“body” : “Mmmmmakase” }, { “id” : “2” , “body” : “I prefer unagi” }, { “id”
: “3” , “body” : “What’s Omakase?” }, { “id” : “4” , “body” : “Parley is a
discussion, especially one between enemies” }, { “id” : “5” , “body” : “The
parsley letter” }, { “id” : “6” , “body” : “Dependency Injection is Not a
Vice” }] } }

[1] Bug #8343: API resource_scope ignores options - Foreman
[2] http://jsonapi.org/

David

You received this message because you are subscribed to the Google Groups
"foreman-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com .

For more options, visit https://groups.google.com/d/optout .


You received this message because you are subscribed to the Google Groups
"foreman-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com .
For more options, visit https://groups.google.com/d/optout .