The candlepin team[1] recently implemented a mechanism to control which fields get returned in the json responses to their api[2]. Some fields, while needed in some situations, can be either very large or very expensive to calculate. I'd like to get some feedback on whether consumers of the foreman/foretello api are interested in this and have opinions on the either the style or implementation.
–
“The leader must aim high, see big, judge widely, thus setting himself apart form the ordinary people who debate in narrow confines.” ~ Charles De Gaulle
“Leadership is about making others better as a result of your presence and making sure that impact lasts in your absence.” ~ Harvard Business School
I think this is a important feature when there's a certain API call is very
expensive like you mentioned. In the areas besides performance, I think
it's a "nice to have" and not "must have"
I started a branch on this several months ago. What is the branch that you
started? I'll take a look at it.
Joseph
···
On Wednesday, July 2, 2014 11:12:11 PM UTC+3, Tom McKay wrote:
>
>
> The candlepin team[1] recently implemented a mechanism to control which
> fields get returned in the json responses to their api[2]. Some fields,
> while needed in some situations, can be either very large or very expensive
> to calculate. I'd like to get some feedback on whether consumers of the
> foreman/foretello api are interested in this and have opinions on the
> either the style or implementation.
>
> Thanks!
>
>
> [1] http://candlepinproject.org
> [2]
> http://www.candlepinproject.org/docs/candlepin/json_response_filtering.html
>
> --
> @thomasmckay
>
> --
> "The leader must aim high, see big, judge widely, thus setting himself
> apart form the ordinary people who debate in narrow confines." ~ Charles De
> Gaulle
>
> "Leadership is about making others better as a result of your presence and
> making sure that impact lasts in your absence." ~ Harvard Business School
>
Would it be possible to refactor that filtering into a library that we
could use?
···
On Wed, Jul 2, 2014 at 10:12 PM, Tom McKay wrote:
The candlepin team[1] recently implemented a mechanism to control which
fields get returned in the json responses to their api[2]. Some fields,
while needed in some situations, can be either very large or very expensive
to calculate. I’d like to get some feedback on whether consumers of the
foreman/foretello api are interested in this and have opinions on the
either the style or implementation.
–
“The leader must aim high, see big, judge widely, thus setting himself
apart form the ordinary people who debate in narrow confines.” ~ Charles De
Gaulle
“Leadership is about making others better as a result of your presence and
making sure that impact lasts in your absence.” ~ Harvard Business School
Before selectively turning off large fields, I’d try using gzip
compression. Chances are it will do the trick.
Without seeing concrete cases, I’m tempted to say that presence of
“expensive to compute” fields is a modelling issue — such fileds
should probably be refactored into resources. They can be represented
as a relation in the return data after that.
-d
···
On Thu, Jul 3, 2014 at 10:20 AM, wrote:
> Hi Tom,
>
> I think this is a important feature when there's a certain API call is very
> expensive like you mentioned. In the areas besides performance, I think it's
> a "nice to have" and not "must have"
>
> I started a branch on this several months ago. What is the branch that you
> started? I'll take a look at it.
>
> Joseph
>
>
>
>
>
> On Wednesday, July 2, 2014 11:12:11 PM UTC+3, Tom McKay wrote:
>>
>>
>> The candlepin team[1] recently implemented a mechanism to control which
>> fields get returned in the json responses to their api[2]. Some fields,
>> while needed in some situations, can be either very large or very expensive
>> to calculate. I'd like to get some feedback on whether consumers of the
>> foreman/foretello api are interested in this and have opinions on the either
>> the style or implementation.
>>
>> Thanks!
>>
>>
>> [1] http://candlepinproject.org
>> [2]
>> http://www.candlepinproject.org/docs/candlepin/json_response_filtering.html
>>
>> --
>> @thomasmckay
>>
>> --
>> "The leader must aim high, see big, judge widely, thus setting himself
>> apart form the ordinary people who debate in narrow confines." ~ Charles De
>> Gaulle
>>
>> "Leadership is about making others better as a result of your presence and
>> making sure that impact lasts in your absence." ~ Harvard Business School
>
> --
> You received this message because you are subscribed to the Google Groups
> "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to foreman-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
This was built into RHEV for the following use case:
We needed a list of all VMs and our show disk usage for every VM. By
default the disks collection is just a link since including it is quite
heavy to calculate.
Now as an API user that meant we had to do an API call for every VM
resulting in 1 + n calls where n is the number of VMs. The solution:
GET /api/vms HTTP/1.1
Accept: application/xml; detail=disks
Now instead of a link the disks collection is expanded with the content
that would normally show up in /api/vms/UUID/disks and we only have to
do 1 API call. This saves the overhead of doing n API calls.
Now that is the RHEV use case. For Foreman I can imagine that you may
want to include all puppet classes for every host which is quite heavy
for every normal call, but I can imagine an overview which lists the
classes can be very useful for some dashboards.
···
On Thu, Jul 03, 2014 at 11:10:58AM +0100, Dmitri Dolguikh wrote:
> I don’t think I like the idea:
>
> - Before selectively turning off large fields, I’d try using gzip
> compression. Chances are it will do the trick.
> - Without seeing concrete cases, I’m tempted to say that presence of
> “expensive to compute” fields is a modelling issue — such fileds
> should probably be refactored into resources. They can be represented
> as a relation in the return data after that.
>
> -d
>
> On Thu, Jul 3, 2014 at 10:20 AM, wrote:
> > Hi Tom,
> >
> > I think this is a important feature when there's a certain API call is very
> > expensive like you mentioned. In the areas besides performance, I think it's
> > a "nice to have" and not "must have"
> >
> > I started a branch on this several months ago. What is the branch that you
> > started? I'll take a look at it.
> >
> > Joseph
> >
> >
> >
> >
> >
> > On Wednesday, July 2, 2014 11:12:11 PM UTC+3, Tom McKay wrote:
> >>
> >>
> >> The candlepin team[1] recently implemented a mechanism to control which
> >> fields get returned in the json responses to their api[2]. Some fields,
> >> while needed in some situations, can be either very large or very expensive
> >> to calculate. I'd like to get some feedback on whether consumers of the
> >> foreman/foretello api are interested in this and have opinions on the either
> >> the style or implementation.
> >>
> >> Thanks!
> >>
> >>
> >> [1] http://candlepinproject.org
> >> [2]
> >> http://www.candlepinproject.org/docs/candlepin/json_response_filtering.html
> >>
> >> --
> >> @thomasmckay
> >>
> >> --
> >> "The leader must aim high, see big, judge widely, thus setting himself
> >> apart form the ordinary people who debate in narrow confines." ~ Charles De
> >> Gaulle
> >>
> >> "Leadership is about making others better as a result of your presence and
> >> making sure that impact lasts in your absence." ~ Harvard Business School
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "foreman-dev" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to foreman-dev+unsubscribe@googlegroups.com.
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
I can see a value in being able to trigger eager/lazy loading of
attributes. Definitely preferable to complete filtering of attributes.
-d
···
On Thu, Jul 3, 2014 at 1:25 PM, Ewoud Kohl van Wijngaarden wrote:
> Have you seen how RHEV does this?
>
> https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.4/html/Technical_Guide/Listing_Extended_Resource_Sub-Collections1.html
>
> This was built into RHEV for the following use case:
>
> We needed a list of all VMs and our show disk usage for every VM. By
> default the disks collection is just a link since including it is quite
> heavy to calculate.
>
> GET /api/vms HTTP/1.1
> Accept: application/xml
>
>
> [...]
> href="/api/vms/082c794b-771f-452f-83c9-b2b5a19c0399">
> [...]
> href="/api/vms/082c794b-771f-452f-83c9-b2b5a19c0399/disks"/>
> [...]
>
> [...]
>
>
> Now as an API user that meant we had to do an API call for every VM
> resulting in 1 + n calls where n is the number of VMs. The solution:
>
> GET /api/vms HTTP/1.1
> Accept: application/xml; detail=disks
>
> Now instead of a link the disks collection is expanded with the content
> that would normally show up in /api/vms/UUID/disks and we only have to
> do 1 API call. This saves the overhead of doing n API calls.
>
> Now that is the RHEV use case. For Foreman I can imagine that you may
> want to include all puppet classes for every host which is quite heavy
> for every normal call, but I can imagine an overview which lists the
> classes can be very useful for some dashboards.
>
> On Thu, Jul 03, 2014 at 11:10:58AM +0100, Dmitri Dolguikh wrote:
>> I don’t think I like the idea:
>>
>> - Before selectively turning off large fields, I’d try using gzip
>> compression. Chances are it will do the trick.
>> - Without seeing concrete cases, I’m tempted to say that presence of
>> “expensive to compute” fields is a modelling issue — such fileds
>> should probably be refactored into resources. They can be represented
>> as a relation in the return data after that.
>>
>> -d
>>
>> On Thu, Jul 3, 2014 at 10:20 AM, wrote:
>> > Hi Tom,
>> >
>> > I think this is a important feature when there's a certain API call is very
>> > expensive like you mentioned. In the areas besides performance, I think it's
>> > a "nice to have" and not "must have"
>> >
>> > I started a branch on this several months ago. What is the branch that you
>> > started? I'll take a look at it.
>> >
>> > Joseph
>> >
>> >
>> >
>> >
>> >
>> > On Wednesday, July 2, 2014 11:12:11 PM UTC+3, Tom McKay wrote:
>> >>
>> >>
>> >> The candlepin team[1] recently implemented a mechanism to control which
>> >> fields get returned in the json responses to their api[2]. Some fields,
>> >> while needed in some situations, can be either very large or very expensive
>> >> to calculate. I'd like to get some feedback on whether consumers of the
>> >> foreman/foretello api are interested in this and have opinions on the either
>> >> the style or implementation.
>> >>
>> >> Thanks!
>> >>
>> >>
>> >> [1] http://candlepinproject.org
>> >> [2]
>> >> http://www.candlepinproject.org/docs/candlepin/json_response_filtering.html
>> >>
>> >> --
>> >> @thomasmckay
>> >>
>> >> --
>> >> "The leader must aim high, see big, judge widely, thus setting himself
>> >> apart form the ordinary people who debate in narrow confines." ~ Charles De
>> >> Gaulle
>> >>
>> >> "Leadership is about making others better as a result of your presence and
>> >> making sure that impact lasts in your absence." ~ Harvard Business School
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups
>> > "foreman-dev" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an
>> > email to foreman-dev+unsubscribe@googlegroups.com.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "foreman-dev" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
