I've put together draft of dynamic bindings for apipie-documented APIs.
My intention is to develop replacement for static bindings (current
foreman_api and katello_api gems).
It should bring developers access to current API without rebuilding the
bindings all the time,
it should help to solve problems with API extensions. It should be also
useful for other apipie documented projects.
I don't understand the cache here. You are caching the apipie
documentation? Why? Because on a production instance, this will never
change, thus it does not make sense to create cache from content. I'd
just generate some kind of identifier (unix timestamp) during the start
and this can be used to find out if the content does not change.
For those who change API during development (in the DEVELOPMENT rails
mode), a restart would help which is I think okay and we can live with
this just fine. I'd prefer that before md5-suming content during
startup. Our startup is slow now, let's not do it slower.
I don't have comments for other features Looks like this is the main
feature in this case
LZ
···
On Tue, Dec 10, 2013 at 06:08:50PM +0100, Martin Bačovský wrote:
> Hi,
>
> I've put together draft of dynamic bindings for apipie-documented APIs.
>
> My intention is to develop replacement for static bindings (current
> foreman_api and katello_api gems).
>
> It should bring developers access to current API without rebuilding
> the bindings all the time,
> it should help to solve problems with API extensions. It should be
> also useful for other apipie documented projects.
>
> If you are interested in early preview, check it on my github:
> https://github.com/mbacovsky/apipie_bindings_draft
>
> There is quick example in the README, more can be seen from the tests.
>
> I'd appreciate any comments (bindings interface, overall design,
> must-have features, etc)
>
> Cheers,
>
> Martin
>
> --
> You received this message because you are subscribed to the Google Groups "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
> Martin,
>
> I don't understand the cache here. You are caching the apipie
> documentation? Why? Because on a production instance, this will never
> change, thus it does not make sense to create cache from content. I'd
> just generate some kind of identifier (unix timestamp) during the start
> and this can be used to find out if the content does not change.
>
> For those who change API during development (in the DEVELOPMENT rails
> mode), a restart would help which is I think okay and we can live with
> this just fine. I'd prefer that before md5-suming content during
> startup. Our startup is slow now, let's not do it slower.
>
> I don't have comments for other features Looks like this is the main
> feature in this case
>
> LZ
Thanks for the comment.
I'm not sure I understand it correctly, so I try to sum up what is going
on with cache:
The dynamic bindings are on the client side (e.g hammer). For the
bindings to do its job we need it to know how the API on the server
looks like. For that we use the apidoc (json description of the params
of the API) and we load it from the server. To not to have to load it
before every request we cache it (client side).
To make the caching effective I've suggested the following change on the
server:
At the server start we should create some identifier and send it in
headers with every response.
MD5 of the apidoc seems to be most natural, but I get your point that it
is a time consuming. Using timestamp, as you suggested, would improve
start time on the server, but may cause unnecessary cache reloading for
all the clients, when the API didn't change between restarts. If we
agree on that server start up time is the priority, I'm okay with
sending timestamp instead.
Another good point is that this approach wouldn't work well in
development mode. The identifier is computed on start up so we have to
be able to trigger the refresh also from elsewhere. Ivan suggested
apidoc controller could recompute the hash during API docs serving.
Did it answer your questions?
Martin
···
On 12/12/2013 03:19 PM, Lukas Zapletal wrote:
On Tue, Dec 10, 2013 at 06:08:50PM +0100, Martin Bačovský wrote:
Hi,
I’ve put together draft of dynamic bindings for apipie-documented APIs.
My intention is to develop replacement for static bindings (current
foreman_api and katello_api gems).
It should bring developers access to current API without rebuilding
the bindings all the time,
it should help to solve problems with API extensions. It should be
also useful for other apipie documented projects.
> MD5 of the apidoc seems to be most natural, but I get your point
> that it is a time consuming. Using timestamp, as you suggested,
> would improve start time on the server, but may cause unnecessary
> cache reloading for all the clients, when the API didn't change
> between restarts. If we agree on that server start up time is the
> priority, I'm okay with sending timestamp instead.
Good point, but how much CPU work is the md5 sum of a directory of files (or
whatever are you going to hash) compared to one HTTP(s) request to get
the JSON with the definition again? I mean, how big this file is
supposed to be? 10kB? 100kB? If we drop it in the public folder which is
being served by Apache and provide gzipped file, that it's literally
nothing.
> Another good point is that this approach wouldn't work well in
> development mode. The identifier is computed on start up so we have
> to be able to trigger the refresh also from elsewhere. Ivan
> suggested apidoc controller could recompute the hash during API docs
> serving.
I don't think it's issue to need to restart the server when API
changes. You'd need to do it anyway if you would add a route perhaps.
I think restarting could be very annoying - it is slow. I would like to
be able to avoid it if I can. Time-stamp sounds good to me - it supports
caching and it's cpu friendly. Also if time of creation instead of
server start-up is used then I think it avoids unnecessary
re-downloading in production (stamp stays same).
···
On 13.12.13 18:24, Lukas Zapletal wrote:
>> MD5 of the apidoc seems to be most natural, but I get your point
>> that it is a time consuming. Using timestamp, as you suggested,
>> would improve start time on the server, but may cause unnecessary
>> cache reloading for all the clients, when the API didn't change
>> between restarts. If we agree on that server start up time is the
>> priority, I'm okay with sending timestamp instead.
>
> Good point, but how much CPU work is the md5 sum of a directory of files (or
> whatever are you going to hash) compared to one HTTP(s) request to get
> the JSON with the definition again? I mean, how big this file is
> supposed to be? 10kB? 100kB? If we drop it in the public folder which is
> being served by Apache and provide gzipped file, that it's literally
> nothing.
>
>> Another good point is that this approach wouldn't work well in
>> development mode. The identifier is computed on start up so we have
>> to be able to trigger the refresh also from elsewhere. Ivan
>> suggested apidoc controller could recompute the hash during API docs
>> serving.
>
> I don't think it's issue to need to restart the server when _API_
> changes. You'd need to do it anyway if you would add a route perhaps.
>
From what I've learnt after chat with Martin, I am fine with calculating
content hash from the apipie json document as it is just one document or
string and it is small.
We slightly discussed how this should be implemented (cli should perhaps
send the hashtag in the request parameters and server should compare
it), but this is another story.
TL;DR So, I think what Martin recommends works for me
LZ
···
On Tue, Dec 17, 2013 at 07:12:41PM +0100, Petr Chalupa wrote:
>
>
> On 13.12.13 18:24, Lukas Zapletal wrote:
> >>MD5 of the apidoc seems to be most natural, but I get your point
> >>that it is a time consuming. Using timestamp, as you suggested,
> >>would improve start time on the server, but may cause unnecessary
> >>cache reloading for all the clients, when the API didn't change
> >>between restarts. If we agree on that server start up time is the
> >>priority, I'm okay with sending timestamp instead.
> >
> >Good point, but how much CPU work is the md5 sum of a directory of files (or
> >whatever are you going to hash) compared to one HTTP(s) request to get
> >the JSON with the definition again? I mean, how big this file is
> >supposed to be? 10kB? 100kB? If we drop it in the public folder which is
> >being served by Apache and provide gzipped file, that it's literally
> >nothing.
> >
> >>Another good point is that this approach wouldn't work well in
> >>development mode. The identifier is computed on start up so we have
> >>to be able to trigger the refresh also from elsewhere. Ivan
> >>suggested apidoc controller could recompute the hash during API docs
> >>serving.
> >
> >I don't think it's issue to need to restart the server when _API_
> >changes. You'd need to do it anyway if you would add a route perhaps.
> >
>
> I think restarting could be very annoying - it is slow. I would like
> to be able to avoid it if I can. Time-stamp sounds good to me - it
> supports caching and it's cpu friendly. Also if time of creation
> instead of server start-up is used then I think it avoids
> unnecessary re-downloading in production (stamp stays same).
>
> --
> You received this message because you are subscribed to the Google Groups "foreman-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
Looks like this is the main