Change in REST API behavior in develop branch

So I just upgraded our foreman instance to from 0.4.2 to latest develop
branch, and there seems to be a significant change in how the API outputs
the default hosts collection.

Previously, it outputted a text hostlist as the default output, and a
semi-useless json list of hosts if you requested JSON output. Now it seems
the default is JSON, and it is a more correct JSON output that includes:
name:, id:, hostgroup_id:, and operatingsystem_id:.

This breaks the default behavior of foremancli --hosts, which previously
output a text list of hosts, that was very easy to parse with classic shell
scripting.

Any thoughts on how to get back the old foremancli behavior? (Ideally, I
would be able to get the old behavior via API versioning, but am open to
additional thoughts.)

This is pretty important, because this functionality is the main reason I
wrote foremancli in the first place. (A simple text host list.) All the
other stuff, I added was to make it more useful to other people.

Thanks,
Brian

··· --

I suppose you could add a param to the API call

http://foreman/hosts?format=json&apiversion=042

then in the format.json branch just add a condition to check for the apiversion param and use the old method.

Otherwise, how many people would this affect? Is it too much to ask the users to update the foremancli gem?

  format.json { 
if params[:apiversion] == "042"
   render :json => search.all(:select => "hosts.name", :include => included_associations).map(&:name) 
    else
 # should you ever need more attributes just add to the :only array or specify :methods, :include, :except to the options hash
 	 format.json { render :json => search.includes(included_associations).to_json({:only => [:name, :id, :hostgroup_id, :operatingsystem_id]}) }
             end
	}

Corey Osman
corey@logicminds.biz

Green IT and Datacenter Automation Specialist

··· On Apr 19, 2012, at 2:28 PM, Brian Gupta wrote:

So I just upgraded our foreman instance to from 0.4.2 to latest develop branch, and there seems to be a significant change in how the API outputs the default hosts collection.

Previously, it outputted a text hostlist as the default output, and a semi-useless json list of hosts if you requested JSON output. Now it seems the default is JSON, and it is a more correct JSON output that includes: name:, id:, hostgroup_id:, and operatingsystem_id:.

This breaks the default behavior of foremancli --hosts, which previously output a text list of hosts, that was very easy to parse with classic shell scripting.

Any thoughts on how to get back the old foremancli behavior? (Ideally, I would be able to get the old behavior via API versioning, but am open to additional thoughts.)

This is pretty important, because this functionality is the main reason I wrote foremancli in the first place. (A simple text host list.) All the other stuff, I added was to make it more useful to other people.

Thanks,
Brian

  1. We don't have API versioning today, so if we don't make the
    default behavior backward compatible we break existing integrations for
    every API change we make. This is going to be a bigger and bigger issue as
    more people integrate with Foreman. We shouldn't just tell them to rewrite
    every time we make API changes. This is not best practice for API design.

  2. By using html headers, to call an API version, we can add the header
    call from the client, and older servers will just ignore it.

  3. I envision that the latest version of foremancli, should work with older
    versions of foreman, as there are going to be environments that need to
    work with multiple foreman servers running different versions of foreman. I
    know we have a difference of opinion here, in that you only want to suport
    the latest version of foreman with RemoteAdmin. As this project matures we
    are going to see more and more people running older versions of Foreman
    longer than we might necessarily like, and I want to support that with the
    tools I am working on. (Basically I feel that for foremancli, it should
    support versions of foreman going back to 0.4 where it was introduced.
    Hammer will likely be coded to the 0.5 API, but likely won't make the 0.5
    ship.)

When following REST principles, APIs shouldn't break backwards
compatibility. Being that this is the real world, and we don't have a large
set of resources to get this all right from the gitgo, we are going to need
to make changes that aren't compatible. One way to support this is through
API versioning. However, from everything I have been researching embedding
API version in the URI in any form, doesn't follow REST design principles.
Using a version header seems to be the best option.

The one question is if you call the URI without the version header, which
version of the API do you default to? Generally speaking the thought is to
respond with the latest version, unless API versioning wasn't part of the
API from the beginning. IE: If you didn't initially provide API consumers
with the capability to lock clients to a specific API version, you
shouldn't break the default behavior.

Thanks,
Brian

··· On Thu, Apr 19, 2012 at 6:01 PM, Corey Osman wrote:

I suppose you could add a param to the API call

http://foreman/hosts?format=json&apiversion=042

then in the format.json branch just add a condition to check for the
apiversion param and use the old method.

Otherwise, how many people would this affect? Is it too much to ask the
users to update the foremancli gem?

  format.json {

if params[:apiversion] == "042"
render :json => search.all(:select => “hosts.name”, :include =>
included_associations).map(&:name)
else

should you ever need more attributes just add to the :only array or specify :methods, :include, :except to the options hash

 	 format.json { render :json => search.includes(included_associations).to_json({:only => [:name, :id, :hostgroup_id, :operatingsystem_id]}) }

             end

}

Corey Osman
corey@logicminds.biz

Green IT and Datacenter Automation Specialist

On Apr 19, 2012, at 2:28 PM, Brian Gupta wrote:

So I just upgraded our foreman instance to from 0.4.2 to latest develop
branch, and there seems to be a significant change in how the API outputs
the default hosts collection.

Previously, it outputted a text hostlist as the default output, and a
semi-useless json list of hosts if you requested JSON output. Now it seems
the default is JSON, and it is a more correct JSON output that includes:
name:, id:, hostgroup_id:, and operatingsystem_id:.

This breaks the default behavior of foremancli --hosts, which previously
output a text list of hosts, that was very easy to parse with classic shell
scripting.

Any thoughts on how to get back the old foremancli behavior? (Ideally, I
would be able to get the old behavior via API versioning, but am open to
additional thoughts.)

This is pretty important, because this functionality is the main reason I
wrote foremancli in the first place. (A simple text host list.) All the
other stuff, I added was to make it more useful to other people.

Thanks,
Brian


http://aws.amazon.com/solutions/solution-providers/brandorr/


http://aws.amazon.com/solutions/solution-providers/brandorr/

Well let's just reverse the conditions i outlined and the 042 version would be the default.

Or let's have an Orc discussion to figure everything out.

··· Sent from my iPhone

On Apr 20, 2012, at 7:01 AM, Brian Gupta brian.gupta@brandorr.com wrote:

  1. We don’t have API versioning today, so if we don’t make the default behavior backward compatible we break existing integrations for every API change we make. This is going to be a bigger and bigger issue as more people integrate with Foreman. We shouldn’t just tell them to rewrite every time we make API changes. This is not best practice for API design.

  2. By using html headers, to call an API version, we can add the header call from the client, and older servers will just ignore it.

  3. I envision that the latest version of foremancli, should work with older versions of foreman, as there are going to be environments that need to work with multiple foreman servers running different versions of foreman. I know we have a difference of opinion here, in that you only want to suport the latest version of foreman with RemoteAdmin. As this project matures we are going to see more and more people running older versions of Foreman longer than we might necessarily like, and I want to support that with the tools I am working on. (Basically I feel that for foremancli, it should support versions of foreman going back to 0.4 where it was introduced. Hammer will likely be coded to the 0.5 API, but likely won’t make the 0.5 ship.)

When following REST principles, APIs shouldn’t break backwards compatibility. Being that this is the real world, and we don’t have a large set of resources to get this all right from the gitgo, we are going to need to make changes that aren’t compatible. One way to support this is through API versioning. However, from everything I have been researching embedding API version in the URI in any form, doesn’t follow REST design principles. Using a version header seems to be the best option.

The one question is if you call the URI without the version header, which version of the API do you default to? Generally speaking the thought is to respond with the latest version, unless API versioning wasn’t part of the API from the beginning. IE: If you didn’t initially provide API consumers with the capability to lock clients to a specific API version, you shouldn’t break the default behavior.

Thanks,
Brian

On Thu, Apr 19, 2012 at 6:01 PM, Corey Osman corey@logicminds.biz wrote:
I suppose you could add a param to the API call

http://foreman/hosts?format=json&apiversion=042

then in the format.json branch just add a condition to check for the apiversion param and use the old method.

Otherwise, how many people would this affect? Is it too much to ask the users to update the foremancli gem?

  format.json { 

if params[:apiversion] == "042"
render :json => search.all(:select => “hosts.name”, :include => included_associations).map(&:name)
else

should you ever need more attributes just add to the :only array or specify :methods, :include, :except to the options hash

 	 format.json { render :json => search.includes(included_associations).to_json({:only => [:name, :id, :hostgroup_id, :operatingsystem_id]}) }


             end
  }

Corey Osman
corey@logicminds.biz

Green IT and Datacenter Automation Specialist

On Apr 19, 2012, at 2:28 PM, Brian Gupta wrote:

So I just upgraded our foreman instance to from 0.4.2 to latest develop branch, and there seems to be a significant change in how the API outputs the default hosts collection.

Previously, it outputted a text hostlist as the default output, and a semi-useless json list of hosts if you requested JSON output. Now it seems the default is JSON, and it is a more correct JSON output that includes: name:, id:, hostgroup_id:, and operatingsystem_id:.

This breaks the default behavior of foremancli --hosts, which previously output a text list of hosts, that was very easy to parse with classic shell scripting.

Any thoughts on how to get back the old foremancli behavior? (Ideally, I would be able to get the old behavior via API versioning, but am open to additional thoughts.)

This is pretty important, because this functionality is the main reason I wrote foremancli in the first place. (A simple text host list.) All the other stuff, I added was to make it more useful to other people.

Thanks,
Brian

Had a quick chat with Ohad. Sounds like going forward we are going to need
to support API versioning, but that requires refactoring so that the API is
separate from the core rails controller.

For my particular issue, Ohad has agreed to help me refactor foremancli
before the 0.5 release. I'm thinking that as a temporary measure, we will
have foremancli query the server status to get the version of foreman, and
have a special workaround for the host list behavior. (Icky I know, but
there doesn't seem to be any choice here.)

It is my current thought that since hammer is designed to support all API
features, that we need to develop hammer against a stable API. In our
onging discussions, I am going to push to get signoff from Ohad that the
0.5 API will be the basis for the first stable Foreman API. The thinking
is that we will only use new API versions if we change/break
existing behavior, and we don't actually have to implement the API
versioning layer, until we want to break the 0.5 API behavior. This will
allow us to extend the API, but existing behavior should not change, unless
we are prepared to go the route of full API versioning. Mmm. actually we
probably want to implement at least some little bit of versioning now.
Enough so that we have a spec that people can specify the API version now.
Otherwise, we would have to commit to having the default behaviour for the
unversioned API calls never break. IE: If we want unversioned API calls to
call the latest API, stabilization of the API should allow devs to specify
the API version number. (Even if we don't fully implement versioning on the
backend, we should publish the first stable API.)

I guess the tricky thing right now, is we don't actually know how many
people are using the API, so we don't know how much pain API changes cause
today, vs the pain of implementing versioning. That said if we want more
people to use the API, we have to give them some basic promises about the
API stability.

-Brian

··· On Fri, Apr 20, 2012 at 1:30 PM, Corey Osman wrote:

Well let’s just reverse the conditions i outlined and the 042 version
would be the default.

Or let’s have an Orc discussion to figure everything out.

Sent from my iPhone

On Apr 20, 2012, at 7:01 AM, Brian Gupta brian.gupta@brandorr.com wrote:

  1. We don’t have API versioning today, so if we don’t make the
    default behavior backward compatible we break existing integrations for
    every API change we make. This is going to be a bigger and bigger issue as
    more people integrate with Foreman. We shouldn’t just tell them to rewrite
    every time we make API changes. This is not best practice for API design.

  2. By using html headers, to call an API version, we can add the header
    call from the client, and older servers will just ignore it.

  3. I envision that the latest version of foremancli, should work with
    older versions of foreman, as there are going to be environments that need
    to work with multiple foreman servers running different versions of
    foreman. I know we have a difference of opinion here, in that you only want
    to suport the latest version of foreman with RemoteAdmin. As this project
    matures we are going to see more and more people running older versions of
    Foreman longer than we might necessarily like, and I want to support that
    with the tools I am working on. (Basically I feel that for foremancli, it
    should support versions of foreman going back to 0.4 where it was
    introduced. Hammer will likely be coded to the 0.5 API, but likely won’t
    make the 0.5 ship.)

When following REST principles, APIs shouldn’t break backwards
compatibility. Being that this is the real world, and we don’t have a large
set of resources to get this all right from the gitgo, we are going to need
to make changes that aren’t compatible. One way to support this is through
API versioning. However, from everything I have been researching embedding
API version in the URI in any form, doesn’t follow REST design principles.
Using a version header seems to be the best option.

The one question is if you call the URI without the version header, which
version of the API do you default to? Generally speaking the thought is to
respond with the latest version, unless API versioning wasn’t part of the
API from the beginning. IE: If you didn’t initially provide API consumers
with the capability to lock clients to a specific API version, you
shouldn’t break the default behavior.

Thanks,
Brian

On Thu, Apr 19, 2012 at 6:01 PM, Corey Osman corey@logicminds.biz wrote:

I suppose you could add a param to the API call

http://foreman/hosts?format=json&apiversion=042

then in the format.json branch just add a condition to check for the
apiversion param and use the old method.

Otherwise, how many people would this affect? Is it too much to ask the
users to update the foremancli gem?

  format.json {

if params[:apiversion] == "042"
render :json => search.all(:select => “hosts.name”, :include =>
included_associations).map(&:name)
else

should you ever need more attributes just add to the :only array or specify :methods, :include, :except to the options hash

 	 format.json { render :json => search.includes(included_associations).to_json({:only => [:name, :id, :hostgroup_id, :operatingsystem_id]}) }


             end

}

Corey Osman
corey@logicminds.biz

Green IT and Datacenter Automation Specialist

On Apr 19, 2012, at 2:28 PM, Brian Gupta wrote:

So I just upgraded our foreman instance to from 0.4.2 to latest develop
branch, and there seems to be a significant change in how the API outputs
the default hosts collection.

Previously, it outputted a text hostlist as the default output, and a
semi-useless json list of hosts if you requested JSON output. Now it seems
the default is JSON, and it is a more correct JSON output that includes:
name:, id:, hostgroup_id:, and operatingsystem_id:.

This breaks the default behavior of foremancli --hosts, which previously
output a text list of hosts, that was very easy to parse with classic shell
scripting.

Any thoughts on how to get back the old foremancli behavior? (Ideally, I
would be able to get the old behavior via API versioning, but am open to
additional thoughts.)

This is pretty important, because this functionality is the main reason I
wrote foremancli in the first place. (A simple text host list.) All the
other stuff, I added was to make it more useful to other people.

Thanks,
Brian


http://aws.amazon.com/solutions/solution-providers/brandorr/


http://aws.amazon.com/solutions/solution-providers/brandorr/


http://aws.amazon.com/solutions/solution-providers/brandorr/