Foreman and katello UI future

[thomasmckay]

With the upcoming coordinated release of foreman-1.6 and katello-2.0, there will be a chance to reflect on the past year of hard work and consider paths going forward for both communities. Clearly the collaboration has had tangible benefits to both sides of development. Specifically, the installer is awesome, hammer is a joy to use and extend, improvements on API consistency, and much more. One topic that we've tried to pretend is not an issue, though, is the UI.

I've been very vocal internally about the need for a consistent UI across the combined project. Perhaps this is more of a nice-to-have for an upstream endeavor, but in my mind it is a requirement for the downstream enterprise product. Admittedly I could be wrong and may be drawing too heavily on my years of closed-source work. On the other hand, I think a unified UI for the combined upstream would be pretty great too.

So, I'd like to ask the community of developers what their opinions are in regards to the future of coding in the UI area. I ask the developers because presumably users will like improvements in ease of use and functionality regardless of the code style it is written in.

What we have today are two very different UI implementations. I will apologize if I offend authors on either side; please correct me where my facts are wrong.

ERB - Foreman's UI is rendered largely server side in the form of "erb" files that can directly access server code and the database to populate names, lists, forms, etc. There are some portions that are "ajax" where the rendered browser page will make calls back to the server to update something or get additional information. A good example of this is the organization edit page. You'll see, if you examine the html in the browser, that all the various association tabs and their choices are hard coded into the html. For example, if there are 150 users you will see all of them in the source.

JS - Katello's UI is rendered entirely client side using angular.js javascript (wrapped in an engine called "bastion", which is the name I'll use here). Bastion provides an MVC layer on the client and populates its data by making api calls to get json which it then uses to fill in the html. If the organization page were to be rewritten in bastion, viewing the page source would look almost empty. The difference would be that when the Users tab was selected in the UI, an ajax api call would be made to the server to get the users that were available to add to the org and rendered then. Clicking a user to add them could also make another ajax call to update the org and reflect that change right away in the UI.

For reference, katello's UI started out exactly like foreman's is currently. With each release of the product through Red Hat, though, customers asked for more and more features that the erb and javascript struggled to implement. When the code reached a snarly mess of jquery ajax and hacks, the developers undertook the effort to start fresh. This is when angular.js was started, some 18 months (or more) ago. I suspect that the devs that have contributed to the existing foreman UI will recognize the same cloud looming over their current infrastructure. Perhaps I'm wrong, though.

There are pros and cons to both styles that I'd like to try to enumerate here for discussion, addition, and argument.

ERB pro - Very simple to write
ERB pro - Can access entire code and database readily when building the UI
ERB pro - Plugins can use 'deface' package to make changes
ERB con - Plugins have to use 'deface' package to make changes
ERB con - Two server side controllers, one for API and one for UI
ERB con - Full page reloads very common

JS con - Has a much steeper learning curve to write
JS pro - Html portions is very similar to erb in its simplicity
JS con - All data must be available via API in JSON (could be a pro)
JS pro - UI consumes same documented REST API, also used by hammer
JS pro - UI is very dynamic, not requiring full page reloads
JS con - Limited to provided hooks to tweak UI, cannot use 'deface'
JS pro - Cannot use 'deface'
JS con - Existing 'deface' plugins will need a rewrite

While it is not strictly necessary that unification of the UI implementations needs to ever happen, it will be a point of divergence if resources are invested in replacing a portion that the other half has chosen not to use. (eg. Katello rewrites the roles UI in bastion and foreman chooses not to include it.)

There is a very elegant choice with the introduction of bastion UI plugins. A UI overhaul can be one-sided to katello and not impact the standalone foreman at all. I, for one, am vehemently against this and would argue strongly that it should be one UI where the resources, be they upstream or downstream, be focused. I haven't won that argument and don't see anyone wanting to make the call. The plugin, then, seems like a good compromise and let the community decide piece by piece as they can compare old to new.

···

-- @thomasmckay

–
“The leader must aim high, see big, judge widely, thus setting himself apart form the ordinary people who debate in narrow confines.” ~ Charles De Gaulle

“Leadership is about making others better as a result of your presence and making sure that impact lasts in your absence.” ~ Harvard Business School

[thomasmckay]

Quick demo of current foreman roles UI and a katello style bastion UI via plugin

I understand that the roles UI has not gotten a lot of attention since the developers were repurposed over to staypuft. I'm suggesting that if/when resources are available to revisit it that the decision as to which underlying UI framework to fix things in is important to decide. At the moment, the interested "resource" is a community member: me-in-the-evening.

···

----- Original Message ----- > > With the upcoming coordinated release of foreman-1.6 and katello-2.0, there > will be a chance to reflect on the past year of hard work and consider paths > going forward for both communities. Clearly the collaboration has had > tangible benefits to both sides of development. Specifically, the installer > is awesome, hammer is a joy to use and extend, improvements on API > consistency, and much more. One topic that we've tried to pretend is not an > issue, though, is the UI. > > I've been very vocal internally about the need for a consistent UI across the > combined project. Perhaps this is more of a nice-to-have for an upstream > endeavor, but in my mind it is a requirement for the downstream enterprise > product. Admittedly I could be wrong and may be drawing too heavily on my > years of closed-source work. On the other hand, I think a unified UI for the > combined upstream would be pretty great too. > > So, I'd like to ask the community of developers what their opinions are in > regards to the future of coding in the UI area. I ask the developers because > presumably users will like improvements in ease of use and functionality > regardless of the code style it is written in. > > What we have today are two very different UI implementations. I will > apologize if I offend authors on either side; please correct me where my > facts are wrong. > > ERB - Foreman's UI is rendered largely server side in the form of "erb" files > that can directly access server code and the database to populate names, > lists, forms, etc. There are some portions that are "ajax" where the > rendered browser page will make calls back to the server to update something > or get additional information. A good example of this is the organization > edit page. You'll see, if you examine the html in the browser, that all the > various association tabs and their choices are hard coded into the html. For > example, if there are 150 users you will see all of them in the source. > > JS - Katello's UI is rendered entirely client side using angular.js > javascript (wrapped in an engine called "bastion", which is the name I'll > use here). Bastion provides an MVC layer on the client and populates its > data by making api calls to get json which it then uses to fill in the html. > If the organization page were to be rewritten in bastion, viewing the page > source would look almost empty. The difference would be that when the Users > tab was selected in the UI, an ajax api call would be made to the server to > get the users that were available to add to the org and rendered then. > Clicking a user to add them could also make another ajax call to update the > org and reflect that change right away in the UI. > > For reference, katello's UI started out exactly like foreman's is currently. > With each release of the product through Red Hat, though, customers asked > for more and more features that the erb and javascript struggled to > implement. When the code reached a snarly mess of jquery ajax and hacks, the > developers undertook the effort to start fresh. This is when angular.js was > started, some 18 months (or more) ago. I suspect that the devs that have > contributed to the existing foreman UI will recognize the same cloud looming > over their current infrastructure. Perhaps I'm wrong, though. > > There are pros and cons to both styles that I'd like to try to enumerate here > for discussion, addition, and argument. > > ERB pro - Very simple to write > ERB pro - Can access entire code and database readily when building the UI > ERB pro - Plugins can use 'deface' package to make changes > ERB con - Plugins have to use 'deface' package to make changes > ERB con - Two server side controllers, one for API and one for UI > ERB con - Full page reloads very common > > JS con - Has a much steeper learning curve to write > JS pro - Html portions is very similar to erb in its simplicity > JS con - All data must be available via API in JSON (could be a pro) > JS pro - UI consumes same documented REST API, also used by hammer > JS pro - UI is very dynamic, not requiring full page reloads > JS con - Limited to provided hooks to tweak UI, cannot use 'deface' > JS pro - Cannot use 'deface' > JS con - Existing 'deface' plugins will need a rewrite > > While it is not strictly necessary that unification of the UI implementations > needs to ever happen, it will be a point of divergence if resources are > invested in replacing a portion that the other half has chosen not to use. > (eg. Katello rewrites the roles UI in bastion and foreman chooses not to > include it.) > > There is a very elegant choice with the introduction of bastion UI plugins. A > UI overhaul can be one-sided to katello and not impact the standalone > foreman at all. I, for one, am vehemently against this and would argue > strongly that it should be one UI where the resources, be they upstream or > downstream, be focused. I haven't won that argument and don't see anyone > wanting to make the call. The plugin, then, seems like a good compromise and > let the community decide piece by piece as they can compare old to new. > > > > > -- > @thomasmckay > > -- > "The leader must aim high, see big, judge widely, thus setting himself apart > form the ordinary people who debate in narrow confines." ~ Charles De Gaulle > > "Leadership is about making others better as a result of your presence and > making sure that impact lasts in your absence." ~ Harvard Business School > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscribe@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. >

[lzap]

Hey Tom,

> JS - Katello's UI is rendered entirely client side using angular.js
> javascript (wrapped in an engine called "bastion", which is the name
> I'll use here). Bastion provides an MVC layer on the client and
> populates its data by making api calls to get json which it then uses
> to fill in the html. If the organization page were to be rewritten in
> bastion, viewing the page source would look almost empty. The
> difference would be that when the Users tab was selected in the UI, an
> ajax api call would be made to the server to get the users that were
> available to add to the org and rendered then. Clicking a user to add
> them could also make another ajax call to update the org and reflect
> that change right away in the UI.

Being total noob in the JS world (thus AngularJS), I don't get what
drives the need for Bastion. I've read its description but I still don't
understand why you don't use plain AngularJS.

> ERB pro - Very simple to write
> ERB pro - Can access entire code and database readily when building the UI
> ERB pro - Plugins can use 'deface' package to make changes
> ERB con - Plugins have to use 'deface' package to make changes

Not if we build APIs - and we've done that previously. We need an API
for plugins to extend things no matter if we have ERB or JS.

> ERB con - Two server side controllers, one for API and one for UI
> ERB con - Full page reloads very common

Why JavaScript people tend to list this as a problem? Really, page
refreshing is no big deal, if the app is properly coded, browser cache
works for all assets and pages are clean, it's almost instant. At least
that's my take from working on Foreman, I have to admit that static
portions of JS pages load faster but once you have some data on a page,
you need to wait for it anyway.

ERB pro - It is the standard way of doing things in Rails with all the
support and integration. I clearly remember Katello on MongoDB which was
swimming against the current and this could be similar case.

ERB pro - Server-side HTML generation was there years ago, will be there
years looking forward. I am bit worried if AngularJS is not another hype
that will bubble out next year. Google is introducing some Web
Components, Material Design and stuff (haven't read much about this
yet), will this work with AngularJS? Isn't there another yet-better
anglular-like framework emerging?

ERB cons - We will always need JavaScript therefore we mix two things
together often. Staying on one stack is definitely preferred.

> JS con - Has a much steeper learning curve to write

Than what? I am bit afraid here.

> JS pro - Html portions is very similar to erb in its simplicity
> JS con - All data must be available via API in JSON (could be a pro)
> JS pro - UI consumes same documented REST API, also used by hammer
> JS pro - UI is very dynamic, not requiring full page reloads
> JS con - Limited to provided hooks to tweak UI, cannot use 'deface'
> JS pro - Cannot use 'deface'

How would plugins extend the UI then?

> JS con - Existing 'deface' plugins will need a rewrite

I watched your video (nice) and I am afraid we are not comparing
technologies but rather better UI workflow that Katello apparently has.
Global search box, everything dynamic and rewritten from the ground up.
Foreman had to evolve from very basic pages. This should be improved and
Kyle already started on some designs. It's on the radar.

My current opinion, which can change during this tread, is that I like
Katello pages (looks really nice and more polished) but I still like the
standard way of doing things. I am biased because I know Rails and I
don't know Angular.

···

-- Later, Lukas #lzap Zapletal

[iNecas]

> Hey Tom,
>
> > JS - Katello's UI is rendered entirely client side using angular.js
> > javascript (wrapped in an engine called "bastion", which is the name
> > I'll use here). Bastion provides an MVC layer on the client and
> > populates its data by making api calls to get json which it then uses
> > to fill in the html. If the organization page were to be rewritten in
> > bastion, viewing the page source would look almost empty. The
> > difference would be that when the Users tab was selected in the UI, an
> > ajax api call would be made to the server to get the users that were
> > available to add to the org and rendered then. Clicking a user to add
> > them could also make another ajax call to update the org and reflect
> > that change right away in the UI.
>
> Being total noob in the JS world (thus AngularJS), I don't get what
> drives the need for Bastion. I've read its description but I still don't
> understand why you don't use plain AngularJS.

In my understanding, the bastion is set of tools and ways how to organize
the Angular code and couple of helpers on top of it. You can go many
ways with Angular, and bastion chooses some of them. It's more like
preset angular environment that has already some structure. When starting
with angular from scratch, you would probably soon end up with something very similar.

>
> > ERB pro - Very simple to write
> > ERB pro - Can access entire code and database readily when building the UI
> > ERB pro - Plugins can use 'deface' package to make changes
> > ERB con - Plugins have to use 'deface' package to make changes
>
> Not if we build APIs - and we've done that previously. We need an API
> for plugins to extend things no matter if we have ERB or JS.

True: the question is whether this API is forming. Might be interesting
to see if the Angular approach could bring some interesting features here.

>
> > ERB con - Two server side controllers, one for API and one for UI
> > ERB con - Full page reloads very common
>
> Why JavaScript people tend to list this as a problem? Really, page
> refreshing is no big deal, if the app is properly coded, browser cache
> works for all assets and pages are clean, it's almost instant. At least
> that's my take from working on Foreman, I have to admit that static
> portions of JS pages load faster but once you have some data on a page,
> you need to wait for it anyway.
>
> ERB pro - It is the standard way of doing things in Rails with all the
> support and integration. I clearly remember Katello on MongoDB which was
> swimming against the current and this could be similar case.

I don't think this has any relevance in this discussion. It's nothing new
in Rails to build just the API part on server GitHub - rails-api/rails-api: Rails for API only applications
and have different approaches for the UI.

> ERB pro - Server-side HTML generation was there years ago, will be there
> years looking forward. I am bit worried if AngularJS is not another hype
> that will bubble out next year. Google is introducing some Web
> Components, Material Design and stuff (haven't read much about this
> yet), will this work with AngularJS? Isn't there another yet-better
> anglular-like framework emerging?

Of source there is (you can ask authors of every competing framework).
Angular worked great for Katello so far and there is big community around it.
Of course, there are other frameworks, all having pros and cons. There are also other languages
that all have pros and cons. Should we rewrite Foreman in Go?

>
> ERB cons - We will always need JavaScript therefore we mix two things
> together often. Staying on one stack is definitely preferred.
>
> > JS con - Has a much steeper learning curve to write
>
> Than what? I am bit afraid here.

Than pure html in erb, for sure. On the other hand, looking at
some more complex javascript in Foreman, what
approach has steeper learning curve is questionanble: from my
experience, once you have the angular basics, it's quite easy
to add new stuff in there, as well as figure out what other
code is doing. Of course, there is some magic every now and then
that needs some more experienced developer to help with.

> > JS pro - Html portions is very similar to erb in its simplicity
> > JS con - All data must be available via API in JSON (could be a pro)
> > JS pro - UI consumes same documented REST API, also used by hammer
> > JS pro - UI is very dynamic, not requiring full page reloads
> > JS con - Limited to provided hooks to tweak UI, cannot use 'deface'
> > JS pro - Cannot use 'deface'
>
> How would plugins extend the UI then?
>
> > JS con - Existing 'deface' plugins will need a rewrite
>
> I watched your video (nice) and I am afraid we are not comparing
> technologies but rather better UI workflow that Katello apparently has.
> Global search box, everything dynamic and rewritten from the ground up.
> Foreman had to evolve from very basic pages. This should be improved and
> Kyle already started on some designs. It's on the radar.

Yes, for the full comparison, we would need to write the same workflow
with the Rails erb way. So far, the Rails way have influenced the Foreman
UI a lot: basically the current roles UI is what you get when you follow
the Rails conventions (which might not be a bad thing). However, if the
workflows are going to be enhanced based on UXD reviews, and not respecting
the Rails conventions that much, there might be some challenges that would
show the angular approach in different light.

On the other hand, the long Foreman's ERB history formed very useful
set of helpers that one can use: we would need to work more on the bastion
to make the usage more developer-friendly: angular has mechanisms for that.
Now I think it makes sense to look around and extract common UI components
into widgets to DRYify the bastion html templates a bit.

>
> My current opinion, which can change during this tread, is that I like
> Katello pages (looks really nice and more polished) but I still like the
> standard way of doing things. I am biased because I know Rails and I
> don't know Angular.

The good thing about this is, we can have both approaches in place for some time:
and if the Foreman's rest API works for starting writing the angular pages
for the foreman functionality, there shouldn't be a big influence on the Foreman core.

I had a chance to work both with Rails erb's + jQuery and the angular's approach and
I must say I like the angular's way much better, when it comes to building
more complex UI pages.

I also had chance to work with deface and I'm looking forward
for the proposal on the extendability approach from the Bastion:
without a good extendability story there is low chance to compete with the current UI
(no matter how bad the deface approach is).

I see a huge benefit in not having to have two sets of controllers, UI and API. Also,
if there are plans for making the Foreman pages more dynamic,
I think the developers deserve to be able to use appropriate tools.

I don't believe all the Foreman users/developers would want to have their
UI they love so much have rewritten into single-page application over night.
However, I believe, when having a chance to choose which one to use, the
Angular one might get quite popular very quickly. And if I was the one
to update the Foreman pages based on UXD suggestions, I would definitely
go the angular's approach.

Just my $0.02

– Ivan

···

----- Original Message -----

–
Later,
Lukas #lzap Zapletal

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[iNecas]

>
>
> > Hey Tom,
> >
> > > JS - Katello's UI is rendered entirely client side using angular.js
> > > javascript (wrapped in an engine called "bastion", which is the name
> > > I'll use here). Bastion provides an MVC layer on the client and
> > > populates its data by making api calls to get json which it then uses
> > > to fill in the html. If the organization page were to be rewritten in
> > > bastion, viewing the page source would look almost empty. The
> > > difference would be that when the Users tab was selected in the UI, an
> > > ajax api call would be made to the server to get the users that were
> > > available to add to the org and rendered then. Clicking a user to add
> > > them could also make another ajax call to update the org and reflect
> > > that change right away in the UI.
> >
> > Being total noob in the JS world (thus AngularJS), I don't get what
> > drives the need for Bastion. I've read its description but I still don't
> > understand why you don't use plain AngularJS.
>
> In my understanding, the bastion is set of tools and ways how to organize
> the Angular code and couple of helpers on top of it. You can go many
> ways with Angular, and bastion chooses some of them. It's more like
> preset angular environment that has already some structure. When starting
> with angular from scratch, you would probably soon end up with something very
> similar.
>
> >
> > > ERB pro - Very simple to write
> > > ERB pro - Can access entire code and database readily when building the
> > > UI
> > > ERB pro - Plugins can use 'deface' package to make changes
> > > ERB con - Plugins have to use 'deface' package to make changes
> >
> > Not if we build APIs - and we've done that previously. We need an API
> > for plugins to extend things no matter if we have ERB or JS.
>
> True: the question is whether this API is forming. Might be interesting
> to see if the Angular approach could bring some interesting features here.
>
> >
> > > ERB con - Two server side controllers, one for API and one for UI
> > > ERB con - Full page reloads very common
> >
> > Why JavaScript people tend to list this as a problem? Really, page
> > refreshing is no big deal, if the app is properly coded, browser cache
> > works for all assets and pages are clean, it's almost instant. At least
> > that's my take from working on Foreman, I have to admit that static
> > portions of JS pages load faster but once you have some data on a page,
> > you need to wait for it anyway.
> >
> > ERB pro - It is the standard way of doing things in Rails with all the
> > support and integration. I clearly remember Katello on MongoDB which was
> > swimming against the current and this could be similar case.
>
> I don't think this has any relevance in this discussion. It's nothing new
> in Rails to build just the API part on server
> GitHub - rails-api/rails-api: Rails for API only applications
> and have different approaches for the UI.
>
> > ERB pro - Server-side HTML generation was there years ago, will be there
> > years looking forward. I am bit worried if AngularJS is not another hype
> > that will bubble out next year. Google is introducing some Web
> > Components, Material Design and stuff (haven't read much about this
> > yet), will this work with AngularJS? Isn't there another yet-better
> > anglular-like framework emerging?
>
> Of source there is (you can ask authors of every competing framework).
> Angular worked great for Katello so far and there is big community around it.
> Of course, there are other frameworks, all having pros and cons. There are
> also other languages
> that all have pros and cons. Should we rewrite Foreman in Go?

Talking about rewriting Foreman to Go, it would be much easier
with the angular approach - for those not knowing the context: it's not happening.

···

----- Original Message ----- > ----- Original Message -----

ERB cons - We will always need JavaScript therefore we mix two things
together often. Staying on one stack is definitely preferred.

JS con - Has a much steeper learning curve to write

Than what? I am bit afraid here.

Than pure html in erb, for sure. On the other hand, looking at
some more complex javascript in Foreman, what
approach has steeper learning curve is questionanble: from my
experience, once you have the angular basics, it’s quite easy
to add new stuff in there, as well as figure out what other
code is doing. Of course, there is some magic every now and then
that needs some more experienced developer to help with.

JS pro - Html portions is very similar to erb in its simplicity
JS con - All data must be available via API in JSON (could be a pro)
JS pro - UI consumes same documented REST API, also used by hammer
JS pro - UI is very dynamic, not requiring full page reloads
JS con - Limited to provided hooks to tweak UI, cannot use ‘deface’
JS pro - Cannot use ‘deface’

How would plugins extend the UI then?

JS con - Existing ‘deface’ plugins will need a rewrite

I watched your video (nice) and I am afraid we are not comparing
technologies but rather better UI workflow that Katello apparently has.
Global search box, everything dynamic and rewritten from the ground up.
Foreman had to evolve from very basic pages. This should be improved and
Kyle already started on some designs. It’s on the radar.

Yes, for the full comparison, we would need to write the same workflow
with the Rails erb way. So far, the Rails way have influenced the Foreman
UI a lot: basically the current roles UI is what you get when you follow
the Rails conventions (which might not be a bad thing). However, if the
workflows are going to be enhanced based on UXD reviews, and not respecting
the Rails conventions that much, there might be some challenges that would
show the angular approach in different light.

On the other hand, the long Foreman’s ERB history formed very useful
set of helpers that one can use: we would need to work more on the bastion
to make the usage more developer-friendly: angular has mechanisms for that.
Now I think it makes sense to look around and extract common UI components
into widgets to DRYify the bastion html templates a bit.

My current opinion, which can change during this tread, is that I like
Katello pages (looks really nice and more polished) but I still like the
standard way of doing things. I am biased because I know Rails and I
don’t know Angular.

The good thing about this is, we can have both approaches in place for some
time:
and if the Foreman’s rest API works for starting writing the angular pages
for the foreman functionality, there shouldn’t be a big influence on the
Foreman core.

I had a chance to work both with Rails erb’s + jQuery and the angular’s
approach and
I must say I like the angular’s way much better, when it comes to building
more complex UI pages.

I also had chance to work with deface and I’m looking forward
for the proposal on the extendability approach from the Bastion:
without a good extendability story there is low chance to compete with the
current UI
(no matter how bad the deface approach is).

I see a huge benefit in not having to have two sets of controllers, UI and
API. Also,
if there are plans for making the Foreman pages more dynamic,
I think the developers deserve to be able to use appropriate tools.

I don’t believe all the Foreman users/developers would want to have their
UI they love so much have rewritten into single-page application over night.
However, I believe, when having a chance to choose which one to use, the
Angular one might get quite popular very quickly. And if I was the one
to update the Foreman pages based on UXD suggestions, I would definitely
go the angular’s approach.

Just my $0.02

– Ivan

–
Later,
Lukas #lzap Zapletal

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ehelms]

All,

I will include some specific responses inline below, however, as the
person who originally pushed Katello down this road, I'd like to give
some insight into the history of the choice we made.

During the days that Katello was the open-source project for what
became Cloud Forms 1.0, we had a UI that reflected the typical 960px
centered view but with lots of progressive JavaScript mechanisms such
as, for those that know, tupane. Over time, users and developers alike
started requesting more and more dynamic functionality. This led to
creating a swath of spaghetti that interlinked Rails views, UI only
controllers and and JavaScript within one another (e.g. the original
tupane code katello/app/assets/javascripts/common/panel.js at KATELLO-1.3 · Katello/katello · GitHub).

Once we hashed out, via design meetings, what new design we wanted for
all pages that represented a table + object details I began to
investigate if there was a better way to build this new design. I had
grown tired of the increasing effort to sustain the spaghetti due to
how inter-weaved all the pieces (views, JS, controller, helpers) had
become and looked for what new tools were out there. This led me to
investigate a number of JavaScript frameworks to see if we could do
better. I took a basic premise: build a table view that only changes
out the data. I then built this using Backbone, AngularJS, Knockout,
and early Ember. At the time, these were the most popular frameworks
that my research could discern. I quickly found that I was building
the AngularJS application faster and more understandably than the
rest. However, I did not stop there. I also considered:

a) Does the framework have longevity? Will there be support 2 years from now?
b) How easy is to build JS components?
c) How easy is it for a new developer to get on board?
d) Can the code written with said framework be tested?

The reasons I inevitably went with Angularjs, and should address some
concerns folks have mentioned are:

a) A primary goal of the Angularjs team is to build Angularjs using
methods and functionality that future Javascript specs are declaring
such that when those are implemented natively by the browser, the
Angularjs team can remove that functionality from the library and thus
reduce the foot print. And, where they see a concept created by them
that could be useful to the larger community, pushing those changes to
the browser spec so as to remove it from their library. A good example
of this is the watch/observe functionality that is now making it's way
into the browser.

Some stats:
27.2k stars on Github
928 contributors
4 years old
54k tagged questions on stack overflow
('angularjs' tag wiki - Stack Overflow)
75k+ hits on youtube when searching "angularjs"

b) Directives made creating components easy, and although the toughest
part of angular, this was easier than building components using basic
JS.

c) I'll let other developers who have built things subsequently chime
in here. For me, things are much quicker and have not degraded over
time like before. There are lots of tools for learning, plenty of
tutorials, videos and tons of short videos. Websites such as
http://angular.codeschool.com/. They also have case studies from third
parties - https://builtwith.angularjs.org/.

d) Angularjs came built with testing in mind via their test runner
Karma. We have since been able to write tests for all aspects of our
JS, generate coverage, run a Jenkins job on each PR for the testing.
Further, the tests can be run from the command line and run in less
than a minute.

After considering all of the above, I decided to build out the design
that had been created and spec'd out with Angularjs for the most
complex object we had (i.e. systems). I did this in order to
illuminate any pitfalls that may occur when applied to a complex
object. Further, I did not radically overhaul and push this upon the
project. Rather, I took an approach of showing off the steps along the
way to developers, asking questions of developers when I faced design
choices, and eventually providing this new look'n'feel and interface
via an experimental UI mode.

This experimental mode was a toggle-able user preference that when
enabled hid the old UI and displayed the new UI components. This
allowed users to play with and try out the new pieces without having
to completely give up (or break) old functionality. After we got the
experimental UI to the point of being ready for prime time, we then
flipped the scenario such that the old UI was still around but in a
legacy mode that a user could toggle on or off. This allowed us to
step through the building with a new implementation style as well as
test and use new pieces without having to make radical change.

As others have mentioned, a side benefit of this approach was that we
only needed to maintain one set of API controllers. This meant that we
found gaps in the logic that our UI controllers and API controllers
were assuming as well as gaps in the authentication by centralizing to
a single entry point.

The reason, in the previous thread, that I brought up the idea of UI
plugins was based upon our previous experience with experimental mode.
Creating UI plugins seems like a great way to not disrupt the core,
while simultaneously providing a method for the community to try out
this new way of creating and new design patterns. If, in the end, we
can convince you of what we believe with respect to how we should
build UI's in the Foreman community, then we can easily pull these
changes into the project as the main method.

I have, at times, got the impression that some feel as if we did not
take a measured and reasoned approach towards converting how we built
the Katello UI in the past to how we build it now. I hope this helps
explain a bit about where we are coming from, why we made the choices
we did and why we believe in the approach that we took.

Eric

>
>
>> Hey Tom,
>>
>> > JS - Katello's UI is rendered entirely client side using angular.js
>> > javascript (wrapped in an engine called "bastion", which is the name
>> > I'll use here). Bastion provides an MVC layer on the client and
>> > populates its data by making api calls to get json which it then uses
>> > to fill in the html. If the organization page were to be rewritten in
>> > bastion, viewing the page source would look almost empty. The
>> > difference would be that when the Users tab was selected in the UI, an
>> > ajax api call would be made to the server to get the users that were
>> > available to add to the org and rendered then. Clicking a user to add
>> > them could also make another ajax call to update the org and reflect
>> > that change right away in the UI.
>>
>> Being total noob in the JS world (thus AngularJS), I don't get what
>> drives the need for Bastion. I've read its description but I still don't
>> understand why you don't use plain AngularJS.
>
> In my understanding, the bastion is set of tools and ways how to organize
> the Angular code and couple of helpers on top of it. You can go many
> ways with Angular, and bastion chooses some of them. It's more like
> preset angular environment that has already some structure. When starting
> with angular from scratch, you would probably soon end up with something very similar.

Another way to think of the "core" aspects of Bastion, are if you took
the application controller, api controller, application helper and
other bits such as authorization from the Foreman rails application.
With those pieces you can use mostly regular Rails to build out
individual entities but you still need those common pieces that you
don't want to re-invent for each controller/model pair or even plugin.
The core of Bastion provides some of these common functionality and
helpers to make re-using UI patterns easier. The goal is that as a
basic UI developer you should only have to care about the business
logic.

>
>>
>> > ERB pro - Very simple to write
>> > ERB pro - Can access entire code and database readily when building the UI
>> > ERB pro - Plugins can use 'deface' package to make changes
>> > ERB con - Plugins have to use 'deface' package to make changes
>>
>> Not if we build APIs - and we've done that previously. We need an API
>> for plugins to extend things no matter if we have ERB or JS.
>
> True: the question is whether this API is forming. Might be interesting
> to see if the Angular approach could bring some interesting features here.
>
>>
>> > ERB con - Two server side controllers, one for API and one for UI
>> > ERB con - Full page reloads very common
>>
>> Why JavaScript people tend to list this as a problem? Really, page
>> refreshing is no big deal, if the app is properly coded, browser cache
>> works for all assets and pages are clean, it's almost instant. At least
>> that's my take from working on Foreman, I have to admit that static
>> portions of JS pages load faster but once you have some data on a page,
>> you need to wait for it anyway.

I don't see this as a problem. But there is something nice about only
changing the part of the page that needs to be changed. The way I look
at it are things like:

a) If the header, and tabs don't need to change then why do a full
page refresh when I could instead just change out the details of a
particular panel and keep the user in context?
b) Why reload the entire page just to change the dataset in the table
based on a search or filter?

>>
>> ERB pro - It is the standard way of doing things in Rails with all the
>> support and integration. I clearly remember Katello on MongoDB which was
>> swimming against the current and this could be similar case.
>
> I don't think this has any relevance in this discussion. It's nothing new
> in Rails to build just the API part on server GitHub - rails-api/rails-api: Rails for API only applications
> and have different approaches for the UI.
>
>> ERB pro - Server-side HTML generation was there years ago, will be there
>> years looking forward. I am bit worried if AngularJS is not another hype
>> that will bubble out next year. Google is introducing some Web
>> Components, Material Design and stuff (haven't read much about this
>> yet), will this work with AngularJS? Isn't there another yet-better
>> anglular-like framework emerging?
>
> Of source there is (you can ask authors of every competing framework).
> Angular worked great for Katello so far and there is big community around it.
> Of course, there are other frameworks, all having pros and cons. There are also other languages
> that all have pros and cons. Should we rewrite Foreman in Go?
>
>>
>> ERB cons - We will always need JavaScript therefore we mix two things
>> together often. Staying on one stack is definitely preferred.
>>
>> > JS con - Has a much steeper learning curve to write
>>
>> Than what? I am bit afraid here.
>
> Than pure html in erb, for sure. On the other hand, looking at
> some more complex javascript in Foreman, what
> approach has steeper learning curve is questionanble: from my
> experience, once you have the angular basics, it's quite easy
> to add new stuff in there, as well as figure out what other
> code is doing. Of course, there is some magic every now and then
> that needs some more experienced developer to help with.

I don't like statements like this due to the fact that it requires a
qualifier. Steeper learning curve than WHAT? A new developer? A
seasoned Rails developer? A ruby developer?

···

On Wed, Sep 10, 2014 at 6:25 AM, Ivan Necas wrote: > ----- Original Message -----

JS pro - Html portions is very similar to erb in its simplicity
JS con - All data must be available via API in JSON (could be a pro)
JS pro - UI consumes same documented REST API, also used by hammer
JS pro - UI is very dynamic, not requiring full page reloads
JS con - Limited to provided hooks to tweak UI, cannot use ‘deface’
JS pro - Cannot use ‘deface’

How would plugins extend the UI then?

JS con - Existing ‘deface’ plugins will need a rewrite

I watched your video (nice) and I am afraid we are not comparing
technologies but rather better UI workflow that Katello apparently has.
Global search box, everything dynamic and rewritten from the ground up.
Foreman had to evolve from very basic pages. This should be improved and
Kyle already started on some designs. It’s on the radar.

Yes, for the full comparison, we would need to write the same workflow
with the Rails erb way. So far, the Rails way have influenced the Foreman
UI a lot: basically the current roles UI is what you get when you follow
the Rails conventions (which might not be a bad thing). However, if the
workflows are going to be enhanced based on UXD reviews, and not respecting
the Rails conventions that much, there might be some challenges that would
show the angular approach in different light.

On the other hand, the long Foreman’s ERB history formed very useful
set of helpers that one can use: we would need to work more on the bastion
to make the usage more developer-friendly: angular has mechanisms for that.
Now I think it makes sense to look around and extract common UI components
into widgets to DRYify the bastion html templates a bit.

My current opinion, which can change during this tread, is that I like
Katello pages (looks really nice and more polished) but I still like the
standard way of doing things. I am biased because I know Rails and I
don’t know Angular.

The good thing about this is, we can have both approaches in place for some time:
and if the Foreman’s rest API works for starting writing the angular pages
for the foreman functionality, there shouldn’t be a big influence on the Foreman core.

I had a chance to work both with Rails erb’s + jQuery and the angular’s approach and
I must say I like the angular’s way much better, when it comes to building
more complex UI pages.

I also had chance to work with deface and I’m looking forward
for the proposal on the extendability approach from the Bastion:
without a good extendability story there is low chance to compete with the current UI
(no matter how bad the deface approach is).

I see a huge benefit in not having to have two sets of controllers, UI and API. Also,
if there are plans for making the Foreman pages more dynamic,
I think the developers deserve to be able to use appropriate tools.

I don’t believe all the Foreman users/developers would want to have their
UI they love so much have rewritten into single-page application over night.
However, I believe, when having a chance to choose which one to use, the
Angular one might get quite popular very quickly. And if I was the one
to update the Foreman pages based on UXD suggestions, I would definitely
go the angular’s approach.

Just my $0.02

– Ivan

–
Later,
Lukas #lzap Zapletal

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

–
You received this message because you are subscribed to the Google Groups “foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Walden_Raines]

> Why JavaScript people tend to list this as a problem? Really, page
> refreshing is no big deal, if the app is properly coded, browser cache
> works for all assets and pages are clean, it's almost instant. At least
> that's my take from working on Foreman, I have to admit that static
> portions of JS pages load faster but once you have some data on a page,
> you need to wait for it anyway.

It's more of a problem on a typical website that is accessed by multiple users. The belief is that it's better to get something in front of people's faces sooner (even with a loading spinner) on an initial page load than it is to have them wait for an entire page load. Plus subsequent page loads are much faster, even with caching of static assets. This is a big deal when you're talking about a B to C business where conversions matter.

> ERB pro - Server-side HTML generation was there years ago, will be there
> years looking forward. I am bit worried if AngularJS is not another hype
> that will bubble out next year. Google is introducing some Web
> Components, Material Design and stuff (haven't read much about this
> yet), will this work with AngularJS? Isn't there another yet-better
> anglular-like framework emerging?

AngularJS' directives [1] are basically a JS implementation of web components. You can create a directive and then reference it in the html like so:

<image-gallery width="300"></image-gallery>

Not sure what Material Design has to do with this debate.

JS (well, angular) pro - the view code is tested.

Cheers,
Walden

[1] AngularJS

···

----- Original Message ----- From: "Lukas Zapletal" To: foreman-dev@googlegroups.com Sent: Wednesday, September 10, 2014 5:08:52 AM Subject: Re: [foreman-dev] foreman and katello UI future

Hey Tom,

JS - Katello’s UI is rendered entirely client side using angular.js
javascript (wrapped in an engine called “bastion”, which is the name
I’ll use here). Bastion provides an MVC layer on the client and
populates its data by making api calls to get json which it then uses
to fill in the html. If the organization page were to be rewritten in
bastion, viewing the page source would look almost empty. The
difference would be that when the Users tab was selected in the UI, an
ajax api call would be made to the server to get the users that were
available to add to the org and rendered then. Clicking a user to add
them could also make another ajax call to update the org and reflect
that change right away in the UI.

Being total noob in the JS world (thus AngularJS), I don’t get what
drives the need for Bastion. I’ve read its description but I still don’t
understand why you don’t use plain AngularJS.

ERB pro - Very simple to write
ERB pro - Can access entire code and database readily when building the UI
ERB pro - Plugins can use ‘deface’ package to make changes
ERB con - Plugins have to use ‘deface’ package to make changes

Not if we build APIs - and we’ve done that previously. We need an API
for plugins to extend things no matter if we have ERB or JS.

ERB con - Two server side controllers, one for API and one for UI
ERB con - Full page reloads very common

Why JavaScript people tend to list this as a problem? Really, page
refreshing is no big deal, if the app is properly coded, browser cache
works for all assets and pages are clean, it’s almost instant. At least
that’s my take from working on Foreman, I have to admit that static
portions of JS pages load faster but once you have some data on a page,
you need to wait for it anyway.

ERB pro - It is the standard way of doing things in Rails with all the
support and integration. I clearly remember Katello on MongoDB which was
swimming against the current and this could be similar case.

ERB pro - Server-side HTML generation was there years ago, will be there
years looking forward. I am bit worried if AngularJS is not another hype
that will bubble out next year. Google is introducing some Web
Components, Material Design and stuff (haven’t read much about this
yet), will this work with AngularJS? Isn’t there another yet-better
anglular-like framework emerging?

ERB cons - We will always need JavaScript therefore we mix two things
together often. Staying on one stack is definitely preferred.

JS con - Has a much steeper learning curve to write

Than what? I am bit afraid here.

JS pro - Html portions is very similar to erb in its simplicity
JS con - All data must be available via API in JSON (could be a pro)
JS pro - UI consumes same documented REST API, also used by hammer
JS pro - UI is very dynamic, not requiring full page reloads
JS con - Limited to provided hooks to tweak UI, cannot use ‘deface’
JS pro - Cannot use ‘deface’

How would plugins extend the UI then?

JS con - Existing ‘deface’ plugins will need a rewrite

I watched your video (nice) and I am afraid we are not comparing
technologies but rather better UI workflow that Katello apparently has.
Global search box, everything dynamic and rewritten from the ground up.
Foreman had to evolve from very basic pages. This should be improved and
Kyle already started on some designs. It’s on the radar.

My current opinion, which can change during this tread, is that I like
Katello pages (looks really nice and more polished) but I still like the
standard way of doing things. I am biased because I know Rails and I
don’t know Angular.

–
Later,
Lukas #lzap Zapletal

–
You received this message because you are subscribed to the Google Groups “foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marek_Hulan]

Hello,

my 2c below in text

> > ERB pro - Very simple to write
> > ERB pro - Can access entire code and database readily when building the UI
> > ERB pro - Plugins can use 'deface' package to make changes
> > ERB con - Plugins have to use 'deface' package to make changes
>
> Not if we build APIs - and we've done that previously. We need an API
> for plugins to extend things no matter if we have ERB or JS.
>
> > ERB con - Two server side controllers, one for API and one for UI

Is ERB the real cause? I don't see a reason why not sharing the code and just
generate different view format. Sometimes there's a need for specific logic, but
this is minority and can be easily handled by respond blocks.

ERB pro - most of rails plugins/gems use ERB, we don't have much of them with
views, AFAIK will_paginate is only one atm, but this would mean we'd have to
reimplement all of those views.
ERB con - usually harder to do some advanced forms (e.g. host form in Foreman)

> > ERB con - Full page reloads very common
>
> Why JavaScript people tend to list this as a problem? Really, page
> refreshing is no big deal, if the app is properly coded, browser cache
> works for all assets and pages are clean, it's almost instant. At least
> that's my take from working on Foreman, I have to admit that static
> portions of JS pages load faster but once you have some data on a page,
> you need to wait for it anyway.
>
> ERB pro - It is the standard way of doing things in Rails with all the
> support and integration. I clearly remember Katello on MongoDB which was
> swimming against the current and this could be similar case.
>
> ERB pro - Server-side HTML generation was there years ago, will be there
> years looking forward. I am bit worried if AngularJS is not another hype
> that will bubble out next year. Google is introducing some Web
> Components, Material Design and stuff (haven't read much about this
> yet), will this work with AngularJS? Isn't there another yet-better
> anglular-like framework emerging?
>
> ERB cons - We will always need JavaScript therefore we mix two things
> together often. Staying on one stack is definitely preferred.
>
> > JS con - Has a much steeper learning curve to write
>
> Than what? I am bit afraid here.
>
> > JS pro - Html portions is very similar to erb in its simplicity
> > JS con - All data must be available via API in JSON (could be a pro)
> > JS pro - UI consumes same documented REST API, also used by hammer
> > JS pro - UI is very dynamic, not requiring full page reloads
> > JS con - Limited to provided hooks to tweak UI, cannot use 'deface'
> > JS pro - Cannot use 'deface'
>
> How would plugins extend the UI then?
>
> > JS con - Existing 'deface' plugins will need a rewrite
>
> I watched your video (nice) and I am afraid we are not comparing
> technologies but rather better UI workflow that Katello apparently has.
> Global search box, everything dynamic and rewritten from the ground up.
> Foreman had to evolve from very basic pages. This should be improved and
> Kyle already started on some designs. It's on the radar.

I might be biased since I'm responsible for current Foreman roles UI. I think
that issues you showed are mostly easy to fix. Role edit form where you can
change just name is really not much useful, easy to add user/usergroup
management. This is basically the only difference between those two UIs in
terms of how you find the assignment, you expect to assign role to user in role
form while in Foreman style we currently have to assign user in user form.
Lukáš already covered search and reloading of a page, also regarding the
speed, there's much bigger difference in development and production mode.

To be honest, I'm not sure whether rewriting the whole role UI is more efficient
than fixing issues you identified.

Also this may be very subjective but Foreman UI seems very clear to me. I know
how to do the task on every page. On the other hand the role UI in Katello
style you showed seems hard to understand from first look. When you're adding a
user, there are three nested pages

• list of roles (gives us action button New Role, 0 selected roles
  information and deselect link)
• role SAM administrator detail (gives us another buttons and tabs)
• user management of that role (gives us another buttons, another tabs,
  another selected info and links)

It's hard to compare based on just part of the whole app. I'm not against
improving the UI in Foreman. Even though I'm not a fan of full JS UI I would
be interested in modernization of what we have. But I think we should decide
to based on all pros and cons and either improve existing UI (evolution) or do
the rewrite (revolution). I usually prefer small steps. I guess I'll have to
look at Katello UI from lower perspective first.

···

-- Marek

[ohadlevy]

Just to make sure, please read my reply in positive light as thats how i
wrote it

> All,
>
> I will include some specific responses inline below, however, as the
> person who originally pushed Katello down this road, I'd like to give
> some insight into the history of the choice we made.
>
> During the days that Katello was the open-source project for what
> became Cloud Forms 1.0, we had a UI that reflected the typical 960px
> centered view but with lots of progressive JavaScript mechanisms such
> as, for those that know, tupane. Over time, users and developers alike
> started requesting more and more dynamic functionality. This led to
> creating a swath of spaghetti that interlinked Rails views, UI only
> controllers and and JavaScript within one another (e.g. the original
> tupane code
> katello/app/assets/javascripts/common/panel.js at KATELLO-1.3 · Katello/katello · GitHub
> ).
>

this is indeed true for Katello, but on the other side, Foreman is 5 years
old (today!) and we don't have spaghetti code at all,
the most "complicated" js we have is in the host edit form, totaling about
600 lines, surely that's not that bad.
we can and should improve our UI implementation, but based on my experience
i cant say that ERB/JS is evil or not manageable etc.

> Once we hashed out, via design meetings, what new design we wanted for
> all pages that represented a table + object details I began to
> investigate if there was a better way to build this new design. I had
> grown tired of the increasing effort to sustain the spaghetti due to
> how inter-weaved all the pieces (views, JS, controller, helpers) had
> become and looked for what new tools were out there. This led me to
> investigate a number of JavaScript frameworks to see if we could do
> better. I took a basic premise: build a table view that only changes
> out the data. I then built this using Backbone, AngularJS, Knockout,
> and early Ember. At the time, these were the most popular frameworks
> that my research could discern. I quickly found that I was building
> the AngularJS application faster and more understandably than the
> rest. However, I did not stop there. I also considered:
>
> a) Does the framework have longevity? Will there be support 2 years from
> now?
> b) How easy is to build JS components?
> c) How easy is it for a new developer to get on board?
> d) Can the code written with said framework be tested?
>

That is indeed great, however, at the same time, i assume it took you at
least one year to learn, implement etc.
The benefit of using common / simple UI practices, means that we dont have
to spend a lot of time in design, learning and testing which is outside the
scope of our tool.
It also means, that a lot of existing tools can be used, we use a lot of JS
libs (charting, editor, bootstrap, novnc, spice, multi select, data tables,
etc), that are either based on json( e.g. data tables) or on pure html
(which is also a very good official api such as multi select).

in other words, you've have built a framework based on angular, while it
can be the best framework in the world, the original question remains,
should we spend all of this time building a framework for a new UI workflow?

>
> The reasons I inevitably went with Angularjs, and should address some
> concerns folks have mentioned are:
>
> a) A primary goal of the Angularjs team is to build Angularjs using
> methods and functionality that future Javascript specs are declaring
> such that when those are implemented natively by the browser, the
> Angularjs team can remove that functionality from the library and thus
> reduce the foot print. And, where they see a concept created by them
> that could be useful to the larger community, pushing those changes to
> the browser spec so as to remove it from their library. A good example
> of this is the watch/observe functionality that is now making it's way
> into the browser.
>
> Some stats:
> 27.2k stars on Github
> 928 contributors
> 4 years old
> 54k tagged questions on stack overflow
> ('angularjs' tag wiki - Stack Overflow)
> 75k+ hits on youtube when searching "angularjs"
>
> b) Directives made creating components easy, and although the toughest
> part of angular, this was easier than building components using basic
> JS.
>
> c) I'll let other developers who have built things subsequently chime
> in here. For me, things are much quicker and have not degraded over
> time like before. There are lots of tools for learning, plenty of
> tutorials, videos and tons of short videos. Websites such as
> http://angular.codeschool.com/. They also have case studies from third
> parties - https://builtwith.angularjs.org/.
>
> d) Angularjs came built with testing in mind via their test runner
> Karma. We have since been able to write tests for all aspects of our
> JS, generate coverage, run a Jenkins job on each PR for the testing.
> Further, the tests can be run from the command line and run in less
> than a minute.
>
> After considering all of the above, I decided to build out the design
> that had been created and spec'd out with Angularjs for the most
> complex object we had (i.e. systems). I did this in order to
> illuminate any pitfalls that may occur when applied to a complex
> object. Further, I did not radically overhaul and push this upon the
> project. Rather, I took an approach of showing off the steps along the
> way to developers, asking questions of developers when I faced design
> choices, and eventually providing this new look'n'feel and interface
> via an experimental UI mode.
>
> This experimental mode was a toggle-able user preference that when
> enabled hid the old UI and displayed the new UI components. This
> allowed users to play with and try out the new pieces without having
> to completely give up (or break) old functionality. After we got the
> experimental UI to the point of being ready for prime time, we then
> flipped the scenario such that the old UI was still around but in a
> legacy mode that a user could toggle on or off. This allowed us to
> step through the building with a new implementation style as well as
> test and use new pieces without having to make radical change.
>
> As others have mentioned, a side benefit of this approach was that we
> only needed to maintain one set of API controllers. This meant that we
> found gaps in the logic that our UI controllers and API controllers
> were assuming as well as gaps in the authentication by centralizing to
> a single entry point.
>

I still dont see it as plus, API needs to be stable, and cant be modified,
while UI controllers can evolve quicker and allow breaking changes when you
want to change your UI.

>
> The reason, in the previous thread, that I brought up the idea of UI
> plugins was based upon our previous experience with experimental mode.
> Creating UI plugins seems like a great way to not disrupt the core,
> while simultaneously providing a method for the community to try out
> this new way of creating and new design patterns. If, in the end, we
> can convince you of what we believe with respect to how we should
> build UI's in the Foreman community, then we can easily pull these
> changes into the project as the main method.
>
> I have, at times, got the impression that some feel as if we did not
> take a measured and reasoned approach towards converting how we built
> the Katello UI in the past to how we build it now. I hope this helps
> explain a bit about where we are coming from, why we made the choices
> we did and why we believe in the approach that we took.
>

Overall, I think you guys did what is right for Katello, For foreman, I'm
having a hard time justifying a rewrite, overall the issues i see are:

Bastion:

• make simple pages complex (its much easier to use existing foreman
  helpers for the avg community developer)
• requires a whole different skillset (ruby, rails and now javascript,
  angular, api, json etc)
• is not extendable by plugins
• rewrite most likely will break existing plugins
• normally, takes longer to implement
• has no equivalent helpers like rails / foreman.

on the plus side, I do admit that our JS testing is lacking, and that some
pages are over complicated, refactoring will solve it, but i can't say that
AngularJS is the only solution.

Further, pure client side implementation is not required for a management
app with a limited amount of users… its obviously a plus, and there is a
good chance that I myself would start with Angular if I had started today,
but I have the feeling that the UI does not require that much of an
attention compared to the actual features the application can provide.

IMHO, we need to be as useful as possible (e.g. integrate with X, do Y etc)
while providing a high quality look & feel, I also believe that most users
that use both foreman / katello, do not see / feel the difference between
the pages at first.
and if performance is an issue, there are simple ways using turbolinks [1]
etc, that can mitigate that as well.

lastly, I'm not against improving, I'm mostly worried that it would slow us
down, and introduce a bottleneck, imho we must:

1. unify look and feel, this is needed regardless of the technologies used.
2. provide bastion as a plugin, so it wont work only in the context of
   katello
3. try to provide new ui plugins (as Joseph and Tom are doing with
   Ember/Angular)
4. continue developing the application as a whole, as neither UI will be
   rewritten in the near future.

in the mean while, try to promote bastion (or any other new stuff that you
want to promote), make sure developers can consume it easily and get the
buy in from users/developers.

[1]

another turbolinks attempt

Ohad

···

On Wed, Sep 10, 2014 at 4:12 PM, Eric D Helms wrote:

Eric

On Wed, Sep 10, 2014 at 6:25 AM, Ivan Necas inecas@redhat.com wrote:

----- Original Message -----

Hey Tom,

JS - Katello’s UI is rendered entirely client side using angular.js
javascript (wrapped in an engine called “bastion”, which is the name
I’ll use here). Bastion provides an MVC layer on the client and
populates its data by making api calls to get json which it then uses
to fill in the html. If the organization page were to be rewritten in
bastion, viewing the page source would look almost empty. The
difference would be that when the Users tab was selected in the UI, an
ajax api call would be made to the server to get the users that were
available to add to the org and rendered then. Clicking a user to add
them could also make another ajax call to update the org and reflect
that change right away in the UI.

Being total noob in the JS world (thus AngularJS), I don’t get what
drives the need for Bastion. I’ve read its description but I still don’t
understand why you don’t use plain AngularJS.

In my understanding, the bastion is set of tools and ways how to organize
the Angular code and couple of helpers on top of it. You can go many
ways with Angular, and bastion chooses some of them. It’s more like
preset angular environment that has already some structure. When starting
with angular from scratch, you would probably soon end up with something
very similar.

Another way to think of the “core” aspects of Bastion, are if you took
the application controller, api controller, application helper and
other bits such as authorization from the Foreman rails application.
With those pieces you can use mostly regular Rails to build out
individual entities but you still need those common pieces that you
don’t want to re-invent for each controller/model pair or even plugin.
The core of Bastion provides some of these common functionality and
helpers to make re-using UI patterns easier. The goal is that as a
basic UI developer you should only have to care about the business
logic.

ERB pro - Very simple to write
ERB pro - Can access entire code and database readily when building
the UI
ERB pro - Plugins can use ‘deface’ package to make changes
ERB con - Plugins have to use ‘deface’ package to make changes

Not if we build APIs - and we’ve done that previously. We need an API
for plugins to extend things no matter if we have ERB or JS.

True: the question is whether this API is forming. Might be interesting
to see if the Angular approach could bring some interesting features
here.

ERB con - Two server side controllers, one for API and one for UI
ERB con - Full page reloads very common

Why JavaScript people tend to list this as a problem? Really, page
refreshing is no big deal, if the app is properly coded, browser cache
works for all assets and pages are clean, it’s almost instant. At least
that’s my take from working on Foreman, I have to admit that static
portions of JS pages load faster but once you have some data on a page,
you need to wait for it anyway.

I don’t see this as a problem. But there is something nice about only
changing the part of the page that needs to be changed. The way I look
at it are things like:

a) If the header, and tabs don’t need to change then why do a full
page refresh when I could instead just change out the details of a
particular panel and keep the user in context?
b) Why reload the entire page just to change the dataset in the table
based on a search or filter?

ERB pro - It is the standard way of doing things in Rails with all the
support and integration. I clearly remember Katello on MongoDB which was
swimming against the current and this could be similar case.

I don’t think this has any relevance in this discussion. It’s nothing new
in Rails to build just the API part on server
GitHub - rails-api/rails-api: Rails for API only applications
and have different approaches for the UI.

ERB pro - Server-side HTML generation was there years ago, will be there
years looking forward. I am bit worried if AngularJS is not another hype
that will bubble out next year. Google is introducing some Web
Components, Material Design and stuff (haven’t read much about this
yet), will this work with AngularJS? Isn’t there another yet-better
anglular-like framework emerging?

Of source there is (you can ask authors of every competing framework).
Angular worked great for Katello so far and there is big community
around it.
Of course, there are other frameworks, all having pros and cons. There
are also other languages
that all have pros and cons. Should we rewrite Foreman in Go?

ERB cons - We will always need JavaScript therefore we mix two things
together often. Staying on one stack is definitely preferred.

JS con - Has a much steeper learning curve to write

Than what? I am bit afraid here.

Than pure html in erb, for sure. On the other hand, looking at
some more complex javascript in Foreman, what
approach has steeper learning curve is questionanble: from my
experience, once you have the angular basics, it’s quite easy
to add new stuff in there, as well as figure out what other
code is doing. Of course, there is some magic every now and then
that needs some more experienced developer to help with.

I don’t like statements like this due to the fact that it requires a
qualifier. Steeper learning curve than WHAT? A new developer? A
seasoned Rails developer? A ruby developer?

JS pro - Html portions is very similar to erb in its simplicity
JS con - All data must be available via API in JSON (could be a pro)
JS pro - UI consumes same documented REST API, also used by hammer
JS pro - UI is very dynamic, not requiring full page reloads
JS con - Limited to provided hooks to tweak UI, cannot use ‘deface’
JS pro - Cannot use ‘deface’

How would plugins extend the UI then?

JS con - Existing ‘deface’ plugins will need a rewrite

I watched your video (nice) and I am afraid we are not comparing
technologies but rather better UI workflow that Katello apparently has.
Global search box, everything dynamic and rewritten from the ground up.
Foreman had to evolve from very basic pages. This should be improved and
Kyle already started on some designs. It’s on the radar.

Yes, for the full comparison, we would need to write the same workflow
with the Rails erb way. So far, the Rails way have influenced the Foreman
UI a lot: basically the current roles UI is what you get when you follow
the Rails conventions (which might not be a bad thing). However, if the
workflows are going to be enhanced based on UXD reviews, and not
respecting
the Rails conventions that much, there might be some challenges that
would
sho

[lzap]

> It's more of a problem on a typical website that is accessed by
> multiple users. The belief is that it's better to get something in
> front of people's faces sooner (even with a loading spinner) on an
> initial page load than it is to have them wait for an entire page
> load. Plus subsequent page loads are much faster, even with caching
> of static assets. This is a big deal when you're talking about a B to
> C business where conversions matter.

And that was my point, Satellite 6 will never be B2C. Are have this
advantage of keeping the possibility of server-side rendered content.

···

-- Later, Lukas #lzap Zapletal

[thomasmckay]

> Hello,
>
> my 2c below in text
>
> > > ERB pro - Very simple to write
> > > ERB pro - Can access entire code and database readily when building the
> > > UI
> > > ERB pro - Plugins can use 'deface' package to make changes
> > > ERB con - Plugins have to use 'deface' package to make changes
> >
> > Not if we build APIs - and we've done that previously. We need an API
> > for plugins to extend things no matter if we have ERB or JS.
> >
> > > ERB con - Two server side controllers, one for API and one for UI
>
> Is ERB the real cause? I don't see a reason why not sharing the code and just
> generate different view format. Sometimes there's a need for specific logic,
> but
> this is minority and can be easily handled by respond blocks.
>
> ERB pro - most of rails plugins/gems use ERB, we don't have much of them with
> views, AFAIK will_paginate is only one atm, but this would mean we'd have to
> reimplement all of those views.
> ERB con - usually harder to do some advanced forms (e.g. host form in
> Foreman)
>
> > > ERB con - Full page reloads very common
> >
> > Why JavaScript people tend to list this as a problem? Really, page
> > refreshing is no big deal, if the app is properly coded, browser cache
> > works for all assets and pages are clean, it's almost instant. At least
> > that's my take from working on Foreman, I have to admit that static
> > portions of JS pages load faster but once you have some data on a page,
> > you need to wait for it anyway.
> >
> > ERB pro - It is the standard way of doing things in Rails with all the
> > support and integration. I clearly remember Katello on MongoDB which was
> > swimming against the current and this could be similar case.
> >
> > ERB pro - Server-side HTML generation was there years ago, will be there
> > years looking forward. I am bit worried if AngularJS is not another hype
> > that will bubble out next year. Google is introducing some Web
> > Components, Material Design and stuff (haven't read much about this
> > yet), will this work with AngularJS? Isn't there another yet-better
> > anglular-like framework emerging?
> >
> > ERB cons - We will always need JavaScript therefore we mix two things
> > together often. Staying on one stack is definitely preferred.
> >
> > > JS con - Has a much steeper learning curve to write
> >
> > Than what? I am bit afraid here.
> >
> > > JS pro - Html portions is very similar to erb in its simplicity
> > > JS con - All data must be available via API in JSON (could be a pro)
> > > JS pro - UI consumes same documented REST API, also used by hammer
> > > JS pro - UI is very dynamic, not requiring full page reloads
> > > JS con - Limited to provided hooks to tweak UI, cannot use 'deface'
> > > JS pro - Cannot use 'deface'
> >
> > How would plugins extend the UI then?
> >
> > > JS con - Existing 'deface' plugins will need a rewrite
> >
> > I watched your video (nice) and I am afraid we are not comparing
> > technologies but rather better UI workflow that Katello apparently has.
> > Global search box, everything dynamic and rewritten from the ground up.
> > Foreman had to evolve from very basic pages. This should be improved and
> > Kyle already started on some designs. It's on the radar.
>
> I might be biased since I'm responsible for current Foreman roles UI. I think
> that issues you showed are mostly easy to fix. Role edit form where you can
> change just name is really not much useful, easy to add user/usergroup
> management. This is basically the only difference between those two UIs in
> terms of how you find the assignment, you expect to assign role to user in
> role
> form while in Foreman style we currently have to assign user in user form.
> Lukáš already covered search and reloading of a page, also regarding the
> speed, there's much bigger difference in development and production mode.
>
> To be honest, I'm not sure whether rewriting the whole role UI is more
> efficient
> than fixing issues you identified.

I will suggest that it is much easier. My video left out some other very difficult foreman UI patterns which are present across the UI, not just in the roles page.

• The page number and total are at the bottom of the list so that one must scroll down to see and interact.
  • Yes, easy to fix
• Go to users page and select a user from the list (or another resource page), note that the url does not change to reflect what you are viewing.
  • Probably easy to fix?
• Go to page 4 of users list, select user, change default organization, submit. Note that you're back on page one and must now search for that user again. Url still indicates page 4.
  • Probably easy to fix?
• Simple association lists lack details in their patterns since they are just lists of names you move from one side to another.
  • Probably fixable by adding more columns to these All items / Selected items widget

Nothing that anyone points out as lacking in current foreman UI is not fixable, it all is. My point is that katello has already been there two years ago. It had the same problems of scale. When there are two roles, the current UI is fine. Now add 200 roles to match your users ("permission to manage content hosts registered by activation key with same name as user"). Try cloning an org admin role with 20 filters and changing it to a different org. Yes, or course the current UI can be fixed but maybe we should invest in the platform that has already iterated from vanilla.

That's really my overall concern: There are a lot of things that need fixing and improvement on both sides. I'd like to see the returns from the time being invested by all parties be maximized towards the future. Clearly I am a proponent of the bastion infrastructure, especially after having lived through katello's growth pains and seeing them again in foreman. Please convince me that fixing again all the things previously fixed is a better investment.

···

----- Original Message -----

Also this may be very subjective but Foreman UI seems very clear to me. I
know
how to do the task on every page. On the other hand the role UI in Katello
style you showed seems hard to understand from first look. When you’re adding
a
user, there are three nested pages

• list of roles (gives us action button New Role, 0 selected roles
  information and deselect link)
• role SAM administrator detail (gives us another buttons and tabs)
• user management of that role (gives us another buttons, another tabs,
  another selected info and links)

It’s hard to compare based on just part of the whole app. I’m not against
improving the UI in Foreman. Even though I’m not a fan of full JS UI I would
be interested in modernization of what we have. But I think we should decide
to based on all pros and cons and either improve existing UI (evolution) or
do
the rewrite (revolution). I usually prefer small steps. I guess I’ll have to
look at Katello UI from lower perspective first.

–
Marek

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[iNecas]

> Hello,
>
> my 2c below in text
>
> > > ERB pro - Very simple to write
> > > ERB pro - Can access entire code and database readily when building the
> > > UI
> > > ERB pro - Plugins can use 'deface' package to make changes
> > > ERB con - Plugins have to use 'deface' package to make changes
> >
> > Not if we build APIs - and we've done that previously. We need an API
> > for plugins to extend things no matter if we have ERB or JS.
> >
> > > ERB con - Two server side controllers, one for API and one for UI
>
> Is ERB the real cause? I don't see a reason why not sharing the code and just
> generate different view format. Sometimes there's a need for specific logic,
> but
> this is minority and can be easily handled by respond blocks.

Just a reminder: been there, done that, and there were reasons why
the UI and API controllers were separated (and I'm quite happy with the
decision). You either end up with the UI that's build around
the REST API and the resources it models or build the workflow-based
UI, but inevitably there will be logic needed just for the UI that the API doesn't
care about. Also, there are other requirements for the UI entry points vs. the API:
one expects the backward compatibility on the API level (versioning etc.), while the UI's main
point is to make the usage of the system the easiest possible.

Of course, there are also challenges for the API to be able to support the
UI requirements (mostly performance one for eliminating N+1 requests).
On the other hand, the improvements here get directly to the API
ready for use outside of the web UI world, which is not that bat at the end.

>
> ERB pro - most of rails plugins/gems use ERB, we don't have much of them with
> views, AFAIK will_paginate is only one atm, but this would mean we'd have to
> reimplement all of those views.
> ERB con - usually harder to do some advanced forms (e.g. host form in
> Foreman)
>
> > > ERB con - Full page reloads very common
> >
> > Why JavaScript people tend to list this as a problem? Really, page
> > refreshing is no big deal, if the app is properly coded, browser cache
> > works for all assets and pages are clean, it's almost instant. At least
> > that's my take from working on Foreman, I have to admit that static
> > portions of JS pages load faster but once you have some data on a page,
> > you need to wait for it anyway.
> >
> > ERB pro - It is the standard way of doing things in Rails with all the
> > support and integration. I clearly remember Katello on MongoDB which was
> > swimming against the current and this could be similar case.
> >
> > ERB pro - Server-side HTML generation was there years ago, will be there
> > years looking forward. I am bit worried if AngularJS is not another hype
> > that will bubble out next year. Google is introducing some Web
> > Components, Material Design and stuff (haven't read much about this
> > yet), will this work with AngularJS? Isn't there another yet-better
> > anglular-like framework emerging?
> >
> > ERB cons - We will always need JavaScript therefore we mix two things
> > together often. Staying on one stack is definitely preferred.
> >
> > > JS con - Has a much steeper learning curve to write
> >
> > Than what? I am bit afraid here.
> >
> > > JS pro - Html portions is very similar to erb in its simplicity
> > > JS con - All data must be available via API in JSON (could be a pro)
> > > JS pro - UI consumes same documented REST API, also used by hammer
> > > JS pro - UI is very dynamic, not requiring full page reloads
> > > JS con - Limited to provided hooks to tweak UI, cannot use 'deface'
> > > JS pro - Cannot use 'deface'
> >
> > How would plugins extend the UI then?
> >
> > > JS con - Existing 'deface' plugins will need a rewrite
> >
> > I watched your video (nice) and I am afraid we are not comparing
> > technologies but rather better UI workflow that Katello apparently has.
> > Global search box, everything dynamic and rewritten from the ground up.
> > Foreman had to evolve from very basic pages. This should be improved and
> > Kyle already started on some designs. It's on the radar.
>
> I might be biased since I'm responsible for current Foreman roles UI. I think
> that issues you showed are mostly easy to fix. Role edit form where you can
> change just name is really not much useful, easy to add user/usergroup
> management. This is basically the only difference between those two UIs in
> terms of how you find the assignment, you expect to assign role to user in
> role
> form while in Foreman style we currently have to assign user in user form.
> Lukáš already covered search and reloading of a page, also regarding the
> speed, there's much bigger difference in development and production mode.
>
> To be honest, I'm not sure whether rewriting the whole role UI is more
> efficient
> than fixing issues you identified.
>
> Also this may be very subjective but Foreman UI seems very clear to me. I
> know
> how to do the task on every page. On the other hand the role UI in Katello
> style you showed seems hard to understand from first look. When you're adding
> a
> user, there are three nested pages
> - list of roles (gives us action button New Role, 0 selected roles
> information and deselect link)
> - role SAM administrator detail (gives us another buttons and tabs)
> - user management of that role (gives us another buttons, another tabs,
> another selected info and links)
>
> It's hard to compare based on just part of the whole app. I'm not against
> improving the UI in Foreman. Even though I'm not a fan of full JS UI I would
> be interested in modernization of what we have. But I think we should decide
> to based on all pros and cons and either improve existing UI (evolution) or
> do
> the rewrite (revolution). I usually prefer small steps. I guess I'll have to
> look at Katello UI from lower perspective first.

As bad as it might sound, I really don't think there is other way to settle down on the future steps
without having the same feature implemented twice in the UI. That's also good
opportunity for one side to prove benefits to the other (ideally the measurable ones).
Small competition migth not be that bad thing here. And if the requirements
on the rich-client UI make the API more capable/stable that's a good thing.

Ad. evolution vs. revolution: (provided creationists are not right)
there must have been a time in a history, where
some living creature decided, that there is a good reason to get out of the sea:
that was quite revolutionary evolution

···

----- Original Message -----

–
Marek

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Walden_Raines]

> Also this may be very subjective but Foreman UI seems very clear to me. I know
> how to do the task on every page. On the other hand the role UI in Katello
> style you showed seems hard to understand from first look.

I think that is quite subjective and it's probably because you use Foreman's UI daily.

There is absolutely nothing clear about forms that contains multiple tabs of required information. Just my opinion but I would be surprised if it wasn't supported by data from user testing.

And at the end of the day user testing is as close as you can get in UI design to making decisions based on fact. If a majority agree that A is easier to use than B then it's difficult to argue with that. We should be doing more user testing to help inform this debate.

Cheers,
Walden

···

----- Original Message ----- From: "Marek Hulan" To: foreman-dev@googlegroups.com Sent: Wednesday, September 10, 2014 7:24:51 AM Subject: Re: [foreman-dev] foreman and katello UI future

Hello,

my 2c below in text

ERB pro - Very simple to write
ERB pro - Can access entire code and database readily when building the UI
ERB pro - Plugins can use ‘deface’ package to make changes
ERB con - Plugins have to use ‘deface’ package to make changes

Not if we build APIs - and we’ve done that previously. We need an API
for plugins to extend things no matter if we have ERB or JS.

ERB con - Two server side controllers, one for API and one for UI

Is ERB the real cause? I don’t see a reason why not sharing the code and just
generate different view format. Sometimes there’s a need for specific logic, but
this is minority and can be easily handled by respond blocks.

ERB pro - most of rails plugins/gems use ERB, we don’t have much of them with
views, AFAIK will_paginate is only one atm, but this would mean we’d have to
reimplement all of those views.
ERB con - usually harder to do some advanced forms (e.g. host form in Foreman)

ERB con - Full page reloads very common

Why JavaScript people tend to list this as a problem? Really, page
refreshing is no big deal, if the app is properly coded, browser cache
works for all assets and pages are clean, it’s almost instant. At least
that’s my take from working on Foreman, I have to admit that static
portions of JS pages load faster but once you have some data on a page,
you need to wait for it anyway.

ERB pro - It is the standard way of doing things in Rails with all the
support and integration. I clearly remember Katello on MongoDB which was
swimming against the current and this could be similar case.

ERB pro - Server-side HTML generation was there years ago, will be there
years looking forward. I am bit worried if AngularJS is not another hype
that will bubble out next year. Google is introducing some Web
Components, Material Design and stuff (haven’t read much about this
yet), will this work with AngularJS? Isn’t there another yet-better
anglular-like framework emerging?

ERB cons - We will always need JavaScript therefore we mix two things
together often. Staying on one stack is definitely preferred.

JS con - Has a much steeper learning curve to write

Than what? I am bit afraid here.

JS pro - Html portions is very similar to erb in its simplicity
JS con - All data must be available via API in JSON (could be a pro)
JS pro - UI consumes same documented REST API, also used by hammer
JS pro - UI is very dynamic, not requiring full page reloads
JS con - Limited to provided hooks to tweak UI, cannot use ‘deface’
JS pro - Cannot use ‘deface’

How would plugins extend the UI then?

JS con - Existing ‘deface’ plugins will need a rewrite

I watched your video (nice) and I am afraid we are not comparing
technologies but rather better UI workflow that Katello apparently has.
Global search box, everything dynamic and rewritten from the ground up.
Foreman had to evolve from very basic pages. This should be improved and
Kyle already started on some designs. It’s on the radar.

I might be biased since I’m responsible for current Foreman roles UI. I think
that issues you showed are mostly easy to fix. Role edit form where you can
change just name is really not much useful, easy to add user/usergroup
management. This is basically the only difference between those two UIs in
terms of how you find the assignment, you expect to assign role to user in role
form while in Foreman style we currently have to assign user in user form.
Lukáš already covered search and reloading of a page, also regarding the
speed, there’s much bigger difference in development and production mode.

To be honest, I’m not sure whether rewriting the whole role UI is more efficient
than fixing issues you identified.

Also this may be very subjective but Foreman UI seems very clear to me. I know
how to do the task on every page. On the other hand the role UI in Katello
style you showed seems hard to understand from first look. When you’re adding a
user, there are three nested pages

• list of roles (gives us action button New Role, 0 selected roles
  information and deselect link)
• role SAM administrator detail (gives us another buttons and tabs)
• user management of that role (gives us another buttons, another tabs,
  another selected info and links)

It’s hard to compare based on just part of the whole app. I’m not against
improving the UI in Foreman. Even though I’m not a fan of full JS UI I would
be interested in modernization of what we have. But I think we should decide
to based on all pros and cons and either improve existing UI (evolution) or do
the rewrite (revolution). I usually prefer small steps. I guess I’ll have to
look at Katello UI from lower perspective first.

–
Marek

–
You received this message because you are subscribed to the Google Groups “foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[thomasmckay]

> Just to make sure, please read my reply in positive light as thats how i
> wrote it
>
>
> > All,
> >
> > I will include some specific responses inline below, however, as the
> > person who originally pushed Katello down this road, I'd like to give
> > some insight into the history of the choice we made.
> >
> > During the days that Katello was the open-source project for what
> > became Cloud Forms 1.0, we had a UI that reflected the typical 960px
> > centered view but with lots of progressive JavaScript mechanisms such
> > as, for those that know, tupane. Over time, users and developers alike
> > started requesting more and more dynamic functionality. This led to
> > creating a swath of spaghetti that interlinked Rails views, UI only
> > controllers and and JavaScript within one another (e.g. the original
> > tupane code
> > katello/app/assets/javascripts/common/panel.js at KATELLO-1.3 · Katello/katello · GitHub
> > ).
> >
>
> this is indeed true for Katello, but on the other side, Foreman is 5 years
> old (today!) and we don't have spaghetti code at all,
> the most "complicated" js we have is in the host edit form, totaling about
> 600 lines, surely that's not that bad.
> we can and should improve our UI implementation, but based on my experience
> i cant say that ERB/JS is evil or not manageable etc.

I guess that's part of the question, isn't it? Will katello stay on its own side and foreman on its? I can see how this would make things easier: Foreman continues on its way while katello begins to sculpt things into the larger downstream products of Satellite-6 and Subscription Asset Manager (SAM). From a resources point of view, that means doubling in some areas that don't align. While I've been against this for the past year, I can certainly see why that's an less controversial choice. Maybe that's the right one, though. Is katello the upstream project then?

···

----- Original Message ----- > On Wed, Sep 10, 2014 at 4:12 PM, Eric D Helms wrote:

Once we hashed out, via design meetings, what new design we wanted for
all pages that represented a table + object details I began to
investigate if there was a better way to build this new design. I had
grown tired of the increasing effort to sustain the spaghetti due to
how inter-weaved all the pieces (views, JS, controller, helpers) had
become and looked for what new tools were out there. This led me to
investigate a number of JavaScript frameworks to see if we could do
better. I took a basic premise: build a table view that only changes
out the data. I then built this using Backbone, AngularJS, Knockout,
and early Ember. At the time, these were the most popular frameworks
that my research could discern. I quickly found that I was building
the AngularJS application faster and more understandably than the
rest. However, I did not stop there. I also considered:

a) Does the framework have longevity? Will there be support 2 years from
now?
b) How easy is to build JS components?
c) How easy is it for a new developer to get on board?
d) Can the code written with said framework be tested?

That is indeed great, however, at the same time, i assume it took you at
least one year to learn, implement etc.
The benefit of using common / simple UI practices, means that we dont have
to spend a lot of time in design, learning and testing which is outside the
scope of our tool.
It also means, that a lot of existing tools can be used, we use a lot of JS
libs (charting, editor, bootstrap, novnc, spice, multi select, data tables,
etc), that are either based on json( e.g. data tables) or on pure html
(which is also a very good official api such as multi select).

in other words, you’ve have built a framework based on angular, while it
can be the best framework in the world, the original question remains,
should we spend all of this time building a framework for a new UI workflow?

The reasons I inevitably went with Angularjs, and should address some
concerns folks have mentioned are:

a) A primary goal of the Angularjs team is to build Angularjs using
methods and functionality that future Javascript specs are declaring
such that when those are implemented natively by the browser, the
Angularjs team can remove that functionality from the library and thus
reduce the foot print. And, where they see a concept created by them
that could be useful to the larger community, pushing those changes to
the browser spec so as to remove it from their library. A good example
of this is the watch/observe functionality that is now making it’s way
into the browser.

Some stats:
27.2k stars on Github
928 contributors
4 years old
54k tagged questions on stack overflow
('angularjs' tag wiki - Stack Overflow)
75k+ hits on youtube when searching “angularjs”

b) Directives made creating components easy, and although the toughest
part of angular, this was easier than building components using basic
JS.

c) I’ll let other developers who have built things subsequently chime
in here. For me, things are much quicker and have not degraded over
time like before. There are lots of tools for learning, plenty of
tutorials, videos and tons of short videos. Websites such as
http://angular.codeschool.com/. They also have case studies from third
parties - https://builtwith.angularjs.org/.

d) Angularjs came built with testing in mind via their test runner
Karma. We have since been able to write tests for all aspects of our
JS, generate coverage, run a Jenkins job on each PR for the testing.
Further, the tests can be run from the command line and run in less
than a minute.

After considering all of the above, I decided to build out the design
that had been created and spec’d out with Angularjs for the most
complex object we had (i.e. systems). I did this in order to
illuminate any pitfalls that may occur when applied to a complex
object. Further, I did not radically overhaul and push this upon the
project. Rather, I took an approach of showing off the steps along the
way to developers, asking questions of developers when I faced design
choices, and eventually providing this new look’n’feel and interface
via an experimental UI mode.

This experimental mode was a toggle-able user preference that when
enabled hid the old UI and displayed the new UI components. This
allowed users to play with and try out the new pieces without having
to completely give up (or break) old functionality. After we got the
experimental UI to the point of being ready for prime time, we then
flipped the scenario such that the old UI was still around but in a
legacy mode that a user could toggle on or off. This allowed us to
step through the building with a new implementation style as well as
test and use new pieces without having to make radical change.

As others have mentioned, a side benefit of this approach was that we
only needed to maintain one set of API controllers. This meant that we
found gaps in the logic that our UI controllers and API controllers
were assuming as well as gaps in the authentication by centralizing to
a single entry point.

I still dont see it as plus, API needs to be stable, and cant be modified,
while UI controllers can evolve quicker and allow breaking changes when you
want to change your UI.

The reason, in the previous thread, that I brought up the idea of UI
plugins was based upon our previous experience with experimental mode.
Creating UI plugins seems like a great way to not disrupt the core,
while simultaneously providing a method for the community to try out
this new way of creating and new design patterns. If, in the end, we
can convince you of what we believe with respect to how we should
build UI’s in the Foreman community, then we can easily pull these
changes into the project as the main method.

I have, at times, got the impression that some feel as if we did not
take a measured and reasoned approach towards converting how we built
the Katello UI in the past to how we build it now. I hope this helps
explain a bit about where we are coming from, why we made the choices
we did and why we believe in the approach that we took.

Overall, I think you guys did what is right for Katello, For foreman, I’m
having a hard time justifying a rewrite, overall the issues i see are:

Bastion:

• make simple pages complex (its much easier to use existing foreman
  helpers for the avg community developer)
• requires a whole different skillset (ruby, rails and now javascript,
  angular, api, json etc)
• is not extendable by plugins
• rewrite most likely will break existing plugins
• normally, takes longer to implement
• has no equivalent helpers like rails / foreman.

on the plus side, I do admit that our JS testing is lacking, and that some
pages are over complicated, refactoring will solve it, but i can’t say that
AngularJS is the only solution.

Further, pure client side implementation is not required for a management
app with a limited amount of users… its obviously a plus, and there is a
good chance that I myself would start with Angular if I had started today,
but I have the feeling that the UI does not require that much of an
attention compared to the actual features the application can provide.

IMHO, we need to be as useful as possible (e.g. integrate with X, do Y etc)
while providing a high quality look & feel, I also believe that most users
that use both foreman / katello, do not see / feel the difference between
the pages at first.
and if performance is an issue, there are simple ways using turbolinks [1]
etc, that can mitigate that as well.

lastly, I’m not against improving, I’m mostly worried that it would slow us
down, and introduce a bottleneck, imho we must:

1. unify look and feel, this is needed regardless of the technologies used.
2. provide bastion as a plugin, so it wont work only in the context of
   katello
3. try to provide new ui plugins (as Joseph and Tom are doing with
   Ember/Angular)
4. continue developing the application as a whole, as neither UI will be
   rewritten in the near future.

in the mean while, try to promote bastion (or any other new stuff that you
want to promote), make sure developers can consume it easily and get the
buy in from users/developers.

[1]
another turbolinks attempt · ohadlevy/foreman@cf1498f · GitHub

Ohad

Eric

On Wed, Sep 10, 2014 at 6:25 AM, Ivan Necas inecas@redhat.com wrote:

----- Original Message -----

Hey Tom,

JS - Katello’s UI is rendered entirely client side using angular.js
javascript (wrapped in an engine called “bastion”, which is the name
I’ll use here). Bastion provides an MVC layer on the client and
populates its data by making api calls to get json which it then uses
to fill in the html. If the organization page were to be rewritten in
bastion, viewing the page source would look almost empty. The
difference would be that when the Users tab was selected in the UI, an
ajax api call would be made to the server to get the users that were
available to add to the org and rendered then. Clicking a user to add
them could also make another ajax call to update the org and reflect
that change right away in the UI.

Being total noob in the JS world (thus AngularJS), I don’t get what
drives the need for Bastion. I’ve read its description but I still don’t
understand why you don’t use plain AngularJS.

In my understanding, the bastion is set of tools and ways how to organize
the Angular code and couple of helpers on top of it. You can go many
ways with Angular, and bastion chooses some of them. It’s more like
preset angular environment that has already some structure. When starting
with angular from scratch, you would probably soon end up with something
very similar.

Another way to think of the “core” aspects of Bastion, are if you took
the application controller, api controller, application helper and
other bits such as authorization from the Foreman rails application.
With those pieces you can use mostly regular Rails to build out
individual entities but you still need those common pieces that you
don’t want to re-invent for each controller/model pair or even plugin.
The core of Bastion provides some of these common functionality and
helpers to make re-using UI patterns easier. The goal is that as a
basic UI developer you should only have to care about the business
logic.

ERB pro - Very simple to write
ERB pro - Can access entire code and database readily when building
the UI
ERB pro - Plugins can use ‘deface’ package to make changes
ERB con - Plugins have to use ‘deface’ package to make changes

Not if we build APIs - and we’ve done that previously. We need an API
for plugins to extend things no matter if we have ERB or JS.

True: the question is whether this API is forming. Might be interesting
to see if the Angular approach could bring some interesting features
here.

ERB con - Two server side controllers, one for API and one for UI
ERB con - Full page reloads very common

Why JavaScript people tend to list this as a problem? Really, page
refreshing is no big deal, if the app is properly coded, browser cache
works for all assets and pages are clean, it’s almost instant. At least
that’s my take from working on Foreman, I have to admit that static
portions of JS pages load faster but once you have some data on a page,
you need to wait for it anyway.

I don’t see this as a problem. But there is something nice about only
changing the part of the page that needs to be changed. The way I look
at it are things like:

a) If the header, and tabs don’t need to change then why do a full
page refresh when I could instead just change out the details of a
particular panel and keep the user in context?
b) Why reload the entire page just to change the dataset in the table
based on a search or filter?

ERB pro - It is the standard way of doing things in Rails with all the
support and integration. I clearly remember Katello on MongoDB which was
swimming against the current and this could be similar case.

I don’t think this has any relevance in this discussion. It’s nothing new
in Rails to build just the API part on server
GitHub - rails-api/rails-api: Rails for API only applications
and have different approaches for the UI.

ERB pro - Server-side HTML generation was there years ago, will be there
years looking forward. I am bit worried if AngularJS is not another hype
that will bubble out next year. Google is introducing some Web
Components, Material Design and stuff (haven’t read much about this
yet), will this work with AngularJS? Isn’t there another yet-better
anglular-like framework emerging?

Of source there is (you can ask authors of every competing framework).
Angular worked great for Katello so far and there is big community
around it.
Of course, there are other frameworks, all having pros and cons. There
are also other languages
that all have pros and cons. Should we rewrite Foreman in Go?

ERB cons - We will always need JavaScript therefore we mix two things
together often. Staying on one stack is definitely preferred.

JS con - Has a much steeper learning curve to write

Than what? I am bit afraid here.

Than pure html in erb, for sure. On the other hand, looking at
some more complex javascript in Foreman, what
approach has steeper learning curve is questionanble: from my
experience, once you have the angular basics, it’s quite easy
to add new stuff in there, as well as figure out what other
code is doing. Of course, there is some magic every now and then
that needs some more experienced developer to help with.

I don’t like statements like this due to the fact that it requires a
qualifier. Steeper learning curve than WHAT? A new developer? A
seasoned Rails developer? A ruby developer?

JS pro - Html portions is very similar to erb in its simplicity
JS con - All data must be available via API in JSON (could be a pro)
JS pro - UI consumes same documented REST API, also used by hammer
JS pro - UI is very dynamic, not requiring full page reloads
JS con - Limited to provided hooks to tweak UI, cannot use ‘deface’
JS pro - Cannot use ‘deface’

How would plugins extend the UI then?

JS con - Existing ‘deface’ plugins will need a rewrite

I watched your video (nice) and I am afraid we are not comparing
technologies but rather better UI workflow that Katello apparently has.
Global search box, everything dynamic and rewritten from the ground up.
Foreman had to evolve from very basic pages. This should be improved and
Kyle already started on some de

[ehelms]

Thanks for the thoughts Ohad. I, for one, do not think making a
singular decision is the right approach as you mention as this is true
of any other type of change that would be made (e.g. to switch to
Dynflow for orchestration has taken time and small steps as well). I
do want to clarify a few points below because I think there are some
misconceptions that I have heard time and time again.

> Just to make sure, please read my reply in positive light as thats how i
> wrote it
>
>>
>> All,
>>
>> I will include some specific responses inline below, however, as the
>> person who originally pushed Katello down this road, I'd like to give
>> some insight into the history of the choice we made.
>>
>> During the days that Katello was the open-source project for what
>> became Cloud Forms 1.0, we had a UI that reflected the typical 960px
>> centered view but with lots of progressive JavaScript mechanisms such
>> as, for those that know, tupane. Over time, users and developers alike
>> started requesting more and more dynamic functionality. This led to
>> creating a swath of spaghetti that interlinked Rails views, UI only
>> controllers and and JavaScript within one another (e.g. the original
>> tupane code
>> katello/app/assets/javascripts/common/panel.js at KATELLO-1.3 · Katello/katello · GitHub).
>
>
> this is indeed true for Katello, but on the other side, Foreman is 5 years
> old (today!) and we don't have spaghetti code at all,
> the most "complicated" js we have is in the host edit form, totaling about
> 600 lines, surely that's not that bad.
> we can and should improve our UI implementation, but based on my experience
> i cant say that ERB/JS is evil or not manageable etc.

First, Happy birthday! Second, the combination of JS (via jQuery
selectors), erb, and erb.js is to me spaghetti. The use of erb.js
links JS snippets to your Rails controllers and the data therein
making it hard to determine where a particular chunk of JS
functionality is coming from. jQuery style selectors to produce
workflows is not scalable and leads to the issues that we encountered.
You are required to build views and view chunks that have specific
classes and IDs that your JavaScript then grabs and manipulates that
makes figuring out which file manipulates what view difficult.

>
>>
>> Once we hashed out, via design meetings, what new design we wanted for
>> all pages that represented a table + object details I began to
>> investigate if there was a better way to build this new design. I had
>> grown tired of the increasing effort to sustain the spaghetti due to
>> how inter-weaved all the pieces (views, JS, controller, helpers) had
>> become and looked for what new tools were out there. This led me to
>> investigate a number of JavaScript frameworks to see if we could do
>> better. I took a basic premise: build a table view that only changes
>> out the data. I then built this using Backbone, AngularJS, Knockout,
>> and early Ember. At the time, these were the most popular frameworks
>> that my research could discern. I quickly found that I was building
>> the AngularJS application faster and more understandably than the
>> rest. However, I did not stop there. I also considered:
>>
>> a) Does the framework have longevity? Will there be support 2 years from
>> now?
>> b) How easy is to build JS components?
>> c) How easy is it for a new developer to get on board?
>> d) Can the code written with said framework be tested?
>
>
> That is indeed great, however, at the same time, i assume it took you at
> least one year to learn, implement etc.
> The benefit of using common / simple UI practices, means that we dont have
> to spend a lot of time in design, learning and testing which is outside the
> scope of our tool.
> It also means, that a lot of existing tools can be used, we use a lot of JS
> libs (charting, editor, bootstrap, novnc, spice, multi select, data tables,
> etc), that are either based on json( e.g. data tables) or on pure html
> (which is also a very good official api such as multi select).

Honestly, it took me about 2 months. After a year we were building
prototypes with ease, spinning up new pages and the bulk of our time
and effort was spent getting our API controllers into proper shape
because they had been neglected in favor of our previous UI
controllers. In the Angular world there are plenty of libraries
available for use (http://ngmodules.org/) and wrappers around
existing, popular libraries (e.g.
Angular directives for Bootstrap).

>
>
> in other words, you've have built a framework based on angular, while it can
> be the best framework in the world, the original question remains, should we
> spend all of this time building a framework for a new UI workflow?

We did not build a framework. This is the equivalent of saying that
Foreman is a framework. We built some common tools and functionality
just as a helper or application controller in a Rails application
centralizes common functionality.

>>
>>
>> The reasons I inevitably went with Angularjs, and should address some
>> concerns folks have mentioned are:
>>
>> a) A primary goal of the Angularjs team is to build Angularjs using
>> methods and functionality that future Javascript specs are declaring
>> such that when those are implemented natively by the browser, the
>> Angularjs team can remove that functionality from the library and thus
>> reduce the foot print. And, where they see a concept created by them
>> that could be useful to the larger community, pushing those changes to
>> the browser spec so as to remove it from their library. A good example
>> of this is the watch/observe functionality that is now making it's way
>> into the browser.
>>
>> Some stats:
>> 27.2k stars on Github
>> 928 contributors
>> 4 years old
>> 54k tagged questions on stack overflow
>> ('angularjs' tag wiki - Stack Overflow)
>> 75k+ hits on youtube when searching "angularjs"
>>
>> b) Directives made creating components easy, and although the toughest
>> part of angular, this was easier than building components using basic
>> JS.
>>
>> c) I'll let other developers who have built things subsequently chime
>> in here. For me, things are much quicker and have not degraded over
>> time like before. There are lots of tools for learning, plenty of
>> tutorials, videos and tons of short videos. Websites such as
>> http://angular.codeschool.com/. They also have case studies from third
>> parties - https://builtwith.angularjs.org/.
>>
>> d) Angularjs came built with testing in mind via their test runner
>> Karma. We have since been able to write tests for all aspects of our
>> JS, generate coverage, run a Jenkins job on each PR for the testing.
>> Further, the tests can be run from the command line and run in less
>> than a minute.
>>
>> After considering all of the above, I decided to build out the design
>> that had been created and spec'd out with Angularjs for the most
>> complex object we had (i.e. systems). I did this in order to
>> illuminate any pitfalls that may occur when applied to a complex
>> object. Further, I did not radically overhaul and push this upon the
>> project. Rather, I took an approach of showing off the steps along the
>> way to developers, asking questions of developers when I faced design
>> choices, and eventually providing this new look'n'feel and interface
>> via an experimental UI mode.
>>
>> This experimental mode was a toggle-able user preference that when
>> enabled hid the old UI and displayed the new UI components. This
>> allowed users to play with and try out the new pieces without having
>> to completely give up (or break) old functionality. After we got the
>> experimental UI to the point of being ready for prime time, we then
>> flipped the scenario such that the old UI was still around but in a
>> legacy mode that a user could toggle on or off. This allowed us to
>> step through the building with a new implementation style as well as
>> test and use new pieces without having to make radical change.
>>
>> As others have mentioned, a side benefit of this approach was that we
>> only needed to maintain one set of API controllers. This meant that we
>> found gaps in the logic that our UI controllers and API controllers
>> were assuming as well as gaps in the authentication by centralizing to
>> a single entry point.
>
>
> I still dont see it as plus, API needs to be stable, and cant be modified,
> while UI controllers can evolve quicker and allow breaking changes when you
> want to change your UI.
>>
>>
>> The reason, in the previous thread, that I brought up the idea of UI
>> plugins was based upon our previous experience with experimental mode.
>> Creating UI plugins seems like a great way to not disrupt the core,
>> while simultaneously providing a method for the community to try out
>> this new way of creating and new design patterns. If, in the end, we
>> can convince you of what we believe with respect to how we should
>> build UI's in the Foreman community, then we can easily pull these
>> changes into the project as the main method.
>>
>> I have, at times, got the impression that some feel as if we did not
>> take a measured and reasoned approach towards converting how we built
>> the Katello UI in the past to how we build it now. I hope this helps
>> explain a bit about where we are coming from, why we made the choices
>> we did and why we believe in the approach that we took.
>
>
> Overall, I think you guys did what is right for Katello, For foreman, I'm
> having a hard time justifying a rewrite, overall the issues i see are:
>
> Bastion:
> - make simple pages complex (its much easier to use existing foreman helpers
> for the avg community developer)

Much easier is subjective, and I'd argue this lacks evidence on either side.

> - requires a whole different skillset (ruby, rails and now javascript,
> angular, api, json etc)

Fair point, although JS already exists within Foreman and knowledge of
the API and it's JSON doesn't seem like asking for much compared to
the other knowledge that is required. This, however, can only be
proven/shown over time and with users trying.

> - is not extendable by plugins
> - rewrite most likely will break existing plugins

I'd argue that this is not true for current Foreman core either.
Pointing users to a library (deface) that can manipulate the UI in any
way they want is not providing extension and cannot scale. You can
never know what parts of the UI any user is changing at any given time
and thus any change to the UI is subject to breaking existing plugins.
However, I wouldn't argue for ever doing a drastic, quick change but
rather, like an API change, introduce it over time.

> - normally, takes longer to implement
> - has no equivalent helpers like rails / foreman.

As I always tell Tom when he makes passionate arguments, please
provide data and evidence.

>
> on the plus side, I do admit that our JS testing is lacking, and that some
> pages are over complicated, refactoring will solve it, but i can't say that
> AngularJS is the only solution.
>
> Further, pure client side implementation is not required for a management
> app with a limited amount of users… its obviously a plus, and there is a
> good chance that I myself would start with Angular if I had started today,
> but I have the feeling that the UI does not require that much of an
> attention compared to the actual features the application can provide.

I agree that a client side implementation is not required, and this
argument is not dealing with the scale of users but rather the
requirements around building dynamic components and workflows. The
original aspects that drove our push for client side was due in part
to requirements. We received requirements that increasing requested
dynamic interface components that provided workflows, and data views
and manipulation. Which we found were easier and quicker to build
through pure client side implementations and hitting the API than
attempting to combine Rails controller, views and jQuery.

As you said, the only course of action is to take the open source
approach, keep believing in what we believe and push to help others
see it as we do

···

On Wed, Sep 10, 2014 at 9:56 AM, Ohad Levy wrote: > On Wed, Sep 10, 2014 at 4:12 PM, Eric D Helms wrote:

IMHO, we need to be as useful as possible (e.g. integrate with X, do Y etc)
while providing a high quality look & feel, I also believe that most users
that use both foreman / katello, do not see / feel the difference between
the pages at first.
and if performance is an issue, there are simple ways using turbolinks [1]
etc, that can mitigate that as well.

lastly, I’m not against improving, I’m mostly worried that it would slow us
down, and introduce a bottleneck, imho we must:

1. unify look and feel, this is needed regardless of the technologies used.
2. provide bastion as a plugin, so it wont work only in the context of
   katello
3. try to provide new ui plugins (as Joseph and Tom are doing with
   Ember/Angular)
4. continue developing the application as a whole, as neither UI will be
   rewritten in the near future.

in the mean while, try to promote bastion (or any other new stuff that you
want to promote), make sure developers can consume it easily and get the buy
in from users/developers.

[1]
another turbolinks attempt · ohadlevy/foreman@cf1498f · GitHub

Ohad

Eric

On Wed, Sep 10, 2014 at 6:25 AM, Ivan Necas inecas@redhat.com wrote:

----- Original Message -----

Hey Tom,

JS - Katello’s UI is rendered entirely client side using angular.js
javascript (wrapped in an engine called “bastion”, which is the name
I’ll use here). Bastion provides an MVC layer on the client and
populates its data by making api calls to get json which it then uses
to fill in the html. If the organization page were to be rewritten in
bastion, viewing the page source would look almost empty. The
difference would be that when the Users tab was selected in the UI,
an
ajax api call would be made to the server to get the users that were
available to add to the org and rendered then. Clicking a user to add
them could also make another ajax call to update the org and reflect
that change right away in the UI.

Being total noob in the JS world (thus AngularJS), I don’t get what
drives the need for Bastion. I’ve read its description but I still
don’t
understand why you don’t use plain AngularJS.

In my understanding, the bastion is set of tools and ways how to
organize
the Angular code and couple of helpers on top of it. You can go many
ways with Angular, and bastion chooses some of them. It’s more like
preset angular environment that has already some structure. When
starting
with angular from scratch, you would probably soon end up with something
very similar.

Another way to think of the “core” aspects of Bastion, are if you took
the application controller, api controller, application helper and
other bits such as authorization from the Foreman rails application.
With those pieces you can use mostly regular Rails to build out
individual entities but you still need those common pieces that you
don’t want to re-invent for each controller/model pair or even plugin.
The core of Bastion provides some of these common functionality and
helpers to make re-using UI patterns easier. The goal is that as a
basic UI developer you should only have to care about the business
logic.

ERB pro - Very simple to write
ERB pro - Can access entire code and database readily when building
the UI
ERB pro - Plugins can use ‘deface’ package to make changes
ERB con - Plugins have to use ‘deface’ package to make changes

Not if we build APIs - and we’ve done that previously. We need an API
for plugins to extend things no matter if we have ERB or JS.

True: the question is whether this API is forming. Might be interesting
to see if the Angular approach could bring some interesting features
here.

ERB con - Two server side controllers, one for API and one for UI
ERB con - Full page reloads very common

Why JavaScript people tend to list this as a problem? Really, page
refreshing is no big deal, if the app is properly coded, browser cache
works for all assets and pages are clean, it’s almost instant. At least
that’s my take from working on Foreman, I have to admit that static
portions of JS pages load faster but once you have some data on a page,
you need to wait for it anyway.

I don’t see this as a problem. But there is something nice about only
changing the part of the page that needs to b

[Joseph_Magen]

Good discussion so far, but I think there are different answers depending on the context.

1. new UI Foreman plugin
2. changing Foreman core upstream
3. unifying UI downstream for products Sat6/SAM

Here's my takes on each:

1. new UI Foreman plugin

I'm in favor of Tom developing a new UI plugin for Foreman using Bastion. I'm also working (slowly) on the side for fun on an Ember UI plugin. Btw, I'll leave the Angular vs Ember discussion for another time. In short, I like them both. In the open source world, it's great to have many options within the Foreman ecosystem.

2. changing Foreman core upstream

I'm guessing this will require a UI plugin to be popular within the community to be merged in core.

3. unifying UI downstream for products Sat6/SAM

This is a separate issue. I understand and agree with Tom's rationale. However, since we don't want downstream to divert too much from upstream, this leads us back to context #1 and #2… in a circle…

Regards,

Joseph

···

----- Original Message ----- > From: "Ohad Levy" > To: "foreman-dev" > Sent: Wednesday, September 10, 2014 4:56:33 PM > Subject: Re: [foreman-dev] foreman and katello UI future > > Just to make sure, please read my reply in positive light as thats how i > wrote it :) > > On Wed, Sep 10, 2014 at 4:12 PM, Eric D Helms wrote: > > > All, > > > > I will include some specific responses inline below, however, as the > > person who originally pushed Katello down this road, I'd like to give > > some insight into the history of the choice we made. > > > > During the days that Katello was the open-source project for what > > became Cloud Forms 1.0, we had a UI that reflected the typical 960px > > centered view but with lots of progressive JavaScript mechanisms such > > as, for those that know, tupane. Over time, users and developers alike > > started requesting more and more dynamic functionality. This led to > > creating a swath of spaghetti that interlinked Rails views, UI only > > controllers and and JavaScript within one another (e.g. the original > > tupane code > > https://github.com/Katello/katello/blob/KATELLO-1.3/app/assets/javascripts/common/panel.js > > ). > > > > this is indeed true for Katello, but on the other side, Foreman is 5 years > old (today!) and we don't have spaghetti code at all, > the most "complicated" js we have is in the host edit form, totaling about > 600 lines, surely that's not that bad. > we can and should improve our UI implementation, but based on my experience > i cant say that ERB/JS is evil or not manageable etc. > > > > Once we hashed out, via design meetings, what new design we wanted for > > all pages that represented a table + object details I began to > > investigate if there was a better way to build this new design. I had > > grown tired of the increasing effort to sustain the spaghetti due to > > how inter-weaved all the pieces (views, JS, controller, helpers) had > > become and looked for what new tools were out there. This led me to > > investigate a number of JavaScript frameworks to see if we could do > > better. I took a basic premise: build a table view that only changes > > out the data. I then built this using Backbone, AngularJS, Knockout, > > and early Ember. At the time, these were the most popular frameworks > > that my research could discern. I quickly found that I was building > > the AngularJS application faster and more understandably than the > > rest. However, I did not stop there. I also considered: > > > > a) Does the framework have longevity? Will there be support 2 years from > > now? > > b) How easy is to build JS components? > > c) How easy is it for a new developer to get on board? > > d) Can the code written with said framework be tested? > > > > That is indeed great, however, at the same time, i assume it took you *at > least* one year to learn, implement etc. > The benefit of using common / simple UI practices, means that we dont have > to spend a lot of time in design, learning and testing which is outside the > scope of our tool. > It also means, that a lot of existing tools can be used, we use a lot of JS > libs (charting, editor, bootstrap, novnc, spice, multi select, data tables, > etc), that are either based on json( e.g. data tables) or on pure html > (which is also a very good official api such as multi select). > > > in other words, you've have built a framework based on angular, while it > can be the best framework in the world, the original question remains, > should we spend all of this time building a framework for a new UI workflow? > > > > > The reasons I inevitably went with Angularjs, and should address some > > concerns folks have mentioned are: > > > > a) A primary goal of the Angularjs team is to build Angularjs using > > methods and functionality that future Javascript specs are declaring > > such that when those are implemented natively by the browser, the > > Angularjs team can remove that functionality from the library and thus > > reduce the foot print. And, where they see a concept created by them > > that could be useful to the larger community, pushing those changes to > > the browser spec so as to remove it from their library. A good example > > of this is the watch/observe functionality that is now making it's way > > into the browser. > > > > Some stats: > > 27.2k stars on Github > > 928 contributors > > 4 years old > > 54k tagged questions on stack overflow > > (http://stackoverflow.com/tags/angularjs/info) > > 75k+ hits on youtube when searching "angularjs" > > > > b) Directives made creating components easy, and although the toughest > > part of angular, this was easier than building components using basic > > JS. > > > > c) I'll let other developers who have built things subsequently chime > > in here. For me, things are much quicker and have not degraded over > > time like before. There are lots of tools for learning, plenty of > > tutorials, videos and tons of short videos. Websites such as > > http://angular.codeschool.com/. They also have case studies from third > > parties - https://builtwith.angularjs.org/. > > > > d) Angularjs came built with testing in mind via their test runner > > Karma. We have since been able to write tests for all aspects of our > > JS, generate coverage, run a Jenkins job on each PR for the testing. > > Further, the tests can be run from the command line and run in less > > than a minute. > > > > After considering all of the above, I decided to build out the design > > that had been created and spec'd out with Angularjs for the most > > complex object we had (i.e. systems). I did this in order to > > illuminate any pitfalls that may occur when applied to a complex > > object. Further, I did not radically overhaul and push this upon the > > project. Rather, I took an approach of showing off the steps along the > > way to developers, asking questions of developers when I faced design > > choices, and eventually providing this new look'n'feel and interface > > via an experimental UI mode. > > > > This experimental mode was a toggle-able user preference that when > > enabled hid the old UI and displayed the new UI components. This > > allowed users to play with and try out the new pieces without having > > to completely give up (or break) old functionality. After we got the > > experimental UI to the point of being ready for prime time, we then > > flipped the scenario such that the old UI was still around but in a > > legacy mode that a user could toggle on or off. This allowed us to > > step through the building with a new implementation style as well as > > test and use new pieces without having to make radical change. > > > > As others have mentioned, a side benefit of this approach was that we > > only needed to maintain one set of API controllers. This meant that we > > found gaps in the logic that our UI controllers and API controllers > > were assuming as well as gaps in the authentication by centralizing to > > a single entry point. > > > > I still dont see it as plus, API needs to be stable, and cant be modified, > while UI controllers can evolve quicker and allow breaking changes when you > want to change your UI. > > > > > The reason, in the previous thread, that I brought up the idea of UI > > plugins was based upon our previous experience with experimental mode. > > Creating UI plugins seems like a great way to not disrupt the core, > > while simultaneously providing a method for the community to try out > > this new way of creating and new design patterns. If, in the end, we > > can convince you of what we believe with respect to how we should > > build UI's in the Foreman community, then we can easily pull these > > changes into the project as the main method. > > > > I have, at times, got the impression that some feel as if we did not > > take a measured and reasoned approach towards converting how we built > > the Katello UI in the past to how we build it now. I hope this helps > > explain a bit about where we are coming from, why we made the choices > > we did and why we believe in the approach that we took. > > > > Overall, I think you guys did what is right for Katello, For foreman, I'm > having a hard time justifying a rewrite, overall the issues i see are: > > Bastion: > - make simple pages complex (its much easier to use existing foreman > helpers for the avg community developer) > - requires a whole different skillset (ruby, rails and now javascript, > angular, api, json etc) > - is not extendable by plugins > - rewrite most likely will break existing plugins > - normally, takes longer to implement > - has no equivalent helpers like rails / foreman. > > on the plus side, I do admit that our JS testing is lacking, and that some > pages are over complicated, refactoring will solve it, but i can't say that > AngularJS is the only solution. > > Further, pure client side implementation is not required for a management > app with a limited amount of users.. its obviously a plus, and there is a > good chance that I myself would start with Angular if I had started today, > but I have the feeling that the UI does not require that much of an > attention compared to the actual features the application can provide. > > IMHO, we need to be as useful as possible (e.g. integrate with X, do Y etc) > while providing a high quality look & feel, I also believe that most users > that use both foreman / katello, do not see / feel the difference between > the pages at first. > and if performance is an issue, there are simple ways using turbolinks [1] > etc, that can mitigate that as well. > > lastly, I'm not against improving, I'm mostly worried that it would slow us > down, and introduce a bottleneck, imho we must: > > 1. unify look and feel, this is needed regardless of the technologies used. > 2. provide bastion as a plugin, so it wont work only in the context of > katello > 3. try to provide new ui plugins (as Joseph and Tom are doing with > Ember/Angular) > 4. continue developing the application as a whole, as neither UI will be > rewritten in the near future. > > in the mean while, try to promote bastion (or any other new stuff that you > want to promote), make sure developers can consume it easily and get the > buy in from users/developers. > > > [1] > https://github.com/ohadlevy/foreman/commit/cf1498f91ee47d00dbdbb185ccb63251c8316105 > > Ohad > > > > Eric > > > > On Wed, Sep 10, 2014 at 6:25 AM, Ivan Necas wrote: > > > > > > > > > ----- Original Message ----- > > >> Hey Tom, > > >> > > >> > JS - Katello's UI is rendered entirely client side using angular.js > > >> > javascript (wrapped in an engine called "bastion", which is the name > > >> > I'll use here). Bastion provides an MVC layer on the client and > > >> > populates its data by making api calls to get json which it then uses > > >> > to fill in the html. If the organization page were to be rewritten in > > >> > bastion, viewing the page source would look almost empty. The > > >> > difference would be that when the Users tab was selected in the UI, an > > >> > ajax api call would be made to the server to get the users that were > > >> > available to add to the org and rendered then. Clicking a user to add > > >> > them could also make another ajax call to update the org and reflect > > >> > that change right away in the UI. > > >> > > >> Being total noob in the JS world (thus AngularJS), I don't get what > > >> drives the need for Bastion. I've read its description but I still don't > > >> understand why you don't use plain AngularJS. > > > > > > In my understanding, the bastion is set of tools and ways how to organize > > > the Angular code and couple of helpers on top of it. You can go many > > > ways with Angular, and bastion chooses some of them. It's more like > > > preset angular environment that has already some structure. When starting > > > with angular from scratch, you would probably soon end up with something > > very similar. > > > > Another way to think of the "core" aspects of Bastion, are if you took > > the application controller, api controller, application helper and > > other bits such as authorization from the Foreman rails application. > > With those pieces you can use mostly regular Rails to build out > > individual entities but you still need those common pieces that you > > don't want to re-invent for each controller/model pair or even plugin. > > The core of Bastion provides some of these common functionality and > > helpers to make re-using UI patterns easier. The goal is that as a > > basic UI developer you should only have to care about the business > > logic. > > > > > > > >> > > >> > ERB pro - Very simple to write > > >> > ERB pro - Can access entire code and database readily when building > > the UI > > >> > ERB pro - Plugins can use 'deface' package to make changes > > >> > ERB con - Plugins have to use 'deface' package to make changes > > >> > > >> Not if we build APIs - and we've done that previously. We need an API > > >> for plugins to extend things no matter if we have ERB or JS. > > > > > > True: the question is whether this API is forming. Might be interesting > > > to see if the Angular approach could bring some interesting features > > here. > > > > > >> > > >> > ERB con - Two server side controllers, one for API and one for UI > > >> > ERB con - Full page reloads very common > > >> > > >> Why JavaScript people tend to list this as a problem? :-) Really, page > > >> refreshing is no big deal, if the app is properly coded, browser cache > > >> works for all assets and pages are clean, it's almost instant. At least > > >> that's my take from working on Foreman, I have to admit that static > > >> portions of JS pages load faster but once you have some data on a page, > > >> you need to wait for it anyway. > > > > I don't see this as a problem. But there is something nice about only > > changing the part of the page that needs to be changed. The way I look > > at it are things like: > > > > a) If the header, and tabs don't need to change then why do a full > > page refresh when I could instead just change out the details of a > > particular panel and keep the user in context? > > b) Why reload the entire page just to change the dataset in the table > > based on a search or filter? > > > > >> > > >> ERB pro - It is the standard way of doing things in Rails with all the > > >> support and integration. I clearly remember Katello on MongoDB which was > > >> swimming against the current and this could be similar case. > > > > > > I don't think this has any relevance in this discussion. It's nothing new > > > in Rails to build just the API part on server > > https://github.com/rails-api/rails-api > > > and have different approaches for the UI. > > > > > >> ERB pro - Server-side HTML generation was there years ago, will be there > > >> years looking forward. I am bit worried if AngularJS is not another hype > > >> that will bubble out next year. Google is introducing some Web > > >> Components, Material Design and stuff (haven't read much about this > > >> yet), will this work with AngularJS? Isn't there another yet-better > > >> anglular-like framework emerging? > > > > > > Of source there is (you can ask authors of every competing framework). > > > Angular worked great for Katello so far and there is big community > > around it. > > > Of course, there are other frameworks, all having pros and cons. There > > are also other languages > > > that all have pros and cons. Should we rewrite Foreman in Go? > > > > > >> > > >> ERB cons - We will always need JavaScript therefore we mix two things > > >> together often. Staying on one stack is definitely preferred. > > >> > > >> > JS con - Has a much steeper learning curve to write > > >> > > >> Than what? I am bit afraid here. > > > > > > Than pure html in erb, for sure. On the other hand, looking at > > > some more complex javascript in Foreman, what > > > approach has steeper learning curve is questionanble: from my > > > experience, once you have the angular basics, it's quite easy > > > to add new stuff in there, as well as figure out what other > > > code is doing. Of course, there is some magic every now and then > > > that needs some more experienced developer to help with. > > > > I don't like statements like this due to the fact that it requires a > > qualifier. Steeper learning curve than WHAT? A new developer? A > > seasoned Rails developer? A ruby developer? > > > > > > > >> > JS pro - Html portions is very similar to erb in its simplicity > > >> > JS con - All data must be available via API in JSON (could be a pro) > > >> > JS pro - UI consumes same documented REST API, also used by hammer > > >> > JS pro - UI is very dynamic

[iNecas]

> Thanks for the thoughts Ohad. I, for one, do not think making a
> singular decision is the right approach as you mention as this is true
> of any other type of change that would be made (e.g. to switch to
> Dynflow for orchestration has taken time and small steps as well). I
> do want to clarify a few points below because I think there are some
> misconceptions that I have heard time and time again.
>
> > Just to make sure, please read my reply in positive light as thats how i
> > wrote it

I'm sure we all write and read the posts in positive light. Could we add
some disclaimer about this practice on this mailing list?

> >
> >>
> >> All,
> >>
> >> I will include some specific responses inline below, however, as the
> >> person who originally pushed Katello down this road, I'd like to give
> >> some insight into the history of the choice we made.
> >>
> >> During the days that Katello was the open-source project for what
> >> became Cloud Forms 1.0, we had a UI that reflected the typical 960px
> >> centered view but with lots of progressive JavaScript mechanisms such
> >> as, for those that know, tupane. Over time, users and developers alike
> >> started requesting more and more dynamic functionality. This led to
> >> creating a swath of spaghetti that interlinked Rails views, UI only
> >> controllers and and JavaScript within one another (e.g. the original
> >> tupane code
> >> katello/app/assets/javascripts/common/panel.js at KATELLO-1.3 · Katello/katello · GitHub).
> >
> >
> > this is indeed true for Katello, but on the other side, Foreman is 5 years
> > old (today!) and we don't have spaghetti code at all,
> > the most "complicated" js we have is in the host edit form, totaling about
> > 600 lines, surely that's not that bad.
> > we can and should improve our UI implementation, but based on my experience
> > i cant say that ERB/JS is evil or not manageable etc.
>
> First, Happy birthday! Second, the combination of JS (via jQuery
> selectors), erb, and erb.js is to me spaghetti. The use of erb.js
> links JS snippets to your Rails controllers and the data therein
> making it hard to determine where a particular chunk of JS
> functionality is coming from. jQuery style selectors to produce
> workflows is not scalable and leads to the issues that we encountered.
> You are required to build views and view chunks that have specific
> classes and IDs that your JavaScript then grabs and manipulates that
> makes figuring out which file manipulates what view difficult.
>
> >
> >>
> >> Once we hashed out, via design meetings, what new design we wanted for
> >> all pages that represented a table + object details I began to
> >> investigate if there was a better way to build this new design. I had
> >> grown tired of the increasing effort to sustain the spaghetti due to
> >> how inter-weaved all the pieces (views, JS, controller, helpers) had
> >> become and looked for what new tools were out there. This led me to
> >> investigate a number of JavaScript frameworks to see if we could do
> >> better. I took a basic premise: build a table view that only changes
> >> out the data. I then built this using Backbone, AngularJS, Knockout,
> >> and early Ember. At the time, these were the most popular frameworks
> >> that my research could discern. I quickly found that I was building
> >> the AngularJS application faster and more understandably than the
> >> rest. However, I did not stop there. I also considered:
> >>
> >> a) Does the framework have longevity? Will there be support 2 years from
> >> now?
> >> b) How easy is to build JS components?
> >> c) How easy is it for a new developer to get on board?
> >> d) Can the code written with said framework be tested?
> >
> >
> > That is indeed great, however, at the same time, i assume it took you at
> > least one year to learn, implement etc.
> > The benefit of using common / simple UI practices, means that we dont have
> > to spend a lot of time in design, learning and testing which is outside the
> > scope of our tool.
> > It also means, that a lot of existing tools can be used, we use a lot of JS
> > libs (charting, editor, bootstrap, novnc, spice, multi select, data tables,
> > etc), that are either based on json( e.g. data tables) or on pure html
> > (which is also a very good official api such as multi select).
>
> Honestly, it took me about 2 months. After a year we were building
> prototypes with ease, spinning up new pages and the bulk of our time
> and effort was spent getting our API controllers into proper shape
> because they had been neglected in favor of our previous UI
> controllers. In the Angular world there are plenty of libraries
> available for use (http://ngmodules.org/) and wrappers around
> existing, popular libraries (e.g.
> Angular directives for Bootstrap).
>
> >
> >
> > in other words, you've have built a framework based on angular, while it
> > can
> > be the best framework in the world, the original question remains, should
> > we
> > spend all of this time building a framework for a new UI workflow?
>
> We did not build a framework. This is the equivalent of saying that
> Foreman is a framework. We built some common tools and functionality
> just as a helper or application controller in a Rails application
> centralizes common functionality.
>
> >>
> >>
> >> The reasons I inevitably went with Angularjs, and should address some
> >> concerns folks have mentioned are:
> >>
> >> a) A primary goal of the Angularjs team is to build Angularjs using
> >> methods and functionality that future Javascript specs are declaring
> >> such that when those are implemented natively by the browser, the
> >> Angularjs team can remove that functionality from the library and thus
> >> reduce the foot print. And, where they see a concept created by them
> >> that could be useful to the larger community, pushing those changes to
> >> the browser spec so as to remove it from their library. A good example
> >> of this is the watch/observe functionality that is now making it's way
> >> into the browser.
> >>
> >> Some stats:
> >> 27.2k stars on Github
> >> 928 contributors
> >> 4 years old
> >> 54k tagged questions on stack overflow
> >> ('angularjs' tag wiki - Stack Overflow)
> >> 75k+ hits on youtube when searching "angularjs"
> >>
> >> b) Directives made creating components easy, and although the toughest
> >> part of angular, this was easier than building components using basic
> >> JS.
> >>
> >> c) I'll let other developers who have built things subsequently chime
> >> in here. For me, things are much quicker and have not degraded over
> >> time like before. There are lots of tools for learning, plenty of
> >> tutorials, videos and tons of short videos. Websites such as
> >> http://angular.codeschool.com/. They also have case studies from third
> >> parties - https://builtwith.angularjs.org/.
> >>
> >> d) Angularjs came built with testing in mind via their test runner
> >> Karma. We have since been able to write tests for all aspects of our
> >> JS, generate coverage, run a Jenkins job on each PR for the testing.
> >> Further, the tests can be run from the command line and run in less
> >> than a minute.
> >>
> >> After considering all of the above, I decided to build out the design
> >> that had been created and spec'd out with Angularjs for the most
> >> complex object we had (i.e. systems). I did this in order to
> >> illuminate any pitfalls that may occur when applied to a complex
> >> object. Further, I did not radically overhaul and push this upon the
> >> project. Rather, I took an approach of showing off the steps along the
> >> way to developers, asking questions of developers when I faced design
> >> choices, and eventually providing this new look'n'feel and interface
> >> via an experimental UI mode.
> >>
> >> This experimental mode was a toggle-able user preference that when
> >> enabled hid the old UI and displayed the new UI components. This
> >> allowed users to play with and try out the new pieces without having
> >> to completely give up (or break) old functionality. After we got the
> >> experimental UI to the point of being ready for prime time, we then
> >> flipped the scenario such that the old UI was still around but in a
> >> legacy mode that a user could toggle on or off. This allowed us to
> >> step through the building with a new implementation style as well as
> >> test and use new pieces without having to make radical change.
> >>
> >> As others have mentioned, a side benefit of this approach was that we
> >> only needed to maintain one set of API controllers. This meant that we
> >> found gaps in the logic that our UI controllers and API controllers
> >> were assuming as well as gaps in the authentication by centralizing to
> >> a single entry point.
> >
> >
> > I still dont see it as plus, API needs to be stable, and cant be modified,
> > while UI controllers can evolve quicker and allow breaking changes when you
> > want to change your UI.
> >>
> >>
> >> The reason, in the previous thread, that I brought up the idea of UI
> >> plugins was based upon our previous experience with experimental mode.
> >> Creating UI plugins seems like a great way to not disrupt the core,
> >> while simultaneously providing a method for the community to try out
> >> this new way of creating and new design patterns. If, in the end, we
> >> can convince you of what we believe with respect to how we should
> >> build UI's in the Foreman community, then we can easily pull these
> >> changes into the project as the main method.
> >>
> >> I have, at times, got the impression that some feel as if we did not
> >> take a measured and reasoned approach towards converting how we built
> >> the Katello UI in the past to how we build it now. I hope this helps
> >> explain a bit about where we are coming from, why we made the choices
> >> we did and why we believe in the approach that we took.
> >
> >
> > Overall, I think you guys did what is right for Katello, For foreman, I'm
> > having a hard time justifying a rewrite, overall the issues i see are:
> >
> > Bastion:
> > - make simple pages complex (its much easier to use existing foreman
> > helpers
> > for the avg community developer)
>
> Much easier is subjective, and I'd argue this lacks evidence on either side.
>
> > - requires a whole different skillset (ruby, rails and now javascript,
> > angular, api, json etc)
>
> Fair point, although JS already exists within Foreman and knowledge of
> the API and it's JSON doesn't seem like asking for much compared to
> the other knowledge that is required. This, however, can only be
> proven/shown over time and with users trying.
>
> > - is not extendable by plugins
> > - rewrite most likely will break existing plugins
>
> I'd argue that this is not true for current Foreman core either.
> Pointing users to a library (deface) that can manipulate the UI in any
> way they want is not providing extension and cannot scale. You can
> never know what parts of the UI any user is changing at any given time
> and thus any change to the UI is subject to breaking existing plugins.
> However, I wouldn't argue for ever doing a drastic, quick change but
> rather, like an API change, introduce it over time.
>
> > - normally, takes longer to implement
> > - has no equivalent helpers like rails / foreman.
>
> As I always tell Tom when he makes passionate arguments, please
> provide data and evidence.
>
> >
> > on the plus side, I do admit that our JS testing is lacking, and that some
> > pages are over complicated, refactoring will solve it, but i can't say that
> > AngularJS is the only solution.
> >
> > Further, pure client side implementation is not required for a management
> > app with a limited amount of users… its obviously a plus, and there is a
> > good chance that I myself would start with Angular if I had started today,
> > but I have the feeling that the UI does not require that much of an
> > attention compared to the actual features the application can provide.
>
> I agree that a client side implementation is not required, and this
> argument is not dealing with the scale of users but rather the
> requirements around building dynamic components and workflows. The
> original aspects that drove our push for client side was due in part
> to requirements. We received requirements that increasing requested
> dynamic interface components that provided workflows, and data views
> and manipulation. Which we found were easier and quicker to build
> through pure client side implementations and hitting the API than
> attempting to combine Rails controller, views and jQuery.
>
> As you said, the only course of action is to take the open source
> approach, keep believing in what we believe and push to help others
> see it as we do
>
> >
> > IMHO, we need to be as useful as possible (e.g. integrate with X, do Y etc)
> > while providing a high quality look & feel, I also believe that most users
> > that use both foreman / katello, do not see / feel the difference between
> > the pages at first.
> > and if performance is an issue, there are simple ways using turbolinks [1]
> > etc, that can mitigate that as well.
> >
> > lastly, I'm not against improving, I'm mostly worried that it would slow us
> > down, and introduce a bottleneck, imho we must:
> >
> > 1. unify look and feel, this is needed regardless of the technologies used.
> > 2. provide bastion as a plugin, so it wont work only in the context of
> > katello

+1 on this approach: we took similar one with foreman-tasks (we started first having the
functionality build into the Katello, but we decided quite early we want a
separate plugin so that it can be used outside of Katello realm, which
turned out to be a good decision, as it opened the orchestration possibilities
for Staypuft, MCO, foreman snapshot, all contributing in improving the usability
and stability, and now we're slowly on a journey to the core.

I'm sure the foreman community is able to recognize added value once having something
useful in hands. The Bastion went interesting way so far and now it's good time to share
that knowledge and code with others (and I'm glad to see the other thread about the
progress there).

– Ivan

···

----- Original Message ----- > On Wed, Sep 10, 2014 at 9:56 AM, Ohad Levy wrote: > > On Wed, Sep 10, 2014 at 4:12 PM, Eric D Helms wrote:

3. try to provide new ui plugins (as Joseph and Tom are doing with
   Ember/Angular)
4. continue developing the application as a whole, as neither UI will be
   rewritten in the near future.

in the mean while, try to promote bastion (or any other new stuff that you
want to promote), make sure developers can consume it easily and get the
buy
in from users/developers.

[1]
another turbolinks attempt · ohadlevy/foreman@cf1498f · GitHub

Ohad

Eric

On Wed, Sep 10, 2014 at 6:25 AM, Ivan Necas inecas@redhat.com wrote:

----- Original Message -----

Hey Tom,

JS - Katello’s UI is rendered entirely client side using angular.js
javascript (wrapped in an engine called “bastion”, which is the name
I’ll use here). Bastion provides an MVC layer on the client and
populates its data by making api calls to get json which it then uses
to fill in the html. If the organization page were to be rewritten in
bastion, viewing the page source would look almost empty. The
difference would be that when the Users tab was selected in the UI,
an
ajax api call would be made to the server to get the users that were
available to add to the org and rendered then. Clicking a user to add
them could also make another ajax call to update the org and reflect
that change right away in the UI.

Being total noob in the JS world (thus AngularJS), I don’t get what
drives the need for Bastion. I’ve read its description but I still
don’t
understand why you don’t use plain AngularJS.

In my understanding, the bastion is set of tools and ways how to
organize
the Angular code and couple of helpers on top of it. You can go many
ways with Angular, and bastion chooses some of them. It’s more like
preset angular environment that has already some structure. When
starting
with angular from scratch, you would probably soon end up with something
very similar.

Another way to think of the “core” aspects of Bastion, are if you took
the application controller, api controller, application helper and
other bits such as authorization from the Foreman rails application.
With those pieces you can use mostly regular Rails to build out
individual entities but you still need those common pieces that you
don’t want to re-invent for each controller/model pair or even plugin.
The core of Bastion provides some of these common functionality and
helpers to make re-using UI patterns easier. The goal is that as a
basic UI developer you should only have to care about the business
logic.

ERB pro - Very simple to write
ERB pro - Can access entire code and database readily when building
the UI
ERB pro - Plugins can use ‘deface’ package to make changes
ERB con - Plugins have to use ‘deface’ package to make changes

Not if we build APIs - and we’ve done that previously. We need an API
for plugins to extend things no matter if we have ERB

[Marek_Hulan]

>
> > Hello,
> >
> > my 2c below in text
> >
> > > > ERB pro - Very simple to write
> > > > ERB pro - Can access entire code and database readily when building
> > > > the
> > > > UI
> > > > ERB pro - Plugins can use 'deface' package to make changes
> > > > ERB con - Plugins have to use 'deface' package to make changes
> > >
> > > Not if we build APIs - and we've done that previously. We need an API
> > > for plugins to extend things no matter if we have ERB or JS.
> > >
> > > > ERB con - Two server side controllers, one for API and one for UI
> >
> > Is ERB the real cause? I don't see a reason why not sharing the code and
> > just generate different view format. Sometimes there's a need for
> > specific logic, but
> > this is minority and can be easily handled by respond blocks.
>
> Just a reminder: been there, done that, and there were reasons why
> the UI and API controllers were separated (and I'm quite happy with the
> decision). You either end up with the UI that's build around
> the REST API and the resources it models or build the workflow-based
> UI, but inevitably there will be logic needed just for the UI that the API
> doesn't care about. Also, there are other requirements for the UI entry
> points vs. the API: one expects the backward compatibility on the API level
> (versioning etc.), while the UI's main point is to make the usage of the
> system the easiest possible.

I worked on application that used this approach before and I didn't
encountered these issues. But to be honest, Foreman + Katello have more
complex screens and I didn't get any API usability feedback from users. So
yes, if it's already 'proven', having only API on server side seems to be best
option with respect to what you described in paragraph below.

> Of course, there are also challenges for the API to be able to support the
> UI requirements (mostly performance one for eliminating N+1 requests).
> On the other hand, the improvements here get directly to the API
> ready for use outside of the web UI world, which is not that bat at the end.
> > ERB pro - most of rails plugins/gems use ERB, we don't have much of them
> > with views, AFAIK will_paginate is only one atm, but this would mean we'd
> > have to reimplement all of those views.
> > ERB con - usually harder to do some advanced forms (e.g. host form in
> > Foreman)
> >
> > > > ERB con - Full page reloads very common
> > >
> > > Why JavaScript people tend to list this as a problem? Really, page
> > > refreshing is no big deal, if the app is properly coded, browser cache
> > > works for all assets and pages are clean, it's almost instant. At least
> > > that's my take from working on Foreman, I have to admit that static
> > > portions of JS pages load faster but once you have some data on a page,
> > > you need to wait for it anyway.
> > >
> > > ERB pro - It is the standard way of doing things in Rails with all the
> > > support and integration. I clearly remember Katello on MongoDB which was
> > > swimming against the current and this could be similar case.
> > >
> > > ERB pro - Server-side HTML generation was there years ago, will be there
> > > years looking forward. I am bit worried if AngularJS is not another hype
> > > that will bubble out next year. Google is introducing some Web
> > > Components, Material Design and stuff (haven't read much about this
> > > yet), will this work with AngularJS? Isn't there another yet-better
> > > anglular-like framework emerging?
> > >
> > > ERB cons - We will always need JavaScript therefore we mix two things
> > > together often. Staying on one stack is definitely preferred.
> > >
> > > > JS con - Has a much steeper learning curve to write
> > >
> > > Than what? I am bit afraid here.
> > >
> > > > JS pro - Html portions is very similar to erb in its simplicity
> > > > JS con - All data must be available via API in JSON (could be a pro)
> > > > JS pro - UI consumes same documented REST API, also used by hammer
> > > > JS pro - UI is very dynamic, not requiring full page reloads
> > > > JS con - Limited to provided hooks to tweak UI, cannot use 'deface'
> > > > JS pro - Cannot use 'deface'
> > >
> > > How would plugins extend the UI then?
> > >
> > > > JS con - Existing 'deface' plugins will need a rewrite
> > >
> > > I watched your video (nice) and I am afraid we are not comparing
> > > technologies but rather better UI workflow that Katello apparently has.
> > > Global search box, everything dynamic and rewritten from the ground up.
> > > Foreman had to evolve from very basic pages. This should be improved and
> > > Kyle already started on some designs. It's on the radar.
> >
> > I might be biased since I'm responsible for current Foreman roles UI. I
> > think that issues you showed are mostly easy to fix. Role edit form where
> > you can change just name is really not much useful, easy to add
> > user/usergroup management. This is basically the only difference between
> > those two UIs in terms of how you find the assignment, you expect to
> > assign role to user in role
> > form while in Foreman style we currently have to assign user in user form.
> > Lukáš already covered search and reloading of a page, also regarding the
> > speed, there's much bigger difference in development and production mode.
> >
> > To be honest, I'm not sure whether rewriting the whole role UI is more
> > efficient
> > than fixing issues you identified.
> >
> > Also this may be very subjective but Foreman UI seems very clear to me. I
> > know
> > how to do the task on every page. On the other hand the role UI in Katello
> > style you showed seems hard to understand from first look. When you're
> > adding a
> > user, there are three nested pages
> >
> > - list of roles (gives us action button New Role, 0 selected roles
> >
> > information and deselect link)
> >
> > - role SAM administrator detail (gives us another buttons and tabs)
> > - user management of that role (gives us another buttons, another tabs,
> >
> > another selected info and links)
> >
> > It's hard to compare based on just part of the whole app. I'm not against
> > improving the UI in Foreman. Even though I'm not a fan of full JS UI I
> > would be interested in modernization of what we have. But I think we
> > should decide to based on all pros and cons and either improve existing
> > UI (evolution) or do
> > the rewrite (revolution). I usually prefer small steps. I guess I'll have
> > to look at Katello UI from lower perspective first.
>
> As bad as it might sound, I really don't think there is other way to settle
> down on the future steps without having the same feature implemented twice
> in the UI. That's also good opportunity for one side to prove benefits to
> the other (ideally the measurable ones). Small competition migth not be
> that bad thing here. And if the requirements on the rich-client UI make the
> API more capable/stable that's a good thing.

As long as we compare technical aspects, I'm completely OK with that.

> Ad. evolution vs. revolution: (provided creationists are not right)
> there must have been a time in a history, where
> some living creature decided, that there is a good reason to get out of the
> sea: that was quite revolutionary evolution

Heh, good one. But I suppose fish didn't start to climb the tree first day out
of water

···

On Wednesday 10 of September 2014 08:23:10 Ivan Necas wrote: > ----- Original Message -----

–
Marek

–
Marek

–
You received this message because you are subscribed to the Google Groups
“foreman-dev” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-dev+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

–
Marek

[dLobatog]

A few questions and thoughts after reading the whole thread:

I like the fact Bastion comes with tests and that we could have a tested js
codebase.
What happens to all the js logic AND html helpers we've been using?
Isn't Foreman going to switch to Patternfly at some point? How does that
work when we add Angular to that equation?
Making the UI pluggable (either through deface or changes we introduce in
Foreman core) is key for plugins. Is there a solid alternative to deface at
this point if we made the change?
Do we have people willing to lead this changes? Tom and Joseph?

How fast can we build plugins that starts changing some things (tables, top
bar…) to bastion? If developers are skilled enough to do this, I think
it's reasonable not to rewrite this from scratch but to see how the look
evolves and ask the whole community (developers + users) in the end. Worst
case scenario, the downstream product will look more unified.

Thanks Tom for bringing up the issue, clearly lots of people care about it.

···

On Wed, Sep 10, 2014 at 7:11 PM, Ivan Necas wrote:

----- Original Message -----

Thanks for the thoughts Ohad. I, for one, do not think making a
singular decision is the right approach as you mention as this is true
of any other type of change that would be made (e.g. to switch to
Dynflow for orchestration has taken time and small steps as well). I
do want to clarify a few points below because I think there are some
misconceptions that I have heard time and time again.

On Wed, Sep 10, 2014 at 9:56 AM, Ohad Levy ohadlevy@gmail.com wrote:

Just to make sure, please read my reply in positive light as thats how
i
wrote it

I’m sure we all write and read the posts in positive light. Could we add
some disclaimer about this practice on this mailing list?

On Wed, Sep 10, 2014 at 4:12 PM, Eric D Helms ericdhelms@gmail.com > wrote:

All,

I will include some specific responses inline below, however, as the
person who originally pushed Katello down this road, I’d like to give
some insight into the history of the choice we made.

During the days that Katello was the open-source project for what
became Cloud Forms 1.0, we had a UI that reflected the typical 960px
centered view but with lots of progressive JavaScript mechanisms such
as, for those that know, tupane. Over time, users and developers alike
started requesting more and more dynamic functionality. This led to
creating a swath of spaghetti that interlinked Rails views, UI only
controllers and and JavaScript within one another (e.g. the original
tupane code

katello/app/assets/javascripts/common/panel.js at KATELLO-1.3 · Katello/katello · GitHub
).

this is indeed true for Katello, but on the other side, Foreman is 5
years
old (today!) and we don’t have spaghetti code at all,
the most “complicated” js we have is in the host edit form, totaling
about
600 lines, surely that’s not that bad.
we can and should improve our UI implementation, but based on my
experience
i cant say that ERB/JS is evil or not manageable etc.

First, Happy birthday! Second, the combination of JS (via jQuery
selectors), erb, and erb.js is to me spaghetti. The use of erb.js
links JS snippets to your Rails controllers and the data therein
making it hard to determine where a particular chunk of JS
functionality is coming from. jQuery style selectors to produce
workflows is not scalable and leads to the issues that we encountered.
You are required to build views and view chunks that have specific
classes and IDs that your JavaScript then grabs and manipulates that
makes figuring out which file manipulates what view difficult.

Once we hashed out, via design meetings, what new design we wanted for
all pages that represented a table + object details I began to
investigate if there was a better way to build this new design. I had
grown tired of the increasing effort to sustain the spaghetti due to
how inter-weaved all the pieces (views, JS, controller, helpers) had
become and looked for what new tools were out there. This led me to
investigate a number of JavaScript frameworks to see if we could do
better. I took a basic premise: build a table view that only changes
out the data. I then built this using Backbone, AngularJS, Knockout,
and early Ember. At the time, these were the most popular frameworks
that my research could discern. I quickly found that I was building
the AngularJS application faster and more understandably than the
rest. However, I did not stop there. I also considered:

a) Does the framework have longevity? Will there be support 2 years
from
now?
b) How easy is to build JS components?
c) How easy is it for a new developer to get on board?
d) Can the code written with said framework be tested?

That is indeed great, however, at the same time, i assume it took you
at
least one year to learn, implement etc.
The benefit of using common / simple UI practices, means that we dont
have
to spend a lot of time in design, learning and testing which is
outside the
scope of our tool.
It also means, that a lot of existing tools can be used, we use a lot
of JS
libs (charting, editor, bootstrap, novnc, spice, multi select, data
tables,
etc), that are either based on json( e.g. data tables) or on pure html
(which is also a very good official api such as multi select).

Honestly, it took me about 2 months. After a year we were building
prototypes with ease, spinning up new pages and the bulk of our time
and effort was spent getting our API controllers into proper shape
because they had been neglected in favor of our previous UI
controllers. In the Angular world there are plenty of libraries
available for use (http://ngmodules.org/) and wrappers around
existing, popular libraries (e.g.
Angular directives for Bootstrap).

in other words, you’ve have built a framework based on angular, while
it
can
be the best framework in the world, the original question remains,
should
we
spend all of this time building a framework for a new UI workflow?

We did not build a framework. This is the equivalent of saying that
Foreman is a framework. We built some common tools and functionality
just as a helper or application controller in a Rails application
centralizes common functionality.

The reasons I inevitably went with Angularjs, and should address some
concerns folks have mentioned are:

a) A primary goal of the Angularjs team is to build Angularjs using
methods and functionality that future Javascript specs are declaring
such that when those are implemented natively by the browser, the
Angularjs team can remove that functionality from the library and thus
reduce the foot print. And, where they see a concept created by them
that could be useful to the larger community, pushing those changes to
the browser spec so as to remove it from their library. A good example
of this is the watch/observe functionality that is now making it’s way
into the browser.

Some stats:
27.2k stars on Github
928 contributors
4 years old
54k tagged questions on stack overflow
('angularjs' tag wiki - Stack Overflow)
75k+ hits on youtube when searching “angularjs”

b) Directives made creating components easy, and although the toughest
part of angular, this was easier than building components using basic
JS.

c) I’ll let other developers who have built things subsequently chime
in here. For me, things are much quicker and have not degraded over
time like before. There are lots of tools for learning, plenty of
tutorials, videos and tons of short videos. Websites such as
http://angular.codeschool.com/. They also have case studies from
third
parties - https://builtwith.angularjs.org/.

d) Angularjs came built with testing in mind via their test runner
Karma. We have since been able to write tests for all aspects of our
JS, generate coverage, run a Jenkins job on each PR for the testing.
Further, the tests can be run from the command line and run in less
than a minute.

After considering all of the above, I decided to build out the design
that had been created and spec’d out with Angularjs for the most
complex object we had (i.e. systems). I did this in order to
illuminate any pitfalls that may occur when applied to a complex
object. Further, I did not radically overhaul and push this upon the
project. Rather, I took an approach of showing off the steps along the
way to developers, asking questions of developers when I faced design
choices, and eventually providing this new look’n’feel and interface
via an experimental UI mode.

This experimental mode was a toggle-able user preference that when
enabled hid the old UI and displayed the new UI components. This
allowed users to play with and try out the new pieces without having
to completely give up (or break) old functionality. After we got the
experimental UI to the point of being ready for prime time, we then
flipped the scenario such that the old UI was still around but in a
legacy mode that a user could toggle on or off. This allowed us to
step through the building with a new implementation style as well as
test and use new pieces without having to make radical change.

As others have mentioned, a side benefit of this approach was that we
only needed to maintain one set of API controllers. This meant that we
found gaps in the logic that our UI controllers and API controllers
were assuming as well as gaps in the authentication by centralizing to
a single entry point.

I still dont see it as plus, API needs to be stable, and cant be
modified,
while UI controllers can evolve quicker and allow breaking changes
when you
want to change your UI.

The reason, in the previous thread, that I brought up the idea of UI
plugins was based upon our previous experience with experimental mode.
Creating UI plugins seems like a great way to not disrupt the core,
while simultaneously providing a method for the community to try out
this new way of creating and new design patterns. If, in the end, we
can convince you of what we believe with respect to how we should
build UI’s in the Foreman community, then we can easily pull these
changes into the project as the main method.

I have, at times, got the impression that some feel as if we did not
take a measured and reasoned approach towards converting how we built
the Katello UI in the past to how we build it now. I hope this helps
explain a bit about where we are coming from, why we made the choices
we did and why we believe in the approach that we took.

Overall, I think you guys did what is right for Katello, For foreman,
I’m
having a hard time justifying a rewrite, overall the issues i see are:

Bastion:

• make simple pages complex (its much easier to use existing foreman
  helpers
  for the avg community developer)

Much easier is subjective, and I’d argue this lacks evidence on either
side.

• requires a whole different skillset (ruby, rails and now javascript,
  angular, api, json etc)

Fair point, although JS already exists within Foreman and knowledge of
the API and it’s JSON doesn’t seem like asking for much compared to
the other knowledge that is required. This, however, can only be
proven/shown over time and with users trying.

• is not extendable by plugins
• rewrite most likely will break existing plugins

I’d argue that this is not true for current Foreman core either.
Pointing users to a library (deface) that can manipulate the UI in any
way they want is not providing extension and cannot scale. You can
never know what parts of the UI any user is changing at any given time
and thus any change to the UI is subject to breaking existing plugins.
However, I wouldn’t argue for ever doing a drastic, quick change but
rather, like an API change, introduce it over time.

• normally, takes longer to implement
• has no equivalent helpers like rails / foreman.

As I always tell Tom when he makes passionate arguments, please
provide data and evidence.

on the plus side, I do admit that our JS testing is lacking, and that
some
pages are over complicated, refactoring will solve it, but i can’t say
that
AngularJS is the only solution.

Further, pure client side implementation is not required for a
management
app with a limited amount of users… its obviously a plus, and there
is a
good chance that I myself would start with Angular if I had started
today,
but I have the feeling that the UI does not require that much of an
attention compared to the actual features the application can provide.

I agree that a client side implementation is not required, and this
argument is not dealing with the scale of users but rather the
requirements around building dynamic components and workflows. The
original aspects that drove our push for client side was due in part
to requirements. We received requirements that increasing requested
dynamic interface components that provided workflows, and data views
and manipulation. Which we found were easier and quicker to build
through pure client side implementations and hitting the API than
attempting to combine Rails controller, views and jQuery.

As you said, the only course of action is to take the open source
approach, keep believing in what we believe and push to help others
see it as we do

IMHO, we need to be as useful as possible (e.g. integrate with X, do Y
etc)
while providing a high quality look & feel, I also believe that most
users
that use both foreman / katello, do not see / feel the difference
between
the pages at first.
and if performance is an issue, there are simple ways using turbolinks
[1]
etc, that can mitigate that as well.

lastly, I’m not against improving, I’m mostly worried that it would
slow us
down, and introduce a bottleneck, imho we must:

1. unify look and feel, this is needed regardless of the technologies
   used.
2. provide bastion as a plugin, so it wont work only in the context of
   katello

+1 on this approach: we took similar one with foreman-tasks (we started
first having the
functionality build into the Katello, but we decided quite early we want a
separate plugin so that it can be used outside of Katello realm, which
turned out to be a good decision, as it opened the orchestration
possibilities
for Staypuft, MCO, foreman snapshot, all contributing in improving the
usability
and stability, and now we’re slowly on a journey to the core.

I’m sure the foreman community is able to recognize added value once
having something
useful in hands. The Bastion went interesting way so far and now it’s good
time to share
that knowledge and code with others (and I’m glad to see the other thread
about the
progress there).

– Ivan

3. try to provide new ui plugins (as Joseph and Tom are doing with
   Ember/Angular)
4. continue developing the application as a whole, as neither UI will
   be
   rewritten in the near future.

in the mean while, try to promote bastion (or any other new stuff that
you
want to promote), make sure developers can consume it easily and get
the
buy
in from users/developers.

[1]

another turbolinks attempt · ohadlevy/foreman@cf1498f · GitHub

Ohad

Eric

On Wed, Sep 10, 2014 at 6:25 AM, Ivan Necas inecas@redhat.com > wrote:

[ehelms]

> A few questions and thoughts after reading the whole thread:
>
> I like the fact Bastion comes with tests and that we could have a tested
> js codebase.
> What happens to all the js logic AND html helpers we've been using?
> Isn't Foreman going to switch to Patternfly at some point? How does that
> work when we add Angular to that equation?
>

Patternfly by itself is concerned with styling. Some of the developers of
the JBoss Liveoak project started an angular-patternfly (
GitHub - patternfly/angular-patternfly: This repo contains instructions and the code for a set of Angular 1 components for the PatternFly project.) repo to contribute the
re-usable pieces that have developed. Walden and myself have started
investigating which of our incubated pieces we could push into that
repository to share and centralize.

···

On Wed, Sep 10, 2014 at 1:44 PM, Daniel Lobato wrote:

Making the UI pluggable (either through deface or changes we introduce in
Foreman core) is key for plugins. Is there a solid alternative to deface at
this point if we made the change?
Do we have people willing to lead this changes? Tom and Joseph?

How fast can we build plugins that starts changing some things (tables,
top bar…) to bastion? If developers are skilled enough to do this, I
think it’s reasonable not to rewrite this from scratch but to see how the
look evolves and ask the whole community (developers + users) in the end.
Worst case scenario, the downstream product will look more unified.

Thanks Tom for bringing up the issue, clearly lots of people care about it.

On Wed, Sep 10, 2014 at 7:11 PM, Ivan Necas inecas@redhat.com wrote:

----- Original Message -----

Thanks for the thoughts Ohad. I, for one, do not think making a
singular decision is the right approach as you mention as this is true
of any other type of change that would be made (e.g. to switch to
Dynflow for orchestration has taken time and small steps as well). I
do want to clarify a few points below because I think there are some
misconceptions that I have heard time and time again.

On Wed, Sep 10, 2014 at 9:56 AM, Ohad Levy ohadlevy@gmail.com wrote:

Just to make sure, please read my reply in positive light as thats
how i
wrote it

I’m sure we all write and read the posts in positive light. Could we add
some disclaimer about this practice on this mailing list?

On Wed, Sep 10, 2014 at 4:12 PM, Eric D Helms ericdhelms@gmail.com >> wrote:

All,

I will include some specific responses inline below, however, as the
person who originally pushed Katello down this road, I’d like to give
some insight into the history of the choice we made.

During the days that Katello was the open-source project for what
became Cloud Forms 1.0, we had a UI that reflected the typical 960px
centered view but with lots of progressive JavaScript mechanisms such
as, for those that know, tupane. Over time, users and developers
alike
started requesting more and more dynamic functionality. This led to
creating a swath of spaghetti that interlinked Rails views, UI only
controllers and and JavaScript within one another (e.g. the original
tupane code

katello/app/assets/javascripts/common/panel.js at KATELLO-1.3 · Katello/katello · GitHub
).

this is indeed true for Katello, but on the other side, Foreman is 5
years
old (today!) and we don’t have spaghetti code at all,
the most “complicated” js we have is in the host edit form, totaling
about
600 lines, surely that’s not that bad.
we can and should improve our UI implementation, but based on my
experience
i cant say that ERB/JS is evil or not manageable etc.

First, Happy birthday! Second, the combination of JS (via jQuery
selectors), erb, and erb.js is to me spaghetti. The use of erb.js
links JS snippets to your Rails controllers and the data therein
making it hard to determine where a particular chunk of JS
functionality is coming from. jQuery style selectors to produce
workflows is not scalable and leads to the issues that we encountered.
You are required to build views and view chunks that have specific
classes and IDs that your JavaScript then grabs and manipulates that
makes figuring out which file manipulates what view difficult.

Once we hashed out, via design meetings, what new design we wanted
for
all pages that represented a table + object details I began to
investigate if there was a better way to build this new design. I had
grown tired of the increasing effort to sustain the spaghetti due to
how inter-weaved all the pieces (views, JS, controller, helpers) had
become and looked for what new tools were out there. This led me to
investigate a number of JavaScript frameworks to see if we could do
better. I took a basic premise: build a table view that only changes
out the data. I then built this using Backbone, AngularJS, Knockout,
and early Ember. At the time, these were the most popular frameworks
that my research could discern. I quickly found that I was building
the AngularJS application faster and more understandably than the
rest. However, I did not stop there. I also considered:

a) Does the framework have longevity? Will there be support 2 years
from
now?
b) How easy is to build JS components?
c) How easy is it for a new developer to get on board?
d) Can the code written with said framework be tested?

That is indeed great, however, at the same time, i assume it took you
at
least one year to learn, implement etc.
The benefit of using common / simple UI practices, means that we
dont have
to spend a lot of time in design, learning and testing which is
outside the
scope of our tool.
It also means, that a lot of existing tools can be used, we use a lot
of JS
libs (charting, editor, bootstrap, novnc, spice, multi select, data
tables,
etc), that are either based on json( e.g. data tables) or on pure html
(which is also a very good official api such as multi select).

Honestly, it took me about 2 months. After a year we were building
prototypes with ease, spinning up new pages and the bulk of our time
and effort was spent getting our API controllers into proper shape
because they had been neglected in favor of our previous UI
controllers. In the Angular world there are plenty of libraries
available for use (http://ngmodules.org/) and wrappers around
existing, popular libraries (e.g.
Angular directives for Bootstrap).

in other words, you’ve have built a framework based on angular, while
it
can
be the best framework in the world, the original question remains,
should
we
spend all of this time building a framework for a new UI workflow?

We did not build a framework. This is the equivalent of saying that
Foreman is a framework. We built some common tools and functionality
just as a helper or application controller in a Rails application
centralizes common functionality.

The reasons I inevitably went with Angularjs, and should address some
concerns folks have mentioned are:

a) A primary goal of the Angularjs team is to build Angularjs using
methods and functionality that future Javascript specs are declaring
such that when those are implemented natively by the browser, the
Angularjs team can remove that functionality from the library and
thus
reduce the foot print. And, where they see a concept created by them
that could be useful to the larger community, pushing those changes
to
the browser spec so as to remove it from their library. A good
example
of this is the watch/observe functionality that is now making it’s
way
into the browser.

Some stats:
27.2k stars on Github
928 contributors
4 years old
54k tagged questions on stack overflow
('angularjs' tag wiki - Stack Overflow)
75k+ hits on youtube when searching “angularjs”

b) Directives made creating components easy, and although the
toughest
part of angular, this was easier than building components using basic
JS.

c) I’ll let other developers who have built things subsequently chime
in here. For me, things are much quicker and have not degraded over
time like before. There are lots of tools for learning, plenty of
tutorials, videos and tons of short videos. Websites such as
http://angular.codeschool.com/. They also have case studies from
third
parties - https://builtwith.angularjs.org/.

d) Angularjs came built with testing in mind via their test runner
Karma. We have since been able to write tests for all aspects of our
JS, generate coverage, run a Jenkins job on each PR for the testing.
Further, the tests can be run from the command line and run in less
than a minute.

After considering all of the above, I decided to build out the design
that had been created and spec’d out with Angularjs for the most
complex object we had (i.e. systems). I did this in order to
illuminate any pitfalls that may occur when applied to a complex
object. Further, I did not radically overhaul and push this upon the
project. Rather, I took an approach of showing off the steps along
the
way to developers, asking questions of developers when I faced design
choices, and eventually providing this new look’n’feel and interface
via an experimental UI mode.

This experimental mode was a toggle-able user preference that when
enabled hid the old UI and displayed the new UI components. This
allowed users to play with and try out the new pieces without having
to completely give up (or break) old functionality. After we got the
experimental UI to the point of being ready for prime time, we then
flipped the scenario such that the old UI was still around but in a
legacy mode that a user could toggle on or off. This allowed us to
step through the building with a new implementation style as well as
test and use new pieces without having to make radical change.

As others have mentioned, a side benefit of this approach was that we
only needed to maintain one set of API controllers. This meant that
we
found gaps in the logic that our UI controllers and API controllers
were assuming as well as gaps in the authentication by centralizing
to
a single entry point.

I still dont see it as plus, API needs to be stable, and cant be
modified,
while UI controllers can evolve quicker and allow breaking changes
when you
want to change your UI.

The reason, in the previous thread, that I brought up the idea of UI
plugins was based upon our previous experience with experimental
mode.
Creating UI plugins seems like a great way to not disrupt the core,
while simultaneously providing a method for the community to try out
this new way of creating and new design patterns. If, in the end, we
can convince you of what we believe with respect to how we should
build UI’s in the Foreman community, then we can easily pull these
changes into the project as the main method.

I have, at times, got the impression that some feel as if we did not
take a measured and reasoned approach towards converting how we built
the Katello UI in the past to how we build it now. I hope this helps
explain a bit about where we are coming from, why we made the choices
we did and why we believe in the approach that we took.

Overall, I think you guys did what is right for Katello, For foreman,
I’m
having a hard time justifying a rewrite, overall the issues i see are:

Bastion:

• make simple pages complex (its much easier to use existing foreman
  helpers
  for the avg community developer)

Much easier is subjective, and I’d argue this lacks evidence on either
side.

• requires a whole different skillset (ruby, rails and now javascript,
  angular, api, json etc)

Fair point, although JS already exists within Foreman and knowledge of
the API and it’s JSON doesn’t seem like asking for much compared to
the other knowledge that is required. This, however, can only be
proven/shown over time and with users trying.

• is not extendable by plugins
• rewrite most likely will break existing plugins

I’d argue that this is not true for current Foreman core either.
Pointing users to a library (deface) that can manipulate the UI in any
way they want is not providing extension and cannot scale. You can
never know what parts of the UI any user is changing at any given time
and thus any change to the UI is subject to breaking existing plugins.
However, I wouldn’t argue for ever doing a drastic, quick change but
rather, like an API change, introduce it over time.

• normally, takes longer to implement
• has no equivalent helpers like rails / foreman.

As I always tell Tom when he makes passionate arguments, please
provide data and evidence.

on the plus side, I do admit that our JS testing is lacking, and that
some
pages are over complicated, refactoring will solve it, but i can’t
say that
AngularJS is the only solution.

Further, pure client side implementation is not required for a
management
app with a limited amount of users… its obviously a plus, and there
is a
good chance that I myself would start with Angular if I had started
today,
but I have the feeling that the UI does not require that much of an
attention compared to the actual features the application can provide.

I agree that a client side implementation is not required, and this
argument is not dealing with the scale of users but rather the
requirements around building dynamic components and workflows. The
original aspects that drove our push for client side was due in part
to requirements. We received requirements that increasing requested
dynamic interface components that provided workflows, and data views
and manipulation. Which we found were easier and quicker to build
through pure client side implementations and hitting the API than
attempting to combine Rails controller, views and jQuery.

As you said, the only course of action is to take the open source
approach, keep believing in what we believe and push to help others
see it as we do

IMHO, we need to be as useful as possible (e.g. integrate with X, do
Y etc)
while providing a high quality look & feel, I also believe that most
users
that use both foreman / katello, do not see / feel the difference
between
the pages at first.
and if performance is an issue, there are simple ways using
turbolinks [1]
etc, that can mitigate that as well.

lastly, I’m not against improving, I’m mostly worried that it would
slow us
down, and introduce a bottleneck, imho we must:

1. unify look and feel, this is needed regardless of the technologies
   used.
2. provide bastion as a plugin, so it wont work only in the context of
   katello

+1 on this approach: we took similar one with foreman-tasks (we started
first having the
functionality build into the Katello, but we decided quite early we want a
separate plugin so that it can be used outside of Katello realm, which
turned out to be a good decision, as it opened the orchestration
possibilities
for Staypuft, MCO, foreman snapshot, all contributing in improving the
usability
and stability, and now we’re slowly on a journey to the core