Three and a half years ago I started a pet project, hammer-cli-csv[1], to
export Foreman resources in CSV format with enough detail to be able to
later reimport that data for an exact clone. Although I envisioned broader
customer usage, it was mainly useful for devs that would often tear down
and spin up a new server. With CSV I could repopulate nearly all details of
the orgs, locs, settings, puppet classes, lifecycle environments, hosts,
host groups, etc.[2] Over time, certain resource types were incorporated
into Satellite-6 customer flows, specifically manipulating subscriptions on
content hosts[3]. Some addition aspects that may be interesting listed
below. Many of these are partially complete but worked well enough for my
own usage. As they say, "Pull-requests welcome!"
- Originally designed to do multi-threaded import (added to expose problems
with server stress)
- Allows custom columns to be defined and exported (custom reports)
- Allows a subset of columns to be exported (custom reports)
- Able to take yaml input instead of csv (useful for repopulating with
ansible playbook via forklift)
- Import of Sat-5 export CSV
- Server plugin for hosts to speed up import by a factor of ten
Over the past year other devs have worked on related features in Foreman.
These include rake scripts for export (very fast compared to API that
hammer csv has to use), export button on tables in UI (very convenient and
accessible compared to CLI), and more. Unfortunately all of the work
combined is not compatible. For example, exported CSV from a table can't
then be imported with hammer csv.
The question I have is, should additional dev effort be put into hammer
csv? Is the concept important? Could it be used for custom report
generation or inter-server sync (ISS)? Do we want a coherent vision and
strategy or is it alright to continue with point solutions?
[1] https://github.com/Katello/hammer-cli-csv
[2]
https://docs.google.com/spreadsheets/d/1brcqqUpfdoWCLDHkQc6f-AX-c93mwaqm7qUgzKfuvBM/edit?usp=sharing
[3] https://access.redhat.com/blogs/1169563/posts/2632781
Here is another example of a very convenient use of import[1]:
$ oc create -f
https://raw.githubusercontent.com/jboss-openshift/application-templates/master/jboss-image-streams.json
$ oc process -f
https://raw.githubusercontent.com/siamaksade/cart-service/master/openshift/cart-template.yaml
-v IMAGE_STREAM_NAMESPACE=ci | oc create -f -
This is using openshift command line tool "oc" (equivalent of hammer) to
populate items directly as well as using it to substitute vars in config
files. This makes for very convenient demo setup, blogs, etc.
Anyone have an interest in using such a tool? Would hammer be best or some
other utility?
[1]
https://blog.openshift.com/openshift-pipelines-jenkins-blue-ocean/
ยทยทยท
On Mon, Sep 25, 2017 at 9:32 AM, Tom McKay <thomasmckay@redhat.com> wrote:
Three and a half years ago I started a pet project, hammer-cli-csv[1], to
export Foreman resources in CSV format with enough detail to be able to
later reimport that data for an exact clone. Although I envisioned broader
customer usage, it was mainly useful for devs that would often tear down
and spin up a new server. With CSV I could repopulate nearly all details of
the orgs, locs, settings, puppet classes, lifecycle environments, hosts,
host groups, etc.[2] Over time, certain resource types were incorporated
into Satellite-6 customer flows, specifically manipulating subscriptions on
content hosts[3]. Some addition aspects that may be interesting listed
below. Many of these are partially complete but worked well enough for my
own usage. As they say, "Pull-requests welcome!"
+ Originally designed to do multi-threaded import (added to expose
problems with server stress)
+ Allows custom columns to be defined and exported (custom reports)
+ Allows a subset of columns to be exported (custom reports)
+ Able to take yaml input instead of csv (useful for repopulating with
ansible playbook via forklift)
+ Import of Sat-5 export CSV
+ Server plugin for hosts to speed up import by a factor of ten
Over the past year other devs have worked on related features in Foreman.
These include rake scripts for export (very fast compared to API that
hammer csv has to use), export button on tables in UI (very convenient and
accessible compared to CLI), and more. Unfortunately all of the work
combined is not compatible. For example, exported CSV from a table can't
then be imported with hammer csv.
The question I have is, should additional dev effort be put into hammer
csv? Is the concept important? Could it be used for custom report
generation or inter-server sync (ISS)? Do we want a coherent vision and
strategy or is it alright to continue with point solutions?
[1] https://github.com/Katello/hammer-cli-csv
[2] https://docs.google.com/spreadsheets/d/1brcqqUpfdoWCLDHkQc6f-AX-
c93mwaqm7qUgzKfuvBM/edit?usp=sharing
[3] https://access.redhat.com/blogs/1169563/posts/2632781
