Katello 3.0.2 API content upload not updating info

I am attempting to use the Katello API to upload content into a repository

  • we are using Jenkins to package an RPM and upload it into Katello for
    deployment. This worked fine with the 3.0 RCs, however in 3.0.2 although
    everything seems to complete successfully, I am not seeing the package in
    the GUI -

I followed the "Uploading content to the satellite server" example
from https://access.redhat.com/documentation/en-US/Red_Hat_Satellite/6.1/html/API_Guide/curl-examples.html

The steps and output I do are below (the output is from Jenkins)

  • byte=5000000
  • offset=0
  • /usr/bin/rm uploadlist.000 uploadlist.001
  • split --bytes 5000000 --numeric-suffixes --suffix-length=3
    tomcat8-8.5.4-1.noarch.rpm uploadlist.
  • shopt -s nullglob
  • array=(uploadlist.*)
    ++ curl -s -H Accept:application/json,version=2 -H
    Content-Type:application/json -X POST -u jenkins:removed -k -d '{}'
    https://wellkatello.niwa.local/katello/api/repositories/1406/content_uploads
··· + RESPONSE='{"upload_id":"4376d1f2-a9f0-451d-b888-3b53e496581c","_href":"/pulp/api/v2/content/uploads/4376d1f2-a9f0-451d-b888-3b53e496581c/"}' ++ echo '{"upload_id":"4376d1f2-a9f0-451d-b888-3b53e496581c","_href":"/pulp/api/v2/content/uploads/4376d1f2-a9f0-451d-b888-3b53e496581c/"}' ++ sed 's/"//g' ++ grep -e '.*"upload_id":\s*"\([a-zA-Z0-9-]\)*"' -o ++ sed 's/{"upload_id"://' + REQUEST_ID=4376d1f2-a9f0-451d-b888-3b53e496581c + echo 'Request ID [4376d1f2-a9f0-451d-b888-3b53e496581c] created' Request ID [4376d1f2-a9f0-451d-b888-3b53e496581c] created + for file in '"${array[@]}"' + echo 'Offset is 0' Offset is 0 + echo 'Uploading chunk uploadlist.000' Uploading chunk uploadlist.000 + curl -H Accept:application/json,version=2 -H Content-Type:multipart/form-data -X PUT -u jenkins:removed -k --data-urlencode content@uploadlist.000 --data-urlencode offset=0 https://wellkatello.niwa.local/katello/api/repositories/1406/content_uploads/4376d1f2-a9f0-451d-b888-3b53e496581c % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:--
0
100 11.8M 0 0 100 11.8M 0 10.1M 0:00:01 0:00:01 --:–:--
10.1M
100 11.8M 0 0 100 11.8M 0 5597k 0:00:02 0:00:02 --:–:--
5598k
100 11.8M 0 0 100 11.8M 0 3827k 0:00:03 0:00:03 --:–:--
3828k
100 11.8M 0 0 100 11.8M 0 2907k 0:00:04 0:00:04 --:–:--
2908k
100 11.8M 0 1 100 11.8M 0 2745k 0:00:04 0:00:04 --:–:--
2745k

  • offset=5000000

  • for file in ‘"${array[@]}"’

  • echo 'Offset is 5000000’
    Offset is 5000000

  • echo 'Uploading chunk uploadlist.001’
    Uploading chunk uploadlist.001

  • curl -H Accept:application/json,version=2 -H
    Content-Type:multipart/form-data -X PUT -u jenkins:removed -k
    –data-urlencode content@uploadlist.001 --data-urlencode offset=5000000
    https://wellkatello.niwa.local/katello/api/repositories/1406/content_uploads/4376d1f2-a9f0-451d-b888-3b53e496581c
    % Total % Received % Xferd Average Speed Time Time Time
    Current
    Dload Upload Total Spent Left
    Speed

    0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:--
    0
    100 6590k 0 0 100 6590k 0 5647k 0:00:01 0:00:01 --:–:--
    5652k
    100 6590k 0 0 100 6590k 0 3039k 0:00:02 0:00:02 --:–:--
    3041k
    100 6590k 0 1 100 6590k 0 2643k 0:00:02 0:00:02 --:–:--
    2643k

  • offset=10000000

  • echo 'Importing upload into repository’
    Importing upload into repository

  • curl -H Accept:application/json,version=2 -H
    Content-Type:application/json -X PUT -u jenkins:removed -k -d
    ’{“upload_ids”:[“4376d1f2-a9f0-451d-b888-3b53e496581c”]}'
    https://wellkatello.niwa.local/katello/api/repositories/1406/import_uploads
    % Total % Received % Xferd Average Speed Time Time Time
    Current
    Dload Upload Total Spent Left
    Speed

    0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:--
    0
    100 55 0 0 100 55 0 44 0:00:01 0:00:01 --:–:--
    44
    100 55 0 0 100 55 0 24 0:00:02 0:00:02 --:–:--
    24
    98 56 0 0 100 55 0 22 0:00:02 0:00:02 --:–:--
    22
    100 56 0 1 100 55 0 22 0:00:02 0:00:02 --:–:--
    22

  • echo 'Delete the upload request’
    Delete the upload request

  • curl -H Accept:application/json,version=2 -H
    Content-Type:application/json -X DELETE -d ‘{}’ -u jenkins:removed -k
    https://wellkatello.niwa.local/katello/api/repositories/1406/content_uploads/4376d1f2-a9f0-451d-b888-3b53e496581c
    % Total % Received % Xferd Average Speed Time Time Time
    Current
    Dload Upload Total Spent Left
    Speed

    0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:--
    0
    100 3 100 1 100 2 2 5 --:–:-- --:–:-- --:–:--
    6
    Finished: SUCCESS

I can successfully download the RPM from the repo -

[root@cislave01p upload]# wget
http://wellkatello.niwa.local/pulp/repos/NIWA/Library/custom/NIWA/NIWA_Custom_Applications/tomcat8-8.5.4-1.noarch.rpm
–2016-08-10 17:31:56–
http://wellkatello.niwa.local/pulp/repos/NIWA/Library/custom/NIWA/NIWA_Custom_Applications/tomcat8-8.5.4-1.noarch.rpm
Resolving wellkatello.niwa.local (wellkatello.niwa.local)… 192.168.59.247
Connecting to wellkatello.niwa.local
(wellkatello.niwa.local)|192.168.59.247|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 7716624 (7.4M) [application/octet-stream]
Saving to: ‘tomcat8-8.5.4-1.noarch.rpm.1’

100%[===================================================================================================================================================================================================>]
7,716,624 --.-K/s in 0.1s

2016-08-10 17:32:02 (70.7 MB/s) - ‘tomcat8-8.5.4-1.noarch.rpm.1’ saved
[7716624/7716624]

[root@cislave01p upload]#

A manual upload via the GUI works fine, and the GUI picks up the RPM. Any
suggestions greatly appreciated :slight_smile:

Hi

I have the same issue. To workaround this I had to create local repo on
Katello server with createrepo and then sync it in Katello with
file:///path/to/repo URL. Then it worked.

Edgars

trešdiena, 2016. gada 10. augusts 07:39:05 UTC+2, Dylan Baars rakstīja:

··· > > I am attempting to use the Katello API to upload content into a repository > - we are using Jenkins to package an RPM and upload it into Katello for > deployment. This worked fine with the 3.0 RCs, however in 3.0.2 although > everything seems to complete successfully, I am not seeing the package in > the GUI - > > I followed the "Uploading content to the satellite server" example from > https://access.redhat.com/documentation/en-US/Red_Hat_Satellite/6.1/html/API_Guide/curl-examples.html > > The steps and output I do are below (the output is from Jenkins) > > + byte=5000000 > + offset=0 > + /usr/bin/rm uploadlist.000 uploadlist.001 > + split --bytes 5000000 --numeric-suffixes --suffix-length=3 > tomcat8-8.5.4-1.noarch.rpm uploadlist. > + shopt -s nullglob > + array=(uploadlist.*) > ++ curl -s -H Accept:application/json,version=2 -H > Content-Type:application/json -X POST -u jenkins:removed -k -d '{}' > https://wellkatello.niwa.local/katello/api/repositories/1406/content_uploads > + > RESPONSE='{"upload_id":"4376d1f2-a9f0-451d-b888-3b53e496581c","_href":"/pulp/api/v2/content/uploads/4376d1f2-a9f0-451d-b888-3b53e496581c/"}' > ++ echo > '{"upload_id":"4376d1f2-a9f0-451d-b888-3b53e496581c","_href":"/pulp/api/v2/content/uploads/4376d1f2-a9f0-451d-b888-3b53e496581c/"}' > ++ sed 's/"//g' > ++ grep -e '.*"upload_id":\s*"\([a-zA-Z0-9-]\)*"' -o > ++ sed 's/{"upload_id"://' > + REQUEST_ID=4376d1f2-a9f0-451d-b888-3b53e496581c > + echo 'Request ID [4376d1f2-a9f0-451d-b888-3b53e496581c] created' > Request ID [4376d1f2-a9f0-451d-b888-3b53e496581c] created > + for file in '"${array[@]}"' > + echo 'Offset is 0' > Offset is 0 > + echo 'Uploading chunk uploadlist.000' > Uploading chunk uploadlist.000 > + curl -H Accept:application/json,version=2 -H > Content-Type:multipart/form-data -X PUT -u jenkins:removed -k > --data-urlencode content@uploadlist.000 --data-urlencode offset=0 > https://wellkatello.niwa.local/katello/api/repositories/1406/content_uploads/4376d1f2-a9f0-451d-b888-3b53e496581c > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > 0 > 100 11.8M 0 0 100 11.8M 0 10.1M 0:00:01 0:00:01 --:--:-- > 10.1M > 100 11.8M 0 0 100 11.8M 0 5597k 0:00:02 0:00:02 --:--:-- > 5598k > 100 11.8M 0 0 100 11.8M 0 3827k 0:00:03 0:00:03 --:--:-- > 3828k > 100 11.8M 0 0 100 11.8M 0 2907k 0:00:04 0:00:04 --:--:-- > 2908k > 100 11.8M 0 1 100 11.8M 0 2745k 0:00:04 0:00:04 --:--:-- > 2745k > + offset=5000000 > + for file in '"${array[@]}"' > + echo 'Offset is 5000000' > Offset is 5000000 > + echo 'Uploading chunk uploadlist.001' > Uploading chunk uploadlist.001 > + curl -H Accept:application/json,version=2 -H > Content-Type:multipart/form-data -X PUT -u jenkins:removed -k > --data-urlencode content@uploadlist.001 --data-urlencode offset=5000000 > https://wellkatello.niwa.local/katello/api/repositories/1406/content_uploads/4376d1f2-a9f0-451d-b888-3b53e496581c > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > 0 > 100 6590k 0 0 100 6590k 0 5647k 0:00:01 0:00:01 --:--:-- > 5652k > 100 6590k 0 0 100 6590k 0 3039k 0:00:02 0:00:02 --:--:-- > 3041k > 100 6590k 0 1 100 6590k 0 2643k 0:00:02 0:00:02 --:--:-- > 2643k > + offset=10000000 > + echo 'Importing upload into repository' > Importing upload into repository > + curl -H Accept:application/json,version=2 -H > Content-Type:application/json -X PUT -u jenkins:removed -k -d > '{"upload_ids":["4376d1f2-a9f0-451d-b888-3b53e496581c"]}' > https://wellkatello.niwa.local/katello/api/repositories/1406/import_uploads > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > 0 > 100 55 0 0 100 55 0 44 0:00:01 0:00:01 --:--:-- > 44 > 100 55 0 0 100 55 0 24 0:00:02 0:00:02 --:--:-- > 24 > 98 56 0 0 100 55 0 22 0:00:02 0:00:02 --:--:-- > 22 > 100 56 0 1 100 55 0 22 0:00:02 0:00:02 --:--:-- > 22 > + echo 'Delete the upload request' > Delete the upload request > + curl -H Accept:application/json,version=2 -H > Content-Type:application/json -X DELETE -d '{}' -u jenkins:removed -k > https://wellkatello.niwa.local/katello/api/repositories/1406/content_uploads/4376d1f2-a9f0-451d-b888-3b53e496581c > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > > 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- > 0 > 100 3 100 1 100 2 2 5 --:--:-- --:--:-- --:--:-- > 6 > Finished: SUCCESS > > > > I can successfully download the RPM from the repo - > > [root@cislave01p upload]# wget > http://wellkatello.niwa.local/pulp/repos/NIWA/Library/custom/NIWA/NIWA_Custom_Applications/tomcat8-8.5.4-1.noarch.rpm > --2016-08-10 17:31:56-- > http://wellkatello.niwa.local/pulp/repos/NIWA/Library/custom/NIWA/NIWA_Custom_Applications/tomcat8-8.5.4-1.noarch.rpm > Resolving wellkatello.niwa.local (wellkatello.niwa.local)... 192.168.59.247 > Connecting to wellkatello.niwa.local > (wellkatello.niwa.local)|192.168.59.247|:80... connected. > HTTP request sent, awaiting response... 200 OK > Length: 7716624 (7.4M) [application/octet-stream] > Saving to: ‘tomcat8-8.5.4-1.noarch.rpm.1’ > > 100%[===================================================================================================================================================================================================>] > 7,716,624 --.-K/s in 0.1s > > 2016-08-10 17:32:02 (70.7 MB/s) - ‘tomcat8-8.5.4-1.noarch.rpm.1’ saved > [7716624/7716624] > > [root@cislave01p upload]# > > > > A manual upload via the GUI works fine, and the GUI picks up the RPM. Any > suggestions greatly appreciated :-) >

Can you try an upload with hammer? Do you see the same behavior?

··· On Aug 10, 2016 5:15 AM, "Edgars M." wrote:

Hi

I have the same issue. To workaround this I had to create local repo on
Katello server with createrepo and then sync it in Katello with
file:///path/to/repo URL. Then it worked.

Edgars

trešdiena, 2016. gada 10. augusts 07:39:05 UTC+2, Dylan Baars rakstīja:

I am attempting to use the Katello API to upload content into a
repository - we are using Jenkins to package an RPM and upload it into
Katello for deployment. This worked fine with the 3.0 RCs, however in 3.0.2
although everything seems to complete successfully, I am not seeing the
package in the GUI -

I followed the “Uploading content to the satellite server” example from
https://access.redhat.com/documentation/en-US/Red_Hat_
Satellite/6.1/html/API_Guide/curl-examples.html

The steps and output I do are below (the output is from Jenkins)

  • byte=5000000

  • offset=0

  • /usr/bin/rm uploadlist.000 uploadlist.001

  • split --bytes 5000000 --numeric-suffixes --suffix-length=3
    tomcat8-8.5.4-1.noarch.rpm uploadlist.

  • shopt -s nullglob

  • array=(uploadlist.*)
    ++ curl -s -H Accept:application/json,version=2 -H
    Content-Type:application/json -X POST -u jenkins:removed -k -d ‘{}’
    https://wellkatello.niwa.local/katello/api/repositories/
    1406/content_uploads

  • RESPONSE=‘{“upload_id”:“4376d1f2-a9f0-451d-b888-3b53e496581c
    “,”_href”:“/pulp/api/v2/content/uploads/4376d1f2-a9f0-
    451d-b888-3b53e496581c/”}’
    ++ echo ‘{“upload_id”:“4376d1f2-a9f0-451d-b888-3b53e496581c”,“_href”
    :“/pulp/api/v2/content/uploads/4376d1f2-a9f0-451d-b888-3b53e496581c/”}’
    ++ sed ‘s/"//g’
    ++ grep -e ‘.“upload_id”:\s"([a-zA-Z0-9-])*"’ -o
    ++ sed ‘s/{“upload_id”://’

  • REQUEST_ID=4376d1f2-a9f0-451d-b888-3b53e496581c

  • echo ‘Request ID [4376d1f2-a9f0-451d-b888-3b53e496581c] created’
    Request ID [4376d1f2-a9f0-451d-b888-3b53e496581c] created

  • for file in ‘“${array[@]}”’

  • echo ‘Offset is 0’
    Offset is 0

  • echo ‘Uploading chunk uploadlist.000’
    Uploading chunk uploadlist.000

  • curl -H Accept:application/json,version=2 -H
    Content-Type:multipart/form-data -X PUT -u jenkins:removed -k
    –data-urlencode content@uploadlist.000 --data-urlencode offset=0
    https://wellkatello.niwa.local/katello/api/repositories/
    1406/content_uploads/4376d1f2-a9f0-451d-b888-3b53e496581c
    % Total % Received % Xferd Average Speed Time Time Time
    Current
    Dload Upload Total Spent Left
    Speed

    0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:–
    0
    100 11.8M 0 0 100 11.8M 0 10.1M 0:00:01 0:00:01 --:–:–
    10.1M
    100 11.8M 0 0 100 11.8M 0 5597k 0:00:02 0:00:02 --:–:–
    5598k
    100 11.8M 0 0 100 11.8M 0 3827k 0:00:03 0:00:03 --:–:–
    3828k
    100 11.8M 0 0 100 11.8M 0 2907k 0:00:04 0:00:04 --:–:–
    2908k
    100 11.8M 0 1 100 11.8M 0 2745k 0:00:04 0:00:04 --:–:–
    2745k

  • offset=5000000

  • for file in ‘“${array[@]}”’

  • echo ‘Offset is 5000000’
    Offset is 5000000

  • echo ‘Uploading chunk uploadlist.001’
    Uploading chunk uploadlist.001

  • curl -H Accept:application/json,version=2 -H
    Content-Type:multipart/form-data -X PUT -u jenkins:removed -k
    –data-urlencode content@uploadlist.001 --data-urlencode offset=5000000
    https://wellkatello.niwa.local/katello/api/repositories/
    1406/content_uploads/4376d1f2-a9f0-451d-b888-3b53e496581c
    % Total % Received % Xferd Average Speed Time Time Time
    Current
    Dload Upload Total Spent Left
    Speed

    0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:–
    0
    100 6590k 0 0 100 6590k 0 5647k 0:00:01 0:00:01 --:–:–
    5652k
    100 6590k 0 0 100 6590k 0 3039k 0:00:02 0:00:02 --:–:–
    3041k
    100 6590k 0 1 100 6590k 0 2643k 0:00:02 0:00:02 --:–:–
    2643k

  • offset=10000000

  • echo ‘Importing upload into repository’
    Importing upload into repository

  • curl -H Accept:application/json,version=2 -H
    Content-Type:application/json -X PUT -u jenkins:removed -k -d
    ‘{“upload_ids”:[“4376d1f2-a9f0-451d-b888-3b53e496581c”]}’
    https://wellkatello.niwa.local/katello/api/repositories/
    1406/import_uploads
    % Total % Received % Xferd Average Speed Time Time Time
    Current
    Dload Upload Total Spent Left
    Speed

    0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:–
    0
    100 55 0 0 100 55 0 44 0:00:01 0:00:01 --:–:–
    44
    100 55 0 0 100 55 0 24 0:00:02 0:00:02 --:–:–
    24
    98 56 0 0 100 55 0 22 0:00:02 0:00:02 --:–:–
    22
    100 56 0 1 100 55 0 22 0:00:02 0:00:02 --:–:–
    22

  • echo ‘Delete the upload request’
    Delete the upload request

  • curl -H Accept:application/json,version=2 -H
    Content-Type:application/json -X DELETE -d ‘{}’ -u jenkins:removed -k
    https://wellkatello.niwa.local/katello/api/repositories/
    1406/content_uploads/4376d1f2-a9f0-451d-b888-3b53e496581c
    % Total % Received % Xferd Average Speed Time Time Time
    Current
    Dload Upload Total Spent Left
    Speed

    0 0 0 0 0 0 0 0 --:–:-- --:–:-- --:–:–
    0
    100 3 100 1 100 2 2 5 --:–:-- --:–:-- --:–:–
    6
    Finished: SUCCESS

I can successfully download the RPM from the repo -

[root@cislave01p upload]# wget http://wellkatello.niwa.local/
pulp/repos/NIWA/Library/custom/NIWA/NIWA_Custom_Applications
/tomcat8-8.5.4-1.noarch.rpm
–2016-08-10 17:31:56-- http://wellkatello.niwa.local
/pulp/repos/NIWA/Library/custom/NIWA/NIWA_Custom_Application
s/tomcat8-8.5.4-1.noarch.rpm
Resolving wellkatello.niwa.local (wellkatello.niwa.local)…
192.168.59.247
Connecting to wellkatello.niwa.local (wellkatello.niwa.local)|192.168.59.247|:80…
connected.
HTTP request sent, awaiting response… 200 OK
Length: 7716624 (7.4M) [application/octet-stream]
Saving to: ‘tomcat8-8.5.4-1.noarch.rpm.1’

100%[=======================================================

================================================================================>]
7,716,624 --.-K/s in 0.1s

2016-08-10 17:32:02 (70.7 MB/s) - ‘tomcat8-8.5.4-1.noarch.rpm.1’ saved
[7716624/7716624]

[root@cislave01p upload]#

A manual upload via the GUI works fine, and the GUI picks up the RPM. Any
suggestions greatly appreciated :slight_smile:


You received this message because you are subscribed to the Google Groups
“Foreman users” group.
To unsubscribe from this group and stop receiving emails from it, send an
email to foreman-users+unsubscribe@googlegroups.com.
To post to this group, send email to foreman-users@googlegroups.com.
Visit this group at https://groups.google.com/group/foreman-users.
For more options, visit https://groups.google.com/d/optout.

Hi Eric,

if I try and upload using hammer, I get this output

[root@wellkatello ~]# hammer repository upload-content --id 1453 --path
/root/tomcat8-8.5.4-1.noarch.rpm
Failed to upload file 'tomcat8-8.5.4-1.noarch.rpm' to repository. Please
check the file and try again.

it seems to spawn the upload task twice, the second one failing immediately
because it conflicts with the first task. I've attached the output from
production.log

Dylan

production.log (16.8 KB)

··· On Wednesday, August 10, 2016 at 10:54:09 PM UTC+12, Eric Helms wrote: > > Can you try an upload with hammer? Do you see the same behavior? > >> >>