Hello,
Does anybody have experience syncing the td-agent repository? Or via Shrub
in general?
td-agent repo for EL6
https://shrub.appspot.com/packages.treasuredata.com/2/redhat/6/x86_64/
It's powered by Shrub which is basically a proxy for Amazon S3
https://github.com/gabriel/shrub
When trying to sync, I get the error "mismatched tag: line 61, column 12".
The repodata directory exists and repomd.xml exists.
However, while on Shrub directory listing works, RPM links point to Amazon
S3 URLs.
On the other hand, directory listing does not work on the Amazon S3 URL.
Maybe that is the issue?
Is there a way to work around this?
Here is the full error I'm getting.
Action:
Actions::Pulp::Repository::Sync
Input:
{"pulp_id"=>"company-td-agent-td-agent_EL6_x86_64",
"task_id"=>nil,
"remote_user"=>"david_somers-harris-ec6995a3",
"remote_cp_user"=>"david.somers-harris",
"locale"=>"en"}
Output:
{"pulp_tasks"=>
[{"exception"=>nil,
"task_type"=>"pulp.server.tasks.repository.sync_with_auto_publish",
"_href"=>"/pulp/api/v2/tasks/78faefb0-73b1-49ca-ac0b-9db00408563e/",
"task_id"=>"78faefb0-73b1-49ca-ac0b-9db00408563e",
"tags"=>
["pulp:repository:company-td-agent-td-agent_EL6_x86_64",
"pulp:action:sync"],
"finish_time"=>"2015-01-05T02:35:46Z",
"ns"=>"task_status",
"start_time"=>"2015-01-05T02:35:45Z",
"traceback"=>
"Traceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task\n R = retval = fun(*args, **kwargs)\n File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 306, in call\n return super(Task, self).call(*args, **kwargs)\n File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in protected_call\n return self.run(*args, **kwargs)\n File "/usr/lib/python2.7/site-packages/pulp/server/tasks/repository.py", line 210, in sync_with_auto_publish\n sync_result = managers.repo_sync_manager().sync(repo_id, sync_config_override=overrides)\n File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py", line 113, in sync\n raise PulpExecutionException(('Importer indicated a failed response'))\nPulpExecutionException: Importer indicated a failed response\n",
"spawned_tasks"=>,
"progress_report"=>
{"yum_importer"=>
{"content"=>
{"size_total"=>0,
"items_left"=>0,
"items_total"=>0,
"state"=>"NOT_STARTED",
"size_left"=>0,
"details"=>
{"rpm_total"=>0, "rpm_done"=>0, "drpm_total"=>0, "drpm_done"=>0},
"error_details"=>},
"comps"=>{"state"=>"NOT_STARTED"},
"distribution"=>
{"items_total"=>0,
"state"=>"NOT_STARTED",
"error_details"=>,
"items_left"=>0},
"errata"=>{"state"=>"NOT_STARTED"},
"metadata"=>
{"state"=>"FAILED", "error"=>"mismatched tag: line 61, column 12"}}},
"queue"=>"reserved_resource_worker-2@foreman00.sandbox.local.dq",
"state"=>"error",
"result"=>nil,
"error"=>
{"code"=>"PLP0000",
"data"=>{},
"description"=>"Importer indicated a failed response",
"sub_errors"=>},
"_id"=>{"$oid"=>"54a9f881a331e98006799d83"},
"id"=>"54a9f881a0496d3974cdbe52"}],
"poll_attempts"=>{"total"=>4, "failed"=>3}}
Exception:
Katello::Errors::PulpError: PLP0000: Importer indicated a failed response
Backtrace:
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/pulp/abstract_async_task.rb:100:in block in external_task=' /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/pulp/abstract_async_task.rb:98:in each'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/pulp/abstract_async_task.rb:98:in external_task=' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/polling.rb:86:in poll_external_task_with_rescue'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/polling.rb:15:in run' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/cancellable.rb:9:in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/pulp/abstract_async_task.rb:57:in run' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:443:in block (3 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in call' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in pass' /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/middleware/remote_action.rb:28:in block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/middleware/remote_action.rb:58:in block (2 levels) in as_remote_user' /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/katello/util/thread_session.rb:85:in set_pulp_config'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/middleware/remote_action.rb:44:in as_pulp_user' /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/middleware/remote_action.rb:57:in block in as_remote_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/katello/util/thread_session.rb:92:in set_cp_config' /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/middleware/remote_action.rb:39:in as_cp_user'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/middleware/remote_action.rb:56:in as_remote_user' /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/middleware/remote_action.rb:28:in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in call' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in pass'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in pass' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/progress.rb:30:in with_progress_calculation'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action/progress.rb:16:in run' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:26:in pass' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware.rb:16:in pass'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/middleware/keep_locale.rb:24:in block in run' /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/middleware/keep_locale.rb:35:in with_locale'
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.0.0/app/lib/actions/middleware/keep_locale.rb:24:in run' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/stack.rb:22:in call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/middleware/world.rb:30:in execute' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:442:in block (2 levels) in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:441:in catch' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:441:in block in execute_run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in call' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in block in with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in catch' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:365:in with_error_handling'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:436:in execute_run' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/action.rb:230:in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:9:in block (2 levels) in execute' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract.rb:152:in call'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract.rb:152:in with_meta_calculation' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:8:in block in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:22:in open_action' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:7:in execute'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/executors/parallel/worker.rb:20:in block in on_message' /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:859:in block in assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in tap' /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:858:in assigns'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:138:in match_value' /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:116:in block in match'
/opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in each' /opt/rh/ruby193/root/usr/share/gems/gems/algebrick-0.4.0/lib/algebrick.rb:115:in match'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/executors/parallel/worker.rb:17:in on_message' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:82:in on_envelope'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:72:in receive' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in block (2 levels) in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in loop' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in block in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in catch' /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:99:in run'
/opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.3/lib/dynflow/micro_actor.rb:13:in block in initialize' /opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in call'
/opt/rh/ruby193/root/usr/share/gems/gems/logging-1.8.1/lib/logging/diagnostic_context.rb:323:in `block in create_with_logging_context'
Action:
Actions::Pulp::Repository::DistributorPublish
Input:
{"pulp_id"=>"company-td-agent-td-agent_EL6_x86_64",
"dependency"=>Step(3).output[:pulp_tasks],
"distributor_type_id"=>"nodes_http_distributor",
"remote_user"=>"david_somers-harris-ec6995a3",
"remote_cp_user"=>"david.somers-harris",
"locale"=>"en"}
Output:
{}
Action:
Actions::ElasticSearch::Repository::IndexContent
Input:
{"dependency"=>Step(3).output[:pulp_tasks],
"id"=>209,
"remote_user"=>"david_somers-harris-ec6995a3",
"remote_cp_user"=>"david.somers-harris",
"locale"=>"en"}
Output:
{}
Action:
Actions::Katello::Repository::Sync
Input:
{"repository"=>
{"id"=>209, "name"=>"td-agent EL6 x86_64", "label"=>"td-agent_EL6_x86_64"},
"product"=>
{"id"=>24,
"name"=>"td-agent",
"label"=>"td-agent",
"cp_id"=>"1420425267658"},
"provider"=>{"id"=>3, "name"=>"Anonymous"},
"organization"=>{"id"=>3, "name"=>"Company", "label"=>"company"},
"sync_result"=>Step(3).output,
"locale"=>"en"}
Output:
{}
Thanks!
David