Importing Manifest to organization

Problem: when i try to import manifest (generated form RHN portal) i got this error
Runtime Error could not initialize proxy [org.candlepin.model.UpstreamConsumer#8a81f81c6b6ace8f016c95e720f809c4] - no Session at org.hibernate.proxy.AbstractLazyInitializer.initialize:169

Expected outcome:

Foreman and Proxy versions: 1.20.1

Foreman and Proxy plugin versions: 3.10.0

Distribution and version:

Other relevant data:

Katello::Errors::CandlepinError: Runtime Error could not initialize proxy [org.candlepin.model.UpstreamConsumer#8a81f81c6b6ace8f016c95e720f809c4] - no Session at org.hibernate.proxy.AbstractLazyInitializer.initialize:169

Backtrace:

/opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:21:in rescue in propagate_candlepin_errors' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:18:in propagate_candlepin_errors’ /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/propagate_candlepin_errors.rb:9:in run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:22:in call’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:26:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware.rb:18:in pass’ /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/remote_action.rb:16:in block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/remote_action.rb:40:in block in as_remote_user’ /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/models/katello/concerns/user_extensions.rb:21:in cp_config' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/remote_action.rb:27:in as_cp_user’ /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/remote_action.rb:39:in as_remote_user' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/remote_action.rb:16:in run’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:22:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:26:in pass’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware.rb:18:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.3/app/lib/actions/middleware/rails_executor_wrap.rb:14:in block in run’ /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/activesupport-5.2.1/lib/active_support/execution_wrapper.rb:87:in wrap' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.3/app/lib/actions/middleware/rails_executor_wrap.rb:13:in run’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:22:in call' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:26:in pass’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware.rb:18:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/action/progress.rb:30:in with_progress_calculation’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/action/progress.rb:16:in run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:22:in call’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:26:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware.rb:18:in pass’ /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/keep_locale.rb:11:in block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/keep_locale.rb:22:in with_locale’ /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.10.0/app/lib/actions/middleware/keep_locale.rb:11:in run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:22:in call’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:26:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware.rb:18:in pass’ /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.3/app/lib/actions/middleware/keep_current_user.rb:15:in block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.3/app/lib/actions/middleware/keep_current_user.rb:43:in restore_curent_user’ /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.3/app/lib/actions/middleware/keep_current_user.rb:15:in run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:22:in call’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:26:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware.rb:18:in pass’ /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.3/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in block in run' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.3/app/lib/actions/middleware/keep_current_taxonomies.rb:44:in restore_current_taxonomies’ /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-0.14.3/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:22:in call’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:26:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware.rb:18:in pass’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware.rb:31:in run' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/stack.rb:22:in call’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/middleware/world.rb:30:in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/action.rb:538:in block (2 levels) in execute_run’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/action.rb:537:in catch' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/action.rb:537:in block in execute_run’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/action.rb:451:in block in with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/action.rb:451:in catch’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/action.rb:451:in with_error_handling' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/action.rb:532:in execute_run’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/action.rb:278:in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in block (2 levels) in execute’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/execution_plan/steps/abstract.rb:162:in with_meta_calculation' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in block in execute’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:30:in open_action' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:15:in execute’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/director.rb:43:in execute' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/executors/parallel/worker.rb:14:in block in on_message’ /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/executors.rb:12:in run_user_code' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/executors/parallel/worker.rb:13:in on_message’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/context.rb:46:in on_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/executes_context.rb:7:in on_envelope’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.1.2/lib/dynflow/actor.rb:26:in on_envelope’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/awaits.rb:15:in on_envelope’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in on_envelope’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:38:in process_envelope’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:31:in process_envelopes?' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/buffer.rb:20:in on_envelope’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/termination.rb:55:in on_envelope’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/removes_child.rb:10:in on_envelope’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/abstract.rb:25:in pass' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/behaviour/sets_results.rb:14:in on_envelope’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:161:in process_envelope' /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:95:in block in on_envelope’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:118:in block (2 levels) in schedule_execution' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in block in synchronize’ /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in synchronize' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/synchronization/mri_lockable_object.rb:38:in synchronize’ /opt/theforeman/tfm/root/usr/share/gems/gems/concurrent-ruby-edge-0.2.4/lib/concurrent/actor/core.rb:115:in block in schedule_execution' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:18:in call’ /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:96:in work' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/serialized_execution.rb:77:in block in call_job’ /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:348:in run_task' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:337:in block (3 levels) in create_worker’ /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in loop' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:320:in block (2 levels) in create_worker’ /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in catch' /opt/theforeman/tfm-ror52/root/usr/share/gems/gems/concurrent-ruby-1.0.5/lib/concurrent/executor/ruby_thread_pool_executor.rb:319:in block in create_worker’ /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context’

i have tried upgrading up to 3.13 but issue remains same , i need urgent help please, need urgent help to resolve this issue

can someone from @katello team please take a look?

Can you provide recent logs from /var/log/candlepin/? The Foreman logs don’t contain any useful information when Candlepin has a problem.

Which Java do you have installed?

Thanks for your quick reply @evgeni

the java version i was using is:
openjdk version “1.8.0_191”
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)

i have tried with oracle java same results.
java version “1.8.0_251”
Java™ SE Runtime Environment (build 1.8.0_251-b08)
Java HotSpot™ 64-Bit Server VM (build 25.251-b08, mixed mode)

cant attached logs getting error message new user cant attache

can you post them to a pastebin?

https://1drv.ms/u/s!AtjqakAK_7Dzgtw-iGgyFkTyAjrGCg?e=heDzrJ

here is one drive link please let me know if that works

Thanks! That’s better.

I am not a huge Candlepin expert (maybe @Jonathon_Turel can jump in here?), but I see the following error in the logs you posted (I trimmed the stacktraces, they are useless, but added some of the normal log messages for context):

2020-04-16 10:40:55,892 [thread=http-bio-8443-exec-6] [req=b2a264bb-d6f6-4156-bdf1-0f03d1458782, org=WINGAS, csid=39f17a19] ERROR org.candlepin.sync.ConsumerImporter - Cannot import manifest for org: WINGAS
2020-04-16 10:40:55,892 [thread=http-bio-8443-exec-6] [req=b2a264bb-d6f6-4156-bdf1-0f03d1458782, org=WINGAS, csid=39f17a19] ERROR org.candlepin.sync.ConsumerImporter - Upstream distributor f54a3d98-53ca-438a-8432-c601ea030cab already in used by org: itss
2020-04-16 10:40:55,916 [thread=http-bio-8443-exec-6] [req=b2a264bb-d6f6-4156-bdf1-0f03d1458782, org=WINGAS, csid=39f17a19] ERROR org.candlepin.resource.OwnerResource - Format error of the data in a manifest
org.candlepin.sync.SyncDataFormatException: This subscription management application has already been imported by another owner.
2020-04-16 10:40:55,917 [thread=http-bio-8443-exec-6] [req=b2a264bb-d6f6-4156-bdf1-0f03d1458782, org=WINGAS, csid=39f17a19] ERROR org.candlepin.sync.Importer - Recording import failure
org.candlepin.sync.SyncDataFormatException: This subscription management application has already been imported by another owner.
2020-04-16 10:40:55,920 [thread=http-bio-8443-exec-6] [req=b2a264bb-d6f6-4156-bdf1-0f03d1458782, org=WINGAS, csid=39f17a19] INFO  org.candlepin.resource.OwnerResource - Import attempt completed for owner WINGAS
2020-04-16 10:40:55,921 [thread=http-bio-8443-exec-6] [req=b2a264bb-d6f6-4156-bdf1-0f03d1458782, org=WINGAS, csid=39f17a19] ERROR org.candlepin.common.exceptions.mappers.CandlepinExceptionMapper - Runtime Error could not initialize proxy [org.candlepin.model.UpstreamConsumer#8a81f81c6b6ace8f016c95e720f809c4] - no Session at org.hibernate.proxy.AbstractLazyInitializer.initialize:169
org.hibernate.LazyInitializationException: could not initialize proxy [org.candlepin.model.UpstreamConsumer#8a81f81c6b6ace8f016c95e720f809c4] - no Session
2020-04-16 10:40:55,922 [thread=http-bio-8443-exec-6] [req=b2a264bb-d6f6-4156-bdf1-0f03d1458782, org=WINGAS, csid=39f17a19] WARN  org.candlepin.audit.EventSinkImpl - Rolling back ActiveMQ transaction.
2020-04-16 10:40:55,930 [thread=http-bio-8443-exec-6] [req=b2a264bb-d6f6-4156-bdf1-0f03d1458782, org=WINGAS, csid=39f17a19] INFO  org.candlepin.common.filter.LoggingFilter - Response: status=500, content-type="application/json", time=449

This hints that you have an organization “itss” which already has imported the very same manifest that you now try to import into the “WINGAS” organization. It is not possible to import the same manifest into multiple organizations of the same Katello installation.

It seems that after that, you tried to import a manifest into “itss”, but that failed as the manifest was older than the data already imported:

2020-04-16 10:50:12,014 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=, csid=2ec479ff] INFO  org.candlepin.common.filter.LoggingFilter - Request: verb=POST, uri=/candlepin/owners/itss/imports
2020-04-16 10:50:12,142 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=itss, csid=2ec479ff] INFO  org.candlepin.sync.RulesImporter - Importing new rules from manifest, current version: 5.37 new version: 5.37
2020-04-16 10:50:12,155 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=itss, csid=2ec479ff] INFO  org.candlepin.audit.EventSinkImpl - Created new ActiveMQ producer.
2020-04-16 10:50:12,158 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=itss, csid=2ec479ff] INFO  org.candlepin.policy.js.JsRunnerProvider - Recompiling rules with timestamp: 2020-04-16 10:50:12.144
2020-04-16 10:50:12,317 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=itss, csid=2ec479ff] ERROR org.candlepin.sync.Importer - Conflicts occurred during import that were not overridden:
2020-04-16 10:50:12,317 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=itss, csid=2ec479ff] ERROR org.candlepin.sync.Importer - [MANIFEST_OLD]
2020-04-16 10:50:12,343 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=itss, csid=2ec479ff] ERROR org.candlepin.resource.OwnerResource - Recording import failure
org.candlepin.sync.ImportConflictException: Import is older than existing data
2020-04-16 10:50:12,344 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=itss, csid=2ec479ff] ERROR org.candlepin.sync.Importer - Recording import failure
org.candlepin.sync.ImportConflictException: Import is older than existing data
2020-04-16 10:50:12,351 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=itss, csid=2ec479ff] INFO  org.candlepin.resource.OwnerResource - Import attempt completed for owner ITSS
2020-04-16 10:50:12,358 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=itss, csid=2ec479ff] WARN  org.candlepin.audit.EventSinkImpl - Rolling back ActiveMQ transaction.
2020-04-16 10:50:12,364 [thread=http-bio-8443-exec-4] [req=fb69f661-26ae-4dc4-81b4-e780d9b9f311, org=itss, csid=2ec479ff] INFO  org.candlepin.common.filter.LoggingFilter - Response: status=409, content-type="application/json", time=363

If my understanding is correct, then you’ll have to create two separate subscription distributors in the RH portal and use one per organization. However, I must say that from experience, handling multiple orgs is not trivial and I’d avoid it unless really necessary.

Nice assessment @evgeni - agree on all points. I’ll keep an eye on this thread, so let me know if you have any more questions @thussain.