Migration problem from Katello 3.18 to 4.0 and foreman 2.3 to 2.4

When i try run update procedure i have a error on 1800 step.
I don’t know what do I do bad ?
I updates all repository and run cmd : foreman-installer --scenario katello

[NOTICE] [configure] 1800 out of 1950 done.
[ERROR ] [configure] ‘pulpcore-manager migrate --noinput’ returned 1 instead of one of [0]
[ERROR ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: change from ‘notrun’ to [‘0’] failed: ‘pulpcore-manager migrate --noinput’ returned 1 instead of one of [0]
[NOTICE] [configure] 1900 out of 1950 done.
[NOTICE] [configure] System configuration has finished.

System version:
Centos 7.

smart_proxies: Pulp, Pulpcore, Puppet CA, Puppet, Logs oraz Registration 2.4.0

foreman-tasks The goal of this plugin is to unify the way of showing task statuses across the Foreman instance. It defines Task model for keeping the information about the tasks and Lock for assigning the tasks to resources. The locking allows dealing with preventing multiple colliding tasks to be run on the same resource. It also optionally provides Dynflow infrastructure for using it for managing the tasks. Ivan Nečas 4.0.1
foreman_remote_execution A plugin bringing remote execution to the Foreman, completing the config management functionality with remote management functionality. Foreman Remote Execution team 4.3.0
katello Katello adds Content and Subscription Management to Foreman. For this it relies on Candlepin and Pulp. N/A 4.0.1.1

When i try run cli : pulpcore-manager

Traceback (most recent call last):
File “/usr/lib/python3.6/site-packages/pulpcore/app/settings.py”, line 269, in
CONTENT_ORIGIN
NameError: name ‘CONTENT_ORIGIN’ is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/bin/pulpcore-manager”, line 11, in
load_entry_point(‘pulpcore==3.9.1’, ‘console_scripts’, ‘pulpcore-manager’)()
File “/usr/lib/python3.6/site-packages/pulpcore/app/manage.py”, line 11, in manage
execute_from_command_line(sys.argv)
File “/usr/lib/python3.6/site-packages/django/core/management/init.py”, line 381, in execute_from_command_line
utility.execute()
File “/usr/lib/python3.6/site-packages/django/core/management/init.py”, line 357, in execute
django.setup()
File “/usr/lib/python3.6/site-packages/django/init.py”, line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File “/usr/lib/python3.6/site-packages/django/apps/registry.py”, line 122, in populate
app_config.ready()
File “/usr/lib/python3.6/site-packages/pulpcore/app/apps.py”, line 94, in ready
self.import_viewsets()
File “/usr/lib/python3.6/site-packages/pulpcore/app/apps.py”, line 127, in import_viewsets
from pulpcore.app.viewsets import NamedModelViewSet
File “/usr/lib/python3.6/site-packages/pulpcore/app/viewsets/init.py”, line 1, in
from .base import ( # noqa
File “/usr/lib/python3.6/site-packages/pulpcore/app/viewsets/base.py”, line 14, in
from pulpcore.openapi import PulpAutoSchema
File “/usr/lib/python3.6/site-packages/pulpcore/openapi/init.py”, line 25, in
from pulpcore.app.settings import INSTALLED_PULP_PLUGINS
File “/usr/lib/python3.6/site-packages/pulpcore/app/settings.py”, line 273, in
"CONTENT_ORIGIN is a required setting but it was not configured. This may be caused "
django.core.exceptions.ImproperlyConfigured: CONTENT_ORIGIN is a required setting but it was not configured. This may be caused by invalid read permissions of the settings file. Note that CONTENT_ORIGIN is set by the installer automatically.

Will start to look and see what is going on. Did you see any tracebacks in /var/log/messages from Pulp during the upgrade failure?

Please use markdown code blocks when pasting code/tracebacks.

Have you had a look at the installer log. AFAIK it prints the location at the end if a run failed.

Sadly we use an exec resource which logs the output at info level so it isn’t shown in installer output anymore. You have to dig into the log to see it.

In messages file are nothing special:

Jun 15 15:24:46 katello systemd: pulpcore-worker@2.service failed.
Jun 15 15:24:46 katello pulpcore-worker-1: Error 111 connecting to localhost:6379. Connection refused.
Jun 15 15:24:46 katello systemd: pulpcore-worker@3.service: main process exited, code=exited, status=1/FAILURE4

Jun 15 15:42:50 katello pulpcore-worker-1: Traceback (most recent call last):
Jun 15 15:42:50 katello pulpcore-worker-1: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Jun 15 15:42:50 katello pulpcore-worker-1: return self.cursor.execute(sql, params)
Jun 15 15:42:50 katello pulpcore-worker-1: psycopg2.errors.UndefinedColumn: column core_task.logging_cid does not exist
Jun 15 15:42:50 katello pulpcore-worker-1: LINE 1: ...pdated", "core_task"."state", "core_task"."name", "core_task...
Jun 15 15:42:50 katello pulpcore-worker-1: ^

Jun 15 15:42:46 katello pulpcore-worker-4: Traceback (most recent call last):
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
Jun 15 15:42:46 katello pulpcore-worker-4: return self.cursor.execute(sql, params)
Jun 15 15:42:46 katello pulpcore-worker-4: psycopg2.errors.UndefinedColumn: column core_task.logging_cid does not exist
Jun 15 15:42:46 katello pulpcore-worker-4: LINE 1: ...pdated", "core_task"."state", "core_task"."name", "core_task...
Jun 15 15:42:46 katello pulpcore-worker-4: ^
Jun 15 15:42:46 katello pulpcore-worker-4: The above exception was the direct cause of the following exception:
Jun 15 15:42:46 katello pulpcore-worker-4: Traceback (most recent call last):
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/bin/rq", line 11, in <module>
Jun 15 15:42:46 katello pulpcore-worker-4: load_entry_point('rq==1.7.0', 'console_scripts', 'rq')()
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/click/core.py", line 829, in __call__
Jun 15 15:42:46 katello pulpcore-worker-4: return self.main(*args, **kwargs)
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/click/core.py", line 782, in main
Jun 15 15:42:46 katello pulpcore-worker-4: rv = self.invoke(ctx)
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
Jun 15 15:42:46 katello pulpcore-worker-4: return _process_result(sub_ctx.command.invoke(sub_ctx))
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
Jun 15 15:42:46 katello pulpcore-worker-4: return ctx.invoke(self.callback, **ctx.params)
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/click/core.py", line 610, in invoke
Jun 15 15:42:46 katello pulpcore-worker-4: return callback(*args, **kwargs)
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/cli/cli.py", line 78, in wrapper
Jun 15 15:42:46 katello pulpcore-worker-4: return ctx.invoke(func, cli_config, *args[1:], **kwargs)
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/click/core.py", line 610, in invoke
Jun 15 15:42:46 katello pulpcore-worker-4: return callback(*args, **kwargs)
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/cli/cli.py", line 268, in worker
Jun 15 15:42:46 katello pulpcore-worker-4: max_jobs=max_jobs, with_scheduler=with_scheduler)
Jun 15 15:42:46 katello pulpcore-worker-4: File "/usr/lib/python3.6/site-packages/rq/worker.py", line 535, in work
Jun 15 15:42:46 katello pulpcore-worker-4: self.register_birth()

Jun 15 15:42:44 katello pulpcore-worker-1: pulp [None]: rq.worker:ERROR: Worker rq:worker:18713@katello.xxx.org: found an unhandle
d exception, quitting...

Jun 15 15:42:41 katello server: Jun 15, 2021 3:42:41 PM com.google.inject.internal.ProxyFactory <init>
Jun 15 15:42:41 katello server: WARNING: Method [public void org.candlepin.model.ContentCurator.delete(org.candlepin.model.Persisted
)] is synthetic and is being intercepted by [com.google.inject.persist.jpa.JpaLocalTxnInterceptor@1b39a122]. This could indicate a b
ug.  The method may be intercepted twice, or may not be intercepted at all.4

Jun 15 15:42:33 katello server: Jun 15, 2021 3:42:33 PM org.apache.catalina.loader.WebappClassLoaderBase checkThreadLocalMapForLeaks
Jun 15 15:42:33 katello server: SEVERE: The web application [/candlepin] created a ThreadLocal with key of type [java.lang.ThreadLoc
al] (value [java.lang.ThreadLocal@5f772ae0]) and a value of type [io.netty.util.internal.InternalThreadLocalMap] (value [io.netty.ut
il.internal.InternalThreadLocalMap@1f26e3]) but failed to remove it when the web application was stopped. Threads are going to be re
newed over time to try and avoid a probable memory leak.
Jun 15 15:42:33 katello server: Jun 15, 2021 3:42:33 PM org.apache.coyote.AbstractProtocol stop
Jun 15 15:42:33 katello server: INFO: Stopping ProtocolHandler ["http-bio-127.0.0.1-23443"]


Jun 15 15:42:32 katello foreman: warning: broker sent EOF, and connection not reliable
Jun 15 15:42:32 katello foreman: #<Thread:0x000000001556ca10@/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:471 run> terminated with exception (report_on_exception is true):
Jun 15 15:42:32 katello foreman: /opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/client/utils.rb:198:in `block (2 levels) in start_listeners': Received message is nil, and connection not reliable (Stomp::Error::NilMessageError)
Jun 15 15:42:32 katello foreman: from /opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/client/utils.rb:194:in `loop'
Jun 15 15:42:32 katello foreman: from /opt/theforeman/tfm/root/usr/share/gems/gems/stomp-1.4.9/lib/client/utils.rb:194:in `block in start_listeners'
Jun 15 15:42:32 katello foreman: from /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
Jun 15 15:42:33 katello server: Jun 15, 2021 3:42:33 PM org.apache.catalina.loader.WebappClassLoaderBase checkThreadLocalMapForLeaks
Jun 15 15:42:33 katello server: SEVERE: The web application [/candlepin] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@5f772ae0]) and a value of ty



I did ask for the installer logs, so in /var/log/foreman-installer. That should have the logs of what went wrong with pulpcore-manager.

Sorry, logs from /var/log/foreman-installer/katello.log

2021-06-15 16:05:33 [DEBUG ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/unless: WARNINGS:
2021-06-15 16:05:33 [DEBUG ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/unless: ?: (guardian.W001) Guardian authentication backend is not hooked. You can add this in settings as eg: `AUTHENTICATION_BACKENDS = ('django.contrib.auth.backends.ModelBackend', 'guardian.backends.ObjectPermissionBackend')`.
2021-06-15 16:05:33 [DEBUG ] [configure] Exec[pulpcore-manager migrate --noinput](provider=posix): Executing 'pulpcore-manager migrate --noinput'
2021-06-15 16:05:33 [DEBUG ] [configure] Executing with uid=pulp: 'pulpcore-manager migrate --noinput'
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: System check identified some issues:
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: WARNINGS:
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: ?: (guardian.W001) Guardian authentication backend is not hooked. You can add this in settings as eg: `AUTHENTICATION_BACKENDS = ('django.contrib.auth.backends.ModelBackend', 'guardian.backends.ObjectPermissionBackend')`.
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: Operations to perform:
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   Apply all migrations: admin, auth, certguard, container, contenttypes, core, deb, file, guardian, pulp_2to3_migration, rpm, sessions
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: Running migrations:
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   Applying core.0049_add_file_field_to_uploadchunk...Traceback (most recent call last):
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     return self.cursor.execute(sql, params)


2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: The above exception was the direct cause of the following exception:
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: Traceback (most recent call last):
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/usr/bin/pulpcore-manager", line 11, in <module>
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     load_entry_point('pulpcore==3.9.1', 'console_scripts', 'pulpcore-manager')()
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/pulpcore/app/manage.py", line 11, in manage
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     execute_from_command_line(sys.argv)
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     utility.execute()
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute

grate --noinput]/returns:     return executor(sql, params, many, context)
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     return self.cursor.execute(sql, params)
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     raise dj_exc_value.with_traceback(traceback) from exc_value
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:   File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns:     return self.cursor.execute(sql, params)
2021-06-15 16:05:35 [INFO  ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: django.db.utils.IntegrityError: column "file" contains null values
2021-06-15 16:05:35 [ERROR ] [configure] 'pulpcore-manager migrate --noinput' returned 1 instead of one of [0]
2021-06-15 16:05:35 [ERROR ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]/returns: change from 'notrun' to ['0'] failed: 'pulpcore-manager migrate --noinput' returned 1 instead of one of [0]
2021-06-15 16:05:35 [DEBUG ] [configure] /Stage[main]/Pulpcore::Database/Pulpcore::Admin[migrate --noinput]/Exec[pulpcore-manager migrate --noinput]: Evaluated in 4.88 seconds
2021-06-15 16:05:35 [DEBUG ] [configure] Pulpcore::Admin[migrate --noinput]: Starting to evaluate the resource (1844 of 1950)
2021-06-15 16:05:35 [DEBUG ] [configure] Pulpcore::Admin[migrate --noinput]: Resource is being skipped, unscheduling all events
2021-06-15 16:05:35 [DEBUG ] [configure] Pulpcore::Admin[migrate --noinput]: Evaluated in 0.00 seconds

Hi @ekohl , do you see any problem ? Do you know what is it wrong ?

Ok, when i use command on pulp user, I have this information:

pulpcore-manager check
System check identified some issues:

WARNINGS:
?: (guardian.W001) Guardian authentication backend is not hooked. You can add this in settings as eg: `AUTHENTICATION_BACKENDS = ('django.contrib.auth.backends.ModelBackend', 'guardian.backends.ObjectPermissionBackend')`.

System check identified 1 issue (0 silenced).

In /etc/pulp/settings i have this notice:

AUTHENTICATION_BACKENDS = ['pulpcore.app.authentication.PulpNoCreateRemoteUserBackend']

When i use command:

pulpcore-manager migrate --noinput
System check identified some issues:

WARNINGS:
?: (guardian.W001) Guardian authentication backend is not hooked. You can add this in settings as eg: `AUTHENTICATION_BACKENDS = ('django.contrib.auth.backends.ModelBackend', 'guardian.backends.ObjectPermissionBackend')`.
Operations to perform:
  Apply all migrations: admin, auth, certguard, container, contenttypes, core, deb, file, guardian, pulp_2to3_migration, rpm, sessions
Running migrations:
  Applying core.0049_add_file_field_to_uploadchunk...Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NotNullViolation: column "file" contains null values


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/bin/pulpcore-manager", line 11, in <module>
    load_entry_point('pulpcore==3.9.1', 'console_scripts', 'pulpcore-manager')()
  File "/usr/lib/python3.6/site-packages/pulpcore/app/manage.py", line 11, in manage
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 234, in handle
    fake_initial=fake_initial,
  File "/usr/lib/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/lib/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/lib/python3.6/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/lib/python3.6/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/lib/python3.6/site-packages/django/db/migrations/operations/fields.py", line 112, in database_forwards
    field,
  File "/usr/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 447, in add_field
    self.execute(sql, params)
  File "/usr/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 137, in execute
    cursor.execute(sql, params)
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: column "file" contains null values

When i use command:

pulpcore-manager migrate --noinput
System check identified some issues:

WARNINGS:
?: (guardian.W001) Guardian authentication backend is not hooked. You can add this in settings as eg: `AUTHENTICATION_BACKENDS = ('django.contrib.auth.backends.ModelBackend', 'guardian.backends.ObjectPermissionBackend')`.
Operations to perform:
  Apply all migrations: admin, auth, certguard, container, contenttypes, core, deb, file, guardian, pulp_2to3_migration, rpm, sessions
Running migrations:
  Applying core.0049_add_file_field_to_uploadchunk...Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NotNullViolation: column "file" contains null values


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/bin/pulpcore-manager", line 11, in <module>
    load_entry_point('pulpcore==3.9.1', 'console_scripts', 'pulpcore-manager')()
  File "/usr/lib/python3.6/site-packages/pulpcore/app/manage.py", line 11, in manage
    execute_from_command_line(sys.argv)
  File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/lib/python3.6/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python3.6/site-packages/django/core/management/base.py", line 83, in wrapped
    res = handle_func(*args, **kwargs)
  File "/usr/lib/python3.6/site-packages/django/core/management/commands/migrate.py", line 234, in handle
    fake_initial=fake_initial,
  File "/usr/lib/python3.6/site-packages/django/db/migrations/executor.py", line 117, in migrate
    state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
  File "/usr/lib/python3.6/site-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
    state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
  File "/usr/lib/python3.6/site-packages/django/db/migrations/executor.py", line 245, in apply_migration
    state = migration.apply(state, schema_editor)
  File "/usr/lib/python3.6/site-packages/django/db/migrations/migration.py", line 124, in apply
    operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
  File "/usr/lib/python3.6/site-packages/django/db/migrations/operations/fields.py", line 112, in database_forwards
    field,
  File "/usr/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 447, in add_field
    self.execute(sql, params)
  File "/usr/lib/python3.6/site-packages/django/db/backends/base/schema.py", line 137, in execute
    cursor.execute(sql, params)
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 76, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: column "file" contains null values

I did full migration to pulp3 from 2 and use : foreman-maintain content remove-pulp2, but script try to use all steps migration with pulp_2to3_migration ???
Maybe is it problem ?

Yes. I saw, but i don’t know what is wrong. I thik, is somting wrong with permission to env or variables for DJango. But i don’t know how to resolve this problem. Can you look at my logs in this post.

I think problem is in packages.
When i run :

sudo -u pulp PULP_SETTINGS='/etc/pulp/settings.py' DJANGO_SETTINGS_MODULE='pulpcore.app.settings' pulpcore-manager migrate --noinput
System check identified some issues:

WARNINGS:
?: (guardian.W001) Guardian authentication backend is not hooked. You can add this in settings as eg: `AUTHENTICATION_BACKENDS = ('django.contrib.auth.backends.ModelBackend', 'guardian.backends.ObjectPermissionBackend')`.
Operations to perform:
  Apply all migrations: admin, auth, certguard, container, contenttypes, core, deb, file, guardian, pulp_2to3_migration, rpm, sessions
Running migrations:
  Applying core.0049_add_file_field_to_uploadchunk...Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NotNullViolation: column "file" contains null values

Can anyone tell me what is wrong?
Are packages version incorrect? are there any bad permissions?

Evening
had the exact same problem.

The issue is in the 0049_add_file_field_to_uploadchunk.py
It deletes a column “file” from table “upload”
then add’s it again to table “uploadchunk”…but if records already exist in the uploadchunk table, then this new “file” column will contain NULLs. which is not allowed.

need to add: "null=True, " to the list of options to the AddField in file:

/usr/lib/python3.6/site-packages/pulpcore/app/migrations/0049_add_file_field_to_uploadchunk.py

        migrations.AddField(
            model_name='uploadchunk',
            name='file',
            field=models.FileField(null=True, default=None, max_length=255, upload_to=pulpcore.app.models.upload.UploadChunk.storage_path),
            preserve_default=False,
        ),

Then it runs successfully:

4 Likes