Debian Repo sync end with the error "This field may not be blank"

Hi @maximilian, thanks for your answer but this doesn’t solve my problem because the setting Default Custom Repository download policy was by immediate. I changed it in the past for the deb repos.

Have you another idea?

Not yet. I’ve tested this on Foreman 2.5 with Katello 4.1 and pulp_deb 2.13 myself successfully. I will ask my colleagues about it.

So have you explicitly tried setting the default download policy explicitly for that repo? There is a default setting as you said, but this default setting is not applied to deb repos as of now. This is fixed in the PR that @maximilian mentioned.

Hi @nadjaheitmann, in the version foreman 3.0.1 with katello 4.2.1 can’t i set the Download Policy for a debian repo. This option not exist.

I create a snapshot of my foreman vm and upgraded it to foreman 3.1 with katello 4.3.0.rc4. Now i have the option Download Policy but i get the same error, no matter if the option set to On Demand or Immediate.

The Error:

{"pulp_tasks"=>
  [{"pulp_href"=>"/pulp/api/v3/tasks/f0c4f436-eff3-4ae1-afa5-8e3c7f44239b/",
    "pulp_created"=>"2021-12-22T11:21:31.383+00:00",
    "state"=>"failed",
    "name"=>"pulp_deb.app.tasks.synchronizing.synchronize",
    "logging_cid"=>"c4b74ae4-ee30-4ad2-bd19-f10353dd0bf7",
    "started_at"=>"2021-12-22T11:21:31.671+00:00",
    "finished_at"=>"2021-12-22T11:21:56.701+00:00",
    "error"=>
     {"traceback"=>
       #<Sequel::SQL::Blob:0x34738 bytes=139 start="  File \"/o" end="form_task\n"> +
       #<Sequel::SQL::Blob:0x3474c bytes=35 start="    result" end="**kwargs)\n"> +
       #<Sequel::SQL::Blob:0x34760 bytes=137 start="  File \"/o" end="nchronize\n"> +
       #<Sequel::SQL::Blob:0x34774 bytes=75 start="    DebDec" end=".create()\n"> +
       #<Sequel::SQL::Blob:0x34788 bytes=142 start="  File \"/o" end="in create\n"> +
       #<Sequel::SQL::Blob:0x3479c bytes=38 start="    loop.r" end="pipeline)\n"> +
       #<Sequel::SQL::Blob:0x347b0 bytes=110 start="  File \"/o" end="_complete\n"> +
       #<Sequel::SQL::Blob:0x347c4 bytes=27 start="    return" end=".result()\n"> +
       #<Sequel::SQL::Blob:0x347d8 bytes=135 start="  File \"/o" end="_pipeline\n"> +
       #<Sequel::SQL::Blob:0x347ec bytes=35 start="    await " end="*futures)\n"> +
       #<Sequel::SQL::Blob:0x34800 bytes=127 start="  File \"/o" end=" __call__\n"> +
       #<Sequel::SQL::Blob:0x34814 bytes=21 start="    await " end="elf.run()\n"> +
       #<Sequel::SQL::Blob:0x34828 bytes=129 start="  File \"/o" end="1, in run\n"> +
       #<Sequel::SQL::Blob:0x3483c bytes=26 start="    await " end="o.gather(\n"> +
       #<Sequel::SQL::Blob:0x34850 bytes=146 start="  File \"/o" end="tribution\n"> +
       #<Sequel::SQL::Blob:0x34864 bytes=37 start="    await " end="ub_tasks)\n"> +
       #<Sequel::SQL::Blob:0x34878 bytes=143 start="  File \"/o" end="component\n"> +
       #<Sequel::SQL::Blob:0x3488c bytes=41 start="    await " end="ng_tasks)\n"> +
       #<Sequel::SQL::Blob:0x348a0 bytes=147 start="  File \"/o" end="age_index\n"> +
       #<Sequel::SQL::Blob:0x348b4 bytes=46 start="    serial" end="ion=True)\n"> +
       #<Sequel::SQL::Blob:0x348c8 bytes=128 start="  File \"/o" end=" is_valid\n"> +
       #<Sequel::SQL::Blob:0x348dc bytes=39 start="    raise " end="f.errors)\n">,
      "description"=>
       "{'section': [ErrorDetail(string='This field may not be blank.', code='blank')], 'priority': [ErrorDetail(string='This field may not be blank.', code='blank')]}"},
    "worker"=>"/pulp/api/v3/workers/1c2e6cb9-5268-4846-a7d9-ae76930ce01a/",
    "child_tasks"=>[],
    "progress_reports"=>
     [{"message"=>"Downloading Artifacts",
       "code"=>"sync.downloading.artifacts",
       "state"=>"canceled",
       "done"=>3},
      {"message"=>"Update ReleaseFile units",
       "code"=>"update.release_file",
       "state"=>"canceled",
       "done"=>1},
      {"message"=>"Update PackageIndex units",
       "code"=>"update.packageindex",
       "state"=>"canceled",
       "done"=>1},
      {"message"=>"Associating Content",
       "code"=>"associating.content",
       "state"=>"canceled",
       "done"=>4}],
    "created_resources"=>[],
    "reserved_resources_record"=>
     ["/pulp/api/v3/repositories/deb/apt/e57d7614-7eaf-4f36-aca7-e38687185bb6/",
      "shared:/pulp/api/v3/remotes/deb/apt/39803006-1476-4e6b-8f2c-71f62c6321e5/"]}],
 "create_version"=>true,
 "task_groups"=>[],
 "poll_attempts"=>{"total"=>17, "failed"=>1}}

One more note, I have included other Debian repos and there I have no problem with the sync.

@jeschero Thanks for checking and providing more logs. We will try to reproduce the error once again.

Hey @jeschero
My colleague @m-bucher had a look at this again with the latest version of pulp_deb.
It turns out the hashicorp repo is rather special as the PackageIndex contains two fields that are defined but empty. As of right now, we assume that this edgecase requires a change in pulp_deb.

Pulp Issue Tracker #9658

This has been fixed on master (but not yet released), see: Issue #8770: Package validation fails if package paragraphs contain keys without values - Debian Support - Pulp

Thanks for the information @maximilian and @quba42.

@quba42, can you tell me when it will be released?

This is actually a somewhat complicated question.
Since there was a new pulpcore release, a regular pulp_deb release is overdue, but that won’t help you, since you really want to know when the fix will be available in Katello.

We don’t currently have any set process for backporting pulp_deb fixes, but this fix is probably a good candidate for backporting. Here is what I will do: I will discuss backporting this fix in our next Pulpcore/katello/pulp_deb integration meeting - HackMD. This is currently scheduled for the 4th of January. I will report back after that.

thank you.

Until the time the fix is available have you temporary fix which i can use?

You can try manually applying the fix from the upstream Pulp issue: Drop package paragraph fields without values · pulp/pulp_deb@67cb51a · GitHub
However, I have not tested this, and I do not know if there are any conflicts with the pulp_deb version you are using.

Thanks for this information, i tested it but don’t work for me.

I modifier the function from822 to this:

org

    @classmethod
    def from822(cls, data, **kwargs):
        """
        Translate deb822.Package to a dictionary for class instatiation.
        """
        return cls(
            data={k: data[v] for k, v in cls.TRANSLATION_DICT.items() if v in data}, **kwargs
        )

new

    @classmethod
    def from822(cls, data, **kwargs):
        """
        Translate deb822.Package to a dictionary for class instatiation.
        """
        skip = ["Filename", "MD5sum", "Size", "SHA1", "SHA256", "SHA512"]
        package_fields = {}
        custom_fields = {}
        for k, v in data.items():
            #if k in cls.TRANSLATION_DICT_INV:
            #    key = cls.TRANSLATION_DICT_INV[k]
            #    package_fields[key] = v
            #elif k not in skip:
                # also save the fields not in TRANSLATION_DICT
                custom_fields[k] = v

        unique_package_name = "{}_{}_{}".format(
            package_fields["package"],
            package_fields["version"],
            package_fields["architecture"],
        )

        # Drop keys with empty values
        empty_fields = [k for k, v in package_fields.items() if not v]
        for key in empty_fields:
            message = _('Dropping empty "{}" field from "{}" package!').format(
                key, unique_package_name
            )
            log.warning(message)
            del package_fields[key]

        # Delete package fields with values of incorrect type
        if "installed_size" in package_fields:
            try:
                int(package_fields["installed_size"])
            except (TypeError, ValueError):
                log.warn(
                    _(
                        "Dropping 'Installed-Size' field from '{}', "
                        "since the value '{}' is of incorrect type."
                    ).format(unique_package_name, package_fields["installed_size"])
                )
                del package_fields["installed_size"]
        message = _(
            "Dropping '{}' field from package '{}', "
            "since the value '{}' is not in the allowed values list '{}'"
        )
        bool_values = [value[1] for value in BOOL_CHOICES]
        if "essential" in package_fields and package_fields["essential"] not in bool_values:
            log.warn(
                message.format(
                    "Essential", unique_package_name, package_fields["essential"], bool_values
                )
            )
            del package_fields["essential"]
        if (
            "build_essential" in package_fields
            and package_fields["build_essential"] not in bool_values
        ):
            log.warn(
                message.format(
                    "Build-Essential",
                    unique_package_name,
                    package_fields["build_essential"],
                    bool_values,
                )
            )
            del package_fields["build_essential"]
        if "multi_arch" in package_fields:
            allowed_values = [value[1] for value in BasePackage.MULTIARCH_CHOICES]
            if package_fields["multi_arch"] not in allowed_values:
                log.warn(
                    message.format(
                        "Multi-Arch",
                        unique_package_name,
                        package_fields["multi_arch"],
                        allowed_values,
                    )
                )
                del package_fields["multi_arch"]

        package_fields["custom_fields"] = custom_fields
        return cls(data=package_fields, **kwargs)

I need to modifier the part in for k, v in data.items(): otherwise i get the error Katello::Errors::Pulp3Error: type object 'Package822Serializer' has no attribute 'TRANSLATION_DICT_INV'.

Now i have an other the problem. The repo sync finished, but i can’t see any package in the repo overview. In the sync process, i can see the packages.

@jeschero Ok, the fix you want is now released as part of pulp_deb 2.17.0.

In addition, I have backported it to the pulp_deb 2.16.1 release as well.
I have opened a PR to have the 2.16.1 version packaged as an RPM package here.
Once this PR is merged, and the pipeline is run, pulp_deb 2.16.1 will be available for Katello 4.3 installations. Katello 4.3 is currently on RC4 and expected for a full release any day now.

This is as far as I am willing to backport the fix. This should provide you with an upgrade path to an official release containing the fix within a couple of days. I hope that helps.

@quba42 Thank you for your update. After the release of Katello 4.3, i will test it and we can close this issue.

Note that while my PR for pulpcore-packaging is now merged, there is not yet a built tfm-pulpcore-python3-pulp-deb-2.16.1-1.el7.noarch.rpm package available in the corresponding RPM repos. Make sure you it is available before you upgrade. If it still isn’t available in a couple of days, ping somebody so they can investigate.

Update: The pulp_deb 2.16.1 RPM package is now available in the repo used for Katello 4.3.

Thanks for this update.

Katello 4.3 is not yet released (Katello 4.3.0 RC release process), so i need to wait.

@jeschero Took longer than expected, but: Katello 4.3 GA is available

@quba42 Thanks for this information.

I updated my system to Foreman 3.1.1 and Katello 4.3 and get some new errors:

Exception:

PulpDebClient::ApiError: Error message: the server returns an error
HTTP status code: 400
Response headers: {"date"=>"Sat, 29 Jan 2022 15:17:09 GMT", "server"=>"gunicorn", "content-type"=>"application/json", "vary"=>"Accept,Cookie", "allow"=>"GET, PUT, PATCH, DELETE, HEAD, OPTIONS", "x-frame-options"=>"DENY", "content-length"=>"42", "x-content-type-options"=>"nosniff", "referrer-policy"=>"same-origin", "correlation-id"=>"fbf63995-229c-4461-9429-bce3b4687fd0", "access-control-expose-headers"=>"Correlation-ID", "via"=>"1.1 <foreman-hostname>", "connection"=>"close"}
Response body: {"policy":["This field may not be null."]}

and

Exception:

Katello::Errors::Pulp3Error: A file located at the url https://apt.releases.hashicorp.com/pool/amd64/main/consul-enterprise_1.11.0+ent_amd64.deb failed validation due to checksum.

I found the solution for this error. The value Download Policy was not automatic set after the upgrade.
I change it to Immediate and now it works.

Thanks for your help

1 Like