1. Forum
  2. Usenet
  3. LINUX.DEBIAN.DEVEL.QA

  • Re: Appstream data not in UTF-8?

    From Raphael Hertzog@21:1/5 to Raphael Hertzog on Thu Dec 7 11:10:01 2023
    On Thu, 07 Dec 2023, Raphael Hertzog wrote:
    tracker.debian.org has been failing to import the appstream metadata for a while (since November 26th) with this exception:

    Quick correction. The first time it failed that way was on November 19th
    at 08:13 UTC.

    Cheers,
    --
    ⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hertzog@debian.org>
    ⣾⠁⢠⠒⠀⣿⡁
    ⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/
    ⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Raphael Hertzog@21:1/5 to All on Thu Dec 7 11:10:01 2023
    Hello Matthias,

    tracker.debian.org has been failing to import the appstream metadata for a while (since November 26th) with this exception:

    Task UpdateAppStreamStatsTask failed with the following traceback.

    Traceback (most recent call last):
    File "/srv/tracker.debian.org/distro-tracker/distro_tracker/core/tasks/base.py", line 378, in run_task
    task.execute()
    File "/srv/tracker.debian.org/distro-tracker/distro_tracker/core/tasks/base.py", line 256, in execute
    call_methods_with_prefix(self, 'execute_')
    File "/srv/tracker.debian.org/distro-tracker/distro_tracker/core/utils/misc.py", line 44, in
    call_methods_with_prefix
    method(*args, **kwargs)
    File "/srv/tracker.debian.org/distro-tracker/distro_tracker/vendor/debian/tracker_tasks.py", line 889,
    in execute_main
    self._load_appstream_hint_stats(component, arch, all_stats)
    File "/srv/tracker.debian.org/distro-tracker/distro_tracker/vendor/debian/tracker_tasks.py", line 728,
    in _load_appstream_hint_stats
    hints_json = get_resource_text(url, force_update=self.force_update)
    File "/srv/tracker.debian.org/distro-tracker/distro_tracker/core/utils/http.py", line 356, in
    get_resource_text
    return content.decode(encoding)

    Exception Type: UnicodeDecodeError
    Exception Value: 'utf-8' codec can't decode byte 0xcc in position 990416: invalid continuation byte
    Request data not supplied


    Can you look into this and fix the data? (the precise position of the
    error in the stream varied over the days, here you have the position as of today, December 7th 09:36 UTC)

    This refers to those two lines:
    url = 'https://appstream.debian.org/hints/sid/{}/Hints-{}.json.gz' \
    .format(section, arch)
    hints_json = get_resource_text(url, force_update=self.force_update)

    arch is "amd64", an section is likely "main" but it's not impossible that
    it's in one of the other sections.

    Thank you in advance for your help!
    --
    ⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <hertzog@debian.org>
    ⣾⠁⢠⠒⠀⣿⡁
    ⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/
    ⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Andreas Tille@21:1/5 to All on Thu Dec 7 11:30:01 2023
    Am Thu, Dec 07, 2023 at 11:03:29AM +0100 schrieb Raphael Hertzog:
    On Thu, 07 Dec 2023, Raphael Hertzog wrote:
    tracker.debian.org has been failing to import the appstream metadata for a while (since November 26th) with this exception:

    Quick correction. The first time it failed that way was on November 19th
    at 08:13 UTC.

    I do not remember the time exactly but my gut feeling says this is pretty
    close to the bookworm upgrade.

    Kind regards
    Andreas.

    --
    http://fam-tille.de

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Matthias Klumpp@21:1/5 to All on Sat Dec 9 02:50:01 2023
    Hi!

    Thanks for letting me know about the issue! Apparently some database
    entry got corrupted, and concerningly I do not know why... I updated
    the software though and refreshed the entry, and also added some UTF-8 sanitization for new entries, so hopefully this will stay fixed.
    The issue should be gone already :-)

    Best,
    Matthias

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)