Hi,

Here's a major patchset to the recently committed pypi.eclass (i.e.
the "change while it's still warm and not used much" kind).

It focuses on two significant changes:

1. Normalizing filenames according to the current sdist/wheel standards.

2. Translating Gentoo-specific components in ${PV} to the corresponding
standard Python versions.

The former change is focused on improving correctness (i.e. sometimes
the current API didn't allow for getting the correct URI), the latter
on convenience (i.e. not having to override ${PV} sometimes).

For example, a package called "Test-Package-1.2.3_beta4" that followed
PEP 625 (e.g. by using Hatchling) would require an ebuild calling
the equivalent of:

$(pypi_sdist_url "test_package" "${PV/_beta/b}")

while now it will work out of the box.

Unfortunately, PEP 625 is not yet universally followed, so many packages
will instead need to:

$(pypi_sdist_url --no-normalize)

but I think following the standard immediately and requiring a number
of `--no-normalize` calls is better than doing it the other way around
and then switching the defaults, even if the majority of affected
packages use "legacy" naming right now.

That said, this only affects packages with uppercase letters, "-" or "."
in their names.

More details in eclassdoc.

The patchset also adds tests for the eclass.

Relevant PR (including fixes to existing consumers): https://github.com/gentoo/gentoo/pull/29526

Fortunately, breakage is easy to detect by remanifesting ebuilds.

--
Best regards,
Michał Górny


Michał Górny (7):
pypi.eclass: Use pypi_sdist_url to generate the default SRC_URI
pypi.eclass: Add a name normalization function
pypi.eclass: Normalize wheel filenames
pypi.eclass: Add `--unpack` to usage error in pypi_wheel_url
pypi.eclass: Normalize sdist filenames by default
pypi.eclass: Add a version translation function
pypi.eclass: Translate ${PV} by default

eclass/pypi.eclass | 94 ++++++++++++++++++++++++++++++++++++--------
eclass/tests/pypi.sh | 93 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 171 insertions(+), 16 deletions(-)
create mode 100755 eclass/tests/pypi.sh

--
2.39.1

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
eclass/pypi.eclass | 19 +++++++++++++++++++
eclass/tests/pypi.sh | 7 +++++++
2 files changed, 26 insertions(+)

diff --git a/eclass/pypi.eclass b/eclass/pypi.eclass
index dd24b8337e62..27bd17403100 100644
--- a/eclass/pypi.eclass
+++ b/eclass/pypi.eclass
@@ -54,6 +54,25 @@ pypi_normalize_name() {
echo "${name,,}"
}

+# @FUNCTION: pypi_translate_version
+# @USAGE: <version>
+# @DESCRIPTION:
+# Translate the specified Gentoo version into the usual Python
+# counterpart. Assumes PEP 440 versions.
+#
+# Note that we do not have clear counterparts for the epoch segment,
+# nor for development release segment.
+pypi_translate_version() {
+ [[ ${#} -ne 1 ]] && die "Usage: ${FUNCNAME} <version>"
+
+ local version=${1}
+ version=${version/_alpha/a}
+ version=${version/_beta/b}
+ version=${version/_rc/rc}
+ version=${version/_p/.post}
+ echo "${version}"
+}
+
# @FUNCTION: pypi_sdist_url
# @USAGE: [--no-normalize] [<project> [<version> [<suffix>]]]
# @DESCRIPTION:
diff

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
eclass/pypi.eclass | 5 +++--
eclass/tests/pypi.sh | 33 +++++++++++++++++++++++++++++++++
2 files changed, 36 insertions(+), 2 deletions(-)

diff --git a/eclass/pypi.eclass b/eclass/pypi.eclass
index 3a37214f8977..c7b9604807eb 100644
--- a/eclass/pypi.eclass
+++ b/eclass/pypi.eclass
@@ -82,7 +82,8 @@ pypi_sdist_url() {
# @DESCRIPTION:
# Output the wheel filename for the specified project/version tuple.
#
-# If <package> is unspecified, it defaults to ${PN}.
+# If <package> is unspecified, it defaults to ${PN}. The package name
+# is normalized according to the wheel specification.
#
# If <version> is unspecified, it defaults to ${PV}.
#
@@ -97,7 +98,7 @@ pypi_wheel_name() {
die "Usage: ${FUNCNAME} <project> [<version> [<python-tag> [<abi-platform-tag>]]]"
fi

- local project=${1-"${PN}"}
+ local project=$(pypi_normalize_name "${1-"${PN}"}")
local version=${2-"${PV}"}
local pytag=${3-py3}
local abitag=${4-none-any}
diff --git a/eclass/tests/pypi.sh b/eclas

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
eclass/pypi.eclass | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/eclass/pypi.eclass b/eclass/pypi.eclass
index e9d3eec1268b..d00b1171fd16 100644
--- a/eclass/pypi.eclass
+++ b/eclass/pypi.eclass
@@ -34,10 +34,6 @@ esac
if [[ ! ${_PYPI_ECLASS} ]]; then
_PYPI_ECLASS=1

-SRC_URI="
- https://files.pythonhosted.org/packages/source/${PN::1}/${PN}/${P}.tar.gz
-"
-
# @FUNCTION: pypi_sdist_url
# @USAGE: [<project> [<version> [<suffix>]]]
# @DESCRIPTION:
@@ -132,4 +128,6 @@ pypi_wheel_url() {
fi
}

+SRC_URI="$(pypi_sdist_url)"
+
fi
--
2.39.1

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
eclass/pypi.eclass | 21 +++++++++++++++------
eclass/tests/pypi.sh | 24 ++++++++++++------------
2 files changed, 27 insertions(+), 18 deletions(-)

diff --git a/eclass/pypi.eclass b/eclass/pypi.eclass
index 27bd17403100..be0e498fcbf3 100644
--- a/eclass/pypi.eclass
+++ b/eclass/pypi.eclass
@@ -87,7 +87,10 @@ pypi_translate_version() {
# is normalized according to the specification unless `--no-normalize`
# is passed.
#
-# If <version> is unspecified, it defaults to ${PV}.
+# If <version> is unspecified, it defaults to ${PV} translated
+# via pypi_translate_version. If it is specified, then it is used
+# verbatim (the function can be called explicitly to translate custom
+# version number).
#
# If <format> is unspecified, it defaults to ".tar.gz". Another valid
# value is ".zip" (please remember to add a BDEPEND on app-arch/unzip).
@@ -103,7 +106,7 @@ pypi_sdist_url() {
fi

local project=${1-"${PN}"}
- local version=${2-"${PV}"}
+ local version=${2-"$(pypi_transl

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
eclass/pypi.eclass | 23 +++++++++++++++++++----
eclass/tests/pypi.sh | 27 ++++++++++++++++++++++++---
2 files changed, 43 insertions(+), 7 deletions(-)

diff --git a/eclass/pypi.eclass b/eclass/pypi.eclass
index a3c38aa5f3ec..dd24b8337e62 100644
--- a/eclass/pypi.eclass
+++ b/eclass/pypi.eclass
@@ -55,26 +55,41 @@ pypi_normalize_name() {
}

# @FUNCTION: pypi_sdist_url
-# @USAGE: [<project> [<version> [<suffix>]]]
+# @USAGE: [--no-normalize] [<project> [<version> [<suffix>]]]
# @DESCRIPTION:
# Output the URL to PyPI sdist for specified project/version tuple.
#
-# If <package> is unspecified, it defaults to ${PN}.
+# The `--no-normalize` option disables project name normalization
+# for sdist filename. This may be necessary when dealing with distfiles
+# generated using build systems that did not follow PEP 625
+# (i.e. the sdist name contains uppercase letters, hyphens or dots).
+#
+# If <package> is unspecified, it defaults to ${PN}. The package name
+# is normalized a

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
eclass/pypi.eclass | 20 ++++++++++++++++++++
eclass/tests/pypi.sh | 32 ++++++++++++++++++++++++++++++++
2 files changed, 52 insertions(+)
create mode 100755 eclass/tests/pypi.sh

diff --git a/eclass/pypi.eclass b/eclass/pypi.eclass
index d00b1171fd16..3a37214f8977 100644
--- a/eclass/pypi.eclass
+++ b/eclass/pypi.eclass
@@ -34,6 +34,26 @@ esac
if [[ ! ${_PYPI_ECLASS} ]]; then
_PYPI_ECLASS=1

+# @FUNCTION: pypi_normalize_name
+# @USAGE: <name>
+# @DESCRIPTION:
+# Normalize the project name according to sdist/wheel normalization
+# rules. That is, convert to lowercase and replace runs of [._-]
+# with a single underscore.
+#
+# Based on the spec, as of 2023-02-10:
+# https://packaging.python.org/en/latest/specifications/#package-distribution-file-formats
+pypi_normalize_name() {
+ [[ ${#} -ne 1 ]] && die "Usage: ${FUNCNAME} <name>"
+
+ local name=${1}
+ local shopt_save=$(shopt -p extglob)
+ shopt -s extglob
+ name=${name//+([._-])/_}
+ ${shopt_save}
+ echo "${name,,}

On Sat, 2023-02-11 at 10:16 +0100, Michał Górny wrote:

Hi,

Here's a major patchset to the recently committed pypi.eclass (i.e.
the "change while it's still warm and not used much" kind).

It focuses on two significant changes:

1. Normalizing filenames according to the current sdist/wheel standards.

2. Translating Gentoo-specific components in ${PV} to the corresponding
standard Python versions.

The former change is focused on improving correctness (i.e. sometimes
the current API didn't allow for getting the correct URI), the latter
on convenience (i.e. not having to override ${PV} sometimes).

For example, a package called "Test-Package-1.2.3_beta4" that followed
PEP 625 (e.g. by using Hatchling) would require an ebuild calling
the equivalent of:

$(pypi_sdist_url "test_package" "${PV/_beta/b}")

while now it will work out of the box.

Unfortunately, PEP 625 is not yet universally followed, so many packages
will instead need to:

$(pypi_sdist_url --no-normalize)

but I think following the standard immediately and requiring a number
of `--no-normalize` calls is better than doing it the other way around
and then switching the defaults, even if the majority of affected
packages use "legacy" naming right now.

That said, this only affects packages with uppercase letters, "-" or "."
in their names.

A minute after sending this I've realized that we need to apply the same operations to S as well. In reply, I'm adding two more patches that respectively:

1) perform normalization and translation in S by default (i.e. make it
match the default SRC_URI),

2) add a more convenient PYPI_NO_NORMALIZE variable that can be used to
control whether SRC_URI and S use normalized or legacy names.

--
Best regards,
Michał Górny

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Signed-off-by: Michał Górny <mgorny@gentoo.org>
---
eclass/pypi.eclass | 19 ++++++++++++++-----
eclass/tests/pypi.sh | 3 +++
2 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/eclass/pypi.eclass b/eclass/pypi.eclass
index be0e498fcbf3..f4367b3fbfec 100644
--- a/eclass/pypi.eclass
+++ b/eclass/pypi.eclass
@@ -11,12 +11,20 @@
# @DESCRIPTION:
# The pypi.eclass can be used to easily obtain URLs for artifacts
# uploaded to PyPI.org. When inherited, the eclass defaults SRC_URI
-# to fetch ${P}.tar.gz sdist.
+# and S to fetch .tar.gz sdist. The project filename is normalized
+# by default, and the version is translated using
+# pypi_translate_version.
#
-# If necessary, SRC_URI can be overriden by the ebuild. Two helper
-# functions, pypi_sdist_url and pypi_wheel_url are provided to generate
-# URLs to artifacts of specified type, with customizable project name.
-# Additionally, pypi_wheel_name can be used to generate wheel filename.
+# If necessary, SRC_URI and S can be overriden by the ebuild. Two
+# helper func