I fixed the elpi package by using something a bit hackish: I added git
as dep, and if I don't see a .git in the build directory, I create one!
Here is what the execute_before_dh_auto_build target does:
if test -f .git; then \
@echo "Found .git, ok"; \
else \
touch .false_git; \
git init --initial-branch=main; \
git config user.email "foo@bar.baz"; \
git config user.name "Foo Bar"; \
git add dune; \
git commit -m foo; \
git tag -a v$(DEB_VERSION_UPSTREAM) -m foo; \
fi
and here is what the execute_after_dh_auto_clean does:
if test -f .false_git; then \
rm -rf .false_git .git; \
fi
This is ugliness to paper over ugliness. Please do not!
My approach to this specific problem would be to add to dune the possibility to use some configuration file (or environment variables) as input for the substitutions, instead of directly querying the VCS. This configuration
could then be set up as part of the Debian packaging.
Usually, projects using this trick publish tarballs with
substitutions
applied:
https://github.com/LPCIC/elpi/releases/tag/v2.0.7
(here, elpi-2.0.7.tbz).
I realize my previous email was a bit short: I was wondering if this
.tbz still source code because in the autotools world, package sources
come with configure scripts ready to run, but the good practice in
Debian is to regenerate those from configure.ac.
I fixed the elpi package by using something a bit hackish: I added git
as dep, and if I don't see a .git in the build directory, I create one!
Here is what the execute_before_dh_auto_build target does:
if test -f .git; then \
@echo "Found .git, ok"; \
else \
touch .false_git; \
git init --initial-branch=main; \
git config user.email "foo@bar.baz"; \
git config user.name "Foo Bar"; \
git add dune; \
git commit -m foo; \
git tag -a v$(DEB_VERSION_UPSTREAM) -m foo; \
fi
and here is what the execute_after_dh_auto_clean does:
if test -f .false_git; then \
rm -rf .false_git .git; \
fi
I suppose dh's ocaml_dune building tool could do that trick itself
(using debian/fake_dot_git instead of .false_git, better user.email/user.name...), and that would make sure we don't break.
What do you think about the topic?
I realize my previous email was a bit short: I was wondering if this
.tbz still source code because in the autotools world, package sources
come with configure scripts ready to run, but the good practice in
Debian is to regenerate those from configure.ac.
Well, we enter a philosophical debate that is not specific to OCaml and probably should be discussed elsewhere... Adding debian-devel to get
more opinions.
Summary to other debian-devel readers: we are facing some upstreams that publish "official" tarballs that differ from what is in their git. The differences may include: variable substitutions, generated files... I
guess this is pretty common (cf. autotools). Moreover, the build system behaves differently if it is called from git or not, or from extracted official tarballs or not.
IMHO, traditionnaly, "source code" from Debian point of view is whatever upstream releases as "official" tarballs (i.e. elpi-2.0.7.tbz), which
may differ from what is in upstream git (i.e. v2.0.7.tar.gz). What makes
me think that is the special care that is taken in keeping upstream
tarballs pristine (with their signatures...).
[...]
What do you think about the topic?
My e-mail is very opinionated, I would really like to hear other opinions.
It's from the old mindset of code signing being the only way ofsecurely getting code from upstream. Recent events have shown (instead
FYI: If all upstream wants is git metadata I like to introduce them to the wonderful, but obscure, git `export-subst` feature. See git-attributes(1).
Works with forges, git-archive and everything.
Example: https://github.com/YosysHQ/yosys/commit/222e7a2da345f01980d9261c40c5d50eced4f9ab
thoug this was later improved by others https://github.com/YosysHQ/yosys/commit/9d15f1d6ac4a9ff2e1f87cda8c366659027fb76f
If that's not enough can you point us to what this upstream is doing exactly?
Diff between those two:
https://whatsrc.org/diff-right-trimmed/sha256:146d2d673358b7927d9a3c74e22b6b0e7f9a1aee2a4307afbe6ac07f12764130/sha256:599ff98cbab933a8b3640a084b12a5308a20795c192855ee454a8c1c16fa4dac
My approach to this specific problem would be to add to dune the
possibility to use some configuration file (or environment variables)
as input for the substitutions, instead of directly querying the VCS.
This configuration could then be set up as part of the Debian
packaging.
On the other hand, insisting on using upstream VCS contents can lead to ugly hacks in Debian packaging, such as what you are describing. I must admit I usually use "official" tarballs to avoid these hacks (and maybe a little out of laziness).In my own packages I am happy to add as many ugly hacks are needed to be
Summary to other debian-devel readers: we are facing some upstreams that publish "official" tarballs that differ from what is in their git. The differences may include: variable substitutions, generated files... I guess this is pretty common (cf. autotools). Moreover, the build system behaves differently if it is called from git or not, or from extracted official tarballs or not.
IMHO, traditionnaly, "source code" from Debian point of view is whatever upstream releases as "official" tarballs (i.e. elpi-2.0.7.tbz), which may differ from what is in upstream git (i.e. v2.0.7.tar.gz). What makes me think that is the special care that is taken in keeping upstream tarballs pristine (with their signatures...).
Some may consider that this Debian notion of "source code" differs from the GNU "preferred form of modification", which would rather be what is in upstream git... or is it? In Debian, we "modify" upstreams by applying patches
on top of them, so I argue that using "official" tarballs is fine as long as patches used in Debian packaging apply on upstream git as well.
Anyway, I do think the "GNU" source should be recoverable from the "Debian" source. Technically, this is usually not the case with variable substitutions,
but IMHO it's acceptable to use the substituted sources most of the time.
On the other hand, insisting on using upstream VCS contents can lead to ugly hacks in Debian packaging, such as what you are describing. I must admit I usually use "official" tarballs to avoid these hacks (and maybe a little out of laziness).
I think that basing our work on upstream Git makes our source packages
more useful, and more accurately reflects our commitment to providing
the preferred form of modification for everything in our archive.
If our work is based on upstream Git then users can clone source
packages from salsa (or, better, 'dgit clone' if the maintainer has used 'dgit push-source') and can use powerful tools like 'git blame' and 'git bisect' to understand their bug.
With tarballs the granularity of these tools is so much less.
If you use autotools, you start with configure.ac and .in files. If
upstream prepared the tree (as they generally do), you also get a libtool/configure, etc. From this, you can run ./configure && make &&
make install ; that will do substitutions in the .in files. And if you
want to start anew, you can use autoconf/autoheader/whatever to re-
create the configure/libtool script and then ./configure && make &&
make install.
I have always considered the true way to know if you have the source of
a package is: imagine you're stuck somewhere with a Debian mirror and
no external link. Could you take over the development of the package at
hand?
The potential for additional function is not relevant.
If the upstream intends to distribute it with a tarball, that's the "golden" package that downstream should base code upon.
- the previous version used %%VERSION_NUM%% in only three places, the
new one uses it more, so it broke my previous hack -- ;
- there are other things than the substitutions done by dune when
compiling the package, which do not break the build, but will break
some depending packages later on with strange and misleading errors.
As mentioned somewhere in the thread I proposed to dune upstream a
simple mechanism to bypass this git reliance issue, which will make
packaging much cleaner.
On Sun 16 Feb 2025 at 06:18am -06, rhys wrote:
The potential for additional function is not relevant.
If the upstream intends to distribute it with a tarball, that's
the "golden" package that downstream should base code upon.
The Debian project officially disagrees with you.
The preferred form for modification, which is what NEW cares about, is determined by upstream's actual practices, not by their fiat.
We frequently reject packages from NEW because we have minified files;
we add the source to debian/missing-sources/.
"Actual practices" and "by fiat" are no different here.
They do as they like with their code.
Debian has, for legal and logistical reasons, decided that it cannot distribute upstream Git repositories as its source packages, and
instead chooses to try to condense upstream's preferred form for
modification (back) into source tarballs.
Their .tbz is working nicely -- but it isn't source anymore!
Yes. Most upstream have little clue on what a best practice is, and
need to be explained at length.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 493 |
| Nodes: | 16 (0 / 16) |
| Uptime: | 167:28:59 |
| Calls: | 9,703 |
| Calls today: | 3 |
| Files: | 13,733 |
| Messages: | 6,178,064 |