Good evening,

After some more heavy surgery our FrankenKotlin is not doing too well
:/.

Le 21/02/2025 à 18:15, Julien Plissonneau Duquène a écrit :

backport the whole scripting plugin from Kotlin 1.3.50 into our
FrankenKotlin and see how it goes

That pulled in more things than I expected but was still marginally
manageable, though I haven't (yet?) fixed a few things that broke in the process, including the REPL console and the ability for that compiler to rebuild itself. Some minor progresses were made and gradle's script
compiler is now going a bit further into code generation, but while investigating the current crashes I discovered that it was caused by incorrectly generated code in Kotlin plugins, and that code is generated
by the "regular" Kotlin 1.3.31 (I checked my Kotlin builds as far back
as December and they are almost all affected). I'm suspecting some
interference caused by the JDK 21, so I'm now testing clean rebuilds and
using diffoscope and procyon to find out the extent of damages. This
issue didn't cause compiler rebuilds to fail so far, but the scripting
code is definitely affected.

Picking features and fixes in Kotlin's git history wasn't too difficult,
which is rather good news for its maintainability as a stable package.

Next I will have to fix that compiler and try again to see if I can
start to get some gradle build script class files (correctly) generated
or if it's time to give up on that approach and start packaging Kotlin
2.0.21.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

The FrankenKotlin experiment is now over, and it failed.

Le 2025-02-28 20:20, Julien Plissonneau Duquène a écrit :

incorrectly generated code in Kotlin plugins

This wasn't as bad as it looked like actually: it was caused by the
Kotlin compiler using the default JDK (21) as its target for API
compatibility, so while the bytecode was compatible with Java 8 it used
some API calls that were introduced in later JDKs instead of
reimplementing them as inline code. Using the JDK 11 again solved that
for the purpose of using that compiler for building Gradle.

After fixing a few more little things the next roadblock was a missing
implicit class declaration for the script that caused the
pluginManagement block (a function call actually) to fail to compile
with an internal assertion error:

* Exception is:
java.lang.AssertionError: Undefined parameter referenced: class Settings::this
owner=class Settings_gradle; [class Settings_gradle::this]

at org.jetbrains.kotlin.ir.util.SymbolTable.referenceValueParameter(SymbolTable.kt:420)
at org.jetbrains.kotlin.psi2ir.generators.ArgumentsGenerationUtilsKt$generateReceiver$$inlined$generateDelegatedValue$1$1.invoke(OnceExpressionValue.kt:63)
at org.jetbrains.kotlin.psi2ir.generators.ArgumentsGenerationUtilsKt$generateReceiver$$inlined$generateDelegatedValue$1$1.invoke(OnceExpressionValue.kt:32)
at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)

It seemed like not much would be needed to fix this, but this was
actually a rabbit hole. My cherry-picks started to touch IR code, then
backends code, then plugins code, and I ended up facing walls of build
errors after applying each of them, and fixing these errors required new
picks, which meant going back to new walls of errors on modules I just
barely got to build. After picking 19 commits and adding 9k lines of
patches I estimated that I wasn't even half done fixing that single
issue and decided that it was definitely not worth continuing.

I computed a few stats to illustrate how this Kotlin project was a
moving target (and probably still is) by then. Between v1.3.31
(2019-04-25) and v1.3.70 (2020-03-03), that's an interval of a bit more
than 10 months, there were 1114 commits that affected some source files
under compiler/ir alone (more than 3 commits/calendar day on average),
for 571k lines of git log -p and 204k lines of git diff. Overall there
were 8236 commits for 4.4M lines of git log -p between the same
releases. For comparison over the same period that's a bit more than 10%
of the activity of the Linux kernel tree in number of commits, and about
44% in number of git log -p lines.

This outcome is a bit frustrating but it was the most likely one, and I
learned a few valuable things in the process.

My plan is now to update the Kotlin package to 2.0.21 (renaming it
kotlin2.0) and use it to build Gradle 8. I'm not sure that both package
(and dependencies) will be in a releasable shape in time for Trixie, but
if they miss the deadline they can still be made available later through backports.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Julien,

On 07/03/2025 18:39, Julien Plissonneau Duquène wrote:

Good evening,

The FrankenKotlin experiment is now over, and it failed.

This wasn't as bad as it looked like actually: it was caused by the
Kotlin compiler using the default JDK (21) as its target for API compatibility, so while the bytecode was compatible with Java 8 it used
some API calls that were introduced in later JDKs instead of
reimplementing them as inline code. Using the JDK 11 again solved that
for the purpose of using that compiler for building Gradle.

Did you consider rebuilding the packages with Java 8 since we still have openjdk-8 in sid? We may even clone the gradle package to make a Java 8 compatible version if that helps. Same thing with kotlin.

This outcome is a bit frustrating but it was the most likely one, and I learned a few valuable things in the process.

The simple fact you didn't end in the local hospital after banging your
head on the wall for so long is quite an achievement!

My plan is now to update the Kotlin package to 2.0.21 (renaming it
kotlin2.0) and use it to build Gradle 8. I'm not sure that both package
(and dependencies) will be in a releasable shape in time for Trixie, but
if they miss the deadline they can still be made available later through backports.

Yes it's too late for Trixie.

Emmanuel Bourg

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Emmanuel,

Le 2025-03-10 18:42, Emmanuel Bourg a écrit :

Did you consider rebuilding the packages with Java 8 since we still
have openjdk-8 in sid? We may even clone the gradle package to make a
Java 8 compatible version if that helps. Same thing with kotlin.

That's not needed for the old Gradle, but I'm considering adding the JDK
8 dependency to the old Kotlin just in case (and then using diffoscope
to see if it changes anything). Actually (I have to check that too...) I believe that it's mostly the default JDK that is used for the build
(it's definitely used to run Gradle) and compiling Java, but that the
Kotlin compiler uses whatever it finds in a configured JDK path to
figure out if some methods are implemented in that JRE or if it needs to provide an alternative implementation for kotlin-stdlib API calls. So it
may end up depending on all of default JDK, JDK 11 (to replace the JDK
9) and JDK 8. The upstream build used a bunch of JDK versions as well.

The simple fact you didn't end in the local hospital after banging your
head on the wall for so long is quite an achievement!

I have to admit that I had to take breaks ^ ^.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

Building a functioning gradle using a pre-built Kotlin 2.0.21 is a bit
more work than I expected.

Le 2025-03-07 18:39, Julien Plissonneau Duquène a écrit :

update the Kotlin package to 2.0.21 (renaming it kotlin2.0)

I've started to look around that release and test the upstream build
which worked out of the box. Thinking about it the package name will
rather be 'kotlin2' as new '2.x' releases are expected twice a year and
there is no way we (I?) are going to simultaneously maintain several of
these short-lived branches. Instead I'm planning to keep packaged in
unstable the latest version that is known or tested to be compatible
with Gradle (currently still 2.0.21 in Gradle's master branch), and
eventually later (we'll see when we get there) the latest Kotlin version
in experimental as kotlin2.x co-installable with kotlin2.

Then I got back to Gradle, removed the patches that downgraded its code
to build with the FrankenKotlin, restored the build of the declarative
DSL module, adjusted dependencies versions, and a few more little things
to get it to build again, and in the process discovered that we now need
to package a kotlinx.serialization library, and to find a way to ship an
old release of kotlinx-metadata-jvm alongside the current API that was refactored and renamed with Kotlin 2.

The builds eventually succeeded, but then I was unable to use the
generated binaries to rebuild gradle, as they failed with some
suspicious compilation errors. Debugging these, it turned out that some
classes normally compiled from generated Kotlin sources were missing, as
well as their generated sources, so I now have to investigate that.

in time for Trixie

Well definitely not now, as new packages won't be allowed anymore
starting with the soft freeze week 16, and with the typical NEW review
and migration delays that means they should have been ready since
probably some date in the past. Let's not miss this great opportunity to
quote Douglas Adams: “I love deadlines. I love the whoosing noise they
make as they go by.”

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

Starting next week the updates are going to be about "kotlin 2 in
Debian" for a while.

Le 2025-03-14 19:49, Julien Plissonneau Duquène a écrit :

some classes normally compiled from generated Kotlin sources were
missing

This was caused by the removal of the `qdox` dependency, which was an
error as it is already available and usable in Debian. After restoring
it and adding two Kotlin JARs to the list of downloads it was possible
to rebuild Gradle with itself and a reduced set of pre-built
dependencies.

I then spent some time updating and reviewing the list of dependencies
which is now 189 lines long after I added all those that are dropped
because they are not currently packaged (52 new lines). Most of them are
only needed for code quality checks or automated tests and there are a
few ones that I believe to not be used at all. Only two of them are
making my builds seriously crippled compared to the upstream release:
- the configuration-cache-report plugin aka problems API which is used
deep in the configuration cache code (currently I have to completely
disable the configuration cache for Gradle to work)
- a tomlj library that is necessary to read version catalogs.

The other removed features are far less critical:
- AWS S3 support (resource/object storage)
- GCS support (resource/object storage)
- the entire documentation generation
- a plugin for JVM toolchain management
- wrapper JAR minification
- Scala Zinc compiler support.

Preparing the work on Kotlin 2 I've decided to fix the build of 1.3.31
with the default JDK rather than making it depend on legacy JDKs, the
reason being that these JDKs are not in stable or testing (which I think
is something that should be renegotiated) and that the package has to be (re)buildable with what is (or will be) in stable. So far I managed to
patch the compiler in a rather clean way and most of the differences in generated code are gone but not all. This is rather slow as a full
diffoscope run takes a bit less than 5 hours on my machine. I'm also
planning to fix the current CVE, and got the other closed as it was not applicable to this version.

Expect to see a few new branches including `kotlin2-wip` on my fork of
the package repository fairly soon.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

As goes the saying, a journey of a thousand miles begins with a single
step.

Le 2025-03-21 18:58, Julien Plissonneau Duquène a écrit :

Expect to see a few new branches including `kotlin2-wip` on my fork of
the package repository fairly soon.

I was hoping to be done sooner with this but then some COVID replica hit
me which did no good to my productivity. So far the branch is pushed but
there aren't any WIP docs yet as I'm not done yet writing a first draft. However the Kotlin sources were carefully sieved to reintroduce
/testData/ contents while removing more unwanted objects including
(almost) all pre-built binaries and copies of curl, sqlite, jquery and
yarn. There is still copied code from several other projects but at
least some of it was modified by Kotlin contributors, so I will
eventually get back to these later to find out if they are to be
considered forked or could be factored out of the project.

A not too pleasant surprise was to discover that the upstream project
decided to use LFS to store a few JPEG images used in the documentation
(which is way overkill). Fortunately this is supported by Salsa and
uscan, but it slows down significantly some remote operations. These
images only appeared in 2.1.0 so the 2.0.21 branch should not be
impacted.

I also fixed JDK17+ compatibility issues and the remaining CVE on Kotlin
1.3.31 on a MR which is now pending review (reviewer(s) welcome!) and eventually upload.

Next week I'm now going to start combing through the dependencies of
Kotlin 2.0.21.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Julien,

On 28/03/2025 19:55, Julien Plissonneau Duquène wrote:

I was hoping to be done sooner with this but then some COVID replica hit
me which did no good to my productivity.

I hope you feel better now.

A not too pleasant surprise was to discover that the upstream project
decided to use LFS to store a few JPEG images used in the documentation (which is way overkill). Fortunately this is supported by Salsa and
uscan, but it slows down significantly some remote operations. These
images only appeared in 2.1.0 so the 2.0.21 branch should not be impacted.

I think you can safely skip the documentation stuff and filter out the
JPEG images.

Emmanuel Bourg

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Le 2025-03-29 12:32, Emmanuel Bourg a écrit :

A not too pleasant surprise was to discover that the upstream project
decided to use LFS to store a few JPEG images used in the
documentation (which is way overkill). Fortunately this is supported
by Salsa and uscan, but it slows down significantly some remote
operations. These images only appeared in 2.1.0 so the 2.0.21 branch
should not be impacted.

I think you can safely skip the documentation stuff and filter out the
JPEG images.

I would rather keep the doc if possible, even if we can't build it right
now (I haven't evaluated this yet). I'm considering instead opening an
upstream issue and/or PR with a few janitorial fixes including this one.
I've also opened an issue to try to get them to sign their release tags, currently they aren't even using annotated tags.

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

Not much was achieved on kotlin2 this week after I got distracted by too
many things.

Le 2025-03-28 19:55, Julien Plissonneau Duquène a écrit :

start combing through the dependencies

I'm almost there. So far I've ditched 62 of the 69 patches that were
applied to Kotlin 1.3.31, keeping the remaining 7 aside for now. Most of
the removed patches were to convert the gradle build scripts from Kotlin
DSL to Groovy DSL, an approach that was useful for the initial packaging
of Kotlin but that proved to be a dead end in terms of maintainability.

My current plan is to use Gradle 8.11 for the build, first using the
binary distribution until I get Kotlin to build successfully in offline
mode with only Debian dependencies, and then switch to my own build of
Gradle. Later in order to make the package "bootstrappable" I'm planning
to reuse (and finish) the Makefile generator I've started to write for
Gradle, and then I will try to compile it with older releases of the
compiler to see if the "treasure trail" strategy can be completed to
find a bootstrapping path.

And speaking of old things — last call for a sponsor for a Kotlin 1.3.31 update before I file a formal RFS ;)

Next week-end I'm going to participate to the BSP in Wien, hoping to
meet some of you there, and decrease significantly the count of RC bugs
on Java packages (30 currently) and a few others.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Grüß Gott,

A patch stack of one thousand changes begins with a single file, or
something like that.

I'm starting slowly and carefully with kotlin, trying to reuse some of
the logic I've already written for Gradle 8 as many recipes are going to
be useful there as well. This will probably end up with a lot of
duplicated but slightly different code that I plan to clean up or factor
into the Debian gradle helper later once the build tools are in a
reasonably stable shape.

Now I have to spend some time reviewing and commenting the list of
dependencies and then comparing it to the one that Vladimir Petko
extracted for Kotlin 2.1, while removing the non-critical missing
dependencies from Kotlin build scripts, and eventually starting to drop
some features.

Among the things that will be different from the Kotlin 1.3 package,
the test suites and data sets were imported for this Kotlin 2 package
and I'm planning to make it possible to run at least some of them.

Le 2025-04-04 20:50, Julien Plissonneau Duquène a écrit :

BSP in Wien

Arrived there about 2 hours ago and met a welcoming, colored and
friendly bunch. I'm going to take advantage of the DDs on site to upload
fixes on a few annoying RC bugs, targeting in priority those that
prevent other packages from migrating. I will start with an attempt to
backport a fix for bouncycastle from the unreleased 1.81 branch.

Tschüss,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

There is no progress on kotlin2 to report this week, as other unrelated
things kept me busy even on my way back from the BSP in Wien.

Le 2025-04-11 14:10, Julien Plissonneau Duquène a écrit :

I'm going to take advantage of the DDs on site to upload fixes

Unfortunately that strategy failed there: only one of them volunteered, declined the first two packages for understandable reasons, and then
went back being busy working on his own things. As a result I still have
a small stack of pending fixes that will probably end up as formal RFS
in the following days.

But it was nice meeting folks in person. Interesting discussions
occured, though most of them in a language I didn't practice for too
long, and I have learnt a few things about Wien that most tourist guides
would never have told me.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

Work resumed on kotlin2. At this point I'm still trying to get its build
script to compile in offline mode, while identifying the dependencies
and eventually replacing them or patching them out of the build scripts.

This is sometimes complicated by limitations or maybe bugs in Gradle:
right now I have to figure out why it fails to resolve the kotlin stdlib
when version substitution is applied, and beyond that why it now fails
to resolve (from the root settings.gradle that is still a Groovy script)
a Kotlin class that is confirmed to be present in a plugin that is
supposed to be already loaded.

Fortunately these failures occur early in the build settings evaluation
phase so trial cycles are still short. I though that Kotlin build infrastructure was smaller than that of Gradle but it's actually about 3
times larger, though I think the full project builds a bit faster.

The way the build scripts are structured and the set of used Gradle
features is also slightly different so I have to modify my custom build
plugin that replaces artifact repositories and selectively rewrites dependencies coordinates and then debug it when hitting new issues such
as above.

Meanwhile in California, upstream Gradle 8.14 was released today.

A thank you to werdahias that offered to help on IRC a few hours ago.
Right now I don't have an up-to-date wishlist of packages that could be
worked on (updated, or created) but I will work on that as I progress on kotlin2; for gradle most missing or outdated packages need newer
versions of gradle/kotlin/groovy to be built.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

There was progress on kotlin2, though I'm still removing dependencies on plugins and libraries that are not yet packaged and are not strictly
necessary for the build.

Le 2025-04-25 22:08, Julien Plissonneau Duquène a écrit :

I have to figure out why it fails to resolve

I was stuck on similar issues for a short while, then decided they were
not worth debugging (that is, not for now) so I just worked around them
by patching the build scripts. At some point I suspected that the issues
could be caused by building with Gradle 8.11.1 instead of the nominal
8.8 so I tried a regular build just changing the Gradle version; some
patching was necessary to make that build succeed because of new
deprecations and a renamed API but that didn't reproduce the class
resolution issues. So I still don't know how I introduced them.

I have some hopes to get the configuration phase of that build script to
work offline by next week, as I think I'm more than halfway done on that
front.

On the side quest stack there now only a single remaining package (intellij-community-idea) waiting to be sponsored on mentors.debian.net
if any DD has some availability for that.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Le 2025-05-03 14:53, Pierre Gruet a écrit :

Done! Could you please update the Salsa repo with your changes?

Done, thank you for the upload.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

After hitting a new, more serious case of module resolution failure I
had to engage in several sessions of debugger-driven development, where
I discovered that my logic that replaces remote artifact repositories
with local ones needed to be reworked.

Le 2025-05-02 19:39, Julien Plissonneau Duquène a écrit :

as I think I'm more than halfway done on that front.

I'm not much further right now: the configuration progress bar only
reaches a meager 13%, still better than the 0% of last week. I had to
spend some time fighting IDEA's debugger again (is there any efficient
GUI or TUI java debugger out there that isn't finicky?) that didn't want
to stop at my breakpoints, didn't recognize my custom exception class
and has no known way AFAICT [1] to write a breakpoint condition testing
the raised exception, so I ended up adding patterns to the breakpoint
class filters until I could get where I wanted to and found out that the
issue was caused by the use of exclusiveContent on the bootstrap
repository added by Kotlin build scripts. When this feature is used, a corresponding set of exclude patterns is automatically applied to all
other repositories, including those added later, even after the exclusiveContent repository is removed as this is done through a DomainObjectCollection.all() closure.

I could then test a few ways to get around that. Patching that part out
of the upstream buildscripts was among the possibilities as it's not
necessary for the packaging build, but I wanted to avoid that as that
would mean a dedicated patch to maintain on the long term on a part of
the build script that so far changed several times a year. Unfortunately
Gradle exposes no public API that makes it possible to inspect and
change the filters applied to repositories (you can only add new ones),
and there is no convenient way to reset a repository list or remove an exclusiveContent repository including its effects applied to other repositories. I settled for a call to an internal, package-private
method of a package-private class to clear the exclude filters using reflection. This is not really satisfying in terms of future-proofing
but it does the job for now and I may submit later a Gradle feature
request for this use case.

Hoping again to clear that configuration phase by next week.

Cheers,


[1]: https://youtrack.jetbrains.com/issue/IDEA-206580

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good afternoon,

My MountainCamp attendance disturbed the established Friday evening
routine, but that didn't prevent progress on the kotlin2 front.

Le 2025-05-09 22:12, Julien Plissonneau Duquène a écrit :

the configuration progress bar only reaches a meager 13%

100% were achieved by wednesday, but only on a few tasks like "clean"
and "tasks" that don't actually resolve dependencies for the whole
project: Gradle is designed to use lazy evaluation as much as possible,
so "configurations" (that in gradle speak associate dependencies to
scopes) that are not needed are not resolved. With a few more minor
tweaks it was then possible to successfully sync the project in IDEA,
which enables features that are really helpful when debugging and
modifying build scripts.

Now I'm working on getting the "zipCompiler" task to actually start
building things. So far I spotted another new kotlinx library that will
need to be packaged (kotlinx.collections.immutable), and it looks very
likely that the intellij-community-idea library packages will need to be updated. I'm also planning to eliminate the relocated embedded copies of various libraries (e.g. protobuf, asm, log4j) from the binary kotlin
packages, unlike the way kotlin 1.3.31 was packaged.

Once that next milestone is reached, I will spend some time reviewing
the list of build dependencies and comparing my work with that of
Vladimir Petko for things I could have missed.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

Thirty years ago to this day Java made its first public appearance at
the Sun World Conference in San Francisco. This was right at the peak of
"the network is the computer" era [1], before the Cloud became a thing.
Today Java remains a fairly popular platform, still integrating
innovations, evolving to address new challenges [2], and being trusted
to reliably run mission-critical applications.

Though there is no definitive timeline of when the project that would
become Java was initiated by Sun, it's likely to be a bit before the
Debian Project was announced (1993) and could be as early as 1991, the
year Linus Torvalds started to work on his kernel "hobby" project.

But let's get back to things that I know are expected to build some more
games ;)

Le 2025-05-18 13:41, Julien Plissonneau Duquène a écrit :

getting the "zipCompiler" task to actually start building things

I'm not there yet: I've started to de-vendor some of the dependencies
that would be embedded in an upstream binary release (among them asm,
log4j, fastutil) and then hit a snag as for some subprojects Gradle now
tries to pull in as a transitive dependency a version of kotlin-stdlib
that is not allowed by Kotlin's build script, despite my custom
dependency rewriting logic (that is not even executed in this case) and
the proper presence of `isTransitive = false` in the buildscript; I even
added an `exclude(...)` that was ignored as well. So I'm now
investigating this with a debugger.

Some non-trivial (but not too complicated either) additional patching of
the asm package will be necessary for the compiler to work correctly.

I now have a Mastodon account [3] that you can use to follow me. I'm
probably going to toot there the progress on kotlin and gradle as
milestones are reached. The next one is still the same as last week.

And finally I will submit a Java/JVM BoF proposal to the DebConf25, as I
think we deserve this as a team.

Cheers,


[1]: https://archive.org/details/Schmidt1995
[2]: https://dev.java/community/java-30-anniversary/
[3]: https://chaos.social/@sre4ever

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

This week was mostly spent debugging. Fortunately I could finally figure
out what the issue was with transitive dependencies.

Le 2025-05-23 20:04, Julien Plissonneau Duquène a écrit :

despite my custom dependency rewriting logic (that is not even executed
in this case)

It was in fact "because of", not "despite". Part of that logic works by creating new dependencies and remove existing ones, as in some places
the gradle documented API doesn't offer a way to replace group and
artifact names. There were several defects in that logic that are now
fixed:
- I use null as a return value that means "do not rewrite replace the dependency" (because it's actually the same, or some other issue
happened). But I also implemented a cache that doesn't allow the storage
of null values, and at some point forgot to implement the null value
thing when the value was obtained from the cache. As a result, the
dependency would still be replaced, but only starting from the second
call to the function. This made finding the root cause a bit more
challenging ^ ^
- the replacement dependency did not retain all the properties of the
initial target, and among these transitivity and exclusion rules. They
were not needed in gradle's build, and DependencyHandler.create()
returns an object of the Dependency type, which is actually an interface
that doesn't expose these properties, so I didn't go further at that
time. This is what caused the issue: the dependency was properly
declared as being non-transitive in build scripts, but the custom plugin replaced it with the same dependency, without setting the replacement as non-transitive.
- while I was at it I investigated a previous issue I had with equals()
between Dependency objects, and found that it's probably a bug in
Gradle's implementation.

I also discovered that there is an interesting `resolveDependencies`
task implemented in Kotlin build scripts. It has some issues, but it
will help with mapping the full dependency set.

After shedding some more Android dependencies the build now fails a bit
closer to the point where Gradle would start actually building the
project, but it's still not there yet.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

The milestone that was starting the execution phase of the build script
was finally reached by Wednesday, and the few remaining errors that
prevented the IDEA sync process to complete successfully were fixed not
long after that.

Le 2025-05-30 20:44, Julien Plissonneau Duquène a écrit :

an interesting `resolveDependencies` task

Before getting further into the build I would like to complete the
census of dependencies so I'm now using this task to catch all the
unresolvable ones and either fix the package build dependencies, drop
the unpackaged dependencies that are not needed for critical features,
or download pre-built artifacts to progress with the build before
packaging new dependencies (or upgrading existing ones).

A significant difference with the current Kotlin package is that I'm
planning to remove (if possible) all embedded copies of libraries. This
will require to patch some of them that were forked by JetBrains to fix
or add features.

Next I'm going to compare my work with that of Vladimir Petko to review
the list of dependencies, then start working on the build. A few jars
are already produced, but my build is currently broken, probably by the
removal of the (eventually relocated) copies of libraries, so some build
script rework is expected. Last week Vladimir told me that he was
already able to build the stdlib, so I will probably borrow some of his
work.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

Going through `resolveDependencies` allowed me to map or remove a few
more dependencies, but I'm still not satisfied with the state of the
dependency list so I've started to write a custom plugin to get an
exhaustive dependency report (not including transitive dependencies, but mentioning where dependencies are declared or used). Of course this came
with "interesting" issues while trying to make that report task to work
without disabling the configuration cache as Gradle's custom
serialization has some significant limitations. And I was also bitten on
the way by mistakenly overloading an equals() implementation instead of overriding it. But now it works and generates a nice, detailed 20k lines
report for this project.

Le 2025-06-07 19:14, Julien Plissonneau Duquène a écrit :

review the list of dependencies

I now have to finish this review and add some work on dependencies to
the TODO list. So far it seems that at least the following will be
needed soon in order to build the compiler:
- kotlinx-collections-immutable-jvm (new package)
- updating the intellij-community-idea source package (new binary
packages)
- possibly patching the asm package for compatibility.

I've also started to peek at Vladimir's branch. As I'm planning to ditch
the relocated/shaded jars ASAP we won't converge right out, but there
are certainly a few things that I'm going to pick from there.

Finally some DebConf25 news: the Java/JVM BoF proposal was accepted, so
I hope to see some of you there.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Julien,

On 07/06/2025 19:14, Julien Plissonneau Duquène wrote:

A significant difference with the current Kotlin package is that I'm
planning to remove (if possible) all embedded copies of libraries. This
will require to patch some of them that were forked by JetBrains to fix
or add features.

I'm not sure this is a good idea, because it diverges from upstream,
increases the maintenance burden, and could potentially break kotlin if
such a dependency gets broken and can no longer compile (due to kotlin
being broken). For kotlin I'd recommend keeping it simple and sticking
to the upstream layout as much as possible. Saving a few kilobytes isn't important here.

Emmanuel Bourg

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

Sadly this week there is no progress to report on kotlin2 as I was kept
busy by unrelated matters. Hoping to do better next week!

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Emmanuel,

Le 2025-06-16 15:45, Emmanuel Bourg a écrit :

On 07/06/2025 19:14, Julien Plissonneau Duquène wrote:

A significant difference with the current Kotlin package is that I'm
planning to remove (if possible) all embedded copies of libraries.
This will require to patch some of them that were forked by JetBrains
to fix or add features.

I'm not sure this is a good idea, because it diverges from upstream, increases the maintenance burden, and could potentially break kotlin if
such a dependency gets broken and can no longer compile (due to kotlin
being broken). For kotlin I'd recommend keeping it simple and sticking
to the upstream layout as much as possible. Saving a few kilobytes
isn't important here.

I think it would actually be easier terms of maintenance, space savings
are definitely not the reason behind that move. I would worry more about
the risks of breaking a shared dependency used elsewhere by patching it
for kotlin.

The alternatives here are to either embed the dependencies in the kotlin package (which is the way it's currently done), or package the JetBrains
forks separately.

Embedding the dependencies means that the compiler package has to be
rebuilt at least twice after patching a dependency: a first time to
build an updated package with the old package, then a second time to
rebuild the compiler with a compiler that uses the updated dependency.
And if there is kotlin-specific patching involved of that dependency,
the source package would become multi-tarball again. All these make the
package maintenance more complicated and brittle, e.g. if someone else
updates an embedded dependency package but forgets to (test-)rebuild the compiler twice afterwards, it might fail to build later at an
inconvenient time.

Packaging the JetBrains forks separately avoids the issues above, but
still adds to maintenance burden as now the forked package has to be
maintained in addition to the mainstream package. In most cases the
duplication would make no sense IMO. I will consider it though if there
is no reasonable way to make the regular dependency package useable with
kotlin (by patching either the dependency and/or kotlin).

Diverging from upstream is not ideal, but there is such a gap in
policies and goals between the binary releases from the upstream
projects and Debian that it is completely unavoidable anyway. In several
cases, upstream depends on older releases of libraries than those
shipped in Debian, probably because they have to provide commercial
support on older releases of IDE and tools. My plan is to use the
upstream test suites to detect and mitigate issues that could be
introduced by Debian divergences, and keep these to a minimum where it
actually matters.

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good afternoon,

Some upstream news: Kotlin 2.2.0 was released last week (2025-06-23)
[1]. In the meantime it looks like Gradle 9.0 is going to use Kotlin
2.1.21 though some tests were successfully run with 2.2.

Le 2025-06-20 19:53, Julien Plissonneau Duquène a écrit :

Hoping to do better next week!

Other life demands and priorities kept preempting me last week so I
could not progress, and this week I'm on holiday so not much will be
done. The next two weeks I will be at DebCamp then DebConf, hoping to be
able to sneak some kotlin2 work especially in the first week.

Cheers and take care under these heat domes,


[1]: https://blog.jetbrains.com/kotlin/2025/06/kotlin-2-2-0-released/

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Good evening,

Progress on this project these last 3 weeks was slow but still happened.

Le 2025-07-01 16:49, Julien Plissonneau Duquène a écrit :

take care under these heat domes,

The weather in Brittany over the two weeks of DebCamp/DebConf was just
perfect, and the bath at plage du Dellec was definitely refreshing, as
was the cider ;)

Back to kotlin2, I'm now satisfied with the list of dependencies I
extracted with my custom plugin. There are probably still errors and
omissions lurking in the report but it is now good enough to figure out
what the priorities are, and which features can be dropped until we get
the dependencies they use in Debian.

It now looks like the next big step will be to update the intellij-community-idea source package, and make it build a bunch of new libraries that are now needed by this version of Kotlin. I will get in
touch with Vladimir to discuss that with him, as he has some information
about what JetBrains is planning to make their packaging easier.

This week expect a report and followup about the Java/JVM BoF, and
further progress on kotlin2 build and planning.

Cheers,

--
Julien Plissonneau Duquène

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)