On Tue, Jun 25, 2024 at 10:34:15AM +0200, Fab Stz wrote:

Hello,

Not sure that you are in charge of this, so please forward to whom it may concern.
I noticed that the popcon stats haven't been updated for approximately a month now.

Hello Fab,

I have just checked the server, the archive is complete, there is no holes:

Package example: freefilesync

popcon-2024-06-01.gz:Package: freefilesync 56 98 26 3
popcon-2024-06-02.gz:Package: freefilesync 57 97 25 3
popcon-2024-06-03.gz:Package: freefilesync 57 99 26 4
popcon-2024-06-04.gz:Package: freefilesync 61 94 26 4
popcon-2024-06-05.gz:Package: freefilesync 63 94 27 3
popcon-2024-06-06.gz:Package: freefilesync 63 95 26 3
popcon-2024-06-07.gz:Package: freefilesync 67 97 22 3
popcon-2024-06-08.gz:Package: freefilesync 67 97 25 3
popcon-2024-06-09.gz:Package: freefilesync 68 97 24 3
popcon-2024-06-10.gz:Package: freefilesync 69 96 23 4
popcon-2024-06-11.gz:Package: freefilesync 71 101 20 3
popcon-2024-06-12.gz:Package: freefilesync 68 104 19 3
popcon-2024-06-13.gz:Package: freefilesync 66 110 18 3
popcon-2024-06-14.gz:Package: freefilesync 65 111 18 3
popcon-2024-06-15.gz:Package: freefilesync 66 111 18 3
popcon-2024-06-16.gz:Package: freefilesync 65 114 17 3
popcon-2024-06-17.gz:Package: freefilesync 67 114 16 4
popcon-2024-06-18.gz:Package: freefilesync 67 119 15 3
popcon-2024-06-19.gz:Package: freefilesync 64 121 15 3
popcon-2024-06-20.gz:Package: freefilesync 64 123 13 3
popcon-2024-06-21.gz:Package: freefilesync 67 121 13 3
popcon-2024-06-22.gz:Package: freefilesync 69 120 14 3
popcon-2024-06-23.gz:Package: freefilesync 68 119 16 3
popcon-2024-06-24.gz:Package: freefilesync 68 115 17 3

So I assume the issue is within
https://qa.debian.org/popcon.php
itself, so this should be reported to qa.debian.org.
When I go to
https://qa.debian.org/popcon.php?package=freefilesync
and click on "Raw data for the graph" I am lead to the following URL: https://qa.debian.org/cgi-bin/popcon-data?packages=freefilesync;from_date=2010-07-10;to_date=2024-05-22
So I only get data up to the 2024-05-22 which is about one month ago, indeed.

Cheers,
--
Bill. <ballombe@debian.org>

Imagine a large red swirl here.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, Jun 25, 2024 at 03:35:52PM +0200, Bill Allombert wrote:

On Tue, Jun 25, 2024 at 10:34:15AM +0200, Fab Stz wrote:

Hello,

Not sure that you are in charge of this, so please forward to whom it may concern.
I noticed that the popcon stats haven't been updated for approximately a month now.

Hello Fab,

I have just checked the server, the archive is complete, there is no holes:

So I assume the issue is within
https://qa.debian.org/popcon.php
itself, so this should be reported to qa.debian.org.
When I go to
https://qa.debian.org/popcon.php?package=freefilesync
and click on "Raw data for the graph" I am lead to the following URL: https://qa.debian.org/cgi-bin/popcon-data?packages=freefilesync;from_date=2010-07-10;to_date=2024-05-22
So I only get data up to the 2024-05-22 which is about one month ago, indeed.

I suspect that there is a cronjob or something that is running to update
the database. Looking at the code for this page [1], if the to_date is not
set explicitly then it will try to find the latest date for a given
package.

```
$to_date = $dbh->selectrow_array('SELECT max(p.day) FROM popcon_package pp JOIN popcon p ON (pp.id = p.package_id) JOIN popcon_day pd ON (pd.day = p.day) WHERE package = ANY(?) AND in_debian AND (p.vote > 0 OR p.old > 0 OR p.recent > 0 or p.no_files > 0)'
, undef, \@packages) if !$to_date;
```

There is a perl script called popcon-import [2] that runs somewhere, can someone who has access to these logs [3] take a look?


[1] https://salsa.debian.org/qa/qa/-/blob/master/cgi-bin/popcon-data?ref_type=heads#L45

[2] https://salsa.debian.org/qa/qa/-/blob/master/popcon-graphs/popcon-import?ref_type=heads#L55

[3] my $logfile = "/srv/qa.debian.org/log/popcon.log";

--
Best,
Lev

https://levlaz.org

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

I suspect that there is a cronjob or something that is running to update
the database. Looking at the code for this page [1], if the to_date is not set explicitly then it will try to find the latest date for a given
package.

```
$to_date = $dbh->selectrow_array('SELECT max(p.day) FROM popcon_package pp JOIN popcon p ON (pp.id = p.package_id) JOIN popcon_day pd ON (pd.day = p.day) WHERE package = ANY(?) AND in_debian AND (p.vote > 0 OR p.old > 0 OR p.recent > 0 or p.no_files >

0)', undef, \@packages) if !$to_date;

```

There is a perl script called popcon-import [2] that runs somewhere, can someone who has access to these logs [3] take a look?

[3] my $logfile = "/srv/qa.debian.org/log/popcon.log";

It has been logrotated already. Instead we have

/srv/qa.debian.org/log/popcon.log.0:
Sun May 19 15:54:44 2024 popcon-import[26445] Imported popcon-2024-05-19.gz in 258.5s
Mon May 20 15:54:58 2024 popcon-import[14181] Imported popcon-2024-05-20.gz in 271.7s
Tue May 21 15:55:03 2024 popcon-import[30832] Imported popcon-2024-05-21.gz in 283.0s
Wed May 22 15:55:00 2024 popcon-import[25033] Imported popcon-2024-05-22.gz in 281.9s

Which suggest popcon-import is not run rather than crashing.

Cheers,
--
Bill. <ballombe@debian.org>

Imagine a large red swirl here.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, Jun 29, 2024 at 10:46:27AM +0200, Bill Allombert wrote:

It has been logrotated already. Instead we have

/srv/qa.debian.org/log/popcon.log.0:
Sun May 19 15:54:44 2024 popcon-import[26445] Imported popcon-2024-05-19.gz in 258.5s
Mon May 20 15:54:58 2024 popcon-import[14181] Imported popcon-2024-05-20.gz in 271.7s
Tue May 21 15:55:03 2024 popcon-import[30832] Imported popcon-2024-05-21.gz in 283.0s
Wed May 22 15:55:00 2024 popcon-import[25033] Imported popcon-2024-05-22.gz in 281.9s

Which suggest popcon-import is not run rather than crashing.

Thanks for looking Bill, yes that was my suspicion as well. I assume
this is running on a cron or something, but looks like it has not been
running since this day in May.

--
Lev Lazinskiy
lev@levlaz.org

P.S. What I'm doing now: https://levlaz.org/now/

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)