• betterbird: reading crossposts once?

    From candycanearter07@21:1/5 to All on Sun Oct 8 03:06:52 2023
    XPost: news.software.readers

    Hi,
    For cross-posts, is there any way to automatically mark the message read
    in the other groups its posted to once I read it?
    --
    user <candycane> is generated from /dev/urandom

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Adam H. Kerman@21:1/5 to no@thanks.net on Sun Oct 8 08:31:08 2023
    XPost: news.software.readers

    candycanearter07 <no@thanks.net> wrote:

    For cross-posts, is there any way to automatically mark the message read
    in the other groups its posted to once I read it?

    Look at the Xref: header. It has the server's article number in each
    newsgroup in the crosspost. This will allow your newsreader to update
    its list of read articles in each newsgroup in the crosspost.

    Check your newsreader's settings. Otherwise you'll have to ask in the betterbird developer's support forum.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From candycanearter07@21:1/5 to Adam H. Kerman on Sun Oct 8 06:27:47 2023
    XPost: news.software.readers

    On 10/8/23 03:31, Adam H. Kerman wrote:
    Check your newsreader's settings. Otherwise you'll have to ask in the betterbird developer's support forum.

    I looked, couldn't find anything in settings.

    https://groups.google.com/g/mozilla.support.thunderbird/c/S5tSRiGR0w8
    Seems that the solution proposed is "use filters and check cross posting manually" which kinda sucks. It almost makes me want to switch to Claws,
    but that one is broken too (doesn't display newsgroup list in subscribe
    menu)
    --
    user <candycane> is generated from /dev/urandom

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From candycanearter07@21:1/5 to The Doctor on Sun Oct 8 06:30:58 2023
    XPost: news.software.readers

    On 10/8/23 06:25, The Doctor wrote:
    Yes AFAIK. Which newsware are you using?

    Stated in Subject, Betterbird.
    --
    user <candycane> is generated from /dev/urandom

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From The Doctor@21:1/5 to no@thanks.net on Sun Oct 8 11:25:31 2023
    XPost: news.software.readers

    In article <uftnus$2voq4$1@dont-email.me>,
    candycanearter07 <no@thanks.net> wrote:
    Hi,
    For cross-posts, is there any way to automatically mark the message read
    in the other groups its posted to once I read it?

    Yes AFAIK. Which newsware are you using?

    --
    user <candycane> is generated from /dev/urandom


    --
    Member - Liberal International This is doctor@nk.ca Ici doctor@nk.ca
    Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising! Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b An oil stain on the carpet is not removed by picking up the litter. -unknown Beware https://mindspring.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Adam H. Kerman@21:1/5 to no@thanks.net on Sun Oct 8 18:10:46 2023
    XPost: news.software.readers

    candycanearter07 <no@thanks.net> wrote:
    On 10/8/23 03:31, Adam H. Kerman wrote:

    Check your newsreader's settings. Otherwise you'll have to ask in the >>betterbird developer's support forum.

    I looked, couldn't find anything in settings.

    https://groups.google.com/g/mozilla.support.thunderbird/c/S5tSRiGR0w8
    Seems that the solution proposed is "use filters and check cross posting >manually" which kinda sucks. It almost makes me want to switch to Claws,
    but that one is broken too (doesn't display newsgroup list in subscribe
    menu)

    Betterbird is being actively developed, doing bug fixes that Mozilla has ignored for decades, sometimes.

    What happens if you start with a fresh profile rather than importing
    your Thunderbird profile?

    betterbird.eu

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From candycanearter07@21:1/5 to Adam H. Kerman on Sun Oct 8 14:19:41 2023
    XPost: news.software.readers

    On 10/8/23 13:10, Adam H. Kerman wrote:
    What happens if you start with a fresh profile rather than importing
    your Thunderbird profile?

    betterbird.eu

    I'll try to remember later today.
    --
    user <candycane> is generated from /dev/urandom

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From VanguardLH@21:1/5 to no@thanks.net on Sat Oct 14 05:32:45 2023
    XPost: news.software.readers

    candycanearter07 <no@thanks.net> wrote:

    For cross-posts, is there any way to automatically mark the message read
    in the other groups its posted to once I read it?

    Note that article numbers are independently assigned at each NNTP
    server. There is no synchronization of article numbers across NNTP
    servers. Hence there is no cross-posting across servers. If you are
    seeing the same article in multiple newsgroups, but the newsgroups are subscribed on different servers, it was a multi-post, not a cross-post.

    To tracking cross-posting means having to track article numbers. When cross-posted, only a single copy of a post is sent to the server, and
    the server adds pointers to the same article based on the Newsgroups
    header. More efficient to have multiple pointers to the same article.

    Alas, the client won't have the indexing to know about pointers to
    multiple newsgroups for the same article. The client only has the
    Message-ID to track across its local store of messages, so it is
    possible to connect cross-posts across multiple newsgroups. That would
    require the "mark as read" action to scan all the subscribed newsgroups
    to find the same MID value. How many newsgroups to which a message was cross-posted wouldn't be an impact on performance, but the number of
    subscribed newsgroups would impact performance in how many newsgroups
    the client would have to search. You can delete a message in one
    newsgroup, but it remains in another newsgroup. The client isn't
    creating pointers to articles across newsgroups as does the server.

    Rather than search across many subscribed newsgroups, the client could
    use the Xref header, if present, to determine in which newsgroups a
    message got cross-posted. That would shorten the time for "mark as
    read" to find the other copies of the message. Clicking on a button,
    and having to wait for the action to complete, is counter ease-of-use.

    https://www.rfc-editor.org/rfc/rfc1036
    Section 2.2.13

    https://bugzilla.mozilla.org/show_bug.cgi?id=43278

    A problem noted there is that the client may not yet have populated the
    other newsgroups into which a message got cross-posted. The "mark as
    read" action would be stalled by however long it takes for the client to retrieve all messages (well, those that cover the cross-posted article)
    in the Xref newsgroups before they could also get marked as read. You'd
    punch the "read as mark" button while in one newsgroup, but have to wait
    for its commit to complete until all other Xref newsgroups got polled,
    and newsgroups aren't polled in the same order as Xref. It's almost as
    if "read as mark" would have to act as a post-retrieve-all-subscribed- newsgroups action; i.e., you couldn't mark any message as read until all subscribed newsgroups got retrieved. The same for when marking as
    unread.

    From this 22-year old bug, implementing the read state across multiple newsgroups for cross-posted messages doesn't seem that easy a task.
    Your request isn't new. About a year ago, this ticket changed from
    minor (problem worth reporting & fixing, but do not interfere with
    normal work or use) to S4 (Important, but not that much. May get fixed someday).

    https://wiki.documentfoundation.org/QA/Bugzilla/Fields/Severity https://wiki.documentfoundation.org/QA/Bugzilla/Fields/Priority

    Per this ticket, doesn't look like any dev is still interested in
    implementing the feature. Interest died out. Since Betterbird is a
    fork of Thunderbird, and since the feature request faded in Thunderbird,
    my guess is nothing for it showed up in Betterbird.

    One suggestion in the ticket was to order the folders (newsgroup) as to
    how you visit them, and use rules in following folders that were
    dependent those on you already visiting the prior folders. You visit
    the top folder, mark as read, then visit the 2nd folder, and a rule
    checks the prior folder for read state, and so on. However, this
    requires you always visit the folders in top-down order, and never
    bounce around the folders. See bugzillaroid's post 114 of 5 years ago.
    I do bounce around the folders since I only want to visit those
    newsgroups that show new/unread messages. No point for me to visit
    newsgroups with no new messages (that survived filtering).

    Does Tbird poll all newsgroups, and retrieve all new message in every
    one (shown as a folder in the tree pane)? Or does it poll a newsgroup
    only when the folder (newsgroup) is selected in the tree?

    There is a German forum for Betterbird at:

    https://www.thunderbird-mail.de/forum/board/91-betterbird/

    They would probably know better the capabilities of that Tbird fork,
    like if that project added read-state by MessageID (or article number
    via Xref) across newsgroups. Get ready with Google Translate.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to VanguardLH on Sat Oct 14 09:25:58 2023
    XPost: news.software.readers

    VanguardLH <V@nguard.LH> writes:

    Rather than search across many subscribed newsgroups, the client could
    use the Xref header, if present, to determine in which newsgroups a
    message got cross-posted.

    Indeed, this is what essentially every client does and what they should
    do.

    That would shorten the time for "mark as read" to find the other copies
    of the message.

    It makes it essentially instantaneous, since it means the reader doesn't
    have to find other copies at all. It already knows exactly which article numbers to mark as read.

    A problem noted there is that the client may not yet have populated the
    other newsgroups into which a message got cross-posted. The "mark as
    read" action would be stalled by however long it takes for the client to retrieve all messages (well, those that cover the cross-posted article)
    in the Xref newsgroups before they could also get marked as read.

    The client shouldn't have to retrieve articles to mark them as read. The standard representation on the client that goes all the way back to the
    very early days of Usenet is called a "newsrc". It holds a list of groups (often both the unsubscribed and the subscribed groups so that read
    articles are tracked properly if someone later subscribes to the group)
    and a list of read articles by number in each group. In text form, it
    looks something like this:

    rec.arts.sf.misc! 1-14774,14786,14789
    rec.arts.sf.reviews! 1-2534
    rec.arts.sf.written: 1-876513
    news.answers! 1-199359,213516,215735
    news.announce.newusers! 1-4399
    news.newusers.questions! 1-645661
    news.groups.questions! 1-32676
    news.software.readers! 1-95504,137265,137274,140059,140091,140117
    alt.test! 1-1441498

    ":" indicates a subscribed newsgroup and "!" indicates an unsubscribed newsgroup. This is a widely-used standard format that can be read by
    multiple newsreaders so that you can switch between, say, trn and tin
    freely and keep the same read article information.

    Marking an article as read in the crossposted groups is as simple as
    reading the group and article number pairs from the Xref header of the
    article that was just read and adding the corresponding numbers to the
    list of read articles in each group, coalescing ranges where needed. It doesn't require any server operations and is extremely fast with any sort
    of reasonable data structure on the client.

    Usually the client couples this with a LIST ACTIVE command at each
    startup, which very efficiently returns the full list of groups on the
    server and the high and low water marks for each one. This allows the
    client to discover new groups and to coalesce article ranges once articles
    have expired and the low water mark moves forward.

    --
    Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>

    Please post questions rather than mailing me directly.
    <https://www.eyrie.org/~eagle/faqs/questions.html> explains why.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From VanguardLH@21:1/5 to Russ Allbery on Sat Oct 14 12:49:11 2023
    XPost: news.software.readers

    Russ Allbery <eagle@eyrie.org> wrote:

    VanguardLH <V@nguard.LH> writes:

    Rather than search across many subscribed newsgroups, the client could
    use the Xref header, if present, to determine in which newsgroups a
    message got cross-posted.

    Indeed, this is what essentially every client does and what they should
    do.

    Except that only tells you (the client) in which newsgroups an article
    was cross-posted. You would still need the Message-ID value to find the matching article in each of those newsgroups. Searching takes time.

    From the bug ticket, there was a suggestion to change the records in the
    MSF database files to add the MID to each message, so it could be SQL
    searched by a field in a record in the message database (for each
    folder) instead of having to search through the messages themselves.
    The change would require dumping the old MSF databases to begin building
    new ones. As I recall, there was some resistance to changing the record structure, and require building new MSF files.

    A problem noted there is that the client may not yet have populated the
    other newsgroups into which a message got cross-posted. The "mark as
    read" action would be stalled by however long it takes for the client to
    retrieve all messages (well, those that cover the cross-posted article)
    in the Xref newsgroups before they could also get marked as read.

    The client shouldn't have to retrieve articles to mark them as read. The standard representation on the client that goes all the way back to the
    very early days of Usenet is called a "newsrc". It holds a list of groups (often both the unsubscribed and the subscribed groups so that read
    articles are tracked properly if someone later subscribes to the group)
    and a list of read articles by number in each group. In text form, it
    looks something like this:

    rec.arts.sf.misc! 1-14774,14786,14789

    Does Thunderbird currently store the article numbers of messages in its
    MSF database files? The article number came from the server, but that
    doesn't mean the client retains that info. One of the posters in the
    bug ticket said some info in Xref was superfluous to Tbird. The Xref
    would show the newsgroups where the message was cross-posted, but the
    article number would get discarded as superfluous which hints article
    numbers are not in a field for a record in the message database for a
    message. I don't know the structure (fields) in the records in the MSF
    message database files. I suppose I could do an SQLite query to find
    out what fields are defined in the records, but I'm not so motivated, especially since I gave up on Tbird over 3 years ago after many trials,
    the last one of which was 6 months before I had to find something
    better.

    To eliminate having to rebuild MSF files, seems a field could be added
    to the end of the record for the article number. If the field were
    missing in a record, an SQL query on an article number would miss those
    without a field holding the article number. The flag state for
    cross-posted messages would be inconsistent at first until all records
    got updated to add the new field. I don't know if SQLite requires
    records to all have the same fields in every record (the same structure
    for all records), or if some records could have missing fields. The SQL
    query would be by field name, not by field position (within a record).

    I don't recall Tbird uses .rc files, but I don't have an instance of
    Tbird to check if it has any. Tbird seems first designed as an e-mail
    client, and newsgroups were tacked on. Rather than have a combo client
    (e-mail + newsgroups), perhaps they should slice apart the
    functionality, so they can focus on fixing or enhancing an e-mail client separate of fixing or enhancing a newsreader.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ray Banana@21:1/5 to All on Sat Oct 14 18:32:35 2023
    XPost: news.software.readers

    * VanguardLH wrote:
    Russ Allbery <eagle@eyrie.org> wrote:
    The client shouldn't have to retrieve articles to mark them as read. The
    standard representation on the client that goes all the way back to the
    very early days of Usenet is called a "newsrc". It holds a list of groups >> (often both the unsubscribed and the subscribed groups so that read
    articles are tracked properly if someone later subscribes to the group)
    and a list of read articles by number in each group. In text form, it
    looks something like this:

    rec.arts.sf.misc! 1-14774,14786,14789

    Does Thunderbird currently store the article numbers of messages in its
    MSF database files?

    Not in the MSF files, but in the *.rc files.

    I don't recall Tbird uses .rc files

    From my news.eternal-september.org.rc file: --------------------------------------------
    eternal-september.config: 1-958
    eternal-september.grouprequests: 1-690
    eternal-september.info: 1-64
    eternal-september.moderated: 1-27,29
    eternal-september.newusers: 1-1366
    eternal-september.nocems: 1-27
    eternal-september.talk: 1-1515
    eternal-september.test: 1-10382,10397,10502,10503,10544,10546-10548 eternal-september.where.are.all.the.newsgroups: 1-2
    control.checkgroups: 1-1649
    control.newgroup: 1-875
    control.rmgroup: 1-865
    alt.de.test: 1-6609
    uk.d-i-y: 1-1261447
    eternal-september.support: 1-15165,15204-15209,15434,15438,15448-15450


    This is what Russ was talking about and I can confirm that marking
    a crossposted article as read in one group (or simply opening it)
    also marks the article as read in all groups it was crossposted to.
    It has been like that for more than 15 years, IIRC.

    --
    Пу́тін — хуйло́
    http://www.eternal-september.org

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to VanguardLH on Sat Oct 14 11:51:42 2023
    XPost: news.software.readers

    VanguardLH <V@nguard.LH> writes:

    Except that only tells you (the client) in which newsgroups an article
    was cross-posted. You would still need the Message-ID value to find the matching article in each of those newsgroups. Searching takes time.

    If a news reader needs the message ID to mark an article as read, it's
    doing something fairly odd.

    --
    Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>

    Please post questions rather than mailing me directly.
    <https://www.eyrie.org/~eagle/faqs/questions.html> explains why.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From candycanearter07@21:1/5 to VanguardLH on Sat Oct 14 17:29:32 2023
    XPost: news.software.readers

    On 10/14/23 05:32, VanguardLH wrote:
    Does Tbird poll all newsgroups, and retrieve all new message in every
    one (shown as a folder in the tree pane)? Or does it poll a newsgroup
    only when the folder (newsgroup) is selected in the tree?

    You can set it to poll every x minutes and/or when you boot up TB. It
    does always poll when you open a folder, I believe.
    --
    user <candycane> is generated from /dev/urandom

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From VanguardLH@21:1/5 to Russ Allbery on Sat Oct 14 18:21:21 2023
    XPost: news.software.readers

    Russ Allbery <eagle@eyrie.org> wrote:

    VanguardLH <V@nguard.LH> writes:

    Except that only tells you (the client) in which newsgroups an article
    was cross-posted. You would still need the Message-ID value to find the
    matching article in each of those newsgroups. Searching takes time.

    If a news reader needs the message ID to mark an article as read, it's
    doing something fairly odd.

    Yeah, that's what I felt when reading the 20-year old Tbird bug ticket
    that seems to have petered out about a year ago. Seems the devs were
    building a Rube Goldberg scheme.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From VanguardLH@21:1/5 to Ray Banana on Sat Oct 14 18:19:47 2023
    XPost: news.software.readers

    Ray Banana <rayban@raybanana.net> wrote:

    * VanguardLH wrote:
    Russ Allbery <eagle@eyrie.org> wrote:
    The client shouldn't have to retrieve articles to mark them as read. The >>> standard representation on the client that goes all the way back to the
    very early days of Usenet is called a "newsrc". It holds a list of groups >>> (often both the unsubscribed and the subscribed groups so that read
    articles are tracked properly if someone later subscribes to the group)
    and a list of read articles by number in each group. In text form, it
    looks something like this:

    rec.arts.sf.misc! 1-14774,14786,14789

    Does Thunderbird currently store the article numbers of messages in its
    MSF database files?

    Not in the MSF files, but in the *.rc files.

    I don't recall Tbird uses .rc files

    From my news.eternal-september.org.rc file: --------------------------------------------
    eternal-september.config: 1-958
    eternal-september.grouprequests: 1-690
    eternal-september.info: 1-64
    eternal-september.moderated: 1-27,29
    eternal-september.newusers: 1-1366
    eternal-september.nocems: 1-27
    eternal-september.talk: 1-1515
    eternal-september.test: 1-10382,10397,10502,10503,10544,10546-10548 eternal-september.where.are.all.the.newsgroups: 1-2
    control.checkgroups: 1-1649
    control.newgroup: 1-875
    control.rmgroup: 1-865
    alt.de.test: 1-6609
    uk.d-i-y: 1-1261447
    eternal-september.support: 1-15165,15204-15209,15434,15438,15448-15450

    This is what Russ was talking about and I can confirm that marking
    a crossposted article as read in one group (or simply opening it)
    also marks the article as read in all groups it was crossposted to.
    It has been like that for more than 15 years, IIRC.

    If all that were doable in other e-mail clients, seems inane that Tbird
    hasn't accomplied the same feat for a 22-year old bug ticket. Comes
    back to me thinking Tbird was first designed to be an e-mail client, and newsgroups got tacked on.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From VanguardLH@21:1/5 to no@thanks.net on Sat Oct 14 18:23:20 2023
    XPost: news.software.readers

    candycanearter07 <no@thanks.net> wrote:

    On 10/14/23 05:32, VanguardLH wrote:
    Does Tbird poll all newsgroups, and retrieve all new message in every
    one (shown as a folder in the tree pane)? Or does it poll a newsgroup
    only when the folder (newsgroup) is selected in the tree?

    You can set it to poll every x minutes and/or when you boot up TB. It
    does always poll when you open a folder, I believe.

    That's probably why someone in the bug ticket that mentioned their
    scheme of using rules based on the prior newsgroup visited would have to
    open the folder in the same order as the rules. Bouncing around would
    mean unvisited newsgroups would not get the read-flag updated.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to VanguardLH on Sun Oct 15 12:34:46 2023
    XPost: news.software.readers

    VanguardLH <V@nguard.lh> wrote:
    Ray Banana <rayban@raybanana.net> wrote:

    * VanguardLH wrote:
    Russ Allbery <eagle@eyrie.org> wrote:
    The client shouldn't have to retrieve articles to mark them as read. The >>> standard representation on the client that goes all the way back to the >>> very early days of Usenet is called a "newsrc". It holds a list of groups
    (often both the unsubscribed and the subscribed groups so that read
    articles are tracked properly if someone later subscribes to the group) >>> and a list of read articles by number in each group. In text form, it >>> looks something like this:

    rec.arts.sf.misc! 1-14774,14786,14789

    Does Thunderbird currently store the article numbers of messages in its
    MSF database files?

    Not in the MSF files, but in the *.rc files.

    I don't recall Tbird uses .rc files

    From my news.eternal-september.org.rc file: --------------------------------------------
    eternal-september.config: 1-958
    eternal-september.grouprequests: 1-690
    eternal-september.info: 1-64
    eternal-september.moderated: 1-27,29
    eternal-september.newusers: 1-1366
    eternal-september.nocems: 1-27
    eternal-september.talk: 1-1515
    eternal-september.test: 1-10382,10397,10502,10503,10544,10546-10548 eternal-september.where.are.all.the.newsgroups: 1-2
    control.checkgroups: 1-1649
    control.newgroup: 1-875
    control.rmgroup: 1-865
    alt.de.test: 1-6609
    uk.d-i-y: 1-1261447
    eternal-september.support: 1-15165,15204-15209,15434,15438,15448-15450

    This is what Russ was talking about and I can confirm that marking
    a crossposted article as read in one group (or simply opening it)
    also marks the article as read in all groups it was crossposted to.
    It has been like that for more than 15 years, IIRC.

    If all that were doable in other e-mail clients, seems inane that Tbird hasn't accomplied the same feat for a 22-year old bug ticket. Comes
    back to me thinking Tbird was first designed to be an e-mail client, and newsgroups got tacked on.

    What Ray is saying, is that Thunderbird *does* already mark
    crossposted articles as read. I.e. Thunderbird does use a 'newsrc' style
    file, as does any other newreader on the planet. The file just isn't
    named '[.]newsrc', but <server_name>.rc. So while there may be some old
    bug report, it's doubtful that Thunderbird is generally not marking
    crossposted articles as read.

    So basically the questions are: 1) under which circumstances does
    Thunderbird - allegedly - not do what it should do or/and 2) does - as
    the OP (candycanearter07) implies - Betterbird do something different
    than Thunderbird?

    Like you (and Ray), I don't use Thunderbird as my regular newsreader,
    so it would be nice if some people who *do* use Thunderbird on a regular
    basis, can confirm if this alleged 'bug' - Thunderbird does (sometimes)
    not mark crossposted articles as read - exists at all or that it's just
    and urban legend, FUD, etc..

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From candycanearter07@21:1/5 to Frank Slootweg on Sun Oct 15 12:08:30 2023
    XPost: news.software.readers

    On 10/15/23 07:34, Frank Slootweg wrote:
    Like you (and Ray), I don't use Thunderbird as my regular newsreader,
    so it would be nice if some people who *do* use Thunderbird on a regular basis, can confirm if this alleged 'bug' - Thunderbird does (sometimes)
    not mark crossposted articles as read - exists at all or that it's just
    and urban legend, FUD, etc..

    It usually doesn't work, but lately I haven't noticed it.
    --
    user <candycane> is generated from /dev/urandom

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to no@thanks.net on Sun Oct 15 18:16:14 2023
    XPost: news.software.readers

    In news.software.nntp candycanearter07 <no@thanks.net> wrote:
    On 10/15/23 07:34, Frank Slootweg wrote:
    Like you (and Ray), I don't use Thunderbird as my regular newsreader,
    so it would be nice if some people who *do* use Thunderbird on a regular basis, can confirm if this alleged 'bug' - Thunderbird does (sometimes)
    not mark crossposted articles as read - exists at all or that it's just
    and urban legend, FUD, etc..

    It usually doesn't work, but lately I haven't noticed it.

    Please stop snipping too much context!

    With your response you imply that you experienced this problem in Thunderbird, but in reality you probably experienced it in Betterbird.

    So the questions stands: What do *Thunderbird* users experience?



    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=C3=89LIE?=@21:1/5 to All on Sun Oct 15 21:43:36 2023
    XPost: news.software.readers

    Hi Frank,

    So the questions stands: What do *Thunderbird* users experience?

    As a user of Thunderbird, and reading both news.software.readers and news.software.nntp, I confirm that when I read (and mark as so) an
    article in this thread either in news.software.readers or
    news.software.nntp first, it still shows up unread in the other newsgroup.
    I'm using the latest version of Thunderbird (115.3.2), and this bug was
    present in previous versions too.

    My .rc file is named "newsrc-servername" (newsrc-news.trigofacile.com)
    and seems normal, but not updated when a crossposted article is read in
    one newsgroup.

    --
    Julien ÉLIE

    « The best preparation for tomorrow is to do today's work superbly
    well. » (William Osler)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D@21:1/5 to Julien ELIE on Sun Oct 15 22:54:40 2023
    On Sun, 15 Oct 2023 21:43:36 +0200, Julien ELIE <iulius@nom-de-mon-site.com.invalid> wrote:
    Hi Frank,
    So the questions stands: What do *Thunderbird* users experience?

    As a user of Thunderbird, and reading both news.software.readers and >news.software.nntp, I confirm that when I read (and mark as so) an
    article in this thread either in news.software.readers or
    news.software.nntp first, it still shows up unread in the other newsgroup. >I'm using the latest version of Thunderbird (115.3.2), and this bug was >present in previous versions too.
    My .rc file is named "newsrc-servername" (newsrc-news.trigofacile.com)
    and seems normal, but not updated when a crossposted article is read in
    one newsgroup.

    as a layman with some experience in old-fashioned newsreaders, it's
    been handy to ignore x-posted threads in one group, while following
    the same thread in another, thus eliminating duplication of replies;
    having tested t & b-bird, old 40tude Dialog is still much easier to
    setup and use, works on most versions of Windows, + hamster scoring

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From VanguardLH@21:1/5 to iulius@nom-de-mon-site.com.invalid on Sun Oct 15 20:46:09 2023
    XPost: news.software.readers

    Julien LIE <iulius@nom-de-mon-site.com.invalid> wrote:

    As a user of Thunderbird, and reading both news.software.readers and news.software.nntp, I confirm that when I read (and mark as so) an
    article in this thread either in news.software.readers or
    news.software.nntp first, it still shows up unread in the other newsgroup. I'm using the latest version of Thunderbird (115.3.2), and this bug was present in previous versions too.

    My .rc file is named "newsrc-servername" (newsrc-news.trigofacile.com)
    and seems normal, but not updated when a crossposted article is read in
    one newsgroup.

    That's why I mentioned the 22-year old article trying to figure out how
    to sync read-flag status across multiple newsgroups. While old, looks
    like the bug petered out only a year ago. To me, as a non-user, and
    from Tbird user reports, read-status is not reflected across the
    newsgroups to which the article was cross-posted.

    From those I assume are more intimate with Tbird, it previously used the
    Mork format (https://en.wikipedia.org/wiki/Mork_%28file_format%29) for
    its message store. The read state is stored in the "flags" column. But
    that doesn't mean the read-state migrates of MSF files for other
    folders. Then they switched to SQlite; however, a database for one
    folder will not automatically update fields in records in another
    folder's database file.

    The bug ticket proposed adding flags to the SQLite databases, but still
    there would have to be synchonization between the separate folder
    databases to update the flag states. The user could click on a folder,
    but the sync hasn't yet completed the length of which would depend on
    how many folders existed for all the subscribed newsgroups. The user
    could visit a folder, the same message was not marked read, and they
    sync may happen while viewing that folder, but the folder tree would
    need to get refreshed which could produce flicker. It looked like they
    were looking at sync between folder databases, but noted the change to
    add fields for the flags would require replacing the old MSF files.

    I don't how Tbird utilizes .rc files. Perhaps only some info is
    extracted from there, like just the newsgroups, and not the article
    numbers. Yeah, seems dumb to omit the article numbers which would tell
    you which ones to look at in which newsgroups.

    Is the Xref header really present all the time? From my reading, it is
    added to an article by the server to which the client connects; i.e.,
    the endpoint host in PATH adds the Xref header when the client retrieves
    an article. Okay, I understand that, because article numbers are
    independent on different NNTP servers (no sync on article numbers
    between servers). The endpoint server to which the client know what are
    its article numbers. Are users not happy with cross-post flagging
    because the same post shows up in different folders (newsgroups), but
    those newsgroups are polled from different servers? The article numbers
    in Xref would be worthless for copies of the same post on different
    servers. Could explain why the read-flag sometimes works across
    newsgroups (retrieved from the same server) and sometimes not (retrieved
    from different servers).

    From RFC 2980, section 2.1.7, the 'list overview.fmt' shows the overview headers that get sent by the server. Both ES, and my server
    (individual.net) show "Xref:full". Do any NNTP servers not add this
    overview header to a retrieved article? The same RFC says "Many
    newsreaders work better if Xref: is one of the optional fields." I
    never thought of it as an optional header.

    https://www.rfc-editor.org/rfc/rfc5536.html#page-24 describes the syntax
    of the Xref header, but that section 3.2.14 is under section 3.2 which
    is titled "Optional Header Fields". Maybe the Xref header has become
    ubiqitous in Usenet, but it is an optional header. What happens when a
    client that relies on Xref (and .rc files) hits an endpoint server that
    doesn't add Xref when the client retrieves an article? Maybe that's why
    the Tbird devs didn't want to rely on an optional header.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to VanguardLH on Sun Oct 15 19:22:22 2023
    XPost: news.software.readers

    VanguardLH <V@nguard.LH> writes:

    From RFC 2980, section 2.1.7, the 'list overview.fmt' shows the overview headers that get sent by the server. Both ES, and my server
    (individual.net) show "Xref:full". Do any NNTP servers not add this
    overview header to a retrieved article? The same RFC says "Many
    newsreaders work better if Xref: is one of the optional fields." I
    never thought of it as an optional header.

    Although it's technically optional, I'm fairly doubtful that any widely
    used news server doesn't provide Xref headers and Xref in overview.

    https://www.rfc-editor.org/rfc/rfc5536.html#page-24 describes the syntax
    of the Xref header, but that section 3.2.14 is under section 3.2 which
    is titled "Optional Header Fields". Maybe the Xref header has become ubiqitous in Usenet, but it is an optional header. What happens when a client that relies on Xref (and .rc files) hits an endpoint server that doesn't add Xref when the client retrieves an article?

    I think a fairly obvious thing to do would be to fall back on not updating
    the read message indicator in crossposted groups if there's no Xref
    header.

    Maybe that's why the Tbird devs didn't want to rely on an optional
    header.

    If indeed Thunderbird doesn't mark crossposted messages as read in all
    groups (it sounds from the recent messages like it doesn't, but I don't
    use it so I don't know), another possibility is that this was an
    intentional design choice. Thunderbird is primarily a mail reader, and in
    the mail world, it's normal *not* to mark messages cross-posted to
    multiple mailing lists as read in other mailing lists to which they're
    sent. This is for somewhat obvious technical reasons (usually you receive separate copies of the message for each list, and mail has no Xref
    header), but it's also something mail users are used to.

    It's possible that the original Thunderbird developers decided keeping the typical mail behavior made more sense given their expected user base, or
    at least was a good argument to not bother to implement crosspost
    tracking.

    --
    Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>

    Please post questions rather than mailing me directly.
    <https://www.eyrie.org/~eagle/faqs/questions.html> explains why.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Olivier Miakinen@21:1/5 to All on Mon Oct 16 08:53:25 2023
    XPost: news.software.readers

    Hi Russ,

    Le 16/10/2023 04:22, Russ Allbery a crit :

    If indeed Thunderbird doesn't mark crossposted messages as read in all
    groups (it sounds from the recent messages like it doesn't, but I don't
    use it so I don't know), another possibility is that this was an
    intentional design choice. Thunderbird is primarily a mail reader, [...]

    Though you may be right about the intentional design choice, then I suppose
    it would have been made not in Thunderbird but in Netscape Messenger 4.x : <https://en.wikipedia.org/wiki/Netscape_Mail_%26_Newsgroups>.


    --
    Olivier Miakinen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Olivier Miakinen@21:1/5 to All on Mon Oct 16 08:44:56 2023
    XPost: news.software.readers

    Bonjour Julien,

    Le 15/10/2023 21:43, Julien LIE a crit :

    As a user of Thunderbird, and reading both news.software.readers and news.software.nntp, I confirm that when I read (and mark as so) an
    article in this thread either in news.software.readers or
    news.software.nntp first, it still shows up unread in the other newsgroup.

    Thank you for the info about the latest version.

    I'm using the latest version of Thunderbird (115.3.2), and this bug was present in previous versions too.

    This bug was already present in Netscape Communicator a.k.a. Netscape 4, then in all flavors of Mozilla, and still in SeaMonkey.

    About Thunderbird which is the only one I never used for newsgroups, because
    it is still present now I suppose that it has always been present since the split between various Mozilla products.

    --
    Olivier Miakinen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to iulius@nom-de-mon-site.com.invalid on Mon Oct 16 11:27:16 2023
    XPost: news.software.readers

    Julien LIE <iulius@nom-de-mon-site.com.invalid> wrote:
    Hi Frank,

    So the questions stands: What do *Thunderbird* users experience?

    As a user of Thunderbird, and reading both news.software.readers and news.software.nntp, I confirm that when I read (and mark as so) an
    article in this thread either in news.software.readers or
    news.software.nntp first, it still shows up unread in the other newsgroup. I'm using the latest version of Thunderbird (115.3.2), and this bug was present in previous versions too.

    My .rc file is named "newsrc-servername" (newsrc-news.trigofacile.com)
    and seems normal, but not updated when a crossposted article is read in
    one newsgroup.

    Hi Julien,

    From your response and other recent responses, it indeed seems that
    this is a (very, very long standing) bug and - as Russ indicates -
    perhaps even an ill-conceived 'design choice' (Jamie (Zawinski) was
    known to have ... ahum ... 'special' ideas about how NetNews should
    work :-().

    But to be sure:

    In the above mentioned context, are you reading both groups from the
    same server? (Or better yet, perhaps you have configured only one server
    in Thunderbird?)

    I assume you read both groups from the same server, because you only
    mention one .rc file and one servername, but need to be sure.

    The reason for my question is obvious: Thunderbird should use article
    numbers to mark articles are read in both groups, but (as you are of
    course all too aware of), article numbers are server specific, so
    marking crossposted articles are read across multiple servers is
    (nearly) impossible.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Olivier Miakinen@21:1/5 to All on Mon Oct 16 13:54:08 2023
    XPost: news.software.readers

    Le 16/10/2023 13:27, Frank Slootweg a crit :

    But to be sure:

    In the above mentioned context, are you reading both groups from the
    same server? (Or better yet, perhaps you have configured only one server
    in Thunderbird?)

    From my experience in dozens of years (from Netscape 4 to SeaMonkey), yes,
    it was always about several groups read from a unique server.


    --
    Olivier Miakinen

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ray Banana@21:1/5 to All on Mon Oct 16 14:27:18 2023
    XPost: news.software.readers

    Thus spake Frank Slootweg <this@ddress.is.invalid>



    The reason for my question is obvious: Thunderbird should use article
    numbers to mark articles are read in both groups, but (as you are of
    course all too aware of), article numbers are server specific, so
    marking crossposted articles are read across multiple servers is
    (nearly) impossible.

    Obviously, I was referring to a setup where all groups are read from the
    same server.
    It would have never occurred to me that somebody would expect this to
    work across multiple servers ;-)

    I don't know any newsreader that is able to do that.

    --
    Пу́тін — хуйло́
    http://www.eternal-september.org

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From candycanearter07@21:1/5 to Frank Slootweg on Mon Oct 16 11:03:24 2023
    XPost: news.software.readers

    On 10/16/23 10:49, Frank Slootweg wrote:
    Neither do I, but it's just SMOP (Small Matter Of Programming). All
    you have to do is to remember all the hundred of thousands / millions of message-ids of the postings you've ever 'read'. How hard can *that* be!?
    :-)

    You could just scan for matching ids when you read an article. Or, maybe
    you could limit it to the messages read in the last x days, to cut down
    on processing time.
    --
    user <candycane> is generated from /dev/urandom

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to Ray Banana on Mon Oct 16 15:49:33 2023
    XPost: news.software.readers

    Ray Banana <rayban@raybanana.net> wrote:
    Thus spake Frank Slootweg <this@ddress.is.invalid>



    The reason for my question is obvious: Thunderbird should use article numbers to mark articles are read in both groups, but (as you are of
    course all too aware of), article numbers are server specific, so
    marking crossposted articles are read across multiple servers is
    (nearly) impossible.

    Obviously, I was referring to a setup where all groups are read from the
    same server.
    It would have never occurred to me that somebody would expect this to
    work across multiple servers ;-)

    Well, one of the respondents in this thread talked about a multiple
    server scenario, so I wanted to be sure that Julien (and you and others)
    were talking single server.

    I don't know any newsreader that is able to do that.

    Neither do I, but it's just SMOP (Small Matter Of Programming). All
    you have to do is to remember all the hundred of thousands / millions of message-ids of the postings you've ever 'read'. How hard can *that* be!?
    :-)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jakob Bohm@21:1/5 to All on Mon Oct 16 18:21:12 2023
    XPost: news.software.readers

    On 2023-10-16 18:03, candycanearter07 wrote:
    On 10/16/23 10:49, Frank Slootweg wrote:
       Neither do I, but it's just SMOP (Small Matter Of Programming). All
    you have to do is to remember all the hundred of thousands / millions of
    message-ids of the postings you've ever 'read'. How hard can *that* be!?
    :-)

    You could just scan for matching ids when you read an article. Or, maybe
    you could limit it to the messages read in the last x days, to cut down
    on processing time.

    A more practical optimization is to only retain a database of MIDs of
    messages that are marked "read" in actively kept message storage. Thus
    when a read message is "aged out" of storage, its "read" status is aged
    out too. To avoid old messages transiting in and out of view, keep a
    count of how many copies are marked read for each MID and count that
    down as messages age out. Once in a while, rebuild database from the
    stored messages and their "read" status, then do a second pass to mark duplicates of read messages as read. Bonus: Also index the body
    contents hash to catch identical multi-posting.

    Enjoy

    Jakob
    --
    Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
    Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
    This public discussion message is non-binding and may contain errors.
    WiseMo - Remote Service Management for PCs, Phones and Embedded

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to Jakob Bohm on Mon Oct 16 17:27:56 2023
    XPost: news.software.readers

    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    On 2023-10-16 18:03, candycanearter07 wrote:
    On 10/16/23 10:49, Frank Slootweg wrote:
    Neither do I, but it's just SMOP (Small Matter Of Programming). All
    you have to do is to remember all the hundred of thousands / millions of >> message-ids of the postings you've ever 'read'. How hard can *that* be!? >> :-)

    You could just scan for matching ids when you read an article. Or, maybe you could limit it to the messages read in the last x days, to cut down
    on processing time.

    A more practical optimization is to only retain a database of MIDs of messages that are marked "read" in actively kept message storage. Thus
    when a read message is "aged out" of storage, its "read" status is aged
    out too. To avoid old messages transiting in and out of view, keep a
    count of how many copies are marked read for each MID and count that
    down as messages age out. Once in a while, rebuild database from the
    stored messages and their "read" status, then do a second pass to mark duplicates of read messages as read. Bonus: Also index the body
    contents hash to catch identical multi-posting.

    I said "hundred of thousands / millions of message-ids" for a reason.

    I have well over a million articles in my "message storage". Keeping
    that a amount of messages-ids is already not practical, but *scanning*
    (for read/not_read status) them for each an every new (and old) article
    is impossible.

    But even if you don't keep (that) many articles in your message store,
    when you add a new newsserver which has a long retention time, you *do*
    have to scan all those articles, just in case there's one which you have already marked as read.

    That's why newsservers and newsreaders use article numbers instead of message-ids. It makes things much more simple (actually possible instead
    of impossible). But yes, it works only for a specific server, not
    accross servers and when adding a new group or/and new server you should
    only fetch the latest N articles (which is very simple using article
    numbers).

    Bottom line: Using article numbers is done for a very good reason.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From =?UTF-8?Q?Julien_=C3=89LIE?=@21:1/5 to All on Mon Oct 16 20:54:03 2023
    XPost: news.software.readers

    Hi Frank,

    As a user of Thunderbird, and reading both news.software.readers and
    news.software.nntp, I confirm that when I read (and mark as so) an
    article in this thread either in news.software.readers or
    news.software.nntp first, it still shows up unread in the other newsgroup. >> I'm using the latest version of Thunderbird (115.3.2), and this bug was
    present in previous versions too.

    In the above mentioned context, are you reading both groups from the
    same server?

    Yes.


    (Or better yet, perhaps you have configured only one server in Thunderbird?)

    I have configured several servers, but the test case was on the same
    server (and newsrc file).

    --
    Julien ÉLIE

    « La vie n'est qu'un tissu de coups de poignard qu'il faut savoir boire
    goutte à goutte. »

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jakob Bohm@21:1/5 to Frank Slootweg on Mon Oct 16 22:11:26 2023
    XPost: news.software.readers

    On 2023-10-16 19:27, Frank Slootweg wrote:
    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    On 2023-10-16 18:03, candycanearter07 wrote:
    On 10/16/23 10:49, Frank Slootweg wrote:
       Neither do I, but it's just SMOP (Small Matter Of Programming). All >>>> you have to do is to remember all the hundred of thousands / millions of >>>> message-ids of the postings you've ever 'read'. How hard can *that* be!? >>>> :-)

    You could just scan for matching ids when you read an article. Or, maybe >>> you could limit it to the messages read in the last x days, to cut down
    on processing time.

    A more practical optimization is to only retain a database of MIDs of
    messages that are marked "read" in actively kept message storage. Thus
    when a read message is "aged out" of storage, its "read" status is aged
    out too. To avoid old messages transiting in and out of view, keep a
    count of how many copies are marked read for each MID and count that
    down as messages age out. Once in a while, rebuild database from the
    stored messages and their "read" status, then do a second pass to mark
    duplicates of read messages as read. Bonus: Also index the body
    contents hash to catch identical multi-posting.

    I said "hundred of thousands / millions of message-ids" for a reason.

    I have well over a million articles in my "message storage". Keeping
    that a amount of messages-ids is already not practical, but *scanning*
    (for read/not_read status) them for each an every new (and old) article
    is impossible.

    But even if you don't keep (that) many articles in your message store, when you add a new newsserver which has a long retention time, you *do*
    have to scan all those articles, just in case there's one which you have already marked as read.

    That's why newsservers and newsreaders use article numbers instead of message-ids. It makes things much more simple (actually possible instead
    of impossible). But yes, it works only for a specific server, not
    accross servers and when adding a new group or/and new server you should
    only fetch the latest N articles (which is very simple using article numbers).

    Bottom line: Using article numbers is done for a very good reason.


    You don't understand what I wrote. Idea would be to use some kind of
    available (search optimized) database engine to store the values,
    updated incrementally as messages enter and leave the storage or are
    marked read. Scanning all messages would be a "once in a while"
    database repair in case the database gets out of sync with the message
    storage and be done using O(n*log(n)) code.

    Database lookups would happen during the initial import of messages (to
    see if a newly arrived message was marked read by an earlier user access
    to a duplicate) and when displaying a message or its header on screen
    (to see if it a duplicate was marked read since its arrival).

    Thus common operations change from complexity O(n*n) to O(delta*log(n))
    given typically assumed lookup costs in indexed databases.


    Enjoy

    Jakob
    --
    Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
    Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
    This public discussion message is non-binding and may contain errors.
    WiseMo - Remote Service Management for PCs, Phones and Embedded

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From VanguardLH@21:1/5 to Frank Slootweg on Mon Oct 16 22:34:41 2023
    XPost: news.software.readers

    Frank Slootweg <this@ddress.is.invalid> wrote:

    All you have to do is to remember all the hundred of thousands /
    millions of message-ids of the postings you've ever 'read'. How hard
    can *that* be!?

    Thunderbird uses SQLite for the messages databases, one for each folder (newsgroup). While SQLite is fast for small database sizes, it doesn't
    scale well. The bigger the database, the slow queries get, and the
    slowdown is not linear. SQLite recommends once you pass into the
    terabyte range to find a different solution.

    https://www.dbtalks.com/tutorials/learn-sqlite/what-are-the-limitations-of-sqlite
    A SQLite database can have maximum 2147483646 pages. Hence the maximum
    number of tables in a schema cannot reach more than 2147483646. The
    maximum number of rows in a table is 264. The maximum number of columns
    is 32767 in a table.

    Notice only 264 rows per table. You end up managing thousands of tables
    to retain millions of records. Been way too long since I did any work
    with SQL, but I thought queries were per table. You'd need a macro that
    walked through all the tables to do a query across them all. Maybe for
    the table spec you could use a wildcard, like *, to have the query
    process across all tables.

    Many clients also have an option to either flag or delete articles over
    a threshold in age. I have my client purge posts older than 60 days.
    Older than that, and the discussion becomes stale. For the expiration,
    each table in a database, and for each database (folder/newsgroup),
    you'd have to issue a purge and compact.

    Overall an SQLite database can hold 140 TB. Is that really big enough? Consider these are message databases, so it's the entire message that
    gets stored, and that includes very long messages that have gads of
    quoted content because lots of posters never trim. It would require
    even more space to store binaries. There are 20K+ newsgroups, and tons
    of messages in each. To be reasonable, some cutoff in expiration would
    be needed, but that could result in undoing the read-state tracking, so
    the same article in different newsgroups across servers could get out of
    sync.

    Plus you're talking about caching locally the entire Usenet that a
    provider has. It would only be for the newsgroups to which you
    subscribe, but some users subscribe to a lot a newsgroups. I was up to
    52 at one time, and am now down to 26. Depending on how many newsgroups
    to which you subscribe, you could end up having a storage media
    requirement equivalent to the Usenet providers. How much free disk
    space do you have?

    There are some users that run proxy servers for personal use, like using Hamster to leech from a Usenet provider. That lets them have everything
    the Usenet provider has. A lot of work. Very few users operate their
    own server.

    Hard enough to implement read-state tracking on one server. Massively
    more difficult to do across multiple servers. And we're talking about
    NNTP clients that could be running on underpowered computers.

    I've notice my NNTP client will slow down when its message store gets
    large. Right now it's about 700 MB, but it will grow. That's one
    reason to have a purge of 60-day old messages to keep quick the
    responsiveness of my client. That also means threads that have articles
    older than 60 days and articles younger than 60 days end up getting
    truncated, so I see a partial thread remaining.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to Jakob Bohm on Tue Oct 17 12:26:36 2023
    XPost: news.software.readers

    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    On 2023-10-16 19:27, Frank Slootweg wrote:
    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    On 2023-10-16 18:03, candycanearter07 wrote:
    On 10/16/23 10:49, Frank Slootweg wrote:
    Neither do I, but it's just SMOP (Small Matter Of Programming). All >>>> you have to do is to remember all the hundred of thousands / millions of >>>> message-ids of the postings you've ever 'read'. How hard can *that* be!? >>>> :-)

    You could just scan for matching ids when you read an article. Or, maybe >>> you could limit it to the messages read in the last x days, to cut down >>> on processing time.

    A more practical optimization is to only retain a database of MIDs of
    messages that are marked "read" in actively kept message storage. Thus
    when a read message is "aged out" of storage, its "read" status is aged
    out too. To avoid old messages transiting in and out of view, keep a
    count of how many copies are marked read for each MID and count that
    down as messages age out. Once in a while, rebuild database from the
    stored messages and their "read" status, then do a second pass to mark
    duplicates of read messages as read. Bonus: Also index the body
    contents hash to catch identical multi-posting.

    I said "hundred of thousands / millions of message-ids" for a reason.

    I have well over a million articles in my "message storage". Keeping that a amount of messages-ids is already not practical, but *scanning*
    (for read/not_read status) them for each an every new (and old) article
    is impossible.

    But even if you don't keep (that) many articles in your message store, when you add a new newsserver which has a long retention time, you *do* have to scan all those articles, just in case there's one which you have already marked as read.

    That's why newsservers and newsreaders use article numbers instead of message-ids. It makes things much more simple (actually possible instead
    of impossible). But yes, it works only for a specific server, not
    accross servers and when adding a new group or/and new server you should only fetch the latest N articles (which is very simple using article numbers).

    Bottom line: Using article numbers is done for a very good reason.


    You don't understand what I wrote. Idea would be to use some kind of available (search optimized) database engine to store the values,
    updated incrementally as messages enter and leave the storage or are
    marked read. Scanning all messages would be a "once in a while"
    database repair in case the database gets out of sync with the message storage and be done using O(n*log(n)) code.

    Database lookups would happen during the initial import of messages (to
    see if a newly arrived message was marked read by an earlier user access
    to a duplicate) and when displaying a message or its header on screen
    (to see if it a duplicate was marked read since its arrival).

    Thus common operations change from complexity O(n*n) to O(delta*log(n))
    given typically assumed lookup costs in indexed databases.

    I understood perfectly what you wrote, but you proposed solution does
    not cover the scenarios I described, especially - but not only - the
    scenario of adding a new newsserver which has a long retention time.

    So I'm afraid "You don't understand what I wrote."

    Yes, you can work around that with hacks like not loading old
    articles, but that will require use of article numbers instead of
    message-ids, so you might as well use article numbers from the start.

    Anyway, *if* this issue is a real problem - i.e. not just a minor
    inconvience in some corner cases - for some people, they can try to
    switch to a newsreader which does aggregrate articles from several
    servers (i.e. something which Thunderbird (and Betterbird?) does not do)
    or add a 'proxy' which does the aggregration (like Hamster, which I
    use).

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to VanguardLH on Tue Oct 17 13:29:38 2023
    XPost: news.software.readers

    VanguardLH <V@nguard.lh> wrote:
    Frank Slootweg <this@ddress.is.invalid> wrote:

    All you have to do is to remember all the hundred of thousands /
    millions of message-ids of the postings you've ever 'read'. How hard
    can *that* be!?

    Sigh!

    You post-edited my post by snipping the start and end of my paragraph
    and reformatting the rest. By doing that, you've turned my blatantly
    obvious joke into something serious. Why?

    [...]

    There are some users that run proxy servers for personal use, like using Hamster to leech from a Usenet provider. [...]

    Actually, Hamster is a solution to this 'problem'.

    Hard enough to implement read-state tracking on one server. Massively
    more difficult to do across multiple servers. And we're talking about
    NNTP clients that could be running on underpowered computers.

    Read-state tracking for (not on) one server is trivial. Every
    newsreader does it (via article-number range tracking in .newsrc type
    files). That Thunderbird et al mess it up, doesn't change that.

    Read-state tracking for multiple servers can be done by aggregrating
    articles from several servers. I've seen people imply that some
    newsreaders can do that, but I have no details (Any takers?). And, as I
    said above, Hamster can do this as well. Perhaps Leafnode can do the
    same on Linux (the OP uses Linux).

    [...]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From VanguardLH@21:1/5 to Frank Slootweg on Tue Oct 17 15:26:05 2023
    XPost: news.software.readers

    Frank Slootweg <this@ddress.is.invalid> wrote:

    Read-state tracking for multiple servers can be done by aggregrating articles from several servers. I've seen people imply that some
    newsreaders can do that, but I have no details (Any takers?). And, as I
    said above, Hamster can do this as well. Perhaps Leafnode can do the
    same on Linux (the OP uses Linux).

    At that point as a cross-server aggregator, using something like Hamster
    might suffice. It would collect articles across multiple servers, but
    does Hamster then assign its own article numbers (so it could add the
    Xref header with its article numbers)?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From D@21:1/5 to Frank Slootweg on Wed Oct 18 06:31:48 2023
    On 17 Oct 2023 13:29:38 GMT, Frank Slootweg <this@ddress.is.invalid> wrote: snip
    Read-state tracking for (not on) one server is trivial. Every
    newsreader does it (via article-number range tracking in .newsrc type
    files). That Thunderbird et al mess it up, doesn't change that.
    Read-state tracking for multiple servers can be done by aggregrating
    articles from several servers. I've seen people imply that some
    newsreaders can do that, but I have no details (Any takers?). And, as I
    said above, Hamster can do this as well. Perhaps Leafnode can do the
    same on Linux (the OP uses Linux).

    I'm no expert, but 40tude Dialog might do something like what you're describing, quoting from the Help section on "create virtual groups":
    [begin quote]
    How can I... (Procedural Help)
    Almost virtual groups
    There are no virtual groups in Dialog, however you can simply copy or move >articles from one group to another manually or by using an action rule: >[a.binary.group]
    !move(a.binary.group;NewsserverOne) bytes %>0
    All previous examples used a folder as the target of the copy/move operation. >This rule uses a newsgroup as the target. Note that the name of the target >group and the name of the newsserver where this group is from is separated by >a semicolon.
    Say, you are using two newsservers named "NewsserverOne" and "NewsserverTwo". >Both newsservers have the group "a.binary.group", but while most of the >messages are the same in this group on the two servers, there are some that >are not available on the other server.
    The first line in the preceding sample makes sure that the following rule is >only applied to the "a.binary.group" newsgroup.
    The rule itself simply says to move all articles (since the condition "bytes >%>0" is always met) to the group a.binary.group of NewsserverOne.
    If you retrieve message headers for this group from server NewsserverOne, the >rule is ignored, since all articles go to the correct group anyway. If you >retrieve message headers from NewsserverTwo for this group though, they will >not show up in "a.binary.group (NewsserverTwo)", but in "a.binary.group >(NewsserverOne)".
    The result is that you joined messages from the same group, but different >newsservers into one group in Dialog. This is especially useful for binary >groups to fill missing multipart postings.
    [end quote]

    while that may not be exactly "aggregrating" articles from multiple servers, possibly those "action rules" could be customized to make it work after all?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to VanguardLH on Wed Oct 18 09:22:37 2023
    XPost: news.software.readers

    VanguardLH <V@nguard.lh> wrote:
    Frank Slootweg <this@ddress.is.invalid> wrote:

    Read-state tracking for multiple servers can be done by aggregrating articles from several servers. I've seen people imply that some
    newsreaders can do that, but I have no details (Any takers?). And, as I said above, Hamster can do this as well. Perhaps Leafnode can do the
    same on Linux (the OP uses Linux).

    At that point as a cross-server aggregator, using something like Hamster might suffice. It would collect articles across multiple servers, but
    does Hamster then assign its own article numbers (so it could add the
    Xref header with its article numbers)?

    Yes, for the user/newsreader side, Hamster works like a news server,
    so it indeed assigns its own local article numbers and reports them in a
    Xref header (the old Xref header is retained, but renamed to
    'X-Old-Xref:').

    For the 'real' news server - i.e. News.Individual.Net in my case -,
    Hamster works like a newsreader, i.e. it uses NNTP command like
    LISTGROUP, XOVER, ARTICLE, etc.. So Hamster is a kind of proxy server.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jakob Bohm@21:1/5 to VanguardLH on Wed Oct 18 19:51:52 2023
    XPost: news.software.readers

    On 2023-10-17 05:34, VanguardLH wrote:
    Frank Slootweg <this@ddress.is.invalid> wrote:

    All you have to do is to remember all the hundred of thousands /
    millions of message-ids of the postings you've ever 'read'. How hard
    can *that* be!?

    Thunderbird uses SQLite for the messages databases, one for each folder (newsgroup). While SQLite is fast for small database sizes, it doesn't
    scale well. The bigger the database, the slow queries get, and the
    slowdown is not linear. SQLite recommends once you pass into the
    terabyte range to find a different solution.

    https://www.dbtalks.com/tutorials/learn-sqlite/what-are-the-limitations-of-sqlite
    A SQLite database can have maximum 2147483646 pages. Hence the maximum
    number of tables in a schema cannot reach more than 2147483646. The
    maximum number of rows in a table is 264. The maximum number of columns
    is 32767 in a table.


    According to the official docs at https://www.sqlite.org/limits.html
    this limit is 2 to the 64th power not twohundredandsixtyfour.

    With that and a database table containing only 2 columns: MID and count
    of read flags already set, the database would contain only one small row
    per read message with only the numeric field changing after adding a
    row. Replacing MIDs by fixed-size hashes of MIDs would allow fixed
    length records, thus further easing the DB engine burden. Anyways disk
    space for known messages would hit physical limits before the database
    fills up.

    The goal with this proposal is to handle 2 real world situations:

    1. Cross-posts to multiple newsgroups from one server.

    2. Having multiple newsserver accounts in one xxBird configuration, such
    as news.dotsrc.org (the old SunSite) and eternal-september.org .


    Enjoy

    Jakob
    --
    Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
    Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
    This public discussion message is non-binding and may contain errors.
    WiseMo - Remote Service Management for PCs, Phones and Embedded

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Russ Allbery@21:1/5 to Frank Slootweg on Wed Oct 18 12:14:30 2023
    XPost: news.software.readers

    Frank Slootweg <this@ddress.is.invalid> writes:

    Neither do I, but it's just SMOP (Small Matter Of Programming). All you
    have to do is to remember all the hundred of thousands / millions of message-ids of the postings you've ever 'read'. How hard can *that* be!?
    :-)

    Well, we know exactly how hard this is, because you're describing a news server's history database. News servers also have to remember every
    message they've seen, and they have to do that by message ID. News
    servers limit this to articles within a certain date range and reject all articles older than that date range, but servers that never expire
    essentially keep that data forever.

    There are a bunch of techniques for maintaining that database. It mostly
    uses dedicated data structures optimized for this specific problem, not a SQLite database, although I'd be interested to see someone try with modern SQLite or another SQL database and see if it can be optimized enough,
    since those tools have a lot more general optimizations and way more
    active developers.

    This is certainly *doable*, since it's done all the time, and a modern
    server is often smaller (in disk, memory, and CPU) than a typical laptop,
    let alone desktop. On my news server, it currently takes up about 660MiB
    of disk space, which is smaller than a lot of video games. :) But that's literally every message the server has on disk, and I can assure you that
    I have not read the VAST majority of them, so most history databases for a personal newsreader would be substantially smaller.

    --
    Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>

    Please post questions rather than mailing me directly.
    <https://www.eyrie.org/~eagle/faqs/questions.html> explains why.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to Jakob Bohm on Wed Oct 18 18:35:48 2023
    XPost: news.software.readers

    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    [...]

    According to the official docs at https://www.sqlite.org/limits.html
    this limit is 2 to the 64th power not twohundredandsixtyfour.

    With that and a database table containing only 2 columns: MID and count
    of read flags already set, the database would contain only one small row
    per read message with only the numeric field changing after adding a
    row. Replacing MIDs by fixed-size hashes of MIDs would allow fixed
    length records, thus further easing the DB engine burden. Anyways disk
    space for known messages would hit physical limits before the database
    fills up.

    The goal with this proposal is to handle 2 real world situations:

    As I've mentioned before, but you've ignored, your proposal handles
    only *part* of the 'problems', only for relatively new not-seen
    articles, not for - probably older - other not-yet-seen articles.

    Having repeated that, let me comment on the "2 real world situations":

    1. Cross-posts to multiple newsgroups from one server.

    This is already handled by article-number ranges in .newsrc type
    files.

    That Thunderbird et al inherit a design-flaw from their predecessors
    (as far back as Netscape Communicator or even further), does not change
    this. Fix Thunderbird et al or live with the flaw.

    2. Having multiple newsserver accounts in one xxBird configuration, such
    as news.dotsrc.org (the old SunSite) and eternal-september.org .

    That's only relevant if one's primary NSP is shaky and you need
    another NSP in case as the primary one is down, misses articles, etc.,
    etc.. (Best) Solution: Get a proper NSP.

    But if you can't get a proper NSP, there's - as I mentioned several
    times - Hamster (for Windows) and Leafnode2 (for Linux and Unix-like
    OSs). That solves *both* the "2 real world situations".

    Bottom line: 'Problems' solved. Next problem.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From candycanearter07@21:1/5 to Jakob Bohm on Thu Oct 19 03:11:49 2023
    XPost: news.software.readers

    On 10/18/23 12:51, Jakob Bohm wrote:
    2. Having multiple newsserver accounts in one xxBird configuration, such
    as news.dotsrc.org (the old SunSite) and eternal-september.org .


    XXbird? Does that mean there are more clients based on Thunderbird?
    --
    user <candycane> is generated from /dev/urandom

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to Russ Allbery on Thu Oct 19 13:14:49 2023
    XPost: news.software.readers

    Russ Allbery <eagle@eyrie.org> wrote:
    Frank Slootweg <this@ddress.is.invalid> writes:

    Neither do I, but it's just SMOP (Small Matter Of Programming). All you have to do is to remember all the hundred of thousands / millions of message-ids of the postings you've ever 'read'. How hard can *that* be!? :-)

    Well, we know exactly how hard this is, because you're describing a news server's history database. News servers also have to remember every
    message they've seen, and they have to do that by message ID. News
    servers limit this to articles within a certain date range and reject all articles older than that date range, but servers that never expire essentially keep that data forever.

    There are a bunch of techniques for maintaining that database. It mostly uses dedicated data structures optimized for this specific problem, not a SQLite database, although I'd be interested to see someone try with modern SQLite or another SQL database and see if it can be optimized enough,
    since those tools have a lot more general optimizations and way more
    active developers.

    This is certainly *doable*, since it's done all the time, and a modern
    server is often smaller (in disk, memory, and CPU) than a typical laptop,
    let alone desktop. On my news server, it currently takes up about 660MiB
    of disk space, which is smaller than a lot of video games. :) But that's literally every message the server has on disk, and I can assure you that
    I have not read the VAST majority of them, so most history databases for a personal newsreader would be substantially smaller.

    Hi Russ,

    Just to be sure/clear: I was both joking (hence "SMOP", "How hard can
    *that* be!?" and the smiley) and serious in my responses to those who apparently took my comments seriously.

    And indeed, it's not that hard to do, because it's exactly what a newsserver's history database does. That's why I later talked about
    Hamster (which I use) and Leafnode(2).

    My response was somewhat in jest, because it was a response to Ray's likeminded post:

    It would have never occurred to me that somebody would expect this to
    work across multiple servers ;-)

    So neither Ray, nor I took the original 'problem' seriously. But
    humour and Usenet often don't go well together.

    Hope this clarifies things.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jakob Bohm@21:1/5 to All on Thu Oct 19 16:12:55 2023
    XPost: news.software.readers

    On 2023-10-19 10:11, candycanearter07 wrote:
    On 10/18/23 12:51, Jakob Bohm wrote:
    2. Having multiple newsserver accounts in one xxBird configuration,
    such as news.dotsrc.org (the old SunSite) and eternal-september.org .


    XXbird? Does that mean there are more clients based on Thunderbird?

    Yes, there's at least IceDove, BetterBird (first I heard of it),
    Interlink (dead), Epyrus, Hermopolis, SeaMonkey (and other full Mozilla Communicator implementations).

    I'm currently using Epyrus.



    Enjoy

    Jakob
    --
    Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
    Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
    This public discussion message is non-binding and may contain errors.
    WiseMo - Remote Service Management for PCs, Phones and Embedded

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From candycanearter07@21:1/5 to Jakob Bohm on Thu Oct 19 09:26:33 2023
    XPost: news.software.readers

    On 10/19/23 09:12, Jakob Bohm wrote:
    On 2023-10-19 10:11, candycanearter07 wrote:
    XXbird? Does that mean there are more clients based on Thunderbird?

    Yes, there's at least IceDove, BetterBird (first I heard of it),
    Interlink (dead), Epyrus, Hermopolis, SeaMonkey (and other full Mozilla Communicator implementations).

    I'm currently using Epyrus.

    Thanks, might check those out.
    --
    user <candycane> is generated from /dev/urandom

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jakob Bohm@21:1/5 to Frank Slootweg on Thu Oct 19 16:23:44 2023
    XPost: news.software.readers

    On 2023-10-18 20:35, Frank Slootweg wrote:
    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    [...]


    The goal with this proposal is to handle 2 real world situations:

    As I've mentioned before, but you've ignored, your proposal handles
    only *part* of the 'problems', only for relatively new not-seen
    articles, not for - probably older - other not-yet-seen articles.


    And you keep refusing to understand any proposals that deal with
    imperfect (not necessarily broken) servers.

    Having repeated that, let me comment on the "2 real world situations":

    1. Cross-posts to multiple newsgroups from one server.

    This is already handled by article-number ranges in .newsrc type
    files.

    This assumes the server has similar but more costly logic to identify cross-posted messages arriving at different times from different feeds.

    That Thunderbird et al inherit a design-flaw from their predecessors
    (as far back as Netscape Communicator or even further), does not change
    this. Fix Thunderbird et al or live with the flaw.


    I am not referring to the "each newsgroup for itself" bug.

    2. Having multiple newsserver accounts in one xxBird configuration, such
    as news.dotsrc.org (the old SunSite) and eternal-september.org .

    That's only relevant if one's primary NSP is shaky and you need
    another NSP in case as the primary one is down, misses articles, etc.,
    etc.. (Best) Solution: Get a proper NSP.

    Given the decline of usenet, the need to use more than one NSP is more
    likely to grow than decline. My example specifically mentioned two
    volunteer run NSPs with good reputation but different inclusion
    policies. Another example would be switching to a different NSP or a
    different server at a single NSP while wanting to keep the "read" status
    of messages already received from the previous NSP.



    But if you can't get a proper NSP, there's - as I mentioned several
    times - Hamster (for Windows) and Leafnode2 (for Linux and Unix-like
    OSs). That solves *both* the "2 real world situations".

    That's an arrogant assumption, see above.

    Bottom line: 'Problems' solved. Next problem.



    Enjoy

    Jakob
    --
    Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
    Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
    This public discussion message is non-binding and may contain errors.
    WiseMo - Remote Service Management for PCs, Phones and Embedded

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to Jakob Bohm on Thu Oct 19 15:48:42 2023
    XPost: news.software.readers

    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    On 2023-10-18 20:35, Frank Slootweg wrote:
    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    [...]

    The goal with this proposal is to handle 2 real world situations:

    As I've mentioned before, but you've ignored, your proposal handles
    only *part* of the 'problems', only for relatively new not-seen
    articles, not for - probably older - other not-yet-seen articles.

    And you keep refusing to understand any proposals that deal with
    imperfect (not necessarily broken) servers.

    No, I don't refuse anything and I fully understand what you're saying. However what you seem to be unable (unwilling?) to accept, is that some
    (even many) people in these groups, especially in news.software.nntp,
    have a long and a lot experience in these matters.

    Having repeated that, let me comment on the "2 real world situations":

    1. Cross-posts to multiple newsgroups from one server.

    This is already handled by article-number ranges in .newsrc type
    files.

    This assumes the server has similar but more costly logic to identify cross-posted messages arriving at different times from different feeds.

    Yes, *every* news server has that logic. It's called the history
    database and it's not at all "costly", because it's a special database, specifically designed for this very purpose (see Russ' post). My
    (Hamster) server also has it, because it couldn't be a news server
    without it.

    That Thunderbird et al inherit a design-flaw from their predecessors
    (as far back as Netscape Communicator or even further), does not change this. Fix Thunderbird et al or live with the flaw.

    I am not referring to the "each newsgroup for itself" bug.

    OK.

    2. Having multiple newsserver accounts in one xxBird configuration, such >> as news.dotsrc.org (the old SunSite) and eternal-september.org .

    That's only relevant if one's primary NSP is shaky and you need
    another NSP in case as the primary one is down, misses articles, etc., etc.. (Best) Solution: Get a proper NSP.

    Given the decline of usenet, the need to use more than one NSP is more
    likely to grow than decline. My example specifically mentioned two
    volunteer run NSPs with good reputation but different inclusion
    policies.

    What do you mean by "inclusion policies"? Retention policies or which
    groups they carry?

    Anyway, a proper configuration (for Thunderbird et al) is not to get
    the same groups from multiple servers. If you do want to do that anyway,
    use Hamster/Leafnode(2).

    Another example would be switching to a different NSP or a
    different server at a single NSP while wanting to keep the "read" status
    of messages already received from the previous NSP.

    As said, Hamster/Leafnode(2) solve that problem.

    But if you can't get a proper NSP, there's - as I mentioned several times - Hamster (for Windows) and Leafnode2 (for Linux and Unix-like
    OSs). That solves *both* the "2 real world situations".

    That's an arrogant assumption, see above.

    Why is that "an arrogant assumption"? Because it's a real life,
    working solution to some somewhat farfetched problems, while your
    proposal is a non-existing, partial solution, which is very unlikely to
    be implemented?

    Bottom line: 'Problems' solved. Next problem.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From J.B. Nicholson@21:1/5 to VanguardLH on Tue Oct 24 05:17:05 2023
    XPost: news.software.readers

    VanguardLH <V@nguard.LH> wrote:
    https://www.dbtalks.com/tutorials/learn-sqlite/what-are-the-limitations-of-sqlite

    I read this URL and immediately didn't understand why dbtalks.com is a reasonable place to look up information about or refer others to
    SQLite's limitations.

    A SQLite database can have maximum 2147483646 pages. Hence the maximum
    number of tables in a schema cannot reach more than 2147483646. The
    maximum number of rows in a table is 264. The maximum number of columns
    is 32767 in a table.

    Notice only 264 rows per table.

    I noticed that you seem to have taken that seriously. Something should
    have immediately caught your eye and made you think 'that is not
    reasonable' particularly for a production-ready relational DB as
    widely used as SQLite.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From VanguardLH@21:1/5 to J.B. Nicholson on Tue Oct 24 06:00:26 2023
    XPost: news.software.readers

    "J.B. Nicholson" <jbn@forestfield.org> wrote:

    VanguardLH <V@nguard.LH> wrote:
    https://www.dbtalks.com/tutorials/learn-sqlite/what-are-the-limitations-of-sqlite

    I read this URL and immediately didn't understand why dbtalks.com is a reasonable place to look up information about or refer others to
    SQLite's limitations.

    A SQLite database can have maximum 2147483646 pages. Hence the maximum
    number of tables in a schema cannot reach more than 2147483646. The
    maximum number of rows in a table is 264. The maximum number of columns
    is 32767 in a table.

    Notice only 264 rows per table.

    I noticed that you seem to have taken that seriously. Something should
    have immediately caught your eye and made you think 'that is not
    reasonable' particularly for a production-ready relational DB as
    widely used as SQLite.

    Okay, how about this from SQLite:

    https://www.sqlite.org/limits.html

    2000 columns (fields per record) maximum. Recommended: 100.

    There are possible maximums, and there are feasible maximums based on performance. SQLite is not good for huge or even large databases, and
    even SQLite says that.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From J.B. Nicholson@21:1/5 to VanguardLH on Fri Oct 27 00:11:03 2023
    XPost: news.software.readers

    VanguardLH <V@nguard.LH> wrote:
    Okay, how about this from SQLite:

    https://www.sqlite.org/limits.html

    2000 columns (fields per record) maximum. Recommended: 100.

    I don't understand why you're discussing SQLite's column limits per
    table when I pointed out that you quoted a clearly disreputable source
    about SQLite's row limits per table. And I think that you're
    misstating what https://www.sqlite.org/limits.html says.

    Here's a quote from https://www.sqlite.org/limits.html:
    The default setting for SQLITE_MAX_COLUMN is 2000. You can change it
    at compile time to values as large as 32767. On the other hand, many experienced database designers will argue that a well-normalized
    database will never need more than 100 columns in a table.

    The SQLite.org text is correct in its statement about 100 columns in a
    table but SQLite clearly supports more than an order of magnitude more
    columns per table in its default compiled-in limit. But the
    interesting things in the context of this thread are row limits per
    table, the veracity of the source you picked, and that if one wants to
    know about SQLite one goes to sqlite.org.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Matija Nalis@21:1/5 to J.B. Nicholson on Sun Oct 29 23:11:15 2023
    XPost: news.software.readers

    On Tue, 24 Oct 2023 05:17:05 -0000 (UTC), J.B. Nicholson <jbn@forestfield.org> wrote:
    VanguardLH <V@nguard.LH> wrote:
    Notice only 264 rows per table.

    I noticed that you seem to have taken that seriously. Something should
    have immediately caught your eye and made you think 'that is not
    reasonable' particularly for a production-ready relational DB as
    widely used as SQLite.

    It seems like simple formatting error though, i.e. not supporting superscript correctly.

    So, it is not "264" rows per table, but 2^64 (two to the power of 64, or 18446744073709551616) rows per table.

    Which is evident in other parts of the page too where it says e.g.
    "The current implementation will only support a string or BLOB length up to 231-1 or 2147483647"

    231-1 is 230, and obviously nowhere near 2147483647. 2^31-1 however is exactly 2147483647.


    --
    Opinions above are GNU-copylefted.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Jakob Bohm@21:1/5 to Frank Slootweg on Tue Oct 31 08:13:58 2023
    XPost: news.software.readers

    On 2023-10-19 17:48, Frank Slootweg wrote:
    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    On 2023-10-18 20:35, Frank Slootweg wrote:
    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    [...]

    Having repeated that, let me comment on the "2 real world situations": >>>
    1. Cross-posts to multiple newsgroups from one server.

    This is already handled by article-number ranges in .newsrc type
    files.

    This assumes the server has similar but more costly logic to identify
    cross-posted messages arriving at different times from different feeds.

    Yes, *every* news server has that logic. It's called the history
    database and it's not at all "costly", because it's a special database, specifically designed for this very purpose (see Russ' post). My
    (Hamster) server also has it, because it couldn't be a news server
    without it.
    That Thunderbird et al inherit a design-flaw from their
    predecessors
    (as far back as Netscape Communicator or even further), does not change
    this. Fix Thunderbird et al or live with the flaw.

    I am not referring to the "each newsgroup for itself" bug.

    OK.

    2. Having multiple newsserver accounts in one xxBird configuration, such >>>> as news.dotsrc.org (the old SunSite) and eternal-september.org .

    That's only relevant if one's primary NSP is shaky and you need
    another NSP in case as the primary one is down, misses articles, etc.,
    etc.. (Best) Solution: Get a proper NSP.

    Given the decline of usenet, the need to use more than one NSP is more
    likely to grow than decline. My example specifically mentioned two
    volunteer run NSPs with good reputation but different inclusion
    policies.

    What do you mean by "inclusion policies"? Retention policies or which groups they carry?

    I mean all of groups carried, retention and messages filtered .


    Anyway, a proper configuration (for Thunderbird et al) is not to get
    the same groups from multiple servers. If you do want to do that anyway,
    use Hamster/Leafnode(2).


    I am talking about getting group A from server A and group B from server
    B, then someone crossposts to groups A and B via a server C that carries
    both group A and group B. This means that neither server A nor server B
    will have server side ability to report the crosspost via per-server IDs
    to a client that gets A from A and B from B.

    Another example would be switching to a different NSP or a
    different server at a single NSP while wanting to keep the "read" status
    of messages already received from the previous NSP.

    As said, Hamster/Leafnode(2) solve that problem.

    But we are discussing a client which is neither named Hamster nor
    Leafnode.

    In my case, I was caught unprepared when Giganews suddenly turned off
    their Europe server, forcing me to switch to their global server and
    loose all message states in the existing XxxBird client. This may have
    been caused by the "each newsgroup for itself" but in the user interface
    it was casued by having to resubscribe to the groups when the server
    name changed.


    But if you can't get a proper NSP, there's - as I mentioned several
    times - Hamster (for Windows) and Leafnode2 (for Linux and Unix-like
    OSs). That solves *both* the "2 real world situations".

    That's an arrogant assumption, see above.

    Why is that "an arrogant assumption"? Because it's a real life,
    working solution to some somewhat farfetched problems, while your
    proposal is a non-existing, partial solution, which is very unlikely to
    be implemented?


    And you assume that bad NSPs don't exist and don't have innocent users
    who need their multipurpose client to handle the problem. Running
    additional software before the need arises is a non-solution, unlike
    putting fixes into software that gets installed as part of routine
    software updates .



    Enjoy

    Jakob
    --
    Jakob Bohm, CIO, Partner, WiseMo A/S. https://www.wisemo.com
    Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10
    This public discussion message is non-binding and may contain errors.
    WiseMo - Remote Service Management for PCs, Phones and Embedded

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Frank Slootweg@21:1/5 to Jakob Bohm on Tue Oct 31 11:00:19 2023
    XPost: news.software.readers

    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    On 2023-10-19 17:48, Frank Slootweg wrote:
    Jakob Bohm <jb-usenet@wisemo.com.invalid> wrote:
    On 2023-10-18 20:35, Frank Slootweg wrote:

    [Most endless repetitions deleted.]

    But if you can't get a proper NSP, there's - as I mentioned several >>> times - Hamster (for Windows) and Leafnode2 (for Linux and Unix-like
    OSs). That solves *both* the "2 real world situations".

    That's an arrogant assumption, see above.

    Why is that "an arrogant assumption"? Because it's a real life,
    working solution to some somewhat farfetched problems, while your
    proposal is a non-existing, partial solution, which is very unlikely to
    be implemented?

    And you assume that bad NSPs don't exist and don't have innocent users
    who need their multipurpose client to handle the problem. Running
    additional software before the need arises is a non-solution, unlike
    putting fixes into software that gets installed as part of routine
    software updates .

    Again, please don't put words in my mouth! Of course I know full well
    that "bad NSPs" exist, that's why I said what I said and gave
    alternative solution which *actually exist* and *actually work* (and use
    a "proper NSP" myself).

    You OTOH, continue to talk about paper tigers. As I said, your
    proposal is very unlikely to be implemented (in Thunderbird). If you
    think otherwise, prove me/us wrong by putting your proposal to the
    people who can actually implement it, i.e. Mozilla, instead of
    footstamping in these groups, where it won't change anything.

    Let's agree to disagree.

    (AFAIC,) EOD.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)