1. Forum
  2. Usenet
  3. NEWS.SOFTWARE.NNTP

  • Client-Side Bridge to Web Forums?

    From Computer Nerd Kev@21:1/5 to All on Tue Oct 3 09:03:48 2023
    Web forums keep annoying me more and more with bloated interfaces,
    so I'm using them less and less, yet there's less and less to read
    on Usenet too. So lately I've been considering, not entirely
    seriously, writing a program to scape specific web forums and
    generate a news spool with the content of the forum's latest posts
    for me to read (either locally, or perhaps remotely via NNTP).

    Has anyone done this before? I know there are various web forum
    platforms that support NNTP server-side, but I'm talking about web
    forums hosted by other people who I have no association or
    influence with. Has anyone done something that's purely a
    client-side implementation?

    Simple Machines Forum and Discourse are prime targets for me, maybe
    phpBB too. Most don't have RSS enabled, or the feed only shows the
    start of new posts. The ideal would be a system supporting scrapers
    for multiple forum platforms which can be easily extended.

    Support for posting would be nice, but read-only access in a news
    reader (Tin) would be better than nothing.

    --
    __ __
    #_ < |\| |< _#

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Computer Nerd Kev@21:1/5 to rek2@hispagatos.org.invalid on Tue Oct 3 11:48:39 2023
    rek2 hispagatos <rek2@hispagatos.org.invalid> wrote:
    On 2023-10-02, Computer Nerd Kev <not@telling.you.invalid> wrote:
    Web forums keep annoying me more and more with bloated interfaces,
    so I'm using them less and less, yet there's less and less to read
    on Usenet too. So lately I've been considering, not entirely
    seriously, writing a program to scape specific web forums and
    generate a news spool with the content of the forum's latest posts
    for me to read (either locally, or perhaps remotely via NNTP).

    this is a good idea, but consider doing it from ethical forums
    so let me explain, instead of reddit, use lemmy that is more interesting
    and is the open/libre reddit and part of the fediverse.
    since is a forum/threads style will be much easier I think...

    I don't have much interest in the Reddit sorts of things, although
    I've noticed projects along similar lines for them. It's dedicated
    web forums for specific software projects or enthusiest groups that
    I'd like to access and combine into one interface through my news
    reader. They've got the content that I'd like to read.

    Note that whereas Reddit, and presumably alternatives, apparantly
    has/had a public API, the forum implementations that I'm looking at
    will probably need to have all content scraped from the HTML. That
    makes the process more difficult and fragile. This is much of the
    reason why I'm not really committed to doing it all myself, but if
    there's an existing project to build from then that might make the
    amount of effort required more attractive.

    --
    __ __
    #_ < |\| |< _#

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From rek2 hispagatos@21:1/5 to Computer Nerd Kev on Tue Oct 3 01:31:31 2023
    On 2023-10-02, Computer Nerd Kev <not@telling.you.invalid> wrote:
    Web forums keep annoying me more and more with bloated interfaces,
    so I'm using them less and less, yet there's less and less to read
    on Usenet too. So lately I've been considering, not entirely
    seriously, writing a program to scape specific web forums and
    generate a news spool with the content of the forum's latest posts
    for me to read (either locally, or perhaps remotely via NNTP).

    this is a good idea, but consider doing it from ethical forums
    so let me explain, instead of reddit, use lemmy that is more interesting
    and is the open/libre reddit and part of the fediverse.
    since is a forum/threads style will be much easier I think...



    Has anyone done this before? I know there are various web forum
    platforms that support NNTP server-side, but I'm talking about web
    forums hosted by other people who I have no association or
    influence with. Has anyone done something that's purely a
    client-side implementation?

    I just finished beta of a bot that will check
    my mastodon feed and if I tag #2usenet it will post also to
    the newsgroups I told him in the config.ini file .


    PD: Writing as I speak a simple usenet client
    for Android in GO/GOLANG

    if we had a go newsgroup I will had posted my bot there and such :/

    Happy Hacking
    ReK2

    --
    - {gemini,https}://{,rek2.}hispagatos.org - mastodon: @rek2@hispagatos.space
    - [https|gemini]://2600.Madrid - https://hispagatos.space/@rek2
    - https://keyoxide.org/A31C7CE19D9C58084EA42BA26C0B0D11E9303EC5

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Computer Nerd Kev@21:1/5 to Computer Nerd Kev on Tue Oct 3 20:06:10 2023
    Computer Nerd Kev <not@telling.you.invalid> wrote:
    Note that whereas Reddit, and presumably alternatives, apparantly
    has/had a public API, the forum implementations that I'm looking at
    will probably need to have all content scraped from the HTML. That
    makes the process more difficult and fragile.

    Actually Discourse does have an API, so it might be a good one to
    start out with:
    https://docs.discourse.org/

    Then again it doesn't seem easy (possible?) to get a list of recent
    posts unless you have "an API key from the admin panel", which I
    won't have.

    --
    __ __
    #_ < |\| |< _#

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Marc Olschok@21:1/5 to All on Tue Oct 3 21:30:36 2023
    On Tue, 03 Oct 2023 01:03:48 Computer Nerd Kev wrote:
    Web forums keep annoying me more and more with bloated interfaces,
    so I'm using them less and less, yet there's less and less to read
    on Usenet too. So lately I've been considering, not entirely
    seriously, writing a program to scape specific web forums and
    generate a news spool with the content of the forum's latest posts
    for me to read (either locally, or perhaps remotely via NNTP).

    Has anyone done this before? I know there are various web forum
    platforms that support NNTP server-side, but I'm talking about web
    forums hosted by other people who I have no association or
    influence with. Has anyone done something that's purely a
    client-side implementation?
    [...]
    Support for posting would be nice, but read-only access in a news
    reader (Tin) would be better than nothing.

    Would news.gmane.io be suitable for you? I use it mainly for
    mailing list (groups starting with gmane.*), but it also carries
    articles from web-based blogs (groups starting with gwene.*).
    The latter often only contain only teasers and links to the full
    article, but I find that reading with Tin and invoking
    w3m on the posting is sufficiently non-bloated.

    --
    M.O.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Computer Nerd Kev@21:1/5 to Marc Olschok on Wed Oct 4 08:07:36 2023
    Marc Olschok <nobody@nowhere.invalid> wrote:
    Would news.gmane.io be suitable for you? I use it mainly for
    mailing list (groups starting with gmane.*), but it also carries
    articles from web-based blogs (groups starting with gwene.*).

    Gmane is great for mailing lists and RSS feeds, but subjects that
    have gone quiet or never appeared on Usenet often don't have active
    mailing lists either. I went for a hunt through the groups list on
    Gmane recently with this in mind, but only came back with one more
    mailing list, also fairly inactive, and only vaguely interesting to
    me.

    If forums had complete RSS feeds, that might be good enough to use
    in Gmane, but the ones I'm interested in either don't offer RSS at
    all or only only show the beginning of posts, which makes it
    useless.

    I've now discovered forum-dl, which gets close to what I'm looking
    for:
    https://github.com/mikwielgus/forum-dl

    It does seem to be aimed at one-time full downloads of forum
    contents rather than regularly checking for new posts, and I don't
    like Python. But at least it's an example of how to extract the
    important data fields from all the forum platforms that I'm
    interested in, and it supports output in MH format which is much
    like the news spool format.

    --
    __ __
    #_ < |\| |< _#

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)