1. Forum
  2. FidoNet
  3. SYNC SYSOPS

  • Rspamd integration for Synchronet BBS

    From Icedawg@1:103/705 to All on Mon Dec 28 05:47:59 2020
    Hi,

    Yesterday I've started playing with Synchronet & have made some first steps towards integrating Rspamd, an alternative to SpamAssassin.

    The result is here: https://github.com/fatalbanana/synchronet-rspamd

    What should work:

    * Scanning files by name or sending of data
    * `reject` and `soft reject` actions informed by Rspamd (for rejecting high-probability spam and greylisting/ratelimits/etc)

    What's missing:

    * Outbound scanning (add `User` header if appropriate)
    * Handling for `add header` and `rewrite subject` actions
    * Handling of header addition/removal; DKIM signatures

    If someone may find this interesting/useful I'd appreciate if you'd try it out and fork/send PRs.

    Best Regards,
    Andrew Lewis
    --- SBBSecho 3.12-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Icedawg on Mon Dec 28 14:32:39 2020
    Re: Rspamd integration for Synchronet BBS
    By: Icedawg to All on Mon Dec 28 2020 05:47 am

    Hi,

    Yesterday I've started playing with Synchronet & have made some first steps towards integrating Rspamd, an alternative to SpamAssassin.

    The result is here: https://github.com/fatalbanana/synchronet-rspamd

    Very cool!

    Wouldn't mind having this in the Synchronet git and/or wiki when it's ready.

    One thing I noticed, in your rspam.js the example mailproc.ini syntax isn't right. I think you mean some more like this:

    [rspamc]
    Command=rspamc.js -d 127.0.0.1 -p 11333
    --
    digital man

    Rush quote #49:
    Some will sell their dreams for small desires or lose the race to rats
    Norco, CA WX: 49.1øF, 89.0% humidity, 0 mph S wind, 0.49 inches rain/24hrs
    --- SBBSecho 3.12-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Icedawg@1:103/705 to Digital Man on Tue Dec 29 15:25:25 2020
    Re: Rspamd integration for Synchronet BBS
    By: Digital Man to Icedawg on Mon Dec 28 2020 02:32 pm

    Hi Rob,

    Those were some funny examples indeed, I've corrected them, thank you.

    I'd be happy for you to have this stuff upstream some day too. ;)

    So far I've doubts that outbound filtering & DKIM signing will be particularly useful given that it works merely for SMTP & I'm unsure how to address this.

    Other doubt is about how to deal with parsing/rewriting message headers - I expect I could do it in JS, I just hoped to avoid that.

    Best,
    -AL.

    ---
    þ Synchronet þ HERP DERP BBS
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Icedawg on Tue Dec 29 08:13:05 2020
    Re: Rspamd integration for Synchronet BBS
    By: Icedawg to Digital Man on Tue Dec 29 2020 03:25 pm

    Re: Rspamd integration for Synchronet BBS
    By: Digital Man to Icedawg on Mon Dec 28 2020 02:32 pm

    Hi Rob,

    Those were some funny examples indeed, I've corrected them, thank you.

    I'd be happy for you to have this stuff upstream some day too. ;)

    So far I've doubts that outbound filtering & DKIM signing will be particularly useful given that it works merely for SMTP & I'm unsure how to address this.

    Other doubt is about how to deal with parsing/rewriting message headers - I expect I could do it in JS, I just hoped to avoid that.

    Do you mean for outbound mail? There's really no JS-hook for that. It might be possible to simply have an event that scanned the mail base and either deleted or modified outbound Internet email that hadn't yet been delivered, but there's still a bit of a race-condition there.

    I'm not sure how much sysops really need/want outbound SPAM filtering to be honest. It's never been requested or discussed before.
    --
    digital man

    Sling Blade quote #4:
    Doyle: wimpy-ass kids or mental retards.. she got one of each livin' with her. Norco, CA WX: 47.4øF, 92.0% humidity, 0 mph S wind, 0.71 inches rain/24hrs
    --- SBBSecho 3.12-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Icedawg@1:103/705 to Digital Man on Tue Dec 29 19:09:16 2020
    Re: Rspamd integration for Synchronet BBS
    By: Digital Man to Icedawg on Tue Dec 29 2020 08:13 am

    Hello,

    Yes, the idea is to support scanning of outbound messages- at least to the extent that is possible- so, messages received over SMTP. It belatedly occurred to me that messages might not arrive over SMTP & that I wouldn't know how to deal with these.

    Commercial e-mail providers likely suffer problems with compromised accounts sending spam; no idea about BBS operators but I could believe they likely don't (and would guess that if they did they would most likely see abuse over SMTP).

    Best,
    -AL

    ---
    þ Synchronet þ HERP DERP BBS
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Icedawg on Tue Dec 29 15:11:55 2020
    Re: Rspamd integration for Synchronet BBS
    By: Icedawg to Digital Man on Tue Dec 29 2020 07:09 pm

    Re: Rspamd integration for Synchronet BBS
    By: Digital Man to Icedawg on Tue Dec 29 2020 08:13 am

    Hello,

    Yes, the idea is to support scanning of outbound messages- at least to the extent that is possible- so, messages received over SMTP. It belatedly occurred to me that messages might not arrive over SMTP & that I wouldn't know how to deal with these.

    An outbound (relayed) message submitted via SMTP would be scanned during the initial receipt, so that would already be supported by the mailproc method.
    --
    digital man

    Synchronet "Real Fact" #6:
    Synchronet version 3 for Linux and FreeBSD development began in 2001.
    Norco, CA WX: 57.3øF, 43.0% humidity, 11 mph ESE wind, 0.53 inches rain/24hrs --- SBBSecho 3.12-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Icedawg@1:103/705 to Digital Man on Wed Dec 30 01:28:02 2020
    Re: Rspamd integration for Synchronet BBS
    By: Digital Man to Icedawg on Tue Dec 29 2020 03:11 pm

    Hi,

    Yes, I'm happy that messages relayed via SMTP will be scanned on receipt, I just lament that messages delivered by being saved to the message base would not be. Though I can personally live with it fine.

    Best,
    -AL.

    ---
    þ Synchronet þ HERP DERP BBS
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Icedawg@1:103/705 to Digital Man on Wed Dec 30 01:43:21 2020
    Re: Rspamd integration for Synchronet BBS
    By: Icedawg to Digital Man on Wed Dec 30 2020 01:28 am

    Yes, I'm happy that messages relayed via SMTP will be scanned on receipt, I
    just lament that messages delivered by being saved to the message base
    would not be. Though I can personally live with it fine.

    Uh, otherwise saved to the message base I might rather say (since SMTP does that too) - such as via web or SSH.

    Best,
    -AL.

    ---
    þ Synchronet þ HERP DERP BBS
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Icedawg on Tue Dec 29 16:05:51 2020
    Re: Rspamd integration for Synchronet BBS
    By: Icedawg to Digital Man on Wed Dec 30 2020 01:28 am

    Re: Rspamd integration for Synchronet BBS
    By: Digital Man to Icedawg on Tue Dec 29 2020 03:11 pm

    Hi,

    Yes, I'm happy that messages relayed via SMTP will be scanned on receipt, I just lament that messages delivered by being saved to the message base
    would
    not be. Though I can personally live with it fine.

    Yeah, for the latter, you could have an event that ran and scanned the mailbase for outbound Internet mail and did the filtering then. There's still that potential race condition I mentioned.
    --
    digital man

    Sling Blade quote #25:
    Karl: they seen fit to put me in here and here I've been a great long while. Norco, CA WX: 58.4øF, 38.0% humidity, 7 mph S wind, 0.44 inches rain/24hrs
    --- SBBSecho 3.12-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Icedawg@1:103/705 to Digital Man on Wed Dec 30 02:44:53 2020
    Re: Rspamd integration for Synchronet BBS
    By: Digital Man to Icedawg on Tue Dec 29 2020 04:05 pm

    Yeah, for the latter, you could have an event that ran and scanned the
    mailbase for outbound Internet mail and did the filtering then. There's
    still
    that potential race condition I mentioned.

    Better it should be possible to intentionally cause message saving to fail via some hooks. I'm not suggesting to add some but I wouldn't be against. ;)

    Best,
    -AL.

    ---
    þ Synchronet þ HERP DERP BBS
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Digital Man@1:103/705 to Icedawg on Tue Dec 29 17:10:09 2020
    Re: Rspamd integration for Synchronet BBS
    By: Icedawg to Digital Man on Wed Dec 30 2020 02:44 am

    Re: Rspamd integration for Synchronet BBS
    By: Digital Man to Icedawg on Tue Dec 29 2020 04:05 pm

    Yeah, for the latter, you could have an event that ran and scanned the mailbase for outbound Internet mail and did the filtering then. There's still
    that potential race condition I mentioned.

    Better it should be possible to intentionally cause message saving to fail via some hooks. I'm not suggesting to add some but I wouldn't be against.
    ;)

    In theory, but there are far too many ways in which a message may be added to the 'mail' base to hook into them all.

    Another approach would be to only send (via SMTP) messages that have been "scanned". An event (e.g. in JS) could perform the scanning and either set the (as of yet undefined) "scanned" attribute flag on a message or bounce it back to the sender. And the SMTP sendmail thread could simply ignore any unscanned mail.
    --
    digital man

    Rush quote #44:
    The shifting shafts of shining weave the fabric of their dreams
    Norco, CA WX: 56.2øF, 40.0% humidity, 3 mph SSW wind, 0.22 inches rain/24hrs --- SBBSecho 3.12-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Fernando Toledo@4:902/26 to Digital Man on Wed Dec 30 03:44:13 2020
    El 29/12/20 a las 20:11, Digital Man escribió:
    Re: Rspamd integration for Synchronet BBS
    By: Icedawg to Digital Man on Tue Dec 29 2020 07:09 pm

    > Re: Rspamd integration for Synchronet BBS
    > By: Digital Man to Icedawg on Tue Dec 29 2020 08:13 am
    >
    > Hello,
    >
    > Yes, the idea is to support scanning of outbound messages- at least to
    the
    > extent that is possible- so, messages received over SMTP. It belatedly
    > occurred to me that messages might not arrive over SMTP & that I
    wouldn't
    > know how to deal with these.

    An outbound (relayed) message submitted via SMTP would be scanned during the
    initial receipt, so that would already be supported by the mailproc method.

    outbound hooks events can be useful for rules for rewrite or limits
    examples:

    * add custom headers

    * add global tail/signature "our enterprice move to new office.. and
    blabla"

    * check policy to limit recipient users / domains

    * analsys / stats


    pd: i love hook events everyhere =)
    --- SBBSecho 3.12-Linux
    * Origin: Dock Sud BBS - http://bbs.docksud.com.ar (4:902/26)
  • From Icedawg@1:103/705 to All on Wed Dec 30 16:39:47 2020
    Re: Rspamd integration for Synchronet BBS
    By: Digital Man to Icedawg on Tue Dec 29 2020 05:10 pm


    Hi,

    So I've implemented support for setting spam flag on Rspamd's `add header` action and am calling this version 1. Support for a small number of advanced features: namely DKIM signing, adding/removing headers & the `rewrite subject` action are unimplemented but it should be fine for most purposes.

    Perhaps, it might not even ruin your day if you will try to use it, I'm not sure. ;) Feedback/patches/forks welcome.

    Here is the URL again: https://github.com/fatalbanana/synchronet-rspamd

    Best,
    -AL.

    ---
    þ Synchronet þ HERP DERP BBS
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)