1. Forum
  2. FidoNet
  3. SYNC SYSOPS

  • webv4: consider translating UTF-8 before posting message

    From Andre Robitaille@1:103/705 to GitLab issue in main/sbbs on Sat Apr 2 08:28:14 2022
    open https://gitlab.synchro.net/main/sbbs/-/issues/386

    Most BBS software and clients, including Mystic, don't understand UTF-8. So when some systems post messages with the web client, their use of single right quote is indecipherable for people. Synchronet and SyncTERM do not have a problem posting/displaying.I know at least iOS and iPad OS have the posting issue.https://www.fileformat.info/info/unicode/char/2019/index.htm
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell@1:103/705 to GitLab note in main/sbbs on Sun Apr 3 16:19:51 2022
    https://gitlab.synchro.net/main/sbbs/-/issues/386#note_2512

    Synchronet already indicates the codepage/character set correctly in the metadata of messages exported to networks (e.g. the "CHRS" kludge line for FidoNet Technology Networks).I'm pretty sure Mystic and several other BBSes know how to recognize and handle UTF-8 message text (e.g. convert to ASCII or CP437 as appropriate) that is self-identified in this manner.Since there are many ways a message can make its way into a message base (webv4 is just one of them), the place to translate would be later, e.g. when the message is exported to a system/network that is known to not support UTF-8.Each sub-board already has an "Export ASCII Only" network toggle option. Looking at SBBSecho, it doesn't appear that it would correctly convert UTF-8 to "ASCII only" when that option is enabled (in SCFG), so that should probably be fixed. Would that meet your needs?
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Andre Robitaille@1:103/705 to GitLab note in main/sbbs on Sun Apr 3 17:09:55 2022
    https://gitlab.synchro.net/main/sbbs/-/issues/386#note_2514

    There's something wonky going on even for Mystic people who use SyncTERM as their client.I don't know that there's any particularly clean way to do it. Does export ASCII also strip ANSI colors and whatnot?But yes, I suppose just exporting ASCII only for fsx, and probably for Fido, makes the most sense since they're the ones complaining about it the most.
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Rob Swindell@1:103/705 to GitLab note in main/sbbs on Sun Apr 3 17:39:27 2022
    https://gitlab.synchro.net/main/sbbs/-/issues/386#note_2515

    SyncTERM doesn't support UTF-8, so I'm not clear how that would play a role at all.Mystic should (and I'm pretty sure does) know when the user's client does not support UTF-8 and performs on-the-fly translation as needed. Of course, that is something to discuss or resolve with the Mystic author if that's not working as expected.Exporting "ASCII only" from Synchronet/SBBSecho would limit the exported characters to the ASCII values SPACE through DEL (plus carriage-return and line-feed). It would convert the ESC character used in ANSI escape sequences to a '.' and convert CP437 characters to their closest ASCII equivalent/approximation. So yeah, that'd pretty much hose any ANSI messages that were being exported.
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Andre Robitaille@1:103/705 to GitLab note in main/sbbs on Sun Apr 3 17:57:28 2022
    https://gitlab.synchro.net/main/sbbs/-/issues/386#note_2516

    I dunno. I skimmed your Sync UTF-8 video for a few minutes and remember you saying that it didn't. And yet everyone using Sync/SyncTERM together could see the original message with the UTF-8 character. People on other BBS software saw garbled characters in the original message, which I could see in their replies. (My replies with Sync terminal got translated back to a standard apostrophe).Mystic weren't the only people complaining, they were just the largest crowd. But regardless, your response is a purist one that matches what I said on a board recently... We can't just freeze everything in time because people want to run vintage hardware and software. Running that stuff comes with known tradeoffs.Point being, I agree. The problem isn't Sync, so doing what I'm asking in this ticket is a kludge.
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Andre Robitaille@1:103/705 to GitLab issue in main/sbbs on Sun Apr 3 17:57:28 2022
    close https://gitlab.synchro.net/main/sbbs/-/issues/386
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Michael J. Ryan@1:103/705 to GitLab note in main/sbbs on Mon Apr 4 18:53:33 2022
    https://gitlab.synchro.net/main/sbbs/-/issues/386#note_2523

    @andreMKE do you know if this is when posting via the Runemaster web interface, or eChicken's web4 version?It could just be taking the bytes as-is and not setting the UTF-8 marker appropriate on message creation. Would have to look into the code itself to see/experiment on this... likely it probably *should* be utf-8 for what gets posted via web-ui.
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Andre Robitaille@1:103/705 to GitLab note in main/sbbs on Thu Apr 14 18:11:50 2022
    https://gitlab.synchro.net/main/sbbs/-/issues/386#note_2546

    I haven't tested against the runemaster version. I do know that it happens with the webv4, but I don't know why runemaster would have a different behavior.
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)