open https://gitlab.synchro.net/main/sbbs/-/issues/541

Some characters in messages (such as the apostrophe used apparently by Apple Mail or other software on iPads) don't display properly in DDMsgReader (and show up as odd-looking characters). This may be related to MIME encoding? msglist.js seems to display them properly, so maybe that can be used as an example.Some notes on dumping a message, which may help in debugging:<DigitalMan> 'smbutil v /sbbs/data/mail' will dump all the headers<DigitalMan> 'smbutil r /sbbs/data/mail' will read the message text<DigitalMan> if you know the message number, you can skip all the other messages easily, or just pipe output to a file and edit to suit<nelgin> I did, thank you. uploaded bademail.txt and badheader.txt and I'm using SyncTerm, of course :)<nelgin> The sender was using MuffinTerm on an iphone fwiw.<DigitalMan> the terminals (on the sending or receiving side) shouldn't really matter<DigitalMan> the sender was using Apple Mail (2.3445.9.7) according to those headers<git_nelgin> GitLab issue from Nigel Reed<git_nelgin> open https://gitlab.synchro.net/main/sbbs/-/issues/540<git_nelgin> add ssh passphrase authentication<nelgin> Ah right, he was using muffinterm to access the bbs but yeah, that was via gmail. My bad.<DigitalMan> Can you try viewing that same message using 'smbutil -r r /sbbs/data/mail' ? Does it look the same or add a like of MIME noise to the output?<DigitalMan> -r = display raw message body text (not MIME-decoded)One of DM's recent commits:git_rswindell> https://gitlab.synchro.net/main/sbbs/-/commit/1ca7f708fe87ea825cb0e5af<git_rswindell> Modified src/smblib/smbtxt.c<git_rswindell> Fix decoding of tab-indented 'charset' of multi-part MIME text parts<git_rswindell> Apple Mail apparently uses tabs to indent the charset of nested MIME parts:
--- SBBSecho 3.20-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/541#note_3414

Does DDMsgReader translate UTF-8 encoded messages to CP437 or ASCII when appropriate for the viewing terminal?
--- SBBSecho 3.20-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/541#note_3420

I did add some code quite a while ago that used utf8_cp437() (from utf8_cp437.js) for that purpose, but it seems that wasn't enough. I hadn't gotten back to that until today.I had a look at what msglist.js is doing, and it has the function msg_pmode(). It looks like if the message header's is_utf8 property is true, then it sets the P_UTF8 mode bit, and it looks like that's required to pass to console.print() in order for it to look valid on a terminal.
--- SBBSecho 3.20-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/541#note_3421

Replace "a terminal" with "a non-UTF-8" terminal and yeah, that'd be true. Some terminals support UTF-8 just fine (and in those cases, don't support CP437).
--- SBBSecho 3.20-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

close https://gitlab.synchro.net/main/sbbs/-/issues/541
--- SBBSecho 3.20-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

reopen https://gitlab.synchro.net/main/sbbs/-/issues/541

Some characters in messages (such as the apostrophe used apparently by Apple Mail or other software on iPads) don't display properly in DDMsgReader (and show up as odd-looking characters). This may be related to MIME encoding? msglist.js seems to display them properly, so maybe that can be used as an example.Some notes on dumping a message, which may help in debugging:```<DigitalMan> 'smbutil v /sbbs/data/mail' will dump all the headers<DigitalMan> 'smbutil r /sbbs/data/mail' will read the message text<DigitalMan> if you know the message number, you can skip all the other messages easily, or just pipe output to a file and edit to suit<nelgin> I did, thank you. uploaded bademail.txt and badheader.txt and I'm using SyncTerm, of course :)<nelgin> The sender was using MuffinTerm on an iphone fwiw.<DigitalMan> the terminals (on the sending or receiving side) shouldn't really matter<DigitalMan> the sender was using Apple Mail (2.3445.9.7) according to those headers<nelgin> Ah right, he was using muffinterm to access the bbs but yeah, that was via gmail. My bad.<DigitalMan> Can you try viewing that same message using 'smbutil -r r /sbbs/data/mail' ? Does it look the same or add a like of MIME noise to the output?<DigitalMan> -r = display raw message body text (not MIME-decoded)```One of DM's recent commits:```<git_rswindell> https://gitlab.synchro.net/main/sbbs/-/commit/1ca7f708fe87ea825cb0e5af<git_rswindell> Modified src/smblib/smbtxt.c<git_rswindell> Fix decoding of tab-indented 'charset' of multi-part MIME text parts<git_rswindell> Apple Mail apparently uses tabs to indent the charset of nested MIME parts:```
--- SBBSecho 3.20-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/541#note_3437

Issue not fixed by latest commits.
--- SBBSecho 3.20-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/541#note_3552

Have you found anything new regarding this issue? I'm wondering if this can be closed.
--- SBBSecho 3.20-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/541#note_3553

Closing, as this appears to be fixed and there's no new information for this issue.
--- SBBSecho 3.20-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)