[...] Other than hoping trn would some day have yenc support,
In separate binary groups, eventually I will see an article number save to a >file name, but I am back at a prompt on the same line, no error messages, >nothing obvious when I examin an end of that file. I can seemingly grab that >same article in Alpine.
[...]
Vision of the final result (no particular order):
- True STARTTLS/NNTPS support
- Curses windows used to show all the various things:
- newsgroup selection
- article selection
- thread selection
- KILL files
- etc.
- A GUI for reading news that's as convenient as the trn TUI.
- Let you manipulate the various windows like people do in vim/emacs
allowing you to choose what is on-screen and where
- Client should be asynchronously advancing through the unread articles
at the "speed of light" to apply scoring/threading/KILL file processing
as far ahead as possible while you read.
- As easy to build on Windows as it is on *nix.
I'm not using trn myself, but in the german hierarchy users sometimes
break things with trn. Better support for MIME would be nice.
I'm not using trn myself, but in the german hierarchy users sometimes
break things with trn. Better support for MIME would be nice.
In news.software.readers, Michael Bäuerle <michael.baeuerle@gmx.net> wrote: >> I'm not using trn myself, but in the german hierarchy users sometimes
break things with trn. Better support for MIME would be nice.
The acli fork of trn vastly improves charset support.
Michael Bauerle <michael.baeuerle@gmx.net> wrote:
I'm not using trn myself, but in the german hierarchy users sometimes
break things with trn. Better support for MIME would be nice.
When I declare a character set, I just copy MIME headers into the
article. I can do it with a macro. The headers are right there for the
user to edit.
"Adam H. Kerman" <ahk@chinet.com> spake:
Michael Bauerle <michael.baeuerle@gmx.net> wrote:
I'm not using trn myself, but in the german hierarchy users sometimes >>>break things with trn. Better support for MIME would be nice.
When I declare a character set, I just copy MIME headers into the
article. I can do it with a macro. The headers are right there for the
user to edit.
Presumably the problem is that the reply doesn't contain the necessary
MIME headers that are relevant for the quoed material or for the
content of the reply?
legalize+jeeves@mail.xmission.com (Richard) wrote:
"Adam H. Kerman" <ahk@chinet.com> spake:
Michael Bauerle <michael.baeuerle@gmx.net> wrote:
I'm not using trn myself, but in the german hierarchy users sometimes >>>>break things with trn. Better support for MIME would be nice.
When I declare a character set, I just copy MIME headers into the >>>article. I can do it with a macro. The headers are right there for the >>>user to edit.
Presumably the problem is that the reply doesn't contain the necessary
MIME headers that are relevant for the quoed material or for the
content of the reply?
I don't see why that matters. You're expecting the newsreader to do
something it cannot do and no newsreader can.
Adam H. Kerman <ahk@chinet.com> spake:
legalize+jeeves@mail.xmission.com (Richard) wrote:
Adam H. Kerman <ahk@chinet.com> spake:
Michael Bauerle <michael.baeuerle@gmx.net> wrote:
I'm not using trn myself, but in the german hierarchy users sometimes >>>>>break things with trn. Better support for MIME would be nice.
When I declare a character set, I just copy MIME headers into the >>>>article. I can do it with a macro. The headers are right there for the >>>>user to edit.
Presumably the problem is that the reply doesn't contain the necessary >>>MIME headers that are relevant for the quoed material or for the
content of the reply?
I don't see why that matters. You're expecting the newsreader to do >>something it cannot do and no newsreader can.
I see now, thanks for clarifying.
So when you say "better support for MIME would be nice", what exactly
is missing?
"Adam H. Kerman" <ahk@chinet.com> spake:
legalize+jeeves@mail.xmission.com (Richard) wrote:
So when you say "better support for MIME would be nice", what exactly
is missing?
I didn't say that; Michael did.
Oops, sorry for misattributing.
I'm guessing that what's breaking
articles is a mix of UTF-8 and 8-bit used by different users, then not >>declaring which one is in use in followup.
Didn't we conclude on this thread that it's up to the user to set the
headers correctly?
Technically, trn doesn't do the posting; inews does the posting as a
separate program after having invoked your editor on the post.
I suppose inews could inspect the content of the post more deeply and >complain about use of 8-bit characters and/or detect UTF-8. It could
offer to adjust the headers as best it can and let you re-edit.
You'd need to add a parsing mechanism, then add the appropriate MIME
header to declare the character set. Still, the user has to clean up the >>mess of broken quotes himself.
You could improve this yourself by setting NEWSPOSTER to a different >program/script that does this detecting and munging of the headers
before invoking inews.
legalize+jeeves@mail.xmission.com (Richard) wrote:
So when you say "better support for MIME would be nice", what exactly
is missing?
I didn't say that; Michael did.
I'm guessing that what's breaking
articles is a mix of UTF-8 and 8-bit used by different users, then not >declaring which one is in use in followup.
You'd need to add a parsing mechanism, then add the appropriate MIME
header to declare the character set. Still, the user has to clean up the
mess of broken quotes himself.
[...]
Still, there's just no way to fix up multi-level quotes that used
different character sets mis-translated without the user going back to
the original text to figure out what it was supposed to be and then translating it consistently with the rest of the text.
Adam H. Kerman wrote:
[...]
Still, there's just no way to fix up multi-level quotes that used
different character sets mis-translated without the user going back to
the original text to figure out what it was supposed to be and then >>translating it consistently with the rest of the text.
If it already happened, this is something that is hard to impossible
to repair for some encodings. Most newsreaders don't even try and
such a repair algorithm is not what I had in mind.
What I wanted to propose is that trn does not create and send such
mixture of encodings and correctly label the encoding used according
to MIME (the problem is that trn users produce such broken articles,
not that they cannnot read them).
It looks like the users don't know what they are doing (not really the
fault of trn in this sense) and their editor is not configured for the >encoding of the content that is quoted. Maybe it is too inconvenient to >change the encoding configuration.
I think such mistakes would not occur if trn would automatically convert
the content to quote into the encoding used by the editor.
The source encoding is declared in the MIME header of the article.
The target encoding should be the one the editor is using (manually >configured, if trn cannot automatically detect it).
The conversion itself can be done with iconv.
This would preserve the users choice for the target encoding (would
not enforce the usage of Unicode).
If the user has configured e.g. US-ASCII to quote an article written in >Unicode, replacement characters should be inserted if the encoding
conversion is not possible (instead of simply copying the bytes that are
then interpreted wrong by the editor and the recipients).
Absolutely, especially ASCII punctuation characters for 8-bit or UTF-8 >punctuation characters. Typesetting characters do not belong in plain
text communication.
Michael Bäuerle wrote:
Adam H. Kerman wrote:
[...]
Still, there's just no way to fix up multi-level quotes that used different character sets mis-translated without the user going back to the original text to figure out what it was supposed to be and then translating it consistently with the rest of the text.
If it already happened, this is something that is hard to impossible
to repair for some encodings. Most newsreaders don't even try and
such a repair algorithm is not what I had in mind.
What I wanted to propose is that trn does not create and send such
mixture of encodings and correctly label the encoding used according
to MIME (the problem is that trn users produce such broken articles,
not that they cannnot read them).
How is it trn's fault? trn isn't translating anything! It receives
characters and passes them along to inews -h. It's up to the author to recognize the encoding.
Take a typical situation I encounter: Root article has non-ASCII
characters and declares UTF-8. The author of the followup has his
default character set ISO-8859-1 and has no parsing mechanism. Because
it's an Apple client, it does weird character substitutions, different
from what a Windows client might do. Either way, it ignored the declared character set of the precursor article.
[...]
In followup, if the declared character set truly describes the character
set inherited from the precursor article, at least a decent parser
called by trn could deal with it, or even translate to another character
set if necessary and an outside process could add the matching MIME
header.
Since I'm a traditionalist, I'd like to have another chance to edit to
verify that the parser caught everything, and I still want to get rid of characters that do not belong in plain text, like nonbreaking space.
It looks like the users don't know what they are doing (not really the fault of trn in this sense) and their editor is not configured for the encoding of the content that is quoted. Maybe it is too inconvenient to change the encoding configuration.
The editor is an outside process. It's not built into trn. It's not the
job of the editor to parse, although there seems to be process that
decodes BASE64 or QP, neither of which belong in Usenet.
That happens
before the editor sees it. There'd have to be another process before it
gets to the editor to test that the declared character set of the
precursor article matches what's actually there.
The editor is likely using a character set from an environment variable.
I think such mistakes would not occur if trn would automatically convert the content to quote into the encoding used by the editor.
The source encoding is declared in the MIME header of the article.
Snarf
When it actually matches!
The target encoding should be the one the editor is using (manually configured, if trn cannot automatically detect it).
Not what trn does. An outside process would have to be called.
The conversion itself can be done with iconv.
This would preserve the users choice for the target encoding (would
not enforce the usage of Unicode).
In my opinion, if there are non-ASCII characters but the lowest
denomination character set is 8-bit, that's the character set to use.
My opinion is shared with nearly no one.
Michael Bauerle <michael.baeuerle@gmx.net> wrote:
Adam H. Kerman wrote:
Michael Bauerle wrote:
Adam H. Kerman wrote:
[...]
[...]
In followup, if the declared character set truly describes the character >>>set inherited from the precursor article, at least a decent parser
called by trn could deal with it, or even translate to another character >>>set if necessary and an outside process could add the matching MIME >>>header.
This is what I propose. It should be easy for the user to do this.
The code for it should be shipped with the newsreader, not every user
is a programmer too.
This isn't programming. The best I can do is write macros. Richard was >talking about calling outside processes that already exist, not
requiring the user to write his own parser.
But this text is decades old. If a successor to this RFC would be
written today, it likely would say something like "Always use Unicode
with UTF-8 encoding".
Adam H. Kerman wrote:
Michael Bauerle wrote:
Adam H. Kerman wrote:
[...]
[...]
In followup, if the declared character set truly describes the character >>set inherited from the precursor article, at least a decent parser
called by trn could deal with it, or even translate to another character >>set if necessary and an outside process could add the matching MIME
header.
This is what I propose. It should be easy for the user to do this.
The code for it should be shipped with the newsreader, not every user
is a programmer too.
Since I'm a traditionalist, I'd like to have another chance to edit to >>verify that the parser caught everything, and I still want to get rid of >>characters that do not belong in plain text, like nonbreaking space.
I think this would always be the case, because all the automatic
processing should be finished before the editor is launched.
It looks like the users don't know what they are doing (not really the >>>fault of trn in this sense) and their editor is not configured for the >>>encoding of the content that is quoted. Maybe it is too inconvenient to >>>change the encoding configuration.
The editor is an outside process. It's not built into trn. It's not the
job of the editor to parse, although there seems to be process that
decodes BASE64 or QP, neither of which belong in Usenet.
No longer true since RFC 5536: ><https://www.rfc-editor.org/rfc/rfc5536#section-2.3>
| User agents MUST meet the definition of MIME conformance in [RFC2049]
| and MUST also support [RFC2231]. This level of MIME conformance
| provides support for internationalization and multimedia in message
| bodies [RFC2045], [RFC2046], and [RFC2231], and support for
| internationalization of header fields [RFC2047] and [RFC2231]. [...]
That happens
before the editor sees it. There'd have to be another process before it >>gets to the editor to test that the declared character set of the
precursor article matches what's actually there.
This part should be added.
Decode transfer encoding first.
Then convert the encoding to match the editor.
Create a MIME declaration for the outgoing data, if not US-ASCII.
The editor is likely using a character set from an environment variable.
Maybe not generic enough to be used by the newsreader too.
I think such mistakes would not occur if trn would automatically convert >>>the content to quote into the encoding used by the editor.
The source encoding is declared in the MIME header of the article.
Snarf
When it actually matches!
It does not match in the articles I talk about (the user has not
converted the encoding of the incoming data, but has written his reply
with a different encoding). This is the case that should be easier to
avoid.
. . .
"Adam H. Kerman" <ahk@chinet.com> spake:
Michael Bauerle <michael.baeuerle@gmx.net> wrote:
Adam H. Kerman wrote:
Michael Bauerle wrote:
Adam H. Kerman wrote:
[...]
[...]
In followup, if the declared character set truly describes the character >>>>set inherited from the precursor article, at least a decent parser >>>>called by trn could deal with it, or even translate to another character >>>>set if necessary and an outside process could add the matching MIME >>>>header.
This is what I propose. It should be easy for the user to do this.
The code for it should be shipped with the newsreader, not every user
is a programmer too.
This isn't programming. The best I can do is write macros. Richard was >>talking about calling outside processes that already exist, not
requiring the user to write his own parser.
I think it's a reasonable request for trn's inews program to be
smarter about encodings and analyze the input file and add the
necessary headers to decorate the content to the best of it's ability.
I don't know if acli's fork of trn is already doing these things; I
think that fork has been focused on proper presentation of UTF-8
content in articles. It is my intention to merge those changes into
my fork.
legalize+jeeves@mail.xmission.com (Richard) wrote:
I think it's a reasonable request for trn's inews program to be
smarter about encodings and analyze the input file and add the
necessary headers to decorate the content to the best of it's ability.
I thought we were using inews from INN.
"Adam H. Kerman" <ahk@chinet.com> spake the secret code
I'm guessing that what's breaking articles is a mix of UTF-8 and
8-bit used by different users, then not declaring which one is in use
in followup.
Didn't we conclude on this thread that it's up to the user to set the
headers correctly?
Technically, trn doesn't do the posting; inews does the posting as a
separate program after having invoked your editor on the post.
You could improve this yourself by setting NEWSPOSTER to a different program/script that does this detecting and munging of the headers
before invoking inews.
Thirdly there's a what-gets-sent issue if you don't post with correct
MIME headers and people read with some tool expecting them. That's the >not-declarining issue.
In news.software.readers, Richard <> wrote:
Didn't we conclude on this thread that it's up to the user to set the
headers correctly?
Setting headers is tedious, tedium is best for computers.
Technically, trn doesn't do the posting; inews does the posting as a
separate program after having invoked your editor on the post.
Kinda right, but not always. trn can act as inews for you.
Eli the Bearded <*@eli.users.panix.com> spake
Thirdly there's a what-gets-sent issue if you don't post with correctI'm open to ideas about how trn can handle this issue better; for
MIME headers and people read with some tool expecting them. That's the
not-declarining issue.
instance, it's certainly possible to scan the edited article before
posting and suggest header additions/changes based on content.
I agree, I just don't know how the computer is supposed to distinguish between, say, Big5 encoding or UTF-8. Certainly it can assume UTF-8
and then reject that assumption if the non-ASCII bytes aren't valid
UTF-8 encoded code points.
Yeah, there's an inews executable in the trn source tree, but I
suspect that people don't include that in the trn package when they're packaging things up for linux distros.
posting updates to trn-workers mailing list: <https://sourceforge.net/p/trn/mailman/>
In news.software.readers, Richard <> wrote:
posting updates to trn-workers mailing list:
<https://sourceforge.net/p/trn/mailman/>
I'm subscribed, but I've been a bit lax about reading it.
| Sysop: | Keyop |
|---|---|
| Location: | Huddersfield, West Yorkshire, UK |
| Users: | 546 |
| Nodes: | 16 (2 / 14) |
| Uptime: | 31:20:49 |
| Calls: | 10,391 |
| Calls today: | 2 |
| Files: | 14,064 |
| Messages: | 6,417,107 |