Hi!

neos universal compiler (that can compile any programming language) is successfully running the tokenization stage
tokenizing a program written in the neos reference language. #cpp #coding #compiler #compsci #gamedev

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming language) is successfully running the tokenization stage
tokenizing a program written in the neos reference language. #cpp #coding #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming language) is
successfully running the tokenization stage tokenizing a program
written in the neos reference language. #cpp #coding #compiler #compsci
#gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming language) is
successfully running the tokenization stage tokenizing a program
written in the neos reference language. #cpp #coding #compiler #compsci
#gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 15/03/2025 15:08, Mr Flibble wrote:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming language) is
successfully running the tokenization stage tokenizing a program
written in the neos reference language. #cpp #coding #compiler #compsci
#gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

<https://www.99-bottles-of-beer.net/language-piet-1269.html>

Good luck with that.

--
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 16 Mar 2025 10:35:25 +0000, Richard Heathfield wrote:

On 15/03/2025 15:08, Mr Flibble wrote:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming language)
is successfully running the tokenization stage tokenizing a program
written in the neos reference language. #cpp #coding #compiler
#compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

<https://www.99-bottles-of-beer.net/language-piet-1269.html>

Good luck with that.

No reason why neos couldn't support Piet.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming language)
is successfully running the tokenization stage tokenizing a program
written in the neos reference language. #cpp #coding #compiler
#compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming language.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-16 18:40:02 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 10:35:25 +0000, Richard Heathfield wrote:

On 15/03/2025 15:08, Mr Flibble wrote:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming language)
is successfully running the tokenization stage tokenizing a program
written in the neos reference language. #cpp #coding #compiler
#compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

<https://www.99-bottles-of-beer.net/language-piet-1269.html>

Good luck with that.

No reason why neos couldn't support Piet.

But "could" is not the same as "does".

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming language)
is successfully running the tokenization stage tokenizing a program
written in the neos reference language. #cpp #coding #compiler
#compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming language.

If it is configured the same for each programming language then how
does it know how to tokenize?

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming language) >>>>>> is successfully running the tokenization stage tokenizing a program >>>>>> written in the neos reference language. #cpp #coding #compiler
#compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming language.

If it is configured the same for each programming language then how does
it know how to tokenize?

You configure it by providing a language specific neosBNF schema (grammar)
file (an input to the compilalation process).

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Mr Flibble <flibble@reddwarf.jmc.corp> wrote:

Hi!

neos universal compiler (that can compile any programming language) is successfully running the tokenization stage
tokenizing a program written in the neos reference language. #cpp #coding #compiler #compsci #gamedev

/Flibble

How is neos (the tokenisation stage) essentially different from lex (the
~50 year old unix utility)? What advantages will neos (if completed)
offer over a compiler based on a lex/yacc combination?

--
Alan Mackenzie (Nuremberg, Germany).

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 17 Mar 2025 21:43:47 +0000, Alan Mackenzie wrote:

Mr Flibble <flibble@reddwarf.jmc.corp> wrote:

Hi!

neos universal compiler (that can compile any programming language) is
successfully running the tokenization stage tokenizing a program
written in the neos reference language. #cpp #coding #compiler #compsci
#gamedev

/Flibble

How is neos (the tokenisation stage) essentially different from lex (the
~50 year old unix utility)? What advantages will neos (if completed)
offer over a compiler based on a lex/yacc combination?

neos grammar definition is superior to lex's in that it is based on EBNF (rather than a horrid mixture of C and regex) so is much more human
friendly potentially opening up language design to the masses possibly
making neos a technological singularity in the areas of language and
compiler design.

lex/yacc generates a static frontend (that must be compiled) that is
different for each supported programming language whilst neos has just one frontend which does not require compilation.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Mr Flibble <flibble@reddwarf.jmc.corp> wrote:

On Mon, 17 Mar 2025 21:43:47 +0000, Alan Mackenzie wrote:

Mr Flibble <flibble@reddwarf.jmc.corp> wrote:

Hi!

neos universal compiler (that can compile any programming language) is
successfully running the tokenization stage tokenizing a program
written in the neos reference language. #cpp #coding #compiler #compsci
#gamedev

/Flibble

How is neos (the tokenisation stage) essentially different from lex (the
~50 year old unix utility)? What advantages will neos (if completed)
offer over a compiler based on a lex/yacc combination?

neos grammar definition is superior to lex's in that it is based on EBNF (rather than a horrid mixture of C and regex) so is much more human
friendly potentially opening up language design to the masses possibly
making neos a technological singularity in the areas of language and
compiler design.

lex and yacc both use BNF, or something very like it.

lex/yacc generates a static frontend (that must be compiled) that is different for each supported programming language whilst neos has just one frontend which does not require compilation.

I can't see the essential difference from that paragraph. neos will
have to interpret a syntactic description, just as lex and yacc do.
They may differ in how and when things get compiled, etc., which is to
be expected after 50 years of hardware and software development.

Still, I wish you all the best with this ambitious project.

/Flibble

--
Alan Mackenzie (Nuremberg, Germany).

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming language) >>>>>>> is successfully running the tokenization stage tokenizing a program >>>>>>> written in the neos reference language. #cpp #coding #compiler
#compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming language.

If it is configured the same for each programming language then how does
it know how to tokenize?

You configure it by providing a language specific neosBNF schema (grammar) file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming
language)
is successfully running the tokenization stage tokenizing a
program written in the neos reference language. #cpp #coding
#compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming
language.

If it is configured the same for each programming language then how
does it know how to tokenize?

You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

Not yet.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-18 03:18:25 +0000, Mr Flibble said:

lex/yacc generates a static frontend (that must be compiled) that is different for each supported programming language whilst neos has just one frontend which does not require compilation.

A compiled front end is usually what is wanted. But it is prossible to
use lex or yacc interpreter or both if that is wanted. It is a bit
complicated as the interpreter must be able to interprete C fragments
but it is possible.

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, 18 Mar 2025 14:20:04 +0000, Richard Heathfield wrote:

On 18/03/2025 14:08, Mr Flibble wrote:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming
language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>> program written in the neos reference language. #cpp #coding >>>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming
language.

If it is configured the same for each programming language then how
does it know how to tokenize?

You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

Not yet.

We deduce, then, that neos universal compiler (that can compile any programming language) cannot compile FORTRAN 66 or ALGOL 60.

We can further deduce that you do not consider FORTRAN 66 and ALGOL 60
to be programming languages.

Your deduction is wrong, my universal compiler has the CAPABILITY to
compile any programming language and to add support for a particular
language a schema file has to be provided.

Your "further deduction" predicated on your first deduction is thus also
wrong.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 18/03/2025 14:30, Mr Flibble wrote:

On Tue, 18 Mar 2025 14:20:04 +0000, Richard Heathfield wrote:

On 18/03/2025 14:08, Mr Flibble wrote:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming >>>>>>>>>>> language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>>> program written in the neos reference language. #cpp #coding >>>>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming
language.

If it is configured the same for each programming language then how >>>>>> does it know how to tokenize?

You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

Not yet.

We deduce, then, that neos universal compiler (that can compile any
programming language) cannot compile FORTRAN 66 or ALGOL 60.

We can further deduce that you do not consider FORTRAN 66 and ALGOL 60
to be programming languages.

Your deduction is wrong, my universal compiler has the CAPABILITY to
compile any programming language and to add support for a particular
language a schema file has to be provided.

So what you seem to be saying is that your universal compiler
lacks the CAPABILITY to compile any programming language until
that capability is added. So my first deduction was correct.

Your "further deduction" predicated on your first deduction is thus also wrong.

I don't see why.

Your original claim (which is what I was addressing) was that it
CAN (note: present tense) compile any programming language. Since
by your own admission it can't (yet) currently compile either
FORTRAN 66 or ALGOL 60 until such time as schema files are
devised for those languages, either your original claim was
mistaken or you do not consider FORTRAN 66 and ALGOL 60 to be
programming languages. Since we now learn (by your rejection of
the latter half) that you presumably /do/ consider FORTRAN 66 and
ALGOL 60 to be programming languages, we are forced to conclude
that your original claim was in error.

We now have three examples of languages it can't compile: FORTRAN
66, ALGOL 60, and Piet. You seem to be of the opinion that
potential is achievement. It isn't.

Were you to claim that your "universal" compiler has the
POTENTIAL to compile any programming language, but to add support
for a particular language a schema file has to be provided, that
seems to me to be a much more defensible claim.

--
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 18/03/2025 14:08, Mr Flibble wrote:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming
language)
is successfully running the tokenization stage tokenizing a
program written in the neos reference language. #cpp #coding >>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming
language.

If it is configured the same for each programming language then how
does it know how to tokenize?

You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

Not yet.

We deduce, then, that neos universal compiler (that can compile
any programming language) cannot compile FORTRAN 66 or ALGOL 60.

We can further deduce that you do not consider FORTRAN 66 and
ALGOL 60 to be programming languages.

--
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, 18 Mar 2025 15:01:34 +0000, Richard Heathfield wrote:

On 18/03/2025 14:30, Mr Flibble wrote:

On Tue, 18 Mar 2025 14:20:04 +0000, Richard Heathfield wrote:

On 18/03/2025 14:08, Mr Flibble wrote:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming >>>>>>>>>>>> language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>>>> program written in the neos reference language. #cpp #coding >>>>>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming
language.

If it is configured the same for each programming language then
how does it know how to tokenize?

You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or >>>>> Algol 60?

Not yet.

We deduce, then, that neos universal compiler (that can compile any
programming language) cannot compile FORTRAN 66 or ALGOL 60.

We can further deduce that you do not consider FORTRAN 66 and ALGOL 60
to be programming languages.

Your deduction is wrong, my universal compiler has the CAPABILITY to
compile any programming language and to add support for a particular
language a schema file has to be provided.

So what you seem to be saying is that your universal compiler lacks the CAPABILITY to compile any programming language until that capability is added. So my first deduction was correct.

Your first deduction was incorrect.

Your "further deduction" predicated on your first deduction is thus
also wrong.

I don't see why.

Because your second deduction was predicated in your first deduction which
was incorrect.

Your original claim (which is what I was addressing) was that it CAN
(note: present tense) compile any programming language. Since by your
own admission it can't (yet) currently compile either FORTRAN 66 or
ALGOL 60 until such time as schema files are devised for those
languages, either your original claim was mistaken or you do not
consider FORTRAN 66 and ALGOL 60 to be programming languages. Since we
now learn (by your rejection of the latter half) that you presumably
/do/ consider FORTRAN 66 and ALGOL 60 to be programming languages, we
are forced to conclude that your original claim was in error.

Nope, my universal compiler is CAPABLE of compiling any programming
language once a schema file has been created for that programming
language. There is no reason why my universal compiler couldn't compile FORTRAN 66 or ALGOL 60 if schema files were written for those languages.

"we" are forced to conclude? Speak for yourself not others. Your
conclusion is specious, vacuous, plain wrong.

We now have three examples of languages it can't compile: FORTRAN 66,
ALGOL 60, and Piet. You seem to be of the opinion that potential is achievement. It isn't.

My universal compiler can compile those languages if schema files are
written for them.

Were you to claim that your "universal" compiler has the POTENTIAL to
compile any programming language, but to add support for a particular language a schema file has to be provided, that seems to me to be a much
more defensible claim.

CAPABLE is a more accurate description than POTENTIAL.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 18/03/2025 15:06, Mr Flibble wrote:

On Tue, 18 Mar 2025 15:01:34 +0000, Richard Heathfield wrote:

On 18/03/2025 14:30, Mr Flibble wrote:

On Tue, 18 Mar 2025 14:20:04 +0000, Richard Heathfield wrote:

On 18/03/2025 14:08, Mr Flibble wrote:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming >>>>>>>>>>>>> language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>>>>> program written in the neos reference language. #cpp #coding >>>>>>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming >>>>>>>>> language.

If it is configured the same for each programming language then >>>>>>>> how does it know how to tokenize?

You configure it by providing a language specific neosBNF schema >>>>>>> (grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or >>>>>> Algol 60?

Not yet.

We deduce, then, that neos universal compiler (that can compile any
programming language) cannot compile FORTRAN 66 or ALGOL 60.

We can further deduce that you do not consider FORTRAN 66 and ALGOL 60 >>>> to be programming languages.

Your deduction is wrong, my universal compiler has the CAPABILITY to
compile any programming language and to add support for a particular
language a schema file has to be provided.

So what you seem to be saying is that your universal compiler lacks the
CAPABILITY to compile any programming language until that capability is
added. So my first deduction was correct.

Your first deduction was incorrect.

Tis tisn't tis.

Have a good day.

--
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, 18 Mar 2025 15:26:17 +0000, Richard Heathfield wrote:

On 18/03/2025 15:06, Mr Flibble wrote:

On Tue, 18 Mar 2025 15:01:34 +0000, Richard Heathfield wrote:

On 18/03/2025 14:30, Mr Flibble wrote:

On Tue, 18 Mar 2025 14:20:04 +0000, Richard Heathfield wrote:

On 18/03/2025 14:08, Mr Flibble wrote:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming >>>>>>>>>>>>>> language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>>>>>> program written in the neos reference language. #cpp >>>>>>>>>>>>>> #coding #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming >>>>>>>>>> language.

If it is configured the same for each programming language then >>>>>>>>> how does it know how to tokenize?

You configure it by providing a language specific neosBNF schema >>>>>>>> (grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 >>>>>>> or Algol 60?

Not yet.

We deduce, then, that neos universal compiler (that can compile any
programming language) cannot compile FORTRAN 66 or ALGOL 60.

We can further deduce that you do not consider FORTRAN 66 and ALGOL
60 to be programming languages.

Your deduction is wrong, my universal compiler has the CAPABILITY to
compile any programming language and to add support for a particular
language a schema file has to be provided.

So what you seem to be saying is that your universal compiler lacks
the CAPABILITY to compile any programming language until that
capability is added. So my first deduction was correct.

Your first deduction was incorrect.

Tis tisn't tis.

Have a good day.

It certainly isn't what you think it is but you are one of those people
who are usually fractally wrong.

It will be shown that the universal compiler neos will be a technological singularity: a seminal moment in language design, compiler design and
computer science.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 18/03/2025 16:23, Mr Flibble wrote:

you are one of those people
who are usually fractally wrong.

I hope you have a great evening.

--
Richard Heathfield
Email: rjh at cpax dot org dot uk
"Usenet is a strange place" - dmr 29 July 1999
Sig line 4 vacant - apply within

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 18/03/2025 13:59, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

[...]

You configure it by providing a language specific neosBNF schema (grammar) >> file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

If you ever get such an animal, it will be interesting to see
if it can be adapted to Algol 68, esp [as a matter of practicality] to
the modern version A68G. I shan't hold my breath!

--
Andy Walker, Nottingham.
Andy's music pages: www.cuboid.me.uk/andy/Music
Composer of the day: www.cuboid.me.uk/andy/Music/Composers/Grieg

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Mikko <mikko.levanto@iki.fi> wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

[ .... ]

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example
of something that cannot be defined with a regular expression and is therefore impossible or at least complicated with an ordinary tokenizer.

Would you please be more specific about just what in an Algol 60 string
literal prevents a regexp from parsing it. Not for any special reason,
just that I'm curious. Maybe an example of such a string would be
interesting. Thanks!

--
Mikko

--
Alan Mackenzie (Nuremberg, Germany).

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming
language)
is successfully running the tokenization stage tokenizing a
program written in the neos reference language. #cpp #coding >>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming
language.

If it is configured the same for each programming language then how
does it know how to tokenize?

You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example
of something that cannot be defined with a regular expression and is
therefore impossible or at least complicated with an ordinary tokenizer.

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-19 11:02:49 +0000, Alan Mackenzie said:

Mikko <mikko.levanto@iki.fi> wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

[ .... ]

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example
of something that cannot be defined with a regular expression and is
therefore impossible or at least complicated with an ordinary tokenizer.

Would you please be more specific about just what in an Algol 60 string literal prevents a regexp from parsing it. Not for any special reason,
just that I'm curious. Maybe an example of such a string would be interesting. Thanks!

--
Mikko

Algol 60 has different characters for opening and closing quotes (something like 2018 and 2019 of Unicode) and allows any number of nested quotes.

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming
language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>> program written in the neos reference language. #cpp #coding >>>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming
language.

If it is configured the same for each programming language then how
does it know how to tokenize?

You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example of something that cannot be defined with a regular expression and is
therefore impossible or at least complicated with an ordinary tokenizer.

neos does not use regular expressions and the neos grammar is context
sensitive (i.e. not context free).

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Wed, 19 Mar 2025 14:32:30 +0000, Mr Flibble wrote:

On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming >>>>>>>>>>> language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>>> program written in the neos reference language. #cpp #coding >>>>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming
language.

If it is configured the same for each programming language then how >>>>>> does it know how to tokenize?

You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example of
something that cannot be defined with a regular expression and is
therefore impossible or at least complicated with an ordinary
tokenizer.

neos does not use regular expressions and the neos grammar is context sensitive (i.e. not context free).

/Flibble

Of course I meant to say that parsing process that uses the grammar is
context sensitive, the grammar itself is still a CFG.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-19 14:32:30 +0000, Mr Flibble said:

On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming >>>>>>>>>>> language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>>> program written in the neos reference language. #cpp #coding >>>>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming
language.

If it is configured the same for each programming language then how >>>>>> does it know how to tokenize?

You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or
Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example of
something that cannot be defined with a regular expression and is
therefore impossible or at least complicated with an ordinary tokenizer.

neos does not use regular expressions and the neos grammar is context sensitive (i.e. not context free).

A context free grammar should be enough for tokenizing strings of Algol 60.

Tokenizing statements of FORTRAN 66 is a harder problem.

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Mikko <mikko.levanto@iki.fi> wrote:

On 2025-03-19 11:02:49 +0000, Alan Mackenzie said:

Mikko <mikko.levanto@iki.fi> wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

[ .... ]

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or >>>>> Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example
of something that cannot be defined with a regular expression and is
therefore impossible or at least complicated with an ordinary tokenizer.

Would you please be more specific about just what in an Algol 60 string
literal prevents a regexp from parsing it. Not for any special reason,
just that I'm curious. Maybe an example of such a string would be
interesting. Thanks!

Algol 60 has different characters for opening and closing quotes (something like 2018 and 2019 of Unicode) ....

Most current languages, including C, have different openers and closers
for comments, which is surely analogous.

.... and allows any number of nested quotes.

Ah OK. Regular expressions can't parse arbitrarily nested structures.
But Backus-Nauer Form can express them, and a push-down automaton can
process them.

Are you sure about ordinary tokenizers not being able to handle such arbitrarily nested things in a non-complicated way?

--
Mikko

--
Alan Mackenzie (Nuremberg, Germany).

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-20 11:16:35 +0000, Alan Mackenzie said:

Mikko <mikko.levanto@iki.fi> wrote:

On 2025-03-19 11:02:49 +0000, Alan Mackenzie said:

Mikko <mikko.levanto@iki.fi> wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

[ .... ]

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or >>>>>> Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example
of something that cannot be defined with a regular expression and is
therefore impossible or at least complicated with an ordinary tokenizer.

Would you please be more specific about just what in an Algol 60 string
literal prevents a regexp from parsing it. Not for any special reason,
just that I'm curious. Maybe an example of such a string would be
interesting. Thanks!

Algol 60 has different characters for opening and closing quotes (something >> like 2018 and 2019 of Unicode) ....

Most current languages, including C, have different openers and closers
for comments, which is surely analogous.

.... and allows any number of nested quotes.

Ah OK. Regular expressions can't parse arbitrarily nested structures.
But Backus-Nauer Form can express them, and a push-down automaton can
process them.

Are you sure about ordinary tokenizers not being able to handle such arbitrarily nested things in a non-complicated way?

Yes if "ordinary" is defined as one that can only parse a regular
language.

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2025-03-19 16:11:18 +0000, Mikko said:

On 2025-03-19 14:32:30 +0000, Mr Flibble said:

On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming >>>>>>>>>>>> language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>>>> program written in the neos reference language. #cpp #coding >>>>>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming
language.

If it is configured the same for each programming language then how >>>>>>> does it know how to tokenize?

You configure it by providing a language specific neosBNF schema
(grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 or >>>>> Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example of
something that cannot be defined with a regular expression and is
therefore impossible or at least complicated with an ordinary tokenizer.

neos does not use regular expressions and the neos grammar is context
sensitive (i.e. not context free).

A context free grammar should be enough for tokenizing strings of Algol 60.

Tokenizing statements of FORTRAN 66 is a harder problem.

Some test cases for a FORTRAN IV tokenizer:

REALITY IS, AS I MAY
LOGICALLY SAY, REAL
LOGICAL = WHAT I SAY

Note that these lines begin with 8 spaces.

Tokens on the first line:
keyword REAL
identifier ITYIS
comma
identifier ASIMAY
end of statement

Tokens on the second line:
keyword LOGICAL
identifier LYSAY
comma
identifier REAL
end of statement

Tokens on the third line:
identifier LOGICAL
assignment symbol
identifier WHAT I SAY

Parser needs to know whether an instance of REAL or LOGICAL is an
identifier or keyword.

I don't understand how neos can be configred to tokenize the above
test lines and other FORTRAN IV statements.

--
Mikko

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 28 Mar 2025 14:01:40 +0200, Mikko wrote:

On 2025-03-19 16:11:18 +0000, Mikko said:

On 2025-03-19 14:32:30 +0000, Mr Flibble said:

On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming >>>>>>>>>>>>> language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>>>>> program written in the neos reference language. #cpp #coding >>>>>>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming >>>>>>>>> language.

If it is configured the same for each programming language then >>>>>>>> how does it know how to tokenize?

You configure it by providing a language specific neosBNF schema >>>>>>> (grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66
or Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example
of something that cannot be defined with a regular expression and is
therefore impossible or at least complicated with an ordinary
tokenizer.

neos does not use regular expressions and the neos grammar is context
sensitive (i.e. not context free).

A context free grammar should be enough for tokenizing strings of Algol
60.

Tokenizing statements of FORTRAN 66 is a harder problem.

Some test cases for a FORTRAN IV tokenizer:

REALITY IS, AS I MAY LOGICALLY SAY, REAL LOGICAL = WHAT I SAY

Note that these lines begin with 8 spaces.

Tokens on the first line:
keyword REAL identifier ITYIS comma identifier ASIMAY end of statement

Tokens on the second line:
keyword LOGICAL identifier LYSAY comma identifier REAL end of statement

Tokens on the third line:
identifier LOGICAL assignment symbol identifier WHAT I SAY

Parser needs to know whether an instance of REAL or LOGICAL is an
identifier or keyword.

I don't understand how neos can be configred to tokenize the above test
lines and other FORTRAN IV statements.

If a native FORTRAN IV compiler can parse it then neos can parse it as
long as it is somehow expressable in the neos attribute grammar.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 3/28/2025 8:14 AM, Mr Flibble wrote:

On Fri, 28 Mar 2025 14:01:40 +0200, Mikko wrote:

On 2025-03-19 16:11:18 +0000, Mikko said:

On 2025-03-19 14:32:30 +0000, Mr Flibble said:

On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming >>>>>>>>>>>>>> language)
is successfully running the tokenization stage tokenizing a >>>>>>>>>>>>>> program written in the neos reference language. #cpp #coding >>>>>>>>>>>>>> #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming >>>>>>>>>> language.

If it is configured the same for each programming language then >>>>>>>>> how does it know how to tokenize?

You configure it by providing a language specific neosBNF schema >>>>>>>> (grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 >>>>>>> or Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good example >>>>> of something that cannot be defined with a regular expression and is >>>>> therefore impossible or at least complicated with an ordinary
tokenizer.

neos does not use regular expressions and the neos grammar is context
sensitive (i.e. not context free).

A context free grammar should be enough for tokenizing strings of Algol
60.

Tokenizing statements of FORTRAN 66 is a harder problem.

Some test cases for a FORTRAN IV tokenizer:

REALITY IS, AS I MAY LOGICALLY SAY, REAL LOGICAL = WHAT I SAY

Note that these lines begin with 8 spaces.

Tokens on the first line:
keyword REAL identifier ITYIS comma identifier ASIMAY end of statement

Tokens on the second line:
keyword LOGICAL identifier LYSAY comma identifier REAL end of statement

Tokens on the third line:
identifier LOGICAL assignment symbol identifier WHAT I SAY

Parser needs to know whether an instance of REAL or LOGICAL is an
identifier or keyword.

I don't understand how neos can be configred to tokenize the above test
lines and other FORTRAN IV statements.

If a native FORTRAN IV compiler can parse it then neos can parse it as
long as it is somehow expressable in the neos attribute grammar.

But that was the question: Can it be parsed in a neos expressible
grammar? Note that those early Fortran grammars were 1) not finite
state, 2) not context free, and 3) inherently ambiguous.It was possible,
for example, to write a "format" and an "assignment" statement that were character to character identical. As an aside, a friend who wrote an
early Fortran compiler that compiled such a chimera so that it would do
an assignment if executed and could be referenced as a format from an IO statement.
--
Jeff Barnett

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 28 Mar 2025 13:05:10 -0600, Jeff Barnett wrote:

On 3/28/2025 8:14 AM, Mr Flibble wrote:

On Fri, 28 Mar 2025 14:01:40 +0200, Mikko wrote:

On 2025-03-19 16:11:18 +0000, Mikko said:

On 2025-03-19 14:32:30 +0000, Mr Flibble said:

On Wed, 19 Mar 2025 12:26:20 +0200, Mikko wrote:

On 2025-03-18 14:08:50 +0000, Mr Flibble said:

On Tue, 18 Mar 2025 15:59:45 +0200, Mikko wrote:

On 2025-03-17 16:53:01 +0000, Mr Flibble said:

On Mon, 17 Mar 2025 11:21:05 +0200, Mikko wrote:

On 2025-03-16 18:40:42 +0000, Mr Flibble said:

On Sun, 16 Mar 2025 12:28:24 +0200, Mikko wrote:

On 2025-03-15 15:08:47 +0000, Mr Flibble said:

On Sat, 15 Mar 2025 11:55:52 +0200, Mikko wrote:

On 2025-03-15 04:00:52 +0000, Mr Flibble said:

Hi!

neos universal compiler (that can compile any programming >>>>>>>>>>>>>>> language)
is successfully running the tokenization stage tokenizing >>>>>>>>>>>>>>> a program written in the neos reference language. #cpp >>>>>>>>>>>>>>> #coding #compiler #compsci #gamedev

/Flibble

Can it tokenize FORTRAN 60 or FORTRAN IV ?

ANY programming language.

/Flibble

How is neos configured to tokenize FORTRAN 60 ?

The same way you would configure it for any other programming >>>>>>>>>>> language.

If it is configured the same for each programming language then >>>>>>>>>> how does it know how to tokenize?

You configure it by providing a language specific neosBNF schema >>>>>>>>> (grammar)
file (an input to the compilalation process).

Is there a neosBNF schema that describes the tokens of FORtRAN 66 >>>>>>>> or Algol 60?

Not yet.

The definition of string literal of Algol 60 would be a good
example of something that cannot be defined with a regular
expression and is therefore impossible or at least complicated with >>>>>> an ordinary tokenizer.

neos does not use regular expressions and the neos grammar is
context sensitive (i.e. not context free).

A context free grammar should be enough for tokenizing strings of
Algol 60.

Tokenizing statements of FORTRAN 66 is a harder problem.

Some test cases for a FORTRAN IV tokenizer:

REALITY IS, AS I MAY LOGICALLY SAY, REAL LOGICAL = WHAT I SAY

Note that these lines begin with 8 spaces.

Tokens on the first line:
keyword REAL identifier ITYIS comma identifier ASIMAY end of statement

Tokens on the second line:
keyword LOGICAL identifier LYSAY comma identifier REAL end of
statement

Tokens on the third line:
identifier LOGICAL assignment symbol identifier WHAT I SAY

Parser needs to know whether an instance of REAL or LOGICAL is an
identifier or keyword.

I don't understand how neos can be configred to tokenize the above
test lines and other FORTRAN IV statements.

If a native FORTRAN IV compiler can parse it then neos can parse it as
long as it is somehow expressable in the neos attribute grammar.

But that was the question: Can it be parsed in a neos expressible
grammar? Note that those early Fortran grammars were 1) not finite
state, 2) not context free, and 3) inherently ambiguous.It was possible,
for example, to write a "format" and an "assignment" statement that were character to character identical. As an aside, a friend who wrote an
early Fortran compiler that compiled such a chimera so that it would do
an assignment if executed and could be referenced as a format from an IO statement.

The tokenisation step is entirely optional in neos which instead deploys a configurable multi-stage parsing "pipeline" so again there should be no
problem if FORTRAN IV is expressable in the neos attribute grammar given
the problem is itself not intractable as FORTRAN IV compilers already
exist.

/Flibble

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)