• Re: Add JSON, XML, CSV to Baby X resource compiler

    From bart@21:1/5 to Malcolm McLean on Sun May 19 23:41:36 2024
    On 19/05/2024 11:16, Malcolm McLean wrote:
    The Baby X resource compiler takes data - fonts, images, audio, strings
    - and converts them into C source so that they can be read by C programs without relying on external data files.

    An obvious extension is to take in structured data. Adding SQL and
    querying a database would unfortuately mean extending the program so
    that it could only run on a large machine with a SQL server running, and isn't really a viable proposition. However JSON, XML, and CSV are
    commonly used to pass small to medium amounts of data about.

    I've made a start on supporting CSV with the "<dataframe>" tag. CSV data
    is tabular and two dimensional, and lends itself to an arrray of simple structures. JSON And XML can of course represent more complex data, with hierarchy. The dataframe tag is still very experimental. I've never used
    it for anything practical.

    So what would be the best approach to putting in JSON and XML support?

    I've only briefly used XML.

    The problem with XML is that the data it represents is not just
    hierarchical, but it can be chaotic. You can have one lot of data,
    followed by another for something else with a different structure,
    followed by other. It is just a container for disparate sets of data.

    Even if the file does represent a simple list of records for example,
    you won't know that without reading it and analysing it.

    I looked online at an XML to CSV converter, which I thought would do
    something clever, but it seems to just turn each XML line into one
    string per line.

    Maybe it doesn't matter; the user of your program knows what's in their
    XML file, and will know what to do with the different bits. It's their
    problem.

    But you still have to figure out how to represent an arbitary data
    structure as C data. Plus you have to deal with tag names, and attributes.

    Personally, I would suggest using converters (ones clever than the CSV
    one I tried) to turn XML files into better-organised formats first.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael S@21:1/5 to bart on Mon May 20 11:46:01 2024
    On Sun, 19 May 2024 23:41:36 +0100
    bart <bc@freeuk.com> wrote:

    On 19/05/2024 11:16, Malcolm McLean wrote:
    The Baby X resource compiler takes data - fonts, images, audio,
    strings
    - and converts them into C source so that they can be read by C
    programs without relying on external data files.

    An obvious extension is to take in structured data. Adding SQL and
    querying a database would unfortuately mean extending the program
    so that it could only run on a large machine with a SQL server
    running, and isn't really a viable proposition. However JSON, XML,
    and CSV are commonly used to pass small to medium amounts of data
    about.

    I've made a start on supporting CSV with the "<dataframe>" tag. CSV
    data is tabular and two dimensional, and lends itself to an arrray
    of simple structures. JSON And XML can of course represent more
    complex data, with hierarchy. The dataframe tag is still very
    experimental. I've never used it for anything practical.

    So what would be the best approach to putting in JSON and XML
    support?

    I've only briefly used XML.

    The problem with XML is that the data it represents is not just
    hierarchical, but it can be chaotic. You can have one lot of data,
    followed by another for something else with a different structure,
    followed by other. It is just a container for disparate sets of data.

    Even if the file does represent a simple list of records for example,
    you won't know that without reading it and analysing it.


    JSON is about the same.

    I looked online at an XML to CSV converter, which I thought would do something clever, but it seems to just turn each XML line into one
    string per line.

    Maybe it doesn't matter; the user of your program knows what's in
    their XML file, and will know what to do with the different bits.
    It's their problem.

    But you still have to figure out how to represent an arbitary data
    structure as C data. Plus you have to deal with tag names, and
    attributes.

    Personally, I would suggest using converters (ones clever than the
    CSV one I tried) to turn XML files into better-organised formats
    first.





    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Mikko@21:1/5 to bart on Mon May 20 12:23:28 2024
    On 2024-05-19 22:41:36 +0000, bart said:

    On 19/05/2024 11:16, Malcolm McLean wrote:
    The Baby X resource compiler takes data - fonts, images, audio, strings
    - and converts them into C source so that they can be read by C
    programs without relying on external data files.

    An obvious extension is to take in structured data. Adding SQL and
    querying a database would unfortuately mean extending the program so
    that it could only run on a large machine with a SQL server running,
    and isn't really a viable proposition. However JSON, XML, and CSV are
    commonly used to pass small to medium amounts of data about.

    I've made a start on supporting CSV with the "<dataframe>" tag. CSV
    data is tabular and two dimensional, and lends itself to an arrray of
    simple structures. JSON And XML can of course represent more complex
    data, with hierarchy. The dataframe tag is still very experimental.
    I've never used it for anything practical.

    So what would be the best approach to putting in JSON and XML support?

    I've only briefly used XML.

    The problem with XML is that the data it represents is not just
    hierarchical, but it can be chaotic. You can have one lot of data,
    followed by another for something else with a different structure,
    followed by other. It is just a container for disparate sets of data.

    Not just XML. JSON and may other formats have the same features.

    In order to put a resouce to a C program my preference is to convert
    the resource to an array of characters or bytes and process it the
    same way it would be processed if it were read from a file.


    --
    Mikko

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Scott Lurndal@21:1/5 to bart on Mon May 20 16:14:21 2024
    bart <bc@freeuk.com> writes:
    On 19/05/2024 11:16, Malcolm McLean wrote:
    The Baby X resource compiler takes data - fonts, images, audio, strings
    - and converts them into C source so that they can be read by C programs
    without relying on external data files.

    An obvious extension is to take in structured data. Adding SQL and
    querying a database would unfortuately mean extending the program so
    that it could only run on a large machine with a SQL server running, and
    isn't really a viable proposition. However JSON, XML, and CSV are
    commonly used to pass small to medium amounts of data about.

    I've made a start on supporting CSV with the "<dataframe>" tag. CSV data
    is tabular and two dimensional, and lends itself to an arrray of simple
    structures. JSON And XML can of course represent more complex data, with
    hierarchy. The dataframe tag is still very experimental. I've never used
    it for anything practical.

    So what would be the best approach to putting in JSON and XML support?

    I've only briefly used XML.

    That's clear from what you write below.


    The problem with XML is that the data it represents is not just
    hierarchical, but it can be chaotic. You can have one lot of data,
    followed by another for something else with a different structure,
    followed by other. It is just a container for disparate sets of data.

    Even if the file does represent a simple list of records for example,
    you won't know that without reading it and analysing it.

    Study XML Schema. https://en.wikipedia.org/wiki/XML_Schema_(W3C)

    Then study XSL. https://en.wikipedia.org/wiki/XSLT

    XML is a markup language. A subset of SGML. HTML is a non-proper
    and non-regular subset of XML.

    It's far more flexible and useful than a set of comma-separated-values.


    I looked online at an XML to CSV converter, which I thought would do >something clever, but it seems to just turn each XML line into one
    string per line.

    You use stylesheets (XSL) with a stylesheet processor to make
    arbitrary transformations to an XML document. The output can
    be XML, HTML, CSV, or any custom format required for an application.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From bart@21:1/5 to Scott Lurndal on Mon May 20 20:27:15 2024
    On 20/05/2024 17:14, Scott Lurndal wrote:
    bart <bc@freeuk.com> writes:
    On 19/05/2024 11:16, Malcolm McLean wrote:
    The Baby X resource compiler takes data - fonts, images, audio, strings
    - and converts them into C source so that they can be read by C programs >>> without relying on external data files.

    An obvious extension is to take in structured data. Adding SQL and
    querying a database would unfortuately mean extending the program so
    that it could only run on a large machine with a SQL server running, and >>> isn't really a viable proposition. However JSON, XML, and CSV are
    commonly used to pass small to medium amounts of data about.

    I've made a start on supporting CSV with the "<dataframe>" tag. CSV data >>> is tabular and two dimensional, and lends itself to an arrray of simple
    structures. JSON And XML can of course represent more complex data, with >>> hierarchy. The dataframe tag is still very experimental. I've never used >>> it for anything practical.

    So what would be the best approach to putting in JSON and XML support?

    I've only briefly used XML.

    That's clear from what you write below.

    I have enough experience to know that it CAN represent disparate data
    just like I said, since I've had to generate exactly such files as input
    into another application that required such data.


    Even if the file does represent a simple list of records for example,
    you won't know that without reading it and analysing it.

    Study XML Schema.

    I have no interest in studying XML or every using again. I already
    stated in an earlier post that it's more complicated than it looks.


    https://en.wikipedia.org/wiki/XML_Schema_(W3C)

    Then study XSL. https://en.wikipedia.org/wiki/XSLT

    XML is a markup language. A subset of SGML. HTML is a non-proper
    and non-regular subset of XML.

    It's far more flexible and useful than a set of comma-separated-values.

    And, therefore, 'chaotic' it what can be represented, even if
    technically it can be described by a recursively defined syntax.

    Or you need to know is that XML can represent the syntactic structure of
    most programming languages, and we all know how easy that is to
    represent as a fixed set of initialised C data structures hardcoded into
    a source file.



    I looked online at an XML to CSV converter, which I thought would do
    something clever, but it seems to just turn each XML line into one
    string per line.

    You use stylesheets (XSL) with a stylesheet processor to make
    arbitrary transformations to an XML document. The output can
    be XML, HTML, CSV, or any custom format required for an application.

    The OP wanted to be able to directly process XML; I suggest that it
    first be transformed into something more regular.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Mikko@21:1/5 to Malcolm McLean on Tue May 21 11:27:46 2024
    On 2024-05-20 10:58:20 +0000, Malcolm McLean said:

    On 20/05/2024 10:23, Mikko wrote:
    On 2024-05-19 22:41:36 +0000, bart said:

    On 19/05/2024 11:16, Malcolm McLean wrote:
    The Baby X resource compiler takes data - fonts, images, audio, strings >>>> - and converts them into C source so that they can be read by C
    programs without relying on external data files.

    An obvious extension is to take in structured data. Adding SQL and
    querying a database would unfortuately mean extending the program so
    that it could only run on a large machine with a SQL server running,
    and isn't really a viable proposition. However JSON, XML, and CSV are
    commonly used to pass small to medium amounts of data about.

    I've made a start on supporting CSV with the "<dataframe>" tag. CSV
    data is tabular and two dimensional, and lends itself to an arrray of
    simple structures. JSON And XML can of course represent more complex
    data, with hierarchy. The dataframe tag is still very experimental.
    I've never used it for anything practical.

    So what would be the best approach to putting in JSON and XML support?

    I've only briefly used XML.

    The problem with XML is that the data it represents is not just
    hierarchical, but it can be chaotic. You can have one lot of data,
    followed by another for something else with a different structure,
    followed by other. It is just a container for disparate sets of data.

    Not just XML. JSON and may other formats have the same features.

    In order to put a resouce to a C program my preference is to convert
    the resource to an array of characters or bytes and process it the
    same way it would be processed if it were read from a file.


    And of course the Baby X resource compiler already supports that. You
    can convert XML or JSON to a string, and then run your own parser over
    it at runtime.

    But that isn't really a very good solution.

    Depends on the problem. If an application needs a resource that is
    not needed by many other applications an ad hoc parser can produce
    a result structure that a generic parser cannot do. Even then it may
    be desiderable to use a format that could be parsed as XML or JSON.

    --
    Mikko

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Ben Bacarisse@21:1/5 to Malcolm McLean on Wed May 22 12:43:43 2024
    Malcolm McLean <malcolm.arthur.mclean@gmail.com> writes:

    On 20/05/2024 20:27, bart wrote:
    On 20/05/2024 17:14, Scott Lurndal wrote:
    bart <bc@freeuk.com> writes:
    On 19/05/2024 11:16, Malcolm McLean wrote:
    The Baby X resource compiler takes data - fonts, images, audio, strings >>>>> - and converts them into C source so that they can be read by C
    programs
    without relying on external data files.

    An obvious extension is to take in structured data. Adding SQL and
    querying a database would unfortuately mean extending the program so >>>>> that it could only run on a large machine with a SQL server running, >>>>> and
    isn't really a viable proposition. However JSON, XML, and CSV are
    commonly used to pass small to medium amounts of data about.

    I've made a start on supporting CSV with the "<dataframe>" tag. CSV
    data
    is tabular and two dimensional, and lends itself to an arrray of simple >>>>> structures. JSON And XML can of course represent more complex data,
    with
    hierarchy. The dataframe tag is still very experimental. I've never
    used
    it for anything practical.

    So what would be the best approach to putting in JSON and XML support? >>>>
    I've only briefly used XML.

    That's clear from what you write below.
    I have enough experience to know that it CAN represent disparate data
    just like I said, since I've had to generate exactly such files as input
    into another application that required such data.

    Even if the file does represent a simple list of records for example,
    you won't know that without reading it and analysing it.

    Study XML Schema.
    I have no interest in studying XML or every using again. I already stated
    in an earlier post that it's more complicated than it looks.
       https://en.wikipedia.org/wiki/XML_Schema_(W3C)

    Then study XSL.     https://en.wikipedia.org/wiki/XSLT

    XML is a markup language.   A subset of SGML.  HTML is a non-proper
    and non-regular subset of XML.

    It's far more flexible and useful than a set of comma-separated-values.
    And, therefore, 'chaotic' it what can be represented, even if technically
    it can be described by a recursively defined syntax.
    Or you need to know is that XML can represent the syntactic structure of
    most programming languages, and we all know how easy that is to represent
    as a fixed set of initialised C data structures hardcoded into a source
    file.

    That is one answer. XML and JSON are trees, with certain contraints on the nodes. So we could devise a C struct which represents a node, and spit out
    a rooted tree.



    I looked online at an XML to CSV converter, which I thought would do
    something clever, but it seems to just turn each XML line into one
    string per line.

    You use stylesheets (XSL) with a stylesheet processor to make
    arbitrary transformations to an XML document.  The output can
    be XML, HTML, CSV, or any custom format required for an application.
    The OP wanted to be able to directly process XML; I suggest that it first
    be transformed into something more regular.

    That seems to be a better approach. Data represents something in the real world. And normally there are a lot of constraints on it. A temperature measurement might be missing, but it must be real, it can't be
    imaginary. Even though XML will of course allow you to put "<REAL>" and "<IMAGINARY>" tags under the temperature tag if you so desire.

    It's already been pointed out, but the reason people use XML is that XML
    files can be validated again a schema that can prevent exactly this kind
    of error.

    --
    Ben.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)