• Editing binary data with editors - or is there no difference of text an

    From Janis Papanagnou@21:1/5 to Lawrence D'Oliveiro on Fri Feb 7 10:57:11 2025
    XPost: comp.mobile.android, comp.editors

    On 07.02.2025 06:57, Lawrence D'Oliveiro wrote:
    On Thu, 6 Feb 2025 23:58:04 +0100, Carlos E.R. wrote:

    On 2025-02-06 21:57, Lawrence D'Oliveiro wrote:

    I have done direct editing of binary data in Emacs.

    And I have done so in MsDOS times with primitive text editors, just
    because that was what I had. To change some string.

    Did they preserve null characters in the file?

    Cannot tell for MS DOS environments, but why is that "binary" editing noteworthy in the first place? - Though I may be spoiled by using Vim
    where you can of course also operate on files containing any control
    characters (including ASCII NUL).

    The likely more interesting thing is probably to provide more advanced
    features in _dedicated_ hex editors. - I recall some tools where you
    could edit either the hex values (on the left part of the screen) or
    its string representation (on the right part of the screen). It would
    also make sense to navigate on the binary text in units of "bytes" and
    "byte" offsets.

    In Vim, for example, you either edit the string texts directly. Or use
    any transform tool prior (typical is 'xxd') to operate on hex values.[*]
    (This is of course simple since you just put the respective Vim command
    on some key, to switch to/from hex representation.[**])

    In that respect the poster is correct that you operate on text anyway,
    whether it's the original text (with control characters like NUL) or
    a transformed text (with a hex layout). - But again; a dedicated "hex
    editor" tool might have advantages; it could show data in more than
    one representation, navigate in the binary file more sensibly, etc.

    At least for Vim the distinction between "text" and "binary" files is nonetheless not that important, at least if you are just concerned
    about control characters.[***]

    Janis

    [*] But note that editing the _informative strings_ in hex mode won't
    change the file; in 'xxd' based hex-mode you have to edit the _hex_
    value so that the contents are changed. (There's the difference from
    such more powerful hex editors I mentioned above.)
    Note also that since the "hex-mode" is not a built-in mode in Vim you
    can use arbitrary hex transformation tools. But 'xxd' is the commonly
    used tool with Vim; with that tool you can (for example) also display
    files as binary information, but strangely the function to revert the
    format to store it isn't possible ("sorry, cannot revert this type of hexdump"), which is particularly sad since it would be so simple to
    have it supported by 'xxd'.

    [**] Makes sense in case you do such hex-editing regularly. (I don't
    need that feature; had used it in decades probably only once.)

    [***] For, say, media data files you'd anyway use some media-specific
    (domain specific) "editor".

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Carlos E.R.@21:1/5 to Janis Papanagnou on Fri Feb 7 11:44:05 2025
    XPost: comp.mobile.android, comp.editors

    On 2025-02-07 10:57, Janis Papanagnou wrote:
    On 07.02.2025 06:57, Lawrence D'Oliveiro wrote:
    On Thu, 6 Feb 2025 23:58:04 +0100, Carlos E.R. wrote:

    On 2025-02-06 21:57, Lawrence D'Oliveiro wrote:

    I have done direct editing of binary data in Emacs.

    And I have done so in MsDOS times with primitive text editors, just
    because that was what I had. To change some string.

    Did they preserve null characters in the file?

    Cannot tell for MS DOS environments, but why is that "binary" editing noteworthy in the first place? - Though I may be spoiled by using Vim
    where you can of course also operate on files containing any control characters (including ASCII NUL).

    Nothing, except that some people think it is not possible :-)


    The likely more interesting thing is probably to provide more advanced features in _dedicated_ hex editors. - I recall some tools where you
    could edit either the hex values (on the left part of the screen) or
    its string representation (on the right part of the screen).

    Certainly. PC Tools on plain MsDOS did just that. Probably the Norton
    Utilities did too. What I don't remember doing is inserting a byte/char.

    With PC Tools you could edit a binary file, or directly the raw disk.
    You could edit the FAT table or the directory entries (no structures,
    just raw). A friend of mine created hard linked files that way (which
    would be destroyed by a checkdisk).

    ...

    --
    Cheers, Carlos.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Janis Papanagnou@21:1/5 to Carlos E.R. on Fri Feb 7 14:39:15 2025
    XPost: comp.mobile.android, comp.editors

    On 07.02.2025 11:44, Carlos E.R. wrote:
    On 2025-02-07 10:57, Janis Papanagnou wrote:
    [...]
    The likely more interesting thing is probably to provide more advanced
    features in _dedicated_ hex editors. - I recall some tools where you
    could edit either the hex values (on the left part of the screen) or
    its string representation (on the right part of the screen).

    Certainly. PC Tools on plain MsDOS did just that. Probably the Norton Utilities did too. What I don't remember doing is inserting a byte/char.

    Good point. Inserting is just a normal operation in editors like Vim.
    (And I don't remember that those dedicated hex-editors were capable
    of that. OTOH, there were so many of these specific editors that I'd
    also not be surprised if some supported that.) For certain data that
    feature might be useful, but generally inserting/deleting of binary
    data might likely just create an inconsistent data file.

    But even domain-specific tailored "editors" seem to have issues with
    data consistency.[*]

    Janis

    [*] I recall during the 1990's I had some tools for video processing
    on a Windows computer; the tools were incapable of creating consistent
    results even when staying within the vendor's tools chest. Every single component seemed to do its job correctly on arbitrary data, but one of
    their tool working on the output of another of their tools created just
    trash. It was a well known vendor, but it's name evades my memories.
    On my complaints they had argued that the original data wasn't correct.
    (That was of course the last time that I used their products at all.)

    [...]

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Carlos E.R.@21:1/5 to Janis Papanagnou on Fri Feb 7 19:39:20 2025
    XPost: comp.mobile.android, comp.editors

    On 2025-02-07 14:39, Janis Papanagnou wrote:
    On 07.02.2025 11:44, Carlos E.R. wrote:
    On 2025-02-07 10:57, Janis Papanagnou wrote:
    [...]
    The likely more interesting thing is probably to provide more advanced
    features in _dedicated_ hex editors. - I recall some tools where you
    could edit either the hex values (on the left part of the screen) or
    its string representation (on the right part of the screen).

    Certainly. PC Tools on plain MsDOS did just that. Probably the Norton
    Utilities did too. What I don't remember doing is inserting a byte/char.

    Good point. Inserting is just a normal operation in editors like Vim.
    (And I don't remember that those dedicated hex-editors were capable
    of that. OTOH, there were so many of these specific editors that I'd
    also not be surprised if some supported that.) For certain data that
    feature might be useful, but generally inserting/deleting of binary
    data might likely just create an inconsistent data file.

    Certainly. The common use was to change a message in a program, so one
    edited the text, having the same length. Inserting or deleting would
    foul every jump in the file.


    But even domain-specific tailored "editors" seem to have issues with
    data consistency.[*]

    Janis

    [*] I recall during the 1990's I had some tools for video processing
    on a Windows computer; the tools were incapable of creating consistent results even when staying within the vendor's tools chest. Every single component seemed to do its job correctly on arbitrary data, but one of
    their tool working on the output of another of their tools created just trash. It was a well known vendor, but it's name evades my memories.
    On my complaints they had argued that the original data wasn't correct.
    (That was of course the last time that I used their products at all.)

    {Chuckle}


    [...]



    --
    Cheers, Carlos.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Lawrence D'Oliveiro@21:1/5 to Janis Papanagnou on Sat Feb 8 03:26:39 2025
    XPost: comp.mobile.android, comp.editors

    On Fri, 7 Feb 2025 10:57:11 +0100, Janis Papanagnou wrote:

    The likely more interesting thing is probably to provide more advanced features in _dedicated_ hex editors.

    If you have to develop different editors dedicated to different kinds of
    data, then think of how much of the common functionality you have to
    reinvent: like multibuffer/multifile management, pattern searching,
    selective collapse/expand of parts of buffer contents etc.

    So much less work to add specialized functions for operating on specific
    data types within a generalized editor framework that already has
    provision for such extensibility.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)