• Re: Help with UDF and BluRay disks

    From Thomas Schmitt@21:1/5 to Bob McGowan on Sat Jan 18 09:50:01 2025
    Hi,

    Bob McGowan wrote:
    I've been trying to figure out how to use my BD disc writer to create
    backups of files.

    I do this by Rock Ridge enhanced ISO 9660 on BD-R and BD-RE media,
    following this example from the man page of xorriso:

    xorriso \
    -abort_on FATAL \
    -for_backup -disk_dev_ino on \
    -assert_volid 'PROJECTS_MAIL_*' FATAL \
    -dev /dev/sr0 \
    -volid PROJECTS_MAIL_"$(date '+%Y_%m_%d_%H%M%S')" \
    -not_leaf '*.o' -not_leaf '*.swp' \
    -update_r /home/thomas/projects /projects \
    -update_r /home/thomas/personal_mail /personal_mail \
    -commit -toc -check_md5 FAILURE -- -eject all

    To be used several times on the same medium, whenever an update of the
    two disk trees to the medium is desired. Begin with a blank medium and
    update it until the run fails gracefully due to lack of remaining space
    on the old one.

    This makes sense if the full backup leaves substantial remaining
    capacity on media and if the expected changes are much smaller than the
    full backup. To apply zisofs compression to those data files which get
    newly copied from the local filesystem, insert these commands
    immediately before -commit :
    -hardlinks perform_update \
    -find / -type f -pending_data -exec set_filter --zisofs -- \


    What I first found were instructions to create an empty file of the
    propper size, 'mkudffs file', loop mount it, copy files to it, unmount
    and burn to the BD disc.

    To my experience, UDF offers no practical advantages over ISO 9660 when
    you plan to read the backup by GNU/Linux. Some other free operating
    systems have problems iin ISO 9660 filesystems with files of size 4 GiB or larger. MS-Windows will show the dull ISO 9660 names if you do not give
    xorriso command
    -joliet on
    before the xorriso command -commit.

    You could use the program genisoimage with its option -udf together
    with a burn program like growisofs, cdrecord, or xorrecord. This produces
    an older version of UDF suitable for DVD video but not for Blu-ray video.

    growisofs on Debian uses genisoimage by default. Its man page proposes:

    growisofs -Z /dev/dvd -R -J /some/files

    You would add genisoimage option -udf.

    The other burn programs can take their data strem from a pipe fed by genisoimage. Like:

    genisoimage -R -J -udf /some/files \
    | xorrecord -v dev=/dev/sr0 -waiti -eject -

    Multi-session with genisoimage -udf is known to be problematic (or
    impossible). Your apporach with mkudffs might be capable of taking file
    changes on BD-RE media when they are mounted with rw-premissions.
    (If you can get it mounted, that is.)


    Further searching found that the recommended way to access UDF
    filesystems on optical media is to use "packet writing".

    This old statement applies to writing, not to reading. Further it is
    mostly about writing of CD media and some DVD media types. BD media can
    get written by several write types which all resemble CD packet writing.

    But in your case of mkudffs operating on a filesystem image file there
    is no special medium involved. This approach assumes that you prepare
    a mountable image file and flatly copy it to a medium by whatever
    write type is suitable.


    For the life of me I cannot figure out how to make this work. None of
    the suggestions I've found work with my setup.

    Did you eject your BD and reload it before mounting it for looking at
    its content ?
    The Linux kernel offers no way to tell it to re-assess a freshly written /dev/sr device. You have to force re-assessment by physically reloading
    the medium.

    If this in-out dance does not help, try to mount your UDF filesystem image
    file before copying it to a BD:

    sudo mkdir /mnr/udf
    sudo mount /my/udf_image_file /mnt/udf

    If this works, then you know that your way of writing to BD is flawed.
    In this case, describe it here in more detail.
    If mount already fails with the UDF image file, then your mkudffs run is flawed. In this case, describe exactly how you do it.


    It may be that my BD drive is too old but before going the route of
    changing it out, I'd like to know if there's something I've missed in
    the general process.

    Try a simple xorriso run. Assuming that your drive is at /dev/sr0:

    disk_dir="...path.to.some.directory..."
    iso_dir="...name.for.the.directory.on.the.BD.medium..."

    xorriso -outdev /dev/sr0 -blank as_needed \
    -for_backup \
    -map "$disk_dir" "$iso_dir" \
    -commit -toc -check_md5 FAILURE -- -eject all

    This will copy the tree under "$disk_dir" onto the BD medium and then
    perform a check-read run to verify readbility and matching MD5 checksum
    of the overall ISO 9660 filesystem.
    No UDF involved, of course.

    If your BD burner has a tray motor, then the tray is supposed to come
    out of the drive when the checksum matchies. In case of mismatch it will
    stay in and you will have to eject it manually.
    If the xorriso run states in the end:

    Ok, session data match recorded md5.

    then the medium should be mountable.
    If the drive is bad, then the checksum will hardly match.


    The DB burner is an LG and is quite old, as in maybe 15 years.

    My oldest one is a LG GGW-H20L from 2008. It survived the demise of two
    USB boxes and now lives in its third home.
    It cannot burn modern BD-R media or format modern BD-RE. But it works
    with BD-RE formatted by a LG BH16NS40 from 2013. It fails to write BD-RE
    which were formatted by a newer ASUS BW-16D1HT. In this case the burn run reports errors.


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Sarunas Burdulis@21:1/5 to All on Sat Jan 18 20:30:02 2025
    This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------G7TJxsW85QiwhkUFpnU0Z0a8
    Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64

    QXJlIHlvdSB0cnlpbmcgdG8gc2NyaXB0L2F1dG9tYXRlPw0KDQpJZiBub3QsIGp1c3QgdXNl IHhmYnVybiAoR1VJKS4gSSdtIHdyaXRpbmcgQmx1cmF5IDI1R2lCIE0tZGlza3MgdXNpbmcg DQp4ZmJ1cm4gcmVndWxhcmx5IChhcmNoaXZpbmcgYSBiYWNrdXBzJyBzbmFwc2hvdCBkaXJl Y3Rvcnkgb24gTkFTKS4NCg0KLS0gDQrFoGFyxatuYXMgQnVyZHVsaXMNCkRhcnRtb3V0aCBN YXRoZW1hdGljcw0KaHR0cHM6Ly9tYXRoLmRhcnRtb3V0aC5lZHUvfnNhcnVuYXMNCg0Kwrcg aHR0cHM6Ly91c2VwbGFpbnRleHQuZW1haWwgwrcNCg==

    --------------G7TJxsW85QiwhkUFpnU0Z0a8--

    -----BEGIN PGP SIGNATURE-----

    wsB5BAABCAAjFiEE5ODlqx+pLMu9Wq48Bw+NpurIYD0FAmeMABMFAwAAAAAACgkQBw+NpurIYD3Y 7wgAkLkDswGFTRuhpXGOtahtE7rXmCd+7k+ScNxVKTE47AA2xB6dFd70LoHrYn1chpHdr1gyFiGW m+7weLbl1+3JXCkjwGpH69fYkHPn8ixubgm55v1o6+8V7+2z23Pi5YEicksULguhEcFjHID7TDgl /gk7+B9XmmgqKdQnWPUxzlkuaYTikxVJMDbUDX/R9CSz2sP1qdX81WuAJgvlmDyPD4lck8hSjcuA qcOw2+/33pZatnzjTKS8C//AEIOj5SLQwpqGZpDwIB5clVRaiCgeVGT7Df1qJPXjD61fvJiy/1nn QrDMXovPRRiwHRLmxHpDhhUSmTsRKaiuIBASUaLzaA==
    =Qg7C
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Sarunas Burdulis on Sat Jan 18 21:40:01 2025
    Hi,

    Sarunas Burdulis wrote:
    just use xfburn (GUI).

    Xfburn does indeed Blu-ray by help of libburn.
    But it does no UDF, because it uses libisofs for filesystem production.
    Insofar the result is supposed to be similar to the results of the xorriso
    runs which i proposed, but without MD5 checksums in the filesystem.

    The question remains whether UDF is really an indispensable feature of
    the intended backups.


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Bob McGowan on Sun Jan 19 09:40:02 2025
    Hi,

    Bob McGowan wrote:
    One question, what I would like to see is a duplicated directory/file hierachy on the destination. I have lists of file names in groups just
    short of 25G, but I can't find an easy way to send the file names to
    any of the programs and have them maintain the hierarchy. They just
    dump all the files in the root of the destination.

    I did manage this for a few files with 'xorriso -update_r' option used
    once per file name and with the full destination path and file name
    listed:

    xorriso -update_r /home/me/topdir/file /topdir/file ...

    But doing this for thousands of files doesn't seem like it is a
    manageable or robust solution.

    It's not unfeasible.
    man xorriso says:

    -options_from_file fileaddress
    Read quoted input from fileaddress and execute it like dialog
    lines. Empty lines and lines which begin by # are ignored.
    Normally one line should hold one xorriso command and all its
    parameters. Nevertheless lines may be concatenated by a
    trailing backslash.
    See also section "Command processing", paragraph "Quoted input".

    That paragraph says:

    Quoted input converts whitespace-separated text into words. The double
    quotation mark " and the single quotation mark ' can be used to enclose
    whitespace and make it part of words (e.g. of file names). Each mark
    type can enclose the marks of the other type. A trailing backslash \
    outside quotations or an open quotation cause the next input line to be
    appended.
    [...]

    So you may quote whitespace like blanks, tabs or newline characters like
    in the shell:

    -update_r "/home/me/topdir/filename with blanks" "/topdir/filename with blanks"
    ...more.update_r.commands...

    Since you seem not to strive for adding sessions to already existing
    ISO 9660 filesystems, you may also use -map or -map_single :

    -map "/home/me/topdir/filename with blanks" "/topdir/filename with blanks"
    ...more.map.commands...

    about which the man page says:

    -map disk_path iso_rr_path
    Insert file object disk_path into the ISO image as iso_rr_path.
    If disk_path is a directory then its whole sub tree is inserted
    into the ISO image.

    -map_single disk_path iso_rr_path
    Like -map, but if disk_path is a directory then its sub tree is
    not inserted.

    I.e. these two commands do not ponder whether the iso_rr_path already
    leads to an up-to-date version of the file in the ISO filesystem, but
    rather copy disk_path to iso_rr_path unconditionally.

    You would write all intended xorriso commands into a disk file, e.g.
    /home/me/xorriso_commands
    and then let xorriso process it by command -options_from_file like in:

    xorriso \
    -for_backup \
    -outdev /dev/sr0 \
    -blank as_needed \
    -options_from_file /home/me/xorriso_commands \
    -commit -toc -check_md5 FAILURE -- -eject all

    -----------------------------------------------------------------------

    Another approach is to create a file with a list of pathspecs which are
    a concept from program mkisofs:

    -path_list disk_path
    Like -add but read the parameter words from file disk_path or
    standard input if disk_path is "-". The list must contain
    exactly one pathspec or disk_path pattern per line.

    The command -add inserts files into the emerging ISO image in a way that
    is usual with mkisofs:

    -add pathspec [...] | disk_path [***]
    Insert the given files or directory trees from filesystem into
    the ISO image.
    If -pathspecs is set to "on" or "as_mkisofs" then pattern
    expansion is always disabled and character '=' has a special
    meaning. It separates the ISO image path from the disk path:
    iso_rr_path=disk_path
    Character '=' in the iso_rr_path must be escaped by '\' (i.e. as
    "\=").
    With -pathspecs "on", the character '\' must not be escaped. The
    character '=' in the disk_path must not be escaped.
    With -pathspecs "as_mkisofs", all characters '\' must be escaped
    in both, iso_rr_path and disk_path. The character '=' may or may
    not be escaped in the disk_path.
    If iso_rr_path does not begin with '/' then -cd is prepended.
    If disk_path does not begin with '/' then -cdx is prepended.
    If no '=' is given then the word is used as both, iso_rr_path
    and disk path. If in this case the word does not begin with '/'
    then -cdx is prepended to the disk_path and -cd is prepended to
    the iso_rr_path.
    ...

    -pathspecs "on"|"off"|"as_mkisofs"
    Control parameter interpretation with xorriso actions -add and
    -path_list.
    Mode "as_mkisofs" enables pathspecs of the form
    iso_rr_path=disk_path
    like with program mkisofs -graft-points.
    ...

    With this approach you would write lines like

    /topdir/file=/home/me/topdir/file
    /topdir/filename with blanks=/home/me/topdir/filename with blanks
    /topdir/filename with \= in it=/home/me/topdir/filename with = in it

    into a disk file, e.g.
    /home/me/xorriso_pathspecs
    and perform in -path_list xorriso:

    xorriso \
    -for_backup \
    -outdev /dev/sr0 \
    -blank as_needed \
    -pathspecs as_mkisofs \
    -path_list /home/me/xorriso_pathspecs \
    -commit -toc -check_md5 FAILURE -- -eject all

    -----------------------------------------------------------------------
    How i handle my larger backups:

    I assume that you want to split a larger collection of files into
    groups which fit on the BD media.
    Nearly half a life ago, my own endeavor with ISO 9660 and optical media
    began with creating a tool which does this splitting automatically:

    http://scdbackup.webframe.org/main_eng.html
    http://scdbackup.webframe.org/examples.html

    BD is configured and handled by the DVD configuration and commands of scdbackup.
    Once:

    ./CONFIGURE_DVD

    With each backup, you'd do something like:

    sdvdbackup /topdir=/home/me/topdir -not /home/me/topdir/temp_files

    Back then there was only mkisofs for producing ISO 9660. So sdvdbackup
    uses its pathspecs notation for defining the mapping from disk to BD.
    Of course i meanwhile use xorriso for the roles of cdrecord, growisofs
    and mkisofs.

    I'm still backing up multi-media file collections by help of sdvdbackup. Backups which fill dozens of media might become lengthy. So there is also
    the opportunity to perform incremental backups:

    http://scdbackup.webframe.org/examples.html#incremental

    -----------------------------------------------------------------------

    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Bob McGowan on Mon Jan 20 10:00:01 2025
    Hi,

    Bob McGowan wrote:
    The -options_from_file is
    exactly what I was needing and works perfectly.

    Congrats.
    I'm glad that UDF was not a hard requirement.


    Just one last question. How do you pronounce "xorriso"? :)

    Rarely. :))
    Normally i only write about it. But i think of it with german
    pronounciation. Like an english speaker would read: "ksorr-ee-so".

    The name is a contraction of "X/Open on Rock Ridge enhanced ISO 9660".
    So "ksorr-eye-so" is the proper english pronounciation.
    (Due to its purpose and the proximity to some words from the iberian
    peninsula, i would visualize it as a smiling sausage which burns at
    both ends.)


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Eike Lantzsch ZP5CGE / KY4PZ@21:1/5 to All on Mon Jan 20 12:20:02 2025
    On Monday, 20 January 2025 05:49:19 GMT-4 Thomas Schmitt wrote:
    Hi,

    Bob McGowan wrote:
    The -options_from_file is
    exactly what I was needing and works perfectly.

    Congrats.
    I'm glad that UDF was not a hard requirement.

    Just one last question. How do you pronounce "xorriso"? :)

    Rarely. :))
    Normally i only write about it. But i think of it with german
    pronounciation. Like an english speaker would read: "ksorr-ee-so".

    The name is a contraction of "X/Open on Rock Ridge enhanced ISO 9660".
    So "ksorr-eye-so" is the proper english pronounciation.
    (Due to its purpose and the proximity to some words from the iberian peninsula, i would visualize it as a smiling sausage which burns at
    both ends.)


    Have a nice day :)

    Thomas
    Muchas gracias por el chorriso ;-)

    --
    Eike Lantzsch KY4PZ / ZP5CGE

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From tomas@tuxteam.de@21:1/5 to Thomas Schmitt on Mon Jan 20 17:00:01 2025
    On Mon, Jan 20, 2025 at 09:49:19AM +0100, Thomas Schmitt wrote:

    [...]

    (Due to its purpose and the proximity to some words from the iberian peninsula, i would visualize it as a smiling sausage which burns at
    both ends.)

    Yikes. Chorizo al infierno :-)

    Cheers
    --
    t

    -----BEGIN PGP SIGNATURE-----

    iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCZ45xPQAKCRAFyCz1etHa Rjc3AJ9NBN5rbwefuLiQBH480btH3Spm2wCcCZIzh9gsuH3kUvfVBep9PBRMTMw=
    =Bogc
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Roy J. Tellason, Sr.@21:1/5 to All on Mon Jan 20 21:20:01 2025
    On Sunday 19 January 2025 03:37:06 am Thomas Schmitt wrote:
    Nearly half a life ago, my own endeavor with ISO 9660 and optical media
    began with creating a tool which does this splitting automatically:

      http://scdbackup.webframe.org/main_eng.html
      http://scdbackup.webframe.org/examples.html

    BD is configured and handled by the DVD configuration and commands of scdbackup.
    Once:

      ./CONFIGURE_DVD

    With each backup, you'd do something like:

      sdvdbackup /topdir=/home/me/topdir -not /home/me/topdir/temp_files

    Back then there was only mkisofs for producing ISO 9660. So sdvdbackup
    uses its pathspecs notation for defining the mapping from disk to BD.
    Of course i meanwhile use xorriso for the roles of cdrecord, growisofs
    and mkisofs.

    I'm still backing up multi-media file collections by help of sdvdbackup. Backups which fill dozens of media might become lengthy. So there is also
    the opportunity to perform incremental backups:

      http://scdbackup.webframe.org/examples.html#incremental

    Interesting stuff! All three of the machiines in here have DVD writers in them, so this looks like it might prove useful. That's one hell of a big README file! :-)

    What all are the rest of you folks using for backup?


    --
    Member of the toughest, meanest, deadliest, most unrelenting -- and
    ablest -- form of life in this section of space,  a critter that can
    be killed but can't be tamed.  --Robert A. Heinlein, "The Puppet Masters"
    -
    Information is more dangerous than cannon to a society ruled by lies. --James M Dakin

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Thomas Schmitt@21:1/5 to Sr. on Mon Jan 20 22:10:02 2025
    Hi,

    i wrote:
    Nearly half a life ago, my own endeavor with ISO 9660 and optical media began with creating a tool which does this splitting automatically:

      http://scdbackup.webframe.org/main_eng.html
      http://scdbackup.webframe.org/examples.html

    Roy J. Tellason, Sr. wrote:
    That's one hell of a big README file! :-)

    And it is old. Device addresses like "ATA:0,0,0" or "/dev/hda", Linux file
    size limit of 2 GiB, image buffer file to prevent burn buffer underruns, confusing germanisms in the text, CD-only burners, afio archives, blowfish encryption, Linux kernel module ide-scsi, IDE slave devices, ...
    83 % of the text describe inner details beyond installation and normal operation.

    As compensation there is no man page. :))
    The one of xorriso-1.5.7 has 5993 lines.


    Have a nice day :)

    Thomas

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tim Woodall@21:1/5 to Bob McGowan on Fri Jan 24 19:30:01 2025
    On Fri, 17 Jan 2025, Bob McGowan wrote:

    Hello list,

    I've been trying to figure out how to use my BD disc writer to create
    backups of files.

    What I first found were instructions to create an empty file of the
    propper size, 'mkudffs file', loop mount it, copy files to it, unmount
    and burn to the BD disc.


    I've not read the rest of the thread but I think you need to be careful
    to set the blocksize in this use case.

    In my notes I have:
    mkudffs --blocksize=2048 $LD

    (where LD is the loop device)

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)