1. Forum
  2. Usenet
  3. LINUX.DEBIAN.USER

  • Re: Shell function & variable usage.

    From Greg Wooledge@21:1/5 to peter@easthope.ca on Sat Dec 7 00:50:01 2024
    On Fri, Dec 06, 2024 at 15:11:51 -0700, peter@easthope.ca wrote:
    (2)
    exclusions='*.mp3'
    exclusions+='*.mp4'
    Restore() { \
    source=somewhere/Backup/ ;
    echo "source is $source." ;
    destination=elsewhere/workingDirectory ;
    echo "destination is $destination." ;
    rsync \
    -anuv --exclude=$exclusions $source $destination ; }

    In a realistic case, there are more than two exclusion patterns.

    Store the exclusions in an ARRAY, not in a string. Then create a
    second array which contains the spelled-out --exclude=... options.

    #!/bin/bash
    exclusions=( '*.mp3' '*.mp4' '*.wav' )
    exclude=( "${exclusions[@]/#/--exclude=}" )
    source=...
    destination=...
    rsync -anuv "${exclude[@]}" "$source" "$destination"

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From peter@easthope.ca@21:1/5 to All on Sat Dec 7 00:40:01 2024
    Hypothetical shell usages (1) & (2).

    (1)
    Restore() { \
    source=somewhere/Backup/ ;
    echo "source is $source." ;
    destination=elsewhere/workingDirectory ;
    echo "destination is $destination." ;
    rsync \
    --exclude '*.mp3' \
    --exclude '*.mp4' \
    -anuv $source $destination ; }

    (2)
    exclusions='*.mp3'
    exclusions+='*.mp4'
    Restore() { \
    source=somewhere/Backup/ ;
    echo "source is $source." ;
    destination=elsewhere/workingDirectory ;
    echo "destination is $destination." ;
    rsync \
    -anuv --exclude=$exclusions $source $destination ; }

    In a realistic case, there are more than two exclusion patterns.
    Comments or suggestions about the two possibilities? Astonishingly
    better ideas? =8~o

    Thanks, ... P.

    --
    VoIP: +1 604 670 0140
    work: https://en.wikibooks.org/wiki/User:PeterEasthope

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From eben@gmx.us@21:1/5 to peter@easthope.ca on Sat Dec 7 00:50:02 2024
    On 12/6/24 17:11, peter@easthope.ca wrote:

    These

    rsync \
    --exclude '*.mp3' \
    --exclude '*.mp4' \

    rsync \
    --exclude=$exclusions

    are not the same. In the first one you have "--exclude" for each pattern,
    and in the second you have it once. I suspect the first one is correct, and for that reason I vote for that one, even those typing "--exclude" several times gives several opportunities for error. You could do something overengineered like defining an array "excludepatterns" for patterns, and
    then doing something like

    rsync \
    $(for pattern in ${excludepatterns[@]} ; do
    echo -- "--exclude '$pattern'"
    done)

    Modulo mistakes, of course. And make sure it deals with wildcards, quotes,
    and spaces appropriately.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From eben@gmx.us@21:1/5 to eben@gmx.us on Sat Dec 7 01:00:01 2024
    On 12/6/24 18:46, eben@gmx.us wrote:
    overengineered

    Greg's solution is better.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to eben@gmx.us on Sat Dec 7 01:00:01 2024
    On Fri, Dec 06, 2024 at 18:46:06 -0500, eben@gmx.us wrote:
    You could do something
    overengineered like defining an array "excludepatterns" for patterns, and then doing something like

    rsync \
    $(for pattern in ${excludepatterns[@]} ; do
    echo -- "--exclude '$pattern'"
    done)

    Modulo mistakes, of course.

    This definitely has mistakes. You're expanding each of the patterns
    based on the files in the current working directory.

    hobbit:~$ exclusions=( '*.mp3' '*.mp4' '*.wav' )
    hobbit:~$ echo *.mp3
    christmas-peebs.mp3 inmexico.mp3 Kristie_and_John_-_Bobby_McGee.mp3
    hobbit:~$ for p in ${exclusions[@]}; do echo -- "--exclude '$p'"; done
    -- --exclude 'christmas-peebs.mp3'
    -- --exclude 'inmexico.mp3'
    -- --exclude 'Kristie_and_John_-_Bobby_McGee.mp3'
    -- --exclude '*.mp4'
    -- --exclude '*.wav'

    That's not what you want. (You're also writing a literal -- which is
    not wanted.)

    As I wrote in the previous message, you can generate the second array
    using a single command that doesn't perform pathname expansions:

    hobbit:~$ opts=( "${exclusions[@]/#/--exclude=}" )
    hobbit:~$ declare -p opts
    declare -a opts=([0]="--exclude=*.mp3" [1]="--exclude=*.mp4" [2]="--exclude=*.wav")

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Karl Vogel@21:1/5 to peter@easthope.ca on Sat Dec 7 12:40:01 2024
    On Fri 06 Dec 2024 at 18:30:27 (-0500), peter@easthope.ca wrote:
    Hypothetical shell usages (1) & (2).

    (1)
    Restore() { \
    source=somewhere/Backup/ ;
    destination=elsewhere/workingDirectory ;
    rsync \
    --exclude '*.mp3' \
    --exclude '*.mp4' \
    -anuv $source $destination ; }

    If you want to modify exclusions, you have to edit the code.

    (2)
    exclusions='*.mp3'
    exclusions+='*.mp4'
    Restore() { \
    source=somewhere/Backup/ ;
    destination=elsewhere/workingDirectory ;
    rsync \
    -anuv --exclude=$exclusions $source $destination ; }

    I would put

    *.mp[34]

    in a separate config file. I usually name those after the script to
    make them easier to find:

    #!/usr/bin/env bash
    export PATH=/usr/local/bin:/bin:/usr/bin
    tag=${0##*/}
    ...

    excfile="/usr/local/etc/${tag}-exclude"
    test -s "$excfile" || { echo die screaming; exit 1; }
    ...

    rsync -anuv --exclude-from="$excfile" "$source" "$destination"

    --
    Karl Vogel I don't speak for anyone but myself

    COMMENT: That's an amazing deal, would snap it up if I had the money.
    REPLY: You can eat next month. --Reddit on buying storage, 21 Nov 2024

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From peter@easthope.ca@21:1/5 to All on Sat Dec 7 15:40:02 2024
    From: Greg Wooledge <greg@wooledge.org>
    Date: Fri, 6 Dec 2024 18:44:04 -0500
    Store the exclusions in an ARRAY, not in a string. Then create a
    second array which contains the spelled-out --exclude=... options.

    Unfamiliar to me & interesting. What benefits outweigh the additional complexity?

    Thanks, ... P.

    --
    VoIP: +1 604 670 0140
    work: https://en.wikibooks.org/wiki/User:PeterEasthope

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to peter@easthope.ca on Sat Dec 7 17:10:01 2024
    On Sat, Dec 07, 2024 at 06:19:20 -0700, peter@easthope.ca wrote:
    From: Greg Wooledge <greg@wooledge.org>
    Date: Fri, 6 Dec 2024 18:44:04 -0500
    Store the exclusions in an ARRAY, not in a string. Then create a
    second array which contains the spelled-out --exclude=... options.

    Unfamiliar to me & interesting. What benefits outweigh the additional complexity?

    It's LESS complexity!

    Trying to shove a list of data elements into a string is ALWAYS worse
    than storing your list of data elements in an array.

    Consider what happens when one of your data elements has a space in it.
    What are you going to do then?

    If you're storing the list in a string with spaces between elements,
    then you need to add some sort of quoting or escaping *inside* the
    string.

    Now it's more complex than if you had simply used the array in the
    first place.

    Array assignments in bash also allow you to use free form syntax, and
    even to add comments.

    exclusions=(
    '*.mp3'
    '*.mp4' # added 2024-12-07 due to ...
    '*[Rr]ed [Hh]ot*'
    '*.wav' '*.snd' '*.au'
    )

    Now, you may be thinking "Hey, you said you wouldn't need to add quoting
    inside the array, but those are quotes!" The difference between this
    and a string is the quotes in the array assignment are parsed by the
    shell and removed.

    hobbit:~$ exclusions=(
    '*.mp3'
    '*.mp4' # added 2024-12-07 due to ...
    '*[Rr]ed [Hh]ot*'
    '*.wav' '*.snd' '*.au'
    )
    hobbit:~$ declare -p exclusions
    declare -a exclusions=([0]="*.mp3" [1]="*.mp4" [2]="*[Rr]ed [Hh]ot*" [3]="*.wav" [4]="*.snd" [5]="*.au")

    Each array element is ready to go. No additional parsing is needed.
    If you want to do something like "convert this list of globs into a
    list of --exclude=GLOB options", it's just one parameter expansion.
    I've shown it already, but here it is again:

    "${exclusions[@]//#/--exclude=}"

    In a string, the quotes are literally placed inside the string, and
    become part of the data payload. You have to write your *own* parsing
    code to separate the data elements and remove the quotes. The shell
    won't be able to do it for you.

    hobbit:~$ unset exclusions
    hobbit:~$ exclusions='*.mp3 *.mp4 "*[Rr]ed [Hh]ot*" *.wav *.snd *.au'
    hobbit:~$ declare -p exclusions
    declare -- exclusions="*.mp3 *.mp4 \"*[Rr]ed [Hh]ot*\" *.wav *.snd *.au"

    That's so much harder to deal with. You can't do anything with it until
    you parse it back out into a list. Essentially you'll be converting
    the string into an array, and then using the array operations that I've
    already shown.

    It's so much easier just to use an array in the first place.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From peter@easthope.ca@21:1/5 to All on Sun Dec 8 19:50:01 2024
    From: Greg Wooledge <greg@wooledge.org>
    Date: Sat, 7 Dec 2024 11:08:54 -0500
    "${exclusions[@]//#/--exclude=}"

    In a shell script; correct?

    I'm making a shell function declared in .bashrc. Numerous syntactical variations were unsuccessful. Maybe a simple escape syntax is needed.

    Stretched a test out to this. it works.

    Test() { \
    exclusions=( \
    'file' \
    '*.mp3' \
    '*.mp4' );
    echo "exclusions = ${exclusions[@]}";
    n=${#exclusions[@]};
    i=0;
    while [ "$i" -lt "$n" ]; do
    echo ${exclusions[$i]};
    exclusions[$i]="--exclude=${exclusions[$i]}";
    ((i++));
    done;
    echo "i = $i";
    echo "exclusions = ${exclusions[@]}";
    }

    When I'm happy with Test, will incorporate into the Backup function.

    Thanks for the help, ... P.

    --
    VoIP: +1 604 670 0140
    work: https://en.wikibooks.org/wiki/User:PeterEasthope

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Anssi Saari@21:1/5 to peter@easthope.ca on Wed Dec 11 14:30:02 2024
    peter@easthope.ca writes:

    In a realistic case, there are more than two exclusion patterns.
    Comments or suggestions about the two possibilities? Astonishingly
    better ideas? =8~o

    Put the exlucde patterns in a file and use --exclude-from=file? Works
    for me, so far with tar and bup, rsync has the same support.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)