• Cleanup in a bash script

    From Tim Woodall@21:1/5 to All on Sat Sep 28 16:00:01 2024
    Is there a way in bash to guarantee that a trap gets called for cleanup
    in a script?

    I have a script that works perfectly normally and cleans up after
    itself, even if it goes wrong.

    However on trying to debug something else, I wanted to run it like this:

    ./script |& tee log

    and now it doesn't clean up if I <ctrl c> it.

    Is there a way in bash to guarantee (modulo uncatchable signals) that a
    cleanup routine gets called?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From john doe@21:1/5 to Tim Woodall on Sat Sep 28 16:10:01 2024
    On 9/28/24 15:53, Tim Woodall wrote:
    Is there a way in bash to guarantee that a trap gets called for cleanup
    in a script?

    I have a script that works perfectly normally and cleans up after
    itself, even if it goes wrong.

    However on trying to debug something else, I wanted to run it like this:

    ./script |& tee log

    and now it doesn't clean up if I <ctrl c> it.

    Is there a way in bash to guarantee (modulo uncatchable signals) that a cleanup routine gets called?


    You need to trap the CTRL C and execute that same clean up routine.
    As I don't use Bash you might want to look online for some insperation.

    --
    John Doe

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to Tim Woodall on Sat Sep 28 16:20:01 2024
    On Sat, Sep 28, 2024 at 14:53:10 +0100, Tim Woodall wrote:
    Is there a way in bash to guarantee that a trap gets called for cleanup
    in a script?

    #!/bin/bash
    trap cleanup EXIT
    cleanup() {
    ...
    }

    This works in bash -- i.e., it calls the cleanup function regardless
    of whether the shell exits by calling "exit", or by falling off the
    end of the script, or by receiving a fatal signal. It does NOT work in
    /bin/sh (dash, or any other implementation). You have been warned.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alain D D Williams@21:1/5 to Greg Wooledge on Sat Sep 28 16:20:01 2024
    On Sat, Sep 28, 2024 at 10:13:59AM -0400, Greg Wooledge wrote:
    On Sat, Sep 28, 2024 at 14:53:10 +0100, Tim Woodall wrote:
    Is there a way in bash to guarantee that a trap gets called for cleanup
    in a script?

    #!/bin/bash
    trap cleanup EXIT
    cleanup() {
    ...
    }

    This works in bash -- i.e., it calls the cleanup function regardless
    of whether the shell exits by calling "exit", or by falling off the
    end of the script, or by receiving a fatal signal. It does NOT work in /bin/sh (dash, or any other implementation). You have been warned.

    *ALL* scripts MUST start with a #! line as you have given above.

    Anyone who does not do that deserves what they get.

    --
    Alain Williams
    Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer.
    +44 (0) 787 668 0256 https://www.phcomp.co.uk/
    Parliament Hill Computers. Registration Information: https://www.phcomp.co.uk/Contact.html
    #include <std_disclaimer.h>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to Alain D D Williams on Sat Sep 28 16:30:01 2024
    On Sat, Sep 28, 2024 at 15:17:47 +0100, Alain D D Williams wrote:
    On Sat, Sep 28, 2024 at 10:13:59AM -0400, Greg Wooledge wrote:
    On Sat, Sep 28, 2024 at 14:53:10 +0100, Tim Woodall wrote:
    Is there a way in bash to guarantee that a trap gets called for cleanup in a script?

    #!/bin/bash
    trap cleanup EXIT
    cleanup() {
    ...
    }

    This works in bash -- i.e., it calls the cleanup function regardless
    of whether the shell exits by calling "exit", or by falling off the
    end of the script, or by receiving a fatal signal. It does NOT work in /bin/sh (dash, or any other implementation). You have been warned.

    *ALL* scripts MUST start with a #! line as you have given above.

    Anyone who does not do that deserves what they get.

    Yes, but many, MANY people don't understand the difference between
    #!/bin/bash and #!/bin/sh.

    Tim specifically asked for an answer that works "in bash", and that's
    what I gave. But someone else reading this thread in the archives
    might try to use this with a /bin/sh script -- and that won't work.

    That's why I'm giving the warning. Even if Tim doesn't need it, there
    are thousands of other people who will.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tim Woodall@21:1/5 to Greg Wooledge on Sat Sep 28 17:30:01 2024
    On Sat, 28 Sep 2024, Greg Wooledge wrote:

    On Sat, Sep 28, 2024 at 14:53:10 +0100, Tim Woodall wrote:
    Is there a way in bash to guarantee that a trap gets called for cleanup
    in a script?

    #!/bin/bash
    trap cleanup EXIT
    cleanup() {
    ...
    }

    This works in bash -- i.e., it calls the cleanup function regardless
    of whether the shell exits by calling "exit", or by falling off the
    end of the script, or by receiving a fatal signal. It does NOT work in /bin/sh (dash, or any other implementation). You have been warned.


    That's exactly what I'm doing but somehow it's not working. (See below
    for what the problem seems to be)

    But I also cannot create a small testcase to reproduce.

    It's a pain, I'm trying to debug something that takes a very long time
    to run and the cleanup requires a complex ordering of unmounting,
    deleting loop devices and deleting files which all happens
    automatically normally.

    But as soon as I send the (copious) output to a pipe, then it doesn't
    cleanup and I'm left having to do it by hand.

    I guess I'll have to try and debug what isn't working.

    Hmmm, I've managed to fix it. The problem seems to be related to using
    echo in the exit trap itself while both stderr and stdout are redirected
    - e.g. via |& tee log

    If I change the echo in the trap function to /bin/echo then it works! (I
    don't see the output anywhere but that isn't really a problem in this
    case - I know why it's aborted!)

    I still can't create a small testcase but maybe this gives someone a
    clue what issue I'm hitting?

    In this particular case it's almost certain that the tee program will
    have gone away before the bash script calls the exit handler.

    apt-cache policy bash
    bash:
    Installed: 5.2.15-2+b7

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael =?utf-8?B?S2rDtnJsaW5n?=@21:1/5 to All on Sat Sep 28 19:20:01 2024
    On 28 Sep 2024 16:28 +0100, from debianuser@woodall.me.uk (Tim Woodall):
    Hmmm, I've managed to fix it. The problem seems to be related to using
    echo in the exit trap itself while both stderr and stdout are redirected
    - e.g. via |& tee log

    If I change the echo in the trap function to /bin/echo then it works! (I don't see the output anywhere but that isn't really a problem in this
    case - I know why it's aborted!)

    I still can't create a small testcase but maybe this gives someone a
    clue what issue I'm hitting?

    In this particular case it's almost certain that the tee program will
    have gone away before the bash script calls the exit handler.

    That last sentence seems _very_ relevant here.

    If the other end of the pipe is gone, then the shell builtin `echo`
    probably fails with SIGPIPE/EPIPE. So will /bin/echo too, of course,
    but that will fail just that process, not the script or the /bin/bash
    that is executing the script as is probably the case in your
    situation. I suspect that if you dig into this, you will find that
    everything works as expected up to that `echo` statement.

    Check $? after /bin/echo in the handler (probably non-zero), and do a
    `type echo` from within a script executed in the same way (probably
    "shell builtin").

    If so, there's your difference.

    --
    Michael Kjörling 🔗 https://michael.kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?”

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Anssi Saari@21:1/5 to Tim Woodall on Mon Sep 30 13:10:01 2024
    Tim Woodall <debianuser@woodall.me.uk> writes:

    However on trying to debug something else, I wanted to run it like this:

    ./script |& tee log

    and now it doesn't clean up if I <ctrl c> it.

    Just a point here about tee since I didn't see anyone else mention
    it. tee has had the -i option to ignore interrupt signals for ages. The non-obvious side effect is INT signals pass up the pipe, in your case to
    bash running your script.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to Anssi Saari on Mon Sep 30 13:20:01 2024
    On Mon, Sep 30, 2024 at 14:08:19 +0300, Anssi Saari wrote:
    Tim Woodall <debianuser@woodall.me.uk> writes:

    However on trying to debug something else, I wanted to run it like this:

    ./script |& tee log

    and now it doesn't clean up if I <ctrl c> it.

    Just a point here about tee since I didn't see anyone else mention
    it. tee has had the -i option to ignore interrupt signals for ages. The non-obvious side effect is INT signals pass up the pipe, in your case to
    bash running your script.

    Signals don't "pass up the pipe". A process either receives a signal,
    or it doesn't.

    Using kill(2) or the shell's kill command to send a signal to a process
    only sends the signal to *one* process, unless you use one of the
    features to send signals to a group of processes (e.g. kill() with a non-positive argument).

    Pressing Ctrl-C (or whatever intr is bound to) in a terminal sends the
    SIGINT to all of the "foreground" processes. If you're running a shell
    script as a foreground job in a terminal, and that shell script is running
    some external foreground command, then keyboard Ctrl-C will send SIGINT
    to the external foreground command *and* to the shell.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tim Woodall@21:1/5 to All on Mon Sep 30 18:10:01 2024
    This message is in MIME format. The first part should be readable text,
    while the remaining parts are likely unreadable without MIME-aware tools.

    On Sat, 28 Sep 2024, Michael Kjörling wrote:

    On 28 Sep 2024 16:28 +0100, from debianuser@woodall.me.uk (Tim Woodall):
    Hmmm, I've managed to fix it. The problem seems to be related to using
    echo in the exit trap itself while both stderr and stdout are redirected
    - e.g. via |& tee log

    If I change the echo in the trap function to /bin/echo then it works! (I
    don't see the output anywhere but that isn't really a problem in this
    case - I know why it's aborted!)

    I still can't create a small testcase but maybe this gives someone a
    clue what issue I'm hitting?

    In this particular case it's almost certain that the tee program will
    have gone away before the bash script calls the exit handler.

    That last sentence seems _very_ relevant here.

    If the other end of the pipe is gone, then the shell builtin `echo`
    probably fails with SIGPIPE/EPIPE. So will /bin/echo too, of course,
    but that will fail just that process, not the script or the /bin/bash
    that is executing the script as is probably the case in your
    situation. I suspect that if you dig into this, you will find that
    everything works as expected up to that `echo` statement.

    Check $? after /bin/echo in the handler (probably non-zero), and do a
    `type echo` from within a script executed in the same way (probably
    "shell builtin").

    If so, there's your difference.


    Yes, this does seem to be my problem. However I didn't know about this
    feature of builtin echo and, had I known, I wouldn't have needed to ask
    the question.

    Here's something else where they explicity use /bin/echo in preference
    to bash builtin echo

    https://unix.stackexchange.com/questions/522929/check-if-named-pipe-is-open-for-reading

    The first answer:
    You cannot use the built-in echo (even in a subshell) because the
    SIGPIPE will be either caught or kill the whole shell.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Anssi Saari@21:1/5 to Greg Wooledge on Tue Oct 1 13:00:01 2024
    Greg Wooledge <greg@wooledge.org> writes:

    On Mon, Sep 30, 2024 at 14:08:19 +0300, Anssi Saari wrote:
    Tim Woodall <debianuser@woodall.me.uk> writes:

    However on trying to debug something else, I wanted to run it like this: >> >
    ./script |& tee log

    and now it doesn't clean up if I <ctrl c> it.

    Just a point here about tee since I didn't see anyone else mention
    it. tee has had the -i option to ignore interrupt signals for ages. The
    non-obvious side effect is INT signals pass up the pipe, in your case to
    bash running your script.

    Signals don't "pass up the pipe". A process either receives a signal,
    or it doesn't.

    I'm not sure now, are you just squabbling over semantics or are you
    disputing my description of what happens?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From tomas@tuxteam.de@21:1/5 to Anssi Saari on Tue Oct 1 13:00:01 2024
    On Tue, Oct 01, 2024 at 01:53:26PM +0300, Anssi Saari wrote:
    Greg Wooledge <greg@wooledge.org> writes:

    On Mon, Sep 30, 2024 at 14:08:19 +0300, Anssi Saari wrote:
    Tim Woodall <debianuser@woodall.me.uk> writes:

    However on trying to debug something else, I wanted to run it like this: >> >
    ./script |& tee log

    and now it doesn't clean up if I <ctrl c> it.

    Just a point here about tee since I didn't see anyone else mention
    it. tee has had the -i option to ignore interrupt signals for ages. The
    non-obvious side effect is INT signals pass up the pipe, in your case to >> bash running your script.

    Signals don't "pass up the pipe". A process either receives a signal,
    or it doesn't.

    I'm not sure now, are you just squabbling over semantics or are you
    disputing my description of what happens?

    Explain to us what you really mean by "signals pass up the pipe", then things might become clearer.

    Cheers
    --
    t

    -----BEGIN PGP SIGNATURE-----

    iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCZvvV0QAKCRAFyCz1etHa RsykAJ9qub04Mm8ElC9/MPIQkWF5VAmEEwCfbk8mAmFnCUMj0gyWYCmEpNU6N4s=
    =ijrA
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Anssi Saari@21:1/5 to tomas@tuxteam.de on Wed Oct 2 13:50:01 2024
    <tomas@tuxteam.de> writes:

    On Tue, Oct 01, 2024 at 01:53:26PM +0300, Anssi Saari wrote:
    Greg Wooledge <greg@wooledge.org> writes:

    On Mon, Sep 30, 2024 at 14:08:19 +0300, Anssi Saari wrote:
    Tim Woodall <debianuser@woodall.me.uk> writes:

    However on trying to debug something else, I wanted to run it like this:

    ./script |& tee log

    and now it doesn't clean up if I <ctrl c> it.

    Just a point here about tee since I didn't see anyone else mention
    it. tee has had the -i option to ignore interrupt signals for ages. The >> >> non-obvious side effect is INT signals pass up the pipe, in your case to >> >> bash running your script.

    Signals don't "pass up the pipe". A process either receives a signal,
    or it doesn't.

    I'm not sure now, are you just squabbling over semantics or are you
    disputing my description of what happens?

    Explain to us what you really mean by "signals pass up the pipe", then things might become clearer.

    I realize I didn't spell it out. It's the, to me, obvious solution to
    Tim's problem, as he wrote:

    QUOTE
    ./script |& tee log

    and now it doesn't clean up if I <ctrl c> it."
    END QUOTE

    Running ./script |& tee -i log works as expected. The script gets the
    INT signal and cleans up.

    To me, "signal passing up the pipe" is an apt analogy of how things work
    in practice.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From tomas@tuxteam.de@21:1/5 to Anssi Saari on Wed Oct 2 14:00:01 2024
    On Wed, Oct 02, 2024 at 02:43:11PM +0300, Anssi Saari wrote:
    <tomas@tuxteam.de> writes:

    [...]

    Explain to us what you really mean by "signals pass up the pipe", then things
    might become clearer.

    I realize I didn't spell it out. It's the, to me, obvious solution to
    Tim's problem, as he wrote:

    QUOTE
    ./script |& tee log

    and now it doesn't clean up if I <ctrl c> it."
    END QUOTE

    Running ./script |& tee -i log works as expected. The script gets the
    INT signal and cleans up.

    Understood.

    To me, "signal passing up the pipe" is an apt analogy of how things work
    in practice.

    This triggered the wrong association for me, TBH :)

    What actually happens seems completely different to me: the shell
    gets the EPIPE from the dying tee before it can see the EINTR, right?

    Cheers
    --
    t

    -----BEGIN PGP SIGNATURE-----

    iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCZv01gQAKCRAFyCz1etHa Rr/aAJ0VC12exJRpXrlSFeBUdY+jv10ORwCfa1oPbUCrOpng2quRXC1fdN2CDMo=
    =zdM3
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From tomas@tuxteam.de@21:1/5 to Greg Wooledge on Wed Oct 2 14:30:01 2024
    On Wed, Oct 02, 2024 at 08:17:34AM -0400, Greg Wooledge wrote:

    [...]

    Thanks for your (as always) very exhaustive description.

    What I'm mostly trying to convey here, though, is that signals do not *propagate* from one process to another in any kind of automatic way.

    That's more or less what I was trying to describe with "wrong associations".

    Cheers
    --
    t

    -----BEGIN PGP SIGNATURE-----

    iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCZv07vAAKCRAFyCz1etHa RsDaAJ9UnmAhKsYgNJHaadnlQi8kq6k75ACfYRLrGJmJA9swB81Ks2DQwgMYfz8=
    =PLzY
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to tomas@tuxteam.de on Wed Oct 2 14:20:01 2024
    On Wed, Oct 02, 2024 at 13:59:04 +0200, tomas@tuxteam.de wrote:
    On Wed, Oct 02, 2024 at 02:43:11PM +0300, Anssi Saari wrote:
    Running ./script |& tee -i log works as expected. The script gets the
    INT signal and cleans up.

    Understood.

    To me, "signal passing up the pipe" is an apt analogy of how things work
    in practice.

    This triggered the wrong association for me, TBH :)

    What actually happens seems completely different to me: the shell
    gets the EPIPE from the dying tee before it can see the EINTR, right?

    Let's assume one opens a terminal with an interactive shell running
    inside it, and then types the command "./script |& tee -i log", and then presses Ctrl-C.

    In that case, there are THREE foreground processes which receive a SIGINT:

    * The interactive shell, which ignores it, because interactive shells
    always ignore SIGINT.

    * The ./script process, which is presumably expanded to "/bin/bash ./script".

    * The tee -i log process.

    We know the interactive shell ignores it, and we can reasonably assume
    that the tee will accept it and terminate. So then the question becomes
    what the /bin/bash ./script process will do with it.

    If the tee process terminates, bash won't *know* that it's gone until
    it tries to write to the pipeline. When it tries to write to a broken pipeline, it will receive a SIGPIPE. If it never writes to the broken pipeline, it will never know the tee is dead.

    If the script is NOT actively writing to the pipeline at the time Ctrl-C
    is pressed, but has a trap set up which tries to write to the pipeline
    upon receiving SIGINT, then there's a race condition between the shell's
    trap activating and trying to write to the pipeline, and the tee process
    trying to close the pipeline.

    I'm guessing tee wins that race most of the time.

    The differing behavior between the builtin echo command, and an external /bin/echo command, when writing to a broken pipe in an EXIT trap, is
    news to me, honestly. I've never run into it myself.

    What I'm mostly trying to convey here, though, is that signals do not *propagate* from one process to another in any kind of automatic way.
    It doesn't matter whether the two processes are connected by a pipe,
    or are parent/child to each other, or anything else.

    Killing a parent process does not automatically kill the child.

    Killing one process in a pipeline does not automatically kill any other
    process in the pipeline.

    And finally, pressing Ctrl-C in a terminal sends MANY signals, and
    therefore is immensely different from sending a SIGINT to just one
    process.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Nicolas George@21:1/5 to All on Wed Oct 2 15:20:01 2024
    Greg Wooledge (12024-10-02):
    In that case, there are THREE foreground processes which receive a SIGINT:

    * The interactive shell, which ignores it, because interactive shells
    always ignore SIGINT.

    You are very right on the rest, but this is a mistake: the interactive
    shell is not in the foreground at this point.

    Since foreground / background is about process group, it could not: each pipeline is started in a group, so that it can be moved from the
    foreground to the background and back. The parent shell cannot enter and
    leave these groups at will.

    Regards,

    --
    Nicolas George

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Anssi Saari@21:1/5 to tomas@tuxteam.de on Thu Oct 3 14:20:01 2024
    <tomas@tuxteam.de> writes:

    What actually happens seems completely different to me: the shell
    gets the EPIPE from the dying tee before it can see the EINTR, right?

    That depends. tee -i will ignore SIGINT but ./script gets it. So it can
    keep writing in the pipe, from the script proper or its SIGINT handler
    and exit when it's done and that'll kill tee as well.

    Without -i, tee will die when it gets a SIGINT, the script gets a
    SIGPIPE if it tries to write the pipe now so that would cause the script
    to die before its SIGINT handler can finish.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From tomas@tuxteam.de@21:1/5 to Anssi Saari on Thu Oct 3 15:30:01 2024
    On Thu, Oct 03, 2024 at 03:19:09PM +0300, Anssi Saari wrote:
    <tomas@tuxteam.de> writes:

    What actually happens seems completely different to me: the shell
    gets the EPIPE from the dying tee before it can see the EINTR, right?

    That depends. tee -i will ignore SIGINT but ./script gets it. So it can
    keep writing in the pipe, from the script proper or its SIGINT handler
    and exit when it's done and that'll kill tee as well.

    Without -i, tee will die when it gets a SIGINT, the script gets a
    SIGPIPE if it tries to write the pipe now so that would cause the script
    to die before its SIGINT handler can finish.

    Yes, we agree. I was talking in the OP's context, where they said they
    don't "see" the INT.

    Cheers
    --
    t

    -----BEGIN PGP SIGNATURE-----

    iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCZv6cIwAKCRAFyCz1etHa RrmkAJsFYbhnPIScKmNW0bqu9p8ZBafz5gCfdBaPxqHlcdyVHG0zgLYUkPNfrAI=
    =YuW8
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)