• Re: vwait for several channels

    From Robert Heller@21:1/5 to luisXXXlupeXXX@gmail.com on Mon Jun 3 16:04:42 2024
    At 03 Jun 2024 15:26:46 GMT Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:


    Hi,


    There's a procedure that goes through a list and opens a channel for each
    of its elements.
    I'd need a vwait for each of the channels, like
    vwait exec_0
    vwait exec_1
    ...
    0, 1, ... are the execution ids (exec_id).

    Is there any reason to wait for each channel? Will there be code between the vwaits? Or do you just want to wait until all channels are open?



    How can I do this?



    Luís Mendes



    --
    Robert Heller -- Cell: 413-658-7953 GV: 978-633-5364
    Deepwoods Software -- Custom Software Services
    http://www.deepsoft.com/ -- Linux Administration Services
    heller@deepsoft.com -- Webhosting Services

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rich@21:1/5 to Luis Mendes on Mon Jun 3 18:01:14 2024
    Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
    Hi,


    There's a procedure that goes through a list and opens a channel for each
    of its elements.
    I'd need a vwait for each of the channels, like
    vwait exec_0
    vwait exec_1
    ...
    0, 1, ... are the execution ids (exec_id).

    How can I do this?

    You can't vwait on channels. vwait waits on a variable to be changed.

    What are you /really/ trying to do?

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Harald Oehlmann@21:1/5 to All on Tue Jun 4 13:50:23 2024
    Am 04.06.2024 um 13:34 schrieb Luis Mendes:
    On Mon, 3 Jun 2024 18:01:14 -0000 (UTC), Rich wrote:

    Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
    Hi,


    There's a procedure that goes through a list and opens a channel for
    each of its elements.
    I'd need a vwait for each of the channels, like vwait exec_0 vwait
    exec_1 ...
    0, 1, ... are the execution ids (exec_id).

    How can I do this?

    You can't vwait on channels. vwait waits on a variable to be changed.

    What are you /really/ trying to do?
    The expression was perhaps too simplistic.
    As said in the first message, I want to use several channels at the same
    time to perform some 'exec' commands and want to wait for the complete execution of all of them.

    Then you put a writevent on them and check for the eof and error
    conditions to detect, if they got closed from the other side.

    Harald

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Robert Heller@21:1/5 to luisXXXlupeXXX@gmail.com on Tue Jun 4 12:33:10 2024
    At 04 Jun 2024 11:34:37 GMT Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:


    On Mon, 3 Jun 2024 18:01:14 -0000 (UTC), Rich wrote:

    Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
    Hi,


    There's a procedure that goes through a list and opens a channel for
    each of its elements.
    I'd need a vwait for each of the channels, like vwait exec_0 vwait
    exec_1 ...
    0, 1, ... are the execution ids (exec_id).

    How can I do this?

    You can't vwait on channels. vwait waits on a variable to be changed.

    What are you /really/ trying to do?
    The expression was perhaps too simplistic.
    As said in the first message, I want to use several channels at the same
    time to perform some 'exec' commands and want to wait for the complete execution of all of them.

    You description here is very unclear. 'exec' does not create channels and you can't "vwait" on a channel anyway.

    Maybe what you want is:

    # launch a subprocess, capturing its output with a "pipe", which will be read
    # by a specified script. The script will have the pipe channel appended to
    # it.
    proc pipeCommand {commandString commpletionScript} {
    if {[catch {open "|$commandString" r} pipe]} {
    puts stderr "Error starting $commandString: $pipe"
    return
    }
    fileevent $pipe readable "$commpletionScript $pipe"
    }

    # read a pipe as data becomes available. Close the pipe when the process
    # completes.
    proc typicalCompletionProc {pipe} {
    if {[gets $pipe line] >= 0} {
    puts $line;# one line of output from the process. Maybe parse it?
    } else {
    # EOF or ERROR on pipe: process completed, possibly with an error
    if {[catch {close $pipe} finalResult]} {
    # process completed with an error
    puts stderr "process completed with an error: $finalResult"
    } else {
    # process completed with success
    puts "process completed: $finalResult"
    }
    }
    }

    # fire up some commands in the background, and wait for them to complete pipeCommand "uname -a" typicalCompletionProc
    pipeCommand "ls" typicalCompletionProc
    pipeCommand "find $HOME/" typicalCompletionProc
    pipeCommand "make" typicalCompletionProc

    # enter event loop to allow fileevents to process (read the pipes
    # asyncroniously) The variable Forever will never be changed, so this
    # statement never returns. Note: if this was done in wish or with
    # 'package require Tk', this last line would not be needed at all.
    vwait Forever




    --
    Robert Heller -- Cell: 413-658-7953 GV: 978-633-5364
    Deepwoods Software -- Custom Software Services
    http://www.deepsoft.com/ -- Linux Administration Services
    heller@deepsoft.com -- Webhosting Services

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Harald Oehlmann@21:1/5 to All on Tue Jun 4 16:52:41 2024
    Am 04.06.2024 um 16:38 schrieb Luis Mendes:
    On Tue, 04 Jun 2024 12:33:10 +0000, Robert Heller wrote:

    At 04 Jun 2024 11:34:37 GMT Luis Mendes <luisXXXlupeXXX@gmail.com>
    wrote:


    On Mon, 3 Jun 2024 18:01:14 -0000 (UTC), Rich wrote:

    Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
    Hi,


    There's a procedure that goes through a list and opens a channel for >>>>> each of its elements.
    I'd need a vwait for each of the channels, like vwait exec_0 vwait
    exec_1 ...
    0, 1, ... are the execution ids (exec_id).

    How can I do this?

    You can't vwait on channels. vwait waits on a variable to be
    changed.

    What are you /really/ trying to do?
    The expression was perhaps too simplistic.
    As said in the first message, I want to use several channels at the
    same time to perform some 'exec' commands and want to wait for the
    complete execution of all of them.

    You description here is very unclear. 'exec' does not create channels
    and you can't "vwait" on a channel anyway.

    Maybe what you want is:

    # launch a subprocess, capturing its output with a "pipe", which will be
    read # by a specified script. The script will have the pipe channel
    appended to # it.
    proc pipeCommand {commandString commpletionScript} {
    if {[catch {open "|$commandString" r} pipe]} {
    puts stderr "Error starting $commandString: $pipe"
    return
    }
    fileevent $pipe readable "$commpletionScript $pipe"
    }

    # read a pipe as data becomes available. Close the pipe when the process
    # completes.
    proc typicalCompletionProc {pipe} {
    if {[gets $pipe line] >= 0} {
    puts $line;# one line of output from the process. Maybe parse it?
    } else {
    # EOF or ERROR on pipe: process completed, possibly with an error
    if {[catch {close $pipe} finalResult]} {
    # process completed with an error puts stderr "process
    completed with an error: $finalResult"
    } else {
    # process completed with success puts "process completed:
    $finalResult"
    }
    }
    }

    # fire up some commands in the background, and wait for them to complete
    pipeCommand "uname -a" typicalCompletionProc pipeCommand "ls"
    typicalCompletionProc pipeCommand "find $HOME/" typicalCompletionProc
    pipeCommand "make" typicalCompletionProc

    # enter event loop to allow fileevents to process (read the pipes #
    asyncroniously) The variable Forever will never be changed, so this #
    statement never returns. Note: if this was done in wish or with #
    'package require Tk', this last line would not be needed at all.
    vwait Forever
    I have everything working fine, from opening a channel, to setting up the readable event, to closing it and setting the variable that vwait is
    waiting for, in your example 'Forever'.

    What I'd like to know is how to wait for several Forever_$exec_id
    variables of channels that are open at the same time.
    I just need this last part.

    Hi Luis,
    it is quite uncommon to use a vwait for this.
    The example uses "vwait forever" just to start the event loop.
    If you are using Tk, this is not required. There is always an event loop.

    To find out when all of your commands ended:
    - create a data structure like a global list which keeps track of all
    open tasks. If a task finishes, remove it from the data structure.
    If the list is empty, you are gone. This should be done in the event
    procedure.
    If you want the vwait in the example to end, you may execute in this case:
    set ::forever 1

    As "vwait forever" waits that a variable called "forever" is changed.

    Thank you and take care,
    Harald

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Robert Heller@21:1/5 to luisXXXlupeXXX@gmail.com on Tue Jun 4 15:32:29 2024
    At 04 Jun 2024 14:38:42 GMT Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:


    On Tue, 04 Jun 2024 12:33:10 +0000, Robert Heller wrote:

    At 04 Jun 2024 11:34:37 GMT Luis Mendes <luisXXXlupeXXX@gmail.com>
    wrote:


    On Mon, 3 Jun 2024 18:01:14 -0000 (UTC), Rich wrote:

    Luis Mendes <luisXXXlupeXXX@gmail.com> wrote:
    Hi,


    There's a procedure that goes through a list and opens a channel for
    each of its elements.
    I'd need a vwait for each of the channels, like vwait exec_0 vwait
    exec_1 ...
    0, 1, ... are the execution ids (exec_id).

    How can I do this?

    You can't vwait on channels. vwait waits on a variable to be
    changed.

    What are you /really/ trying to do?
    The expression was perhaps too simplistic.
    As said in the first message, I want to use several channels at the
    same time to perform some 'exec' commands and want to wait for the
    complete execution of all of them.

    You description here is very unclear. 'exec' does not create channels
    and you can't "vwait" on a channel anyway.

    Maybe what you want is:

    # launch a subprocess, capturing its output with a "pipe", which will be read # by a specified script. The script will have the pipe channel appended to # it.
    proc pipeCommand {commandString commpletionScript} {
    if {[catch {open "|$commandString" r} pipe]} {
    puts stderr "Error starting $commandString: $pipe"
    return
    }
    fileevent $pipe readable "$commpletionScript $pipe"
    }

    # read a pipe as data becomes available. Close the pipe when the process
    # completes.
    proc typicalCompletionProc {pipe} {
    if {[gets $pipe line] >= 0} {
    puts $line;# one line of output from the process. Maybe parse it?
    } else {
    # EOF or ERROR on pipe: process completed, possibly with an error
    if {[catch {close $pipe} finalResult]} {
    # process completed with an error puts stderr "process
    completed with an error: $finalResult"
    } else {
    # process completed with success puts "process completed:
    $finalResult"
    }
    }
    }

    # fire up some commands in the background, and wait for them to complete pipeCommand "uname -a" typicalCompletionProc pipeCommand "ls" typicalCompletionProc pipeCommand "find $HOME/" typicalCompletionProc pipeCommand "make" typicalCompletionProc

    # enter event loop to allow fileevents to process (read the pipes # asyncroniously) The variable Forever will never be changed, so this # statement never returns. Note: if this was done in wish or with #
    'package require Tk', this last line would not be needed at all.
    vwait Forever
    I have everything working fine, from opening a channel, to setting up the readable event, to closing it and setting the variable that vwait is
    waiting for, in your example 'Forever'.

    Why are you bothering to set the Forever variable?


    What I'd like to know is how to wait for several Forever_$exec_id
    variables of channels that are open at the same time.
    I just need this last part.

    Exactly why do you need to do this? What are you *really* trying to do? And why? The reader proc can handle any process rundown and whatever post process processing.

    If all you want is for the program to terminate when all of the processes complete, then you can set a counter to the total number of them, and
    decrement that counter in the reader proc when it closes the pipe and then
    have a loop at the end like:

    set totalProc 10

    # 10x start process
    # the commpletion procs decrement totalProc: "incr ::totalProc -1"

    # then:

    while {$totalProc > 0} {
    vwait totalProc
    }

    This should do what you want.

    It is not meaningful to have multiple vwaits -- that just makes no sense. The processes are asyncronious and won't terminate in order, so having the vwaits in order won't work. You should only vwait on a single variable. Using it
    as a global counter will work to wait for each process, even if they finish "out of order". You would stop waiting when the count reaches 0.




    --
    Robert Heller -- Cell: 413-658-7953 GV: 978-633-5364
    Deepwoods Software -- Custom Software Services
    http://www.deepsoft.com/ -- Linux Administration Services
    heller@deepsoft.com -- Webhosting Services

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From et99@21:1/5 to Luis Mendes on Tue Jun 4 21:55:40 2024
    On 6/4/2024 7:38 AM, Luis Mendes wrote:
    -snip-
    I have everything working fine, from opening a channel, to setting up the readable event, to closing it and setting the variable that vwait is
    waiting for, in your example 'Forever'.

    What I'd like to know is how to wait for several Forever_$exec_id
    variables of channels that are open at the same time.
    I just need this last part.


    I do the following to wait for multiple threads to *ALL* complete some job; it should also work with processes.

    Assume some proc, for example startit, creates a processN (or threadN) and endN is the variable that is set when the process (or thread) finishes:

    unset -nocomplain end1 ;# variable that is set to some value when process1 is done
    startit process1 end1

    unset -nocomplain end2 ;# ditto
    startit process2 end2

    unset -nocomplain end3 ;# ditto
    startit process3 end3

    ... and so on

    # now when you want to wait for *ALL* of them (the order of these does not matter)

    if {![info exist end1]} {vwait end1}
    if {![info exist end2]} {vwait end2}
    if {![info exist end3]} {vwait end3}

    .... and so on

    By unsetting each variable first, a potential race condition deadlock is avoided. It also doesn't matter which one finishes first or even if they've all finished by the time you need to wait for all of them (in case if you want to do something else
    before waiting).

    To understand why this approach is needed, suppose that while you are vwaiting for the first one, the 2nd or 3rd ones finish first and set their corresponding variables to some value.

    Upon continuing after end1 is set, if you were to vwait for either of end2 or end3 *after* they were already set, you would wait forever. You avoid that because you are testing for the variable having been already set *before* vwaiting on that variable.
    That's why you unset first, so you can do the [info exist] test before vwaiting.

    The rest of the code you say is complete, and is compatible with the above method.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From et99@21:1/5 to undroidwish on Tue Jun 4 22:11:38 2024
    On 6/4/2024 9:32 PM, undroidwish wrote:
    On 6/3/24 17:26, Luis Mendes wrote:

    ...
    I'd need a vwait for each of the channels, like
    vwait exec_0
    vwait exec_1
    ...
    0, 1, ... are the execution ids (exec_id).

    How can I do this?

    Might be a pattern for which TIP#455 could be of use, see

      https://core.tcl-lang.org/tips/doc/trunk/tip/455.md

    Cheers,
    Christian


    I have considered writing a TIP that would enhance vwait with one more option (I don't see it in TIP 455)

    vwait -ifnotset variable

    This would be equivalent to the code I use where I unset a variable and then do:

    if {![info exist variable]} {vwait variable}

    which works only because I know that the if and info statements are essentially atomic with respect to the event loop. I doubt that would ever change, but it's still an undocumented (afaik) assumption I have been making.

    -e

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From undroidwish@21:1/5 to Luis Mendes on Wed Jun 5 06:32:38 2024
    On 6/3/24 17:26, Luis Mendes wrote:

    ...
    I'd need a vwait for each of the channels, like
    vwait exec_0
    vwait exec_1
    ...
    0, 1, ... are the execution ids (exec_id).

    How can I do this?

    Might be a pattern for which TIP#455 could be of use, see

    https://core.tcl-lang.org/tips/doc/trunk/tip/455.md

    Cheers,
    Christian

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)