In "The Python Language Reference, Release 3.13.0a0",
there is this section:

|6.4 Await expression
|
|Suspend the execution of coroutine on an awaitable object.
|Can only be used inside a coroutine function.
|
|await_expr ::= "await" primary
|
|New in version 3.5.

. And this is the whole section.

What I do not understand:

- Which coroutine is suspended?
- Which object is the object mentioned?
- For what purpose is the value of the primary expression used?
- What does it mean to "suspend something on something"?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

We say that an object is an awaitable object if it can be used in an
await expression. Many asyncio APIs are designed to accept awaitables.

There are three main types of awaitable objects:
coroutines, Tasks, and Futures.

Stefan Ram schrieb:

In "The Python Language Reference, Release 3.13.0a0",
there is this section:

|6.4 Await expression
|
|Suspend the execution of coroutine on an awaitable object.
|Can only be used inside a coroutine function.
|
|await_expr ::= "await" primary
|
|New in version 3.5.

. And this is the whole section.

What I do not understand:

- Which coroutine is suspended?
- Which object is the object mentioned?
- For what purpose is the value of the primary expression used?
- What does it mean to "suspend something on something"?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Maybe consult:

PEP 492 – Coroutines with async and await syntax
Created: 09-Apr-2015
Python-Version: 3.5
https://peps.python.org/pep-0492/

Mild Shock schrieb:

We say that an object is an awaitable object if it can be used in an
await expression. Many asyncio APIs are designed to accept awaitables.

There are three main types of awaitable objects:
coroutines, Tasks, and Futures.

Stefan Ram schrieb:

   In "The Python Language Reference, Release 3.13.0a0",
   there is this section:

|6.4 Await expression
|
|Suspend the execution of coroutine on an awaitable object.
|Can only be used inside a coroutine function.
|
|await_expr ::= "await" primary
|
|New in version 3.5.

   . And this is the whole section.

   What I do not understand:

   - Which coroutine is suspended?
   - Which object is the object mentioned?
   - For what purpose is the value of the primary expression used?
   - What does it mean to "suspend something on something"?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 26 Jan 2024 18:36:50 GMT, Stefan Ram wrote:

In "The Python Language Reference, Release 3.13.0a0",
there is this section:

|6.4 Await expression |
|Suspend the execution of coroutine on an awaitable object.
|Can only be used inside a coroutine function.
|
|await_expr ::= "await" primary |
|New in version 3.5.

. And this is the whole section.

What I do not understand:

- Which coroutine is suspended?

The one currently executing. An “await” expression is only allowed at
the top level of an “async def” function.

- Which object is the object mentioned?

The result returned from the primary.

- For what purpose is the value of the primary expression used?

To return the awaitable object.

- What does it mean to "suspend something on something"?

It returns control to the point of execution of the .send method that
(directly or indirectly) started or resumed the coroutine execution.

I did a diagram that gives an overview of the whole system of
interlocking concepts here <https://www.deviantart.com/default-cube/art/Van-Rossum-s-Triangle-679791228>.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Lawrence D'Oliveiro <ldo@nz.invalid> writes:

On 26 Jan 2024 18:36:50 GMT, Stefan Ram wrote:

In "The Python Language Reference, Release 3.13.0a0",
there is this section:
|6.4 Await expression |
|Suspend the execution of coroutine on an awaitable object.
|Can only be used inside a coroutine function.
|await_expr ::= "await" primary |

...

- What does it mean to "suspend something on something"?

It returns control to the point of execution of the .send method that >(directly or indirectly) started or resumed the coroutine execution.

Thank you!

The specification said: "suspend the execution of coroutine on
an awaitable object".

You said this: "returns control to the point of execution of the
.send method that (directly or indirectly) started or resumed
the coroutine execution.".

But your explanation seems to have no mention of the "something" /
"the awaitable object" part following the preposition "on". Shouldn't
this awaitable object play a rôle in the explanation of what happens?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 27/01/24 10:46 am, Stefan Ram wrote:

But your explanation seems to have no mention of the "something" /
"the awaitable object" part following the preposition "on". Shouldn't
this awaitable object play a rôle in the explanation of what happens?

If it helps at all, you can think of an async function as being
very similar to a generator, and "await" as being very similar to
"yield from". In the current implementation they're almost exactly
the same thing underneath.

--
Greg

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 26 Jan 2024 21:46:47 GMT, Stefan Ram wrote:

Lawrence D'Oliveiro <ldo@nz.invalid> writes:

- Which object is the object mentioned?

The result returned from the primary.

- For what purpose is the value of the primary expression used?

To return the awaitable object.

But your explanation seems to have no mention of the "something" /
"the awaitable object" part following the preposition "on". Shouldn't
this awaitable object play a rôle in the explanation of what happens?

See above.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 27/01/24 10:46 am, Stefan Ram wrote:

But your explanation seems to have no mention of the "something" /
"the awaitable object" part following the preposition "on". Shouldn't
this awaitable object play a r�le in the explanation of what happens?

You can explain a function call without saying much about the called function. Similarly, you can explain "await <expr>" without saying much about
"<expr>".

Important is only: "<expr>" evaluates to an "awaitable".
An "awaitable" (usually an `asyncio.Future`, `asyncio.Task` or
call of an `async` function) eventuelly produces
a value and `await <expr>` waits until
this happens and then returns this value.

Not everything which eventually returns a value is an
"awaitable" -- e.g. a call of `time.sleep` is not an "awaitable".
Special provisions are necessary to be able to wait for a value
(and meanwhile do other things).
`asyncio.sleep` has e.g. this provisions and a call of it is an "awaitable".

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, 27 Jan 2024 12:59:39 +1300, Greg Ewing wrote:

If it helps at all, you can think of an async function as being very
similar to a generator, and "await" as being very similar to "yield
from". In the current implementation they're almost exactly the same
thing underneath.

Similar, yet not quite the same; in no sense can one stand in for the
other.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

dieter.maurer@online.de writes:

You can explain a function call without saying much about the called function. >Similarly, you can explain "await <expr>" without saying much about
"<expr>".

Thanks, Greg and Dieter! The intention of my post was twofold:
To better understand "await", but also to suggest that the
documentation can be improved.

When I have a syntax section and then a semantics section,
the syntax introduces terms that can then be referred to
in the semantics section.

For example:

| Syntax:
|
|expression + expression
|
| Semantic
|
|Evaluates to the sum of the values of the two expressions.

This is clear to me. Notice how "expressions" in the Semantics
section refers to the two "expression" in the Syntax section.
What would be less clear:

| Syntax:
|
|expression + expression
|
| Semantic
|
|Evaluates to the sum of two numbers.

What "two numbers"? Yes, now we can guess that these numbers must
be the values of the expressions, but it's less clear.

I think that the author of the specification could improve
the specification by addressing my questions from the OP,
but I am not able to submit a suggestion for a wording myself,
because I am still learning asyncio.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 28 Jan 2024 15:09:12 GMT, Stefan Ram wrote:

I think that the author of the specification could improve the
specification by addressing my questions from the OP,
but I am not able to submit a suggestion for a wording myself, because I
am still learning asyncio.

It can be a tricky thing to get to grips with, particularly if you are
trying to understand how it works at a low level. That’s why I put
together my “Python Topics Notebooks” series of Jupyter notebooks <https://gitlab.com/ldo/python_topics_notebooks/>.

The one on “Generators & Coroutines” takes you through some stuff you
might already know about (generators), then goes on from there into
coroutines, the send() method and awaitables, and the basics of
implementing your own asyncio-style event loop.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Lawrence D'Oliveiro <ldo@nz.invalid> writes:

It can be a tricky thing to get to grips with, particularly if you are
trying to understand how it works at a low level.

Does "await f()" (immediately) call "f()"?

To find this out, I wrote the following program.

I assume that the next function executed after executing
"await sleep()" is "sleep" and encoded this in assertions.
So far, this assumption never failed.

So, directly after the eternal task executes "await sleep()",
it is always the coroutine sleep() that starts, it never
happens that the other task continues before this happens.

Also, "caller" in "sleep" is "main", "eternal", or "other" as if
"sleep" is called from one of those (as one would naïvely expect).

So, it seems that "await f()" indeed will call f().

import asyncio as _asyncio
import inspect as _inspect
import random as _random

next = ''

async def sleep():
global next
assert next == 'sleep'
next = ''
caller = ''
frameinfo = _inspect.stack( 1 )
if frameinfo: caller = frameinfo[ 1 ].function
await _asyncio.sleep( 0.0000000001 )

async def eternal():
global next
while True:
assert next != 'sleep'
next = 'sleep'
await sleep()

async def other():
global next
while True:
assert next != 'sleep'
next = 'sleep'
await sleep()

async def main():
global next
assert next != 'sleep'
eternal_task = _asyncio.get_running_loop().create_task( eternal() )
next = 'sleep'
await sleep()
other_task = _asyncio.get_running_loop().create_task( other() )
await eternal_task

_asyncio.run( main() )

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 29 Jan 2024 12:53:33 GMT, Stefan Ram wrote:

Does "await f()" (immediately) call "f()"?

Of course. The expression is evaluated exactly as it would normally be in
any other context. The “await”-construct in which it occurs then does something special with the returned value.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

ram@zedat.fu-berlin.de (Stefan Ram) writes:

|await_expr ::= "await" primary
|Suspend the execution of coroutine on an awaitable object.

My ideas of language specifications are that they should
contain all the information to write an implementation of the
language. I am not sure whether the Python Language Reference
already fullfils this criterion. BUT I have made one step
in the direction of better understanding "await":

main.py

import dis

def f(): return g()

dis.dis( f )
print()

async def f(): return await g()

dis.dis( f )

sys.stdout

3 0 LOAD_GLOBAL 0 (g)
2 CALL_FUNCTION 0
4 RETURN_VALUE

8 0 LOAD_GLOBAL 0 (g)
2 CALL_FUNCTION 0
4 GET_AWAITABLE
6 LOAD_CONST 0 (None)
8 YIELD_FROM
10 RETURN_VALUE

. So, now it's down to finding documentation for the "GET_AWAITABLE"
and "YIELD_FROM" opcodes! One can guess that GET_AWAITABLE just gets
a kind of reference to the coroutine g. So the core of await must be
YIELD_FROM. Web:

|In Python 3.11 and above, the YIELD_FROM opcode is replaced
|by a SEND + YIELD_VALUE while loop, as documented in the
|SEND(target_delta).

. Oh, this seems to be a moving target!

|# yield from subgenerator is implemented as the following loop
|# (with None initially at the top of the stack)
|#
|# SEND (sends the top of stack to the subgenerator)
|# YIELD_VALUE (returns the yielded value to the caller)
|# JUMP_BACKWARDS (to SEND)

Ok, but this just seems to spell out what "YIELD_FROM" does,
so it should still be true that "await" boils down to "YIELD_FROM".
(Yes, Greg already kindly answered something to that effect.)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 31 Jan 2024 14:43:15 GMT, Stefan Ram wrote:

My ideas of language specifications are that they should contain all
the information to write an implementation of the language. I am not
sure whether the Python Language Reference already fullfils this
criterion.

Why not ask the various developers of implementations the Python language? There are quite a few different implementations around, and you can see
how successful they were at working from a common spec.

def f(): return g()

async def f(): return await g()

Assuming g() returns something awaitable, these two are logically
equivalent.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

ram@zedat.fu-berlin.de (Stefan Ram) writes:

In "The Python Language Reference, Release 3.13.0a0",
there is this section:
|6.4 Await expression
|Suspend the execution of coroutine on an awaitable object.
|Can only be used inside a coroutine function.
|await_expr ::= "await" primary

A wording I like is what I found in the World-Wide Web where
Victor Skvortsov wrote:

|When we await on some object, await first checks whether the
|object is a native coroutine or a generator-based coroutine,
|in which case it "yields from" the coroutine. Otherwise, it
|"yields from" the iterator returned by the object's
|__await__() method.
Victor Skvortsov (2021).

This actually explains "to wait on some object" (which might be the
same as to "suspend on some object"), and I was not able to find
such an explanation in the venerable Python Language Reference!

Heck, even of the respected members of this newsgroup, IIRC, no one
mentioned "__await__". So, let's give a big shoutout to Victor!

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

ram@zedat.fu-berlin.de (Stefan Ram) writes:

Heck, even of the respected members of this newsgroup, IIRC, no one
mentioned "__await__". So, let's give a big shoutout to Victor!

It should be possible to understand "await" in isolation (in
the spirit of the quotation I posted to "comp.lang.lisp" today.)
For example, use "await" without using "asycnio". So here's
an example program in that direction I just wrote.

main.py

class o_class():
def __await__( self ):
print( "__await__ called." )
return iter( [ 'a', 'b', 'c' ])

o = o_class()

async def f():
while 1: await o

for i, j in enumerate( f().__await__() ):
print( j )
if 5 == i: break

sys.stdout

__await__ called.
a
b
c
__await__ called.
a
b
c

(Then, it should also be interesting to understand how asyncio
uses "await" to implement asynchronous I/O.)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

ram@zedat.fu-berlin.de (Stefan Ram) writes:

(Then, it should also be interesting to understand how asyncio
uses "await" to implement asynchronous I/O.)

I have taken "sleep" from asyncio\tasks.py and simplified it a bit:

async def sleep( delay, result=None, *, loop=None ):

loop = events.get_running_loop()

future = loop.create_future()

h = loop.call_later
( delay, futures._set_result_unless_cancelled, future, result )

try: return await future
finally: h.cancel()

So, this is how the control is transferred to the event
loop after an "await sleep"! Initially, the control goes
to "sleep", but this transfers the control to the event loop
(until "sleep" stops waiting for its future in "await future").

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 1 Feb 2024 10:09:10 GMT, Stefan Ram wrote:

Heck, even of the respected members of this newsgroup, IIRC, no one
mentioned "__await__".

It’s part of the definition of an “awaitable”, if you had looked that up.

<https://docs.python.org/3/glossary.html#term-awaitable>

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2024-02-02, Lawrence D'Oliveiro <ldo@nz.invalid> wrote:

On 1 Feb 2024 10:09:10 GMT, Stefan Ram wrote:

Heck, even of the respected members of this newsgroup, IIRC, no one
mentioned "__await__".

It’s part of the definition of an “awaitable”, if you had looked that up.

<https://docs.python.org/3/glossary.html#term-awaitable>

To be fair, I've been using Python for well over quarter of a century,
and I never knew it had a glossary.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

ram@zedat.fu-berlin.de (Stefan Ram) writes:

So, this is how the control is transferred to the event
loop after an "await sleep"! Initially, the control goes
to "sleep", but this transfers the control to the event loop
(until "sleep" stops waiting for its future in "await future").

And, to answer one final question, let me quote the Web again.
The author name is not indicated clearly, but probably it's "Bharel":

|The final burning question we must answer is - how is the IO
|implemented?
...
|The IO part of the event loop is built upon a single crucial
|function called "select".
...
|When all available tasks are waiting for futures, the event
|loop calls select and waits. When the one of the sockets has
|incoming data, or its send buffer drained up, asyncio checks
|for the future object tied to that socket, and sets it to
|done.
...
probably Bharel.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2 Feb 2024 09:12:06 GMT, Stefan Ram wrote:

|The IO part of the event loop is built upon a single crucial
|function called "select".

select(2) has limitations. Better to use poll(2). Depending on *nix
variant, other more advanced alternatives may also be available <https://docs.python.org/3/library/select.html>.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

And whats the roadmap for an asyncified module
loader, is this on the radar of Python?

Mild Shock schrieb:

I am still waiting for async files in the
style of nodejs that works on windows and

is bundled with the main python distribution.
I am not very  fond on doing something

like adding listeners to a file descriptor,
in nodejs async files are based on callbacks

not on listeners. Whats the roadmap here?

Lawrence D'Oliveiro schrieb:

On 2 Feb 2024 09:12:06 GMT, Stefan Ram wrote:

|The IO part of the event loop is built upon a single crucial
|function called "select".

select(2) has limitations. Better to use poll(2). Depending on *nix
variant, other more advanced alternatives may also be available
<https://docs.python.org/3/library/select.html>.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

I am still waiting for async files in the
style of nodejs that works on windows and

is bundled with the main python distribution.
I am not very fond on doing something

like adding listeners to a file descriptor,
in nodejs async files are based on callbacks

not on listeners. Whats the roadmap here?

Lawrence D'Oliveiro schrieb:

On 2 Feb 2024 09:12:06 GMT, Stefan Ram wrote:

|The IO part of the event loop is built upon a single crucial
|function called "select".

select(2) has limitations. Better to use poll(2). Depending on *nix
variant, other more advanced alternatives may also be available <https://docs.python.org/3/library/select.html>.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sat, 3 Feb 2024 00:45:18 +0100, Mild Shock wrote:

... that works on windows ...

You lost me there.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

The docu tells me:

Windows
loop.add_reader() and loop.add_writer() only accept
socket handles (e.g. pipe file descriptors are not supported). https://docs.python.org/3/library/asyncio-platforms.html

Alternatives are aiofiles and anyio and maybe more,
but not sure whether they span all platforms, i.e. unix,
windows and mac or whether they have similar restrictions.

Theoretically on Windows IOCP should also cover file handles: https://en.wikipedia.org/wiki/Input/output_completion_port

Its use by libuv the basis for node.js.

Lawrence D'Oliveiro schrieb:

On Sat, 3 Feb 2024 00:45:18 +0100, Mild Shock wrote:

... that works on windows ...

You lost me there.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Funny source code tells me IOCP is used;

proactor is only implemented on Windows with IOCP. https://github.com/python/cpython/blob/3.12/Lib/asyncio/proactor_events.py

But maybe the focus is more on networking than file system.

But it has sock_sendfile() that might avoid copying data to userspace.

Mild Shock schrieb:

The docu tells me:

Windows
loop.add_reader() and loop.add_writer() only accept
socket handles (e.g. pipe file descriptors are not supported). https://docs.python.org/3/library/asyncio-platforms.html

Alternatives are aiofiles and anyio and maybe more,
but not sure whether they span all platforms, i.e. unix,
windows and mac or whether they have similar restrictions.

Theoretically on Windows IOCP should also cover file handles: https://en.wikipedia.org/wiki/Input/output_completion_port

Its use by libuv the basis for node.js.

Lawrence D'Oliveiro schrieb:

On Sat, 3 Feb 2024 00:45:18 +0100, Mild Shock wrote:

... that works on windows ...

You lost me there.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)