Example 1 (works as expected)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code adds new record to the beginning of the file,
expected behaviour
file.write("new city\n")

file.close()


Example 2 (weird behaviour)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code DOES NOT add new record TO THE BEGINNING of the
file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
new content should be added to the beginning of the file (as in
Example 1)
file.write("new city\n")

file.readlines()
file.close()

I could not find anything in documentation to explain this strange
behaviour. Why is this happening?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Looks like the data to be written is buffered, so actual write takes
place after readlines(), when close() flushes buffers.

See io package documentation, BufferedIOBase.

The solution is file.flush() after file.write()

Can't deny, such a behaviour looks utterly weird. Try to avoid designing
your software this way.

Axy.

On 19/02/2023 14:03, Azizbek Khamdamov wrote:

Example 1 (works as expected)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code adds new record to the beginning of the file,
expected behaviour
file.write("new city\n")

file.close()

Example 2 (weird behaviour)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code DOES NOT add new record TO THE BEGINNING of the
file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
new content should be added to the beginning of the file (as in
Example 1)
file.write("new city\n")

file.readlines()
file.close()

I could not find anything in documentation to explain this strange
behaviour. Why is this happening?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2023-02-19 16:57:02 +0000, Axy via Python-list wrote:

Looks like the data to be written is buffered, so actual write takes place after readlines(), when close() flushes buffers.

See io package documentation, BufferedIOBase.

The solution is file.flush() after file.write()

Or alternatively, file.seek() to the intended position when switching
between reading and writing. (The C standard says you have to do this. I
can't find it in the Python docs, but apparently Python behaves the
same.)

On 19/02/2023 14:03, Azizbek Khamdamov wrote:

Example 2 (weird behaviour)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code DOES NOT add new record TO THE BEGINNING of the
file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
new content should be added to the beginning of the file (as in
Example 1)
file.write("new city\n")

Also note that you can't ADD anything at the beginning (or in the
middle) of a file. You will overwrite existing content if you try this.
You can only add at the end of the file. If you want to insert
something, you have to rewrite everything from that position.

(So typically, for small files you wouldn't update a file in place, you
would just replace it completely. For large data sets which need to be
updated you would generally use some kind of database.)

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEETtJbRjyPwVTYGJ5k8g5IURL+KF0FAmPyWTQACgkQ8g5IURL+ KF31Jg/9GhLbcEi3xE1rDwYqKuFd14HhQOm9rVb7gfJ4Ya7eXVjZsG0Wdrbn71e+ q02RQzWop9zEfMQbXAdawc4hmEkbPZlZDA/HLEw3nd5jPaOgQVu2oiFQuGPMlCwh W6v81C9JEv7v1agjtV6tDp2VD96Zdlo8PJJLyptCeJSx2eRcY7A5oNVLimcYWHc9 A5NTE2haGmCSrOg5nPrJcgnmD7NuYUH4mb8IJplXTWaYI+cscSbIOlFlQlK3oqdg jMqkiaFRK1rQVW6DGn7o1Uytk3ZSP5Yr9LATd9IRlEKCMlE3rBMzdFrySCcZjdWI 7J66gD+cCesEyMvieUY5heiWDzJU07FLYOIgDC1ZE0hAp3XZHSJV+Bu7/6fC58s8 QbXaddOTWzVNAH9oKLcGi/5KYctGyENd2WFBmAv2bWCwLu1QixONaTf1zO+AKZ6O SH8aRWqet8HlzXs2sfqZTLJRLhzHSUSaSKllqdrnTBa6Yfkr271Pmnrz+Iq0bgUN aLXNnNfslS1dUyYDCyyVAQNJfSHhTpnlSLAvlPlmbPdTlFlZwBkwwtqHiGFymWJm 45FCeGVLZ6FahaaANcsZv2MDW8HkifqDiNgm768FwtEwKX672i0WJtzUaAp2+DHn enfMgkkzIHk+4XgiXGKT/4QAizft3JUQZQ6SQLh

On 2/19/2023 11:57 AM, Axy via Python-list wrote:

Looks like the data to be written is buffered, so actual write takes
place after readlines(), when close() flushes buffers.

See io package documentation, BufferedIOBase.

The solution is file.flush() after file.write()

Another possibility, from the Python docs:

"...TextIOWrapper (i.e., files opened with mode='r+') ... To disable
buffering in TextIOWrapper, consider using the write_through flag for io.TextIOWrapper.reconfigure()"


Also from the docs:

"Warning: Calling f.write() without using the with keyword or calling
f.close() might result in the arguments of f.write() not being
completely written to the disk, even if the program exits successfully."

I realize that in the example, close() was called ... but not
immediately after the write().

Can't deny, such a behaviour looks utterly weird. Try to avoid designing
your software this way.

Axy.

On 19/02/2023 14:03, Azizbek Khamdamov wrote:

Example 1 (works as expected)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code adds new record to the beginning of the file,
expected behaviour
file.write("new city\n")

file.close()


Example 2 (weird behaviour)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code DOES NOT add new record TO THE BEGINNING of the
file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
new content should be added to the beginning of the file (as in
Example 1)
file.write("new city\n")

file.readlines()
file.close()

I could not find anything in documentation to explain this strange
behaviour. Why is this happening?

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2023-02-19 14:03, Azizbek Khamdamov wrote:

Example 1 (works as expected)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code adds new record to the beginning of the file,
expected behaviour
file.write("new city\n")

file.close()

Example 2 (weird behaviour)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code DOES NOT add new record TO THE BEGINNING of the
file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
new content should be added to the beginning of the file (as in
Example 1)
file.write("new city\n")

file.readlines()
file.close()

I could not find anything in documentation to explain this strange
behaviour. Why is this happening?

It works correctly if you use file.flush() or file.tell() before
switching from writing to reading.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
<azizbek.khamdamov@gmail.com> wrote:

Example 1 (works as expected)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities

Side note: You happened to get lucky with P, w, and c, but for the
future, I recommend using forward slashes for your paths:

open("D:/Programming/Python/working_with_files/cities.txt", "r+")

Otherwise, you may run into annoying and hard-to-solve problems. Or alternatively, you'll upgrade to a newer Python and start getting
warnings, which would at least tell you that there's a problem.

ChrisA

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2023-02-19 12:59:43 -0500, Thomas Passin wrote:

On 2/19/2023 11:57 AM, Axy via Python-list wrote:

Looks like the data to be written is buffered, so actual write takes
place after readlines(), when close() flushes buffers.

See io package documentation, BufferedIOBase.

The solution is file.flush() after file.write()

Another possibility, from the Python docs:

"...TextIOWrapper (i.e., files opened with mode='r+') ... To disable buffering in TextIOWrapper, consider using the write_through flag for io.TextIOWrapper.reconfigure()"

That actually doesn't help (I tried it before writing my answer). The
binary layer below the text layer also buffers ...

Also from the docs:

"Warning: Calling f.write() without using the with keyword or calling f.close() might result in the arguments of f.write() not being completely written to the disk, even if the program exits successfully."

He does call file.close():

file.close()

so that doesn't seem relevant.

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEETtJbRjyPwVTYGJ5k8g5IURL+KF0FAmPyeJsACgkQ8g5IURL+ KF0K0g//f6VmQgtqrZ8MuFTkNBX/HhudJJ0Bql3yQJXyCTYLGTnxt58ZezWd+g/W u49nMWlMF0FaOPw8rO2KwpA7vp5Xj7vkBg4SMmE/e8JtV9yPom0TezkDZPuVGBrh yIj5oMfOU+dLF2jnP9e7SQseEQC4H9WxIrGJg2qXNcGa+Wk0mfA0MIsdzv+uXjzv yg2/gRf/MMnaXTJjWiY2itVmCfd80WaJ7FkBEorSqowMWK0HA63YElE0TvKa+reH dy2gZ2VcJ+P8t9agyIWKlYU/bMv5OjIDMoh87kOiu4puzy0Mo2tdGNwE4J57hI5q YstXHWK0mMtBT6LhoeD1Z6ZG8zm4PA1B/EGC8YPxRGBI434sS5VnhIvOTTU6wl68 ipcdzMFPPRR5irsJUakt0/PKuUSo0S9dzyV+VY7m2JLS6Y29qFHd4y/aJpajHHqn O9Z6SgJorKE1vQWBnkKaTVGLNFrPiVfQ/2e0SDbY1APb2eQQLhbTo5b/qFEUWuz+ bF1KeCGV1m7vU1nu5u0J4Vb4p9K8KNAmzmWnDNj4l+jrIiJeMkapNYmeBdAxXFHK nLkK8/f+gwDULL8hjPo/8TV0Ja8LXa1Mt/lcTbKkWcr1n9EvfEWDow/fjGwnkqtN a+x7QUQ2Qq8K1gRvvX2SOksG4Z3PvQhU5IxW5fO

On 2023-02-19 19:31, Chris Angelico wrote:

On Mon, 20 Feb 2023 at 06:24, Thomas Passin <list1@tompassin.net> wrote:

On 2/19/2023 1:53 PM, Chris Angelico wrote:

On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
<azizbek.khamdamov@gmail.com> wrote:

Example 1 (works as expected)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities

Side note: You happened to get lucky with P, w, and c, but for the
future, I recommend using forward slashes for your paths:

open("D:/Programming/Python/working_with_files/cities.txt", "r+")

Otherwise, you may run into annoying and hard-to-solve problems. Or
alternatively, you'll upgrade to a newer Python and start getting
warnings, which would at least tell you that there's a problem.

Or use r'...' strings. If you are copying a path to clipboard from
Windows Explorer - a fairly common operation - it's much easier to
prepend the "r" than to change all the backslashes to forward slashes.

Yep, either way. I personally prefer using forward slashes since
there's no way to end an r-string with a single backslash, which is
often useful when building a path:

# won't work
path = r"c:\path\to\some\"
open(path + "file")

# will work
path = "c:/path/to/some/"
open(path + "file")

When I build a path, I use os.path.join, so it's only a problem if one
of the strings refers to the root of a drive, e.g. "C:\\".

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 20 Feb 2023 at 06:24, Thomas Passin <list1@tompassin.net> wrote:

On 2/19/2023 1:53 PM, Chris Angelico wrote:

On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov <azizbek.khamdamov@gmail.com> wrote:

Example 1 (works as expected)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities

Side note: You happened to get lucky with P, w, and c, but for the
future, I recommend using forward slashes for your paths:

open("D:/Programming/Python/working_with_files/cities.txt", "r+")

Otherwise, you may run into annoying and hard-to-solve problems. Or alternatively, you'll upgrade to a newer Python and start getting
warnings, which would at least tell you that there's a problem.

Or use r'...' strings. If you are copying a path to clipboard from
Windows Explorer - a fairly common operation - it's much easier to
prepend the "r" than to change all the backslashes to forward slashes.

Yep, either way. I personally prefer using forward slashes since
there's no way to end an r-string with a single backslash, which is
often useful when building a path:

# won't work
path = r"c:\path\to\some\"
open(path + "file")

# will work
path = "c:/path/to/some/"
open(path + "file")

ChrisA

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2/19/2023 1:53 PM, Chris Angelico wrote:

On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
<azizbek.khamdamov@gmail.com> wrote:

Example 1 (works as expected)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities

Side note: You happened to get lucky with P, w, and c, but for the
future, I recommend using forward slashes for your paths:

open("D:/Programming/Python/working_with_files/cities.txt", "r+")

Otherwise, you may run into annoying and hard-to-solve problems. Or alternatively, you'll upgrade to a newer Python and start getting
warnings, which would at least tell you that there's a problem.

Or use r'...' strings. If you are copying a path to clipboard from
Windows Explorer - a fairly common operation - it's much easier to
prepend the "r" than to change all the backslashes to forward slashes.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2/19/2023 2:31 PM, Chris Angelico wrote:

On Mon, 20 Feb 2023 at 06:24, Thomas Passin <list1@tompassin.net> wrote:

On 2/19/2023 1:53 PM, Chris Angelico wrote:

On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
<azizbek.khamdamov@gmail.com> wrote:

Example 1 (works as expected)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities

Side note: You happened to get lucky with P, w, and c, but for the
future, I recommend using forward slashes for your paths:

open("D:/Programming/Python/working_with_files/cities.txt", "r+")

Otherwise, you may run into annoying and hard-to-solve problems. Or
alternatively, you'll upgrade to a newer Python and start getting
warnings, which would at least tell you that there's a problem.

Or use r'...' strings. If you are copying a path to clipboard from
Windows Explorer - a fairly common operation - it's much easier to
prepend the "r" than to change all the backslashes to forward slashes.

Yep, either way. I personally prefer using forward slashes since
there's no way to end an r-string with a single backslash, which is
often useful when building a path:

# won't work
path = r"c:\path\to\some\"
open(path + "file")

# will work
path = "c:/path/to/some/"
open(path + "file")

Never hit that one before. To be classified as "Learn something new
every day"!

I've been using pathlib more lately. It does clever things with "/" -

from pathlib import PurePath
p1 = PurePath('c:/this') / 'is' / 'a' /'test'
p1

PureWindowsPath('c:/this/is/a/test')
# on Linux, would be a PurePosixPath

and also

p2 = PurePath('c:/this/') /'is' /'a' /'test'
p2

PureWindowsPath('c:/this/is/a/test')

Despite the apparent forward slashes, the right separators get used for
the OS. And the Paths can be used with open(), etc.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500:

...
Example 2 (weird behaviour)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code DOES NOT add new record TO THE BEGINNING of the
file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
new content should be added to the beginning of the file (as in
Example 1)
file.write("new city\n")

file.readlines()
file.close()

I could not find anything in documentation to explain this strange
behaviour. Why is this happening?

The effect of "r+" (and friends) is specified by the C standard.

The Linux doc (of `fopen`) tells us that ANSI C requires that
a file positioning command (e.g. `seek`) must intervene
between input and output operations. Your example above violates
this condition. Therefore, weird behavior is to be expected.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2/19/23 14:06, Dieter Maurer wrote:

Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500:

...
Example 2 (weird behaviour)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code DOES NOT add new record TO THE BEGINNING of the
file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
new content should be added to the beginning of the file (as in
Example 1)
file.write("new city\n")

file.readlines()
file.close()

I could not find anything in documentation to explain this strange
behaviour. Why is this happening?

The effect of "r+" (and friends) is specified by the C standard.

The Linux doc (of `fopen`) tells us that ANSI C requires that
a file positioning command (e.g. `seek`) must intervene
between input and output operations. Your example above violates
this condition. Therefore, weird behavior is to be expected.

If this isn't sufficiently described, someone should raise an issue
against the Python docs. I know that many concepts are "inherited from" environments generally in the POSIX space and the C language, because
that's where Python was hatched (all of which makes perfect sense to me,
who's been working in those spaces for...ever), but a Python programmer shouldn't have to read the ISO C standard (which is not free, although
you can find copies on-line), or the POSIX standard (which also is not
free, though manpages for systems like Linux cover the same material),
in order to figure out how Python is going to work.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2/19/2023 6:10 PM, Mats Wichmann wrote:

On 2/19/23 14:06, Dieter Maurer wrote:

Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500:

...
Example 2 (weird behaviour)

file = open("D:\Programming\Python\working_with_files\cities.txt",
'r+') ## contains list cities
# the following code DOES NOT add new record TO THE BEGINNING of the
file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
new content should be added to the beginning of the file (as in
Example 1)
file.write("new city\n")

file.readlines()
file.close()

I could not find anything in documentation to explain this strange
behaviour. Why is this happening?

The effect of "r+" (and friends) is specified by the C standard.

The Linux doc (of `fopen`) tells us that ANSI C requires that
a file positioning command (e.g. `seek`) must intervene
between input and output operations. Your example above violates
this condition. Therefore, weird behavior is to be expected.

If this isn't sufficiently described, someone should raise an issue
against the Python docs.  I know that many concepts are "inherited from" environments generally in the POSIX space and the C language, because
that's where Python was hatched (all of which makes perfect sense to me, who's been working in those spaces for...ever), but a Python programmer shouldn't have to read the ISO C standard (which is not free, although
you can find copies on-line), or the POSIX standard (which also is not
free, though manpages for systems like Linux cover the same material),
in order to figure out how Python is going to work.

The Python docs say that text file behavior is done by Python, not C,
and so its behavior will be consistent across all platforms. I didn't
find any words about this case, though.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)