1. Forum
  2. Usenet
  3. LINUX.DEBIAN.USER

  • Instant flush to file

    From Jeff Peng@21:1/5 to All on Tue Jun 18 15:20:01 2024
    Hi

    After write to file with bash shell like:

    echo … > file
    echo … >> file

    Is it possible the file get no update instantly?

    I ask this b/c some commands following echo cannot run well.

    Such as,

    echo … >> dove.db
    doveadm create mailbox from dove.db

    Sometimes it succeeds. Sometimes it fails.

    If I add a line like this:

    echo … >> dove.db
    sleep 1
    doveadm create mailbox from dove.db

    It seems working now.

    Thanks.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to Jeff Peng on Tue Jun 18 15:30:01 2024
    On Tue, Jun 18, 2024 at 21:17:07 +0800, Jeff Peng wrote:
    After write to file with bash shell like:

    echo … > file
    echo … >> file

    Is it possible the file get no update instantly?

    No. Each of those commands opens the file (the first one opens it for
    write, the second opens it for append), writes data, and then closes
    the file.

    If you really have both of those commands in your script, then you are
    in fact opening and closing the file twice, which is not the most
    efficient way to go.

    But your speculation ("maybe there is a delay before the data actually
    gets written by the kernel") is groundless.

    echo … >> dove.db
    doveadm create mailbox from dove.db

    Sometimes it succeeds. Sometimes it fails.

    If I add a line like this:

    echo … >> dove.db
    sleep 1
    doveadm create mailbox from dove.db

    It seems working now.

    If the "dove.db" file is on a local file system, then your problem is
    coming from something else. Maybe there's a previous "doveadm"
    command that starts something up, and that "something" isn't fully
    up and running by the time your second doveadm command is executed?

    If any network layers are involved, then *maybe* there's something in
    one of those layers that could cause the file to be seen in a partially
    written state, but even that would be highly speculative.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael =?utf-8?B?S2rDtnJsaW5n?=@21:1/5 to All on Tue Jun 18 16:00:01 2024
    On 18 Jun 2024 21:17 +0800, from jeff@tls-mail.com (Jeff Peng):
    I ask this b/c some commands following echo cannot run well.

    Such as,

    echo … >> dove.db
    doveadm create mailbox from dove.db

    Sometimes it succeeds. Sometimes it fails.

    If I add a line like this:

    echo … >> dove.db
    sleep 1
    doveadm create mailbox from dove.db

    It seems working now.

    Your problem is almost certainly not related to writes not yet being
    committed to stable storage.

    Caching is used for both reads and writes. If a write syscall has
    finished, then even if the data is still only in a cache, a subsequent
    read syscall will return it.

    You should be able to verify this by replacing your "doveadm"
    invocation with, say, md5sum (or anything else which by necessity must
    read the file in question). If the results or output of that is
    consistent, then the data is being properly persisted and read back
    (again, even if it hasn't been written out to stable storage yet
    because of caching and write coalescing).

    So whatever your problem is, you have either (a) stumbled upon a huge
    bug in the kernel's I/O subsystem (unlikely), or (b) an unrelated
    issue. It's hard to tell from simply a "sometimes it fails" what that
    issue might be; maybe this "doveadm" program is doing some extra
    validation of some kind, and a rapid execution fails this validation?

    --
    Michael Kjörling 🔗 https://michael.kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?”

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)