• Re: Dovecot correct ownership for logs

    From Geert Stappers@21:1/5 to Richard on Mon May 13 22:30:01 2024
    On Mon, May 13, 2024 at 10:16:13PM +0200, Richard wrote:
    Maybe someone here knows how the ownership of these files for Dovecot needs to be in order to work, as various distributions of Dovecot packages seem
    to use different users:
    I'd like Dovecot not to log into syslog, but to dedicated files. Therefore I've created the directory /var/log/dovecot and told dovecot in 10-logging.conf to log info, debug and error messages to separate files.
    But I get error messages from postfix (weird):

    May 13 20:55:37 mail postfix/local[2824184]: 95BCF1000A9: to=<user@domain.de>,
    relay=local, delay=3.2, delays=1.9/0.29/0/1.1, dsn=4.3.0, status=deferred (temporary failure. Command output: lda(user): Error: net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied Can't open log file /var/log/dovecot/error.log: Permission denied )

    This is the content of /var/log/dovecot:
    -rw-r--r-- 1 dovecot dovecot 0 13. Mai 20:50 debug.log
    -rw-r--r-- 1 dovecot dovecot 880 13. Mai 21:21 error.log
    -rw-r--r-- 1 dovecot dovecot 40K 13. Mai 21:20 info.log

    So why is Dovecot complaining?

    :-)


    I only see processes owned by root, dovecot,
    dovenull and user processes of dovecot. And dovenull seems to be not
    relevant for this.


    Reread
    } May 13 20:55:37 mail postfix/local[2824184]: 95BCF1000A9: to=<user@domain.de>,
    } } open log file /var/log/dovecot/error.log: Permission denied )

    Note postfix


    Best
    Richard


    Groeten
    Geert Stappers
    --
    Silence is hard to parse

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to Richard on Mon May 13 22:30:01 2024
    On Mon, May 13, 2024 at 10:16:13PM +0200, Richard wrote:
    May 13 20:55:37 mail postfix/local[2824184]: 95BCF1000A9: to=<user@domain.de>,
    relay=local, delay=3.2, delays=1.9/0.29/0/1.1, dsn=4.3.0, status=deferred (temporary failure. Command output: lda(user): Error: net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied Can't open log file /var/log/dovecot/error.log: Permission denied )

    This is the content of /var/log/dovecot:
    -rw-r--r-- 1 dovecot dovecot 0 13. Mai 20:50 debug.log
    -rw-r--r-- 1 dovecot dovecot 880 13. Mai 21:21 error.log
    -rw-r--r-- 1 dovecot dovecot 40K 13. Mai 21:20 info.log

    The first two things that come to mind are AppArmor, and systemd unit
    files.

    Check to see whether there's an AppArmor profile for this program, or
    any lines from AppArmor in dmesg.

    If that's not it, look at the systemd unit file(s) that start the
    dovecot service(s), if there are any, and see if they're using any of
    the directives that restrict the locations the program can access.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From tomas@tuxteam.de@21:1/5 to Richard on Tue May 14 06:40:01 2024
    On Mon, May 13, 2024 at 10:16:13PM +0200, Richard wrote:
    Maybe someone here knows how the ownership of these files for Dovecot needs to be in order to work, as various distributions of Dovecot packages seem
    to use different users:
    I'd like Dovecot not to log into syslog, but to dedicated files. Therefore I've created the directory /var/log/dovecot and told dovecot in 10-logging.conf to log info, debug and error messages to separate files.
    But I get error messages from postfix (weird):

    I think this Dovecot's LDA (the local delivery agent) [1], which is
    invoked by the MTA (Postfix) and is, therefore, most probably running
    as postfix.

    [...]

    (temporary failure. Command output: lda(user): Error: net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied Can't open log file /var/log/dovecot/error.log: Permission denied )

    This message actually is an indicator against the chroot theory posed
    elsewhere in this thread (in a chroot, you would get "no such file or directory", I guess).

    This is the content of /var/log/dovecot:
    -rw-r--r-- 1 dovecot dovecot 0 13. Mai 20:50 debug.log
    -rw-r--r-- 1 dovecot dovecot 880 13. Mai 21:21 error.log
    -rw-r--r-- 1 dovecot dovecot 40K 13. Mai 21:20 info.log

    Try to set the log file's group to mail (or whatever group Postfix is
    running as) and make them group writable.

    Cheers
    --
    t

    -----BEGIN PGP SIGNATURE-----

    iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCZkLp5gAKCRAFyCz1etHa RisHAJ9NqinAE4/nfmSEuJmSL2YJOSWanACfWMzQUaHKwuqMzMI2GXpNpN4TMZ4=
    =l+D/
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to jeremy ardley on Tue May 14 13:50:02 2024
    On Tue, May 14, 2024 at 07:36:17PM +0800, jeremy ardley wrote:
    Postfix is chrooted (usuallly) to /var/spool/postfix

    If this is true, then how would a local delivery agent work? It needs
    write access to all users' inboxes, which are either in /var/mail or in
    users' home directories.

    I could imagine the Postfix SMTP sending/receiving and queue processing programs being chrooted, but the LDA probably isn't. Or at least not
    chrooted to /var/spool/postfix.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From tomas@tuxteam.de@21:1/5 to Richard on Tue May 14 13:50:01 2024
    On Tue, May 14, 2024 at 01:29:17PM +0200, Richard wrote:
    My guess is that postfix runs as postfix.

    That would be my guess too (or perhaps as some special "Debian-+postfix".

    At least processes like local,
    smtpd, bounce etc run as that user. But beyond that I have no idea how to find that out. At least there's nothing in the postfix.service or postfix@.service
    about that. So I've changed the files to dovecot:postfix 664, but same
    error.

    You might try

    ps -eo pid,user,group,comm | grep postfix

    or similar. Or have a look at Posrfix's log file ownerships.

    You might try making the log files in question world writable just
    to see whether the problem disappears or this approach is a blind
    alley (don't forget to revert that: leaving them world-writable
    seems like asking for trouble).

    Cheers
    --
    t

    -----BEGIN PGP SIGNATURE-----

    iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCZkNO6gAKCRAFyCz1etHa RnG+AJ9eS5z5m5f3dFzbIvlia7bc9HbjsgCfb7i2jneDdQjbvpD+3Z6ym7eQzr0=
    =+/9C
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From tomas@tuxteam.de@21:1/5 to jeremy ardley on Tue May 14 13:50:02 2024
    On Tue, May 14, 2024 at 07:36:17PM +0800, jeremy ardley wrote:

    [...]

    Postfix is chrooted (usuallly) to /var/spool/postfix

    If postfix complains about /var/log/dovecot it's actually complaining about /var/spool/postfix/var/log/dovecot

    I'm sceptical about this -- the error would have been ENOENT, not EPERM (because an intervening directory would be missing).

    But of course, I might be wrong.

    Cheers
    --
    t

    -----BEGIN PGP SIGNATURE-----

    iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCZkNPaQAKCRAFyCz1etHa RpEEAJ9NsUaZmwqGYT4da7u49clS/UYNmgCfe7wzEHlyL9IvC9gQQ3TxOJzDQj4=
    =G1eV
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Florent Rougon@21:1/5 to All on Tue May 14 14:10:02 2024
    Le 14/05/2024, tomas@tuxteam.de a écrit:

    You might try

    ps -eo pid,user,group,comm | grep postfix

    or similar.

    Yep, and beware that the original message mentions a postfix program
    named 'local' (/usr/lib/postfix/sbin/local).

    May 13 20:55:37 mail postfix/local[2824184]: (...)

    Regards

    --
    Florent

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From tomas@tuxteam.de@21:1/5 to Richard on Tue May 14 14:20:01 2024
    On Tue, May 14, 2024 at 02:11:53PM +0200, Richard wrote:

    [...]

    Setting the permissions in /var/log/dovecot to 666 actually didn't
    solve the problem [...]

    This seems to prove (or, at least, strongly suggest) that I was barking
    up the wrong tree. I've currently run out of trees and at $DAYJOB, so
    tight on resources. Good luck :)

    Cheers
    --
    t

    -----BEGIN PGP SIGNATURE-----

    iF0EABECAB0WIQRp53liolZD6iXhAoIFyCz1etHaRgUCZkNWOQAKCRAFyCz1etHa Rg06AJ472IyvFafBPtUssnq4SQQ3c6P9UwCcDy/PmUetXRaxVhW9jpVYwl4IaHc=
    =PR9D
    -----END PGP SIGNATURE-----

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Henning Follmann@21:1/5 to Richard on Tue May 14 16:00:01 2024
    On Tue, May 14, 2024 at 03:11:16PM +0200, Richard wrote:
    "Top posting" (writing the answer above the text that's being replied to)
    is literally industry standard behavior.

    Whatever.
    It is not standard behavior in mailing lists. https://wiki.debian.org/DebianMailingLists#Posting_Rules.2C_Guidelines.2C_and_Tips

    But I just asked you to not to. You can ignore that, as you do.

    Good luck with your issues.

    Also, I don't think you've really cleared out any confusion. Now, how
    exactly can dovecot log to /var/log/dovecot/ without (postfix) throwing errors? Because it clearly is for 2 out of 3 files as visible from the file sizes in my original post. Only the debug file is empty, but maybe nothing relevant enough has been found yet. There are entries from master,
    imap, imap-login, managesieve-login, anvil and various other processes in info and error. But the error messages from postfix still appear, also seemingly at least vastly slowing down emails being delivered to the users, if not bringing that to a complete halt. Only after commenting out these locations in 10-logging.conf the mails show up for the users.



    -H


    --
    Henning Follmann | hfollmann@itcfollmann.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Nicolas George@21:1/5 to All on Tue May 14 17:50:01 2024
    Alain D D Williams (12024-05-14):
    PS: check the dictionary definition of "literally".

    I think you should have checked first that it makes the point you want
    to make and not the opposite:

    2. (degree, figuratively, proscribed, contranym) Used non-literally as
    an intensifier for figurative statements: virtually, so to speak
    (often considered incorrect; see usage notes)
    3. (colloquial) Used to intensify or dramatize non-figurative
    statements.
    4. (colloquial) Used as a generic downtoner: just, merely. <https://en.wiktionary.org/wiki/literally>

    2. in effect : virtually—used in an exaggerated way to emphasize a
    statement or description that is not literally true or possible <https://www.merriam-webster.com/dictionary/literally>

    * used to emphasize what you are saying:
    * simply or just: <https://dictionary.cambridge.org/dictionary/english/literally>

    Regards,

    --
    Nicolas George

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Alain D D Williams@21:1/5 to Richard on Tue May 14 17:40:01 2024
    On Tue, May 14, 2024 at 03:11:16PM +0200, Richard wrote:
    "Top posting" (writing the answer above the text that's being replied
    to) is literally industry standard behavior.

    Many do top post, but many do not.

    Places where it is often frowned on are technical mail lists such as this one. This is because only quoting to the parts of the mail that you reply to and putting you comment underneath can greatly help understanding.

    Read the Netiquette Guidelines (1995):

    https://www.ietf.org/rfc/rfc1855.txt

    Other discussions here:

    https://idallen.com/topposting.html

    https://www.caliburn.nl/topposting.html

    PS: check the dictionary definition of "literally".

    --
    Alain Williams
    Linux/GNU Consultant - Mail systems, Web sites, Networking, Programmer, IT Lecturer.
    +44 (0) 787 668 0256 https://www.phcomp.co.uk/
    Parliament Hill Computers. Registration Information: https://www.phcomp.co.uk/Contact.html
    #include <std_disclaimer.h>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Henning Follmann@21:1/5 to Richard on Wed May 15 14:00:01 2024
    On Wed, May 15, 2024 at 12:23:35PM +0200, Richard wrote:

    [...]

    But that's still not that helpful for the main issue. Why on earth is
    postfix throwing issues about the log files, even when they are world-readable and -writable? It's not that dovecot doesn't log to them,
    but it's also not the case that it's an error message that can just be ignored, as it brings mail delivery to a halt.


    Maybe because you write directly to the logfile. One process trying to write
    t it while a different already holds a lock to it.

    -H

    --
    Henning Follmann | hfollmann@itcfollmann.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Henning Follmann@21:1/5 to Richard on Thu May 16 14:40:01 2024
    On Thu, May 16, 2024 at 01:00:19PM +0200, Richard wrote:
    But why is postfix even holding a lock on it? And how do I prevent that? I never asked it to.
    At least, I don't think there should be a different process holding a lock
    on it.


    I told you where to look, which is more than you deserve after how you
    behave.
    Configure the literal industry standard syslog or journald to use a
    facility to your liking and the problem should resolve itself.

    -H


    --
    Henning Follmann | hfollmann@itcfollmann.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to Henning Follmann on Fri May 17 14:30:01 2024
    On Fri, May 17, 2024 at 08:20:17AM -0400, Henning Follmann wrote:
    No the point is, you are not setting a file path, you are configure dovecot to directly write to these files.
    And dovecot is not just one process, there are multiple running as
    different users all trying t write into one file. Race conditions are
    to be expected. Because these options exist does not mean it is a good decision to use them.

    This is speculation. When writing to log files, one normally opens the
    file in append mode. Multiple processes opening a single log file in
    append mode, and using atomic write() calls, will not have any locking
    or concurrency issues. It'll just work.

    Now, the question is whether Dovecot and Postfix both work this way. I
    don't use those two programs, so I don't know the answer to that.

    It has become apparent that nobody on this mailing list has the intimate technical knowledge of Dovecot and/or Postfix required to assist the OP
    with setting up this nonstandard logging configuration. I would suggest
    that the OP should try on a Dovecot or Postfix support list instead.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Henning Follmann@21:1/5 to Richard on Fri May 17 14:30:01 2024
    On Fri, May 17, 2024 at 10:12:03AM +0200, Richard wrote:
    So now that you don't know any further, you just start lying? Now that's rich.

    I told you where to look, which is more than you deserve after how you
    behave.

    You didn't though.

    Don't call me a liar, you are just too dumb to understand.


    Configure the literal industry standard syslog or journald to use a facility
    to your liking and the problem should resolve itself.
    The point is, Dovecot has an option to write certain types of logs to different files. While it's doing that great, postfix is upset about that capability. It shouldn't even try to access these files. So the issue is
    not being able to log to files, that's already solved, but postfix running crazy when using a very simple setting.


    No the point is, you are not setting a file path, you are configure dovecot
    to directly write to these files.
    And dovecot is not just one process, there are multiple running as
    different users all trying t write into one file. Race conditions are
    to be expected. Because these options exist does not mean it is a good decision to use them.
    And because you clearly don't understand what you are doing, do as being told. Configure syslog or rsyslog to use that file path.


    -H


    --
    Henning Follmann | hfollmann@itcfollmann.com

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to Richard on Sun May 19 17:10:01 2024
    On Sun, May 19, 2024 at 04:55:09PM +0200, Richard wrote:
    Dovecot expects execution permissions on the directory it writes the logs
    to. Because "Standard POSIX permissions for a non-root process to enter a directory." How on earth is that even a thing?

    That's how Unix permissions have always worked. In order to access
    a file, you need +x permissions on *all* of the directories leading
    up to that file, and then appropriate permissions on the file itself.

    If you have read permission on a directory but *not* execute permissions,
    then the only thing you can do is read the contents of that directory --
    the filenames and their inode numbers. You cannot stat() the files,
    so you can't see who owns them or even what kind of files they are.
    Just their names.

    If you have execute permission but *not* read permission on a directory,
    then you can access the files within the directory, but only if you
    already know their names. You can't read the directory to get their
    names.

    Likewise, write permission on a directory allows you to rename or unlink
    files contained within that directory (because the names are not a
    property of the files -- they are part of the *directory*). You don't
    need write permission on a file to unlink it. Only on the directory.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Stefan Monnier@21:1/5 to All on Sun May 19 17:20:01 2024
    If you have read permission on a directory but *not* execute permissions, then the only thing you can do is read the contents of that directory --
    the filenames and their inode numbers. You cannot stat() the files,
    so you can't see who owns them or even what kind of files they are.
    Just their names.

    Never found a situation where this as useful.

    If you have execute permission but *not* read permission on a directory,
    then you can access the files within the directory, but only if you
    already know their names. You can't read the directory to get their
    names.

    This OTOH is very handy, making the filename into a kind of "passwd" to
    access the file's content.


    Stefan

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Greg Wooledge@21:1/5 to Richard on Sun May 19 17:40:01 2024
    On Sun, May 19, 2024 at 05:15:40PM +0200, Richard wrote:
    Then where does the combination rwx come in here? With read the app knows
    the file is there, with write it writes to the file. Question is, where the necessity would be to know the owner of the file or even the kind. The
    logger is supposed to just append text to a file.

    Stop trying to reason out WHY things are the way they are. Just accept it.

    You need execute permission on a directory in order to access anything
    within that directory. That's how it is. That's the decision the
    creators of Unix came up with.

    In order for a program to write (append) to a file that already exists,
    it needs:

    * execute permission on every directory leading up to that file
    * write permission on the file

    If the file does NOT already exist, then the program needs to be able to
    create it, and in that case, it will need:

    * execute permission on every directory leading up to where the file goes
    * write permission on the final (leaf) directory

    File creation, file removal, and file renaming are directory operations,
    and they need write (plus execute, of course) permission on the directory.

    Modifying an existing file is a file operation, so that needs write
    permission on the file (plus execute permission on the directory).

    In EVERY case, you always need execute permission on all the parent
    directories as well. This is well-understood, so we usually don't bother saying it explicitly.

    Kudos to whoever you spoke with on the Dovecot list who thought to go
    all the way back to first principles on this one. We got distracted
    with all of the other complex ways that this setup could have been
    incorrect, and forgot to check one of the most basic ones.

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Michael =?utf-8?B?S2rDtnJsaW5n?=@21:1/5 to All on Sun May 19 18:30:01 2024
    On 19 May 2024 11:11 -0400, from monnier@iro.umontreal.ca (Stefan Monnier):
    If you have read permission on a directory but *not* execute permissions,
    then the only thing you can do is read the contents of that directory --
    the filenames and their inode numbers. You cannot stat() the files,
    so you can't see who owns them or even what kind of files they are.
    Just their names.

    Never found a situation where this as useful.

    Put the user that a HTTP server runs in into each user's group, and
    set g=x on each user's home directory plus ~/public_html. (Your choice
    whether you want g=rx or g=x on the latter.) The web server process is
    now able to access web content from within the user's home directory
    without needing to run anything as root (possibly after dropping
    privileges after binding to ports 80 and 443), and all this without
    giving anyone else any access into other users' home directories.

    Sure, something like www.example.com/~username mapping to a directory
    inside username's home directory isn't that common a pattern these
    days, but it used to be _very_ common. And since it's exposed more or
    less directly to untrusted network inputs, it's nice to be able to
    make the HTTP server drop privileges as much as possible.

    This generalizes to any process having reason to descend into a
    directory but not reason to enumerate the contents of the directory it
    needs to descend into. Another example would be a process wanting to
    manage its own /var/log/$subsystem directory itself; it doesn't need
    to do anything to anything in /var/log, it only needs to be able to
    descend into its own directory.

    Yes, you _can_ do it in other ways. But the above is definitely _one_
    way.

    --
    Michael Kjörling 🔗 https://michael.kjorling.se “Remember when, on the Internet, nobody cared that you were a dog?”

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)