1. Forum
  2. Usenet
  3. LINUX.DEBIAN.BUGS.DIST

  • Bug#1107331: logcheck-database: Many incorrect regular expressions

    From Gabor Kiss@21:1/5 to All on Thu Jun 5 19:50:01 2025
    Package: logcheck-database
    Version: 1.4.2+deb12u1
    Severity: normal

    Dear Maintainer,

    I ran this Bash command in /etc/logcheck/ignore.d.server/ directory:

    for f in *
    do
    echo $f================
    while read line
    do
    grep -E "$line" /dev/null 2>&1
    [[ "$?" = 2 ]] && echo $line
    done < $f
    done

    The grep complained about more than hundred of regular expressions. Sample:

    courier================
    grep: Unmatched [, [^, [:, [., or [=
    ^(w{3} [ :0-9]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ ((imap|pop3)(login|d-ssl)|cou
    riertcpd): (LOGOUT|TIMEOUT|DISCONNECTED), ip=[[.:[:alnum:]]+](, port=[[[:digit:]
    ]+])?, headers=[[:digit:]]+, body=[[:digit:]]+$

    cron-apt================
    grep: Invalid range end
    ^(w{3} [ :[:digit:]]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ cron-apt: CRON-APT RUN [[-[:alnum:]/]+]: w{3} w{3} [ [:digit:]]+ [:[:digit:]]{8} w{3,4} [[:digit:]]{4}$
    grep: Unmatched [, [^, [:, [., or [=

    smartd================
    grep: Unmatched ( or \(
    ^(w{3} [ :[:digit:]]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ smartd[[[:digit:]]+]: Device: /dev/[^[:space:]]+( [[_/[:alnum:][:space:]]+])?( [SAT])?, Temperature changed (-|+)?[1-3]+ Celsius to ([0-4]?[[:digit:]]|5[0-4]) Celsius since last report$

    ssh================
    grep: Invalid character class name
    ^(w{3} [ :[:digit:]]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ sshd[[[:digit:]]+]: reverse mapping checking getaddrinfo for [._[:alnum:]-]+ ([[:.[:xdigit:]]+] )?failed - POSSIBLE BREAK-?IN ATTEMPT!$

    sympa================
    grep: Invalid collation character
    ^(w{3} [ :0-9]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ wwsympa[[0-9]+]: [robot [._[:alnum:]-]+] [client [0-9.]{7,15}] [user [^[:space:]]+] do_home$


    IMHO it would be useful to examine the reported lines.

    Gabor

    -- System Information:
    Debian Release: 12.11
    APT prefers stable-updates
    APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
    Architecture: amd64 (x86_64)

    -- no debconf information

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard Lewis@21:1/5 to kissg@ssg.ki.iif.hu on Thu Jun 5 23:10:01 2025
    On Thu, 5 Jun 2025, 18:49 Gabor Kiss, <kissg@ssg.ki.iif.hu> wrote:



    for f in *
    do
    echo $f================
    while read line
    do
    grep -E "$line" /dev/null 2>&1
    [[ "$?" = 2 ]] && echo $line
    done < $f
    done



    IMHO it would be useful to examine the reported lines.



    some of these have been removed for trixie, but i think not all -- in
    practice grep seems not always give the warning, even though every reported line is matched against every regexp (?) so i suppose.we can live with it
    for trixie? (most have been unchanged for years).

    not for trixie, but we should do this check as part of the autopkgtest

    <div dir="auto"><div><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Thu, 5 Jun 2025, 18:49 Gabor Kiss, &lt;<a href="mailto:kissg@ssg.ki.iif.hu">kissg@ssg.ki.iif.hu</a>&gt; wrote:</div><blockquote class="gmail_quote"
    style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>

    for f in *<br>
    do<br>
            echo $f================<br>
            while read line<br>
            do<br>
                    grep -E &quot;$line&quot; /dev/null 2&gt;&amp;1<br>
                    [[ &quot;$?&quot; = 2 ]] &amp;&amp; echo $line<br>
            done &lt; $f<br>
    done<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote gmail_quote_container"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);
    padding-left:1ex">IMHO it would be useful to examine the reported lines.<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto">some of these have been removed for trixie, but i think not all -- in practice grep
    seems not always give the warning, even though every reported line is matched against every regexp (?) so i suppose.we can live with it for trixie? (most have been unchanged for years).</div><div dir="auto"><br></div><div dir="auto">not for trixie, but
    we should do this check as part of the autopkgtest</div></div>

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Kiss Gabor (Bitman)@21:1/5 to Richard Lewis on Fri Jun 6 07:00:02 2025
    On Thu, 5 Jun 2025, Richard Lewis wrote:

    practice grep seems not always give the warning, even though every reported line is matched against every regexp (?)

    Dear Richard,

    It seems grep regards differently the command line pattern and the
    -f patternfile.

    Command
    for f in * ; do echo $f================; grep -E -f $f /dev/null 2>&1 ; done reports only two problems:
    ...
    login================
    grep: login:8: Unmatched ( or \(
    ...
    systemd================
    grep: systemd:21: Unmatched ( or \(
    ...

    But both comes from comments:
    # old messages, no longer seen (?)
    # services with Restart=always ((eg console-getty.service)



    IMHO the most cases the problem is that literal square brackets
    are unescaped. E.g.

    | courier================
    | grep: Unmatched [, [^, [:, [., or [=
    | ^(w{3} [ :0-9]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ ((imap|pop3)(login|d-ssl)|couriertcpd): (LOGOUT|TIMEOUT|DISCONNECTED), ip=[[.:[:alnum:]]+](, port=[[[:digit:]]+])?, headers=[[:digit:]]+, body=[[:digit:]]+$

    ip=[[.:[:alnum:]]+] and port=[[[:digit:]]+] are suspicious.
    I guess they should be ip=\[[.:[:alnum:]+]\] and \[[[:digit:]+]\] .

    | cron-apt================
    | grep: Invalid range end
    | ^(w{3} [ :[:digit:]]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ cron-apt: CRON-APT RUN [[-[:alnum:]/]+]: w{3} w{3} [ [:digit:]]+ [:[:digit:]]{8} w{3,4} [[:digit:]]{4}$

    [[-[:alnum:]/]+] should be \[[-[:alnum:]/]+\] .

    | dovecot================
    | grep: Unmatched [, [^, [:, [., or [=
    | ^(w{3} [ :[:digit:]]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ (dovecot: )?(imap|pop3)-login: Disconnected [[.:[:xdigit:]]+]$

    I bet on \[[.:[:xdigit:]]+\] .

    | exim4================
    | grep: Unmatched [, [^, [:, [., or [=
    | ^[-0-9]{10} [0-9:]{8} [-[:alnum:]]+ [=-]> [@._[:alnum:]-]+ <?[@._[:alnum:]-]+>? R=dnslookup T=remote_smtp H=[._[:alnum:]-]+ [[.0-9]{7,15}]$

    [[.0-9]{7,15}] is probably \[[.0-9]{7,15}\] .

    | horde3================
    | grep: Unmatched [, [^, [:, [., or [=
    | ^(w{3} [ :0-9]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ .+[[0-9]+]: [horde] Login success for [@._[:alnum:]-]+ [[.0-9]{7,15}] to Horde [on line [0-9]+ of ".+"]$

    \[[.0-9]{7,15}] to Horde [on line [0-9]+ of ".+"\]

    Cheers

    Gabor

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Richard Lewis@21:1/5 to kissg@probatbicol.hu on Fri Jun 6 19:40:01 2025
    On Fri, 6 Jun 2025 at 05:51, Kiss Gabor (Bitman) <kissg@probatbicol.hu> wrote:
    On Thu, 5 Jun 2025, Richard Lewis wrote:

    It seems grep regards differently the command line pattern and the
    -f patternfile.

    this looks like a bug in grep. perhaps you could report it to grep --
    i suppose they never tested invalid lines with -f

    (but the other issues below we should still fix)

    Command
    for f in * ; do echo $f================; grep -E -f $f /dev/null 2>&1 ; done reports only two problems:
    ...
    login================
    grep: login:8: Unmatched ( or \(
    ...
    systemd================
    grep: systemd:21: Unmatched ( or \(
    ...

    But both comes from comments:
    # old messages, no longer seen (?)
    # services with Restart=always ((eg console-getty.service)

    that's good in the sense that we dont need to urgently fix this for
    trixie (although the first one doesnt look unmatched!) -- commented
    lines are dropped by logcheck anyway,

    IMHO the most cases the problem is that literal square brackets
    are unescaped. E.g.

    | courier================
    | grep: Unmatched [, [^, [:, [., or [=
    | ^(w{3} [ :0-9]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ ((imap|pop3)(login|d-ssl)|couriertcpd): (LOGOUT|TIMEOUT|DISCONNECTED), ip=[[.:[:alnum:]]+](, port=[[[:digit:]]+])?, headers=[[:digit:]]+, body=[[:digit:]]+$

    ip=[[.:[:alnum:]]+] and port=[[[:digit:]]+] are suspicious.
    I guess they should be ip=\[[.:[:alnum:]+]\] and \[[[:digit:]+]\] .

    i think you are be right


    | cron-apt================
    | grep: Invalid range end
    | ^(w{3} [ :[:digit:]]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ cron-apt: CRON-APT RUN [[-[:alnum:]/]+]: w{3} w{3} [ [:digit:]]+ [:[:digit:]]{8} w{3,4} [[:digit:]]{4}$

    [[-[:alnum:]/]+] should be \[[-[:alnum:]/]+\] .

    yes - this one is already corrected in the rules that will be in
    trixie, i think

    | dovecot================
    | grep: Unmatched [, [^, [:, [., or [=
    | ^(w{3} [ :[:digit:]]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ (dovecot: )?(imap|pop3)-login: Disconnected [[.:[:xdigit:]]+]$

    I bet on \[[.:[:xdigit:]]+\] .

    looks like it


    | exim4================
    | grep: Unmatched [, [^, [:, [., or [=
    | ^[-0-9]{10} [0-9:]{8} [-[:alnum:]]+ [=-]> [@._[:alnum:]-]+ <?[@._[:alnum:]-]+>? R=dnslookup T=remote_smtp H=[._[:alnum:]-]+ [[.0-9]{7,15}]$

    [[.0-9]{7,15}] is probably \[[.0-9]{7,15}\] .

    this file has other issues -- the helo logging format is quite
    complex, and you can have an ip in or outside [...].

    this file is on the "revise in forky" list anyway i think -- i believe
    that it's not actualy matching anything at the moment, because these
    lines only go in exim4's log, which is not checked by default


    | horde3================
    | grep: Unmatched [, [^, [:, [., or [=
    | ^(w{3} [ :0-9]{11}|[0-9T:.+-]{32}) [._[:alnum:]-]+ .+[[0-9]+]: [horde] Login success for [@._[:alnum:]-]+ [[.0-9]{7,15}] to Horde [on line [0-9]+ of ".+"]$

    \[[.0-9]{7,15}] to Horde [on line [0-9]+ of ".+"\]

    i beleive horde3 is no longer in debian, so this whole file should be
    dropped instead

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)