To:
Brian.Inglis@systematicsw.ab.ca
On Thu, Nov 21, 2024 at 4:56 PM Brian Inglis <
Brian.Inglis@systematicsw.ab.ca> wrote:
On 2024-11-20 14:32, Roger wrote:
On Wed, 20 Nov 2024 19:53:00 -0000 (UTC), Brian Inglis wrote:
Maybe add "iburst preempt" options and drop "poll 11" or perhaps change
to
"maxpoll 11" or higher, unless you have very good reasons to require a longer
interval than the default maximum, instead of adaptive polling based on the error.
Well, the documentation (confopt) tells me that the pool command
"mobilizes a preemptable pool client mode association for the
DNS name specified." Why would adding "preempt" change anything?
It *may* be required and can never hurt:
In fact it won't change anything. The only options propagated from the "
pool" directive in ntp.conf (and thereby set on the prototype pool
association listed with refid POOL in the peers billboard) to the resulting pool server associations are "iburst" and "noselect". See POOL_FLAG_PMASK
in source code file ntp_proto.c.
The preemptible option is forced on for pool servers, so they are
preemptible with or without that option. However, that option doesn't do
much in 4.2.8 as the code intended to preempt useless servers has an
off-by-one error that's corrected in my test 3792 release, so preemption
only happens in the unusual case where there are more than 2 times as many
pool or manycast client associations as "tos maxclock" which defaults to
10. Arguably this could be fixed in the stable 4.2.8 branch but it would
be a substantial change in behavior without any configuration change that
might break existing setups that depend on the off-by-one error.
As an aside, using "preempt" on a non-pool non-manycastclient association (basically, configured via "server" or "peer") seems quixotic to me, as it allows the association to be removed but nothing is done to replace it. I
have a difficult time imagining where that might be useful. It may have
been useful in the pre-2009 implementation of "pool" which I'm having a
hard time remembering because I thought it was primitive and needed improvement, as it did all its work at startup and never changed the
servers selected once up and running. I re-implemented it to the current iteration, but didn't catch that the preemption was suffering the aforementioned off-by-one error, or it wasn't back then.
If you're wondering why I mentioned "manycastclient", it shares much of the implementation with "pool". They use different approaches to finding
servers, but the rest of the code is common. Both are intended to be
automatic server discovery schemes that discard, or preempt, servers which haven't been useful for 10 poll intervals so that another server can be solicited to replace it.
$ grep 'pool.*preempt' ~/src/time/ntp/ntp-4.2.8p18/ntpd/complete.conf.in
pool 2.ubuntu.pool.ntp.org. iburst preempt
complete.conf.in is part of the "make check" tests and is not intended to suggest useful configurations. Rather it's used both to ensure every
keyword in the configuration file parser is covered, and to ensure a configuration can successfully round-trip through ntpd's reading and
applying the configuration and exporting the configuration via the --saveconfigquit command-line option added specifically for that developer
test to catch changes which break that functionality. It's no coincidence it is ordered exactly the same as the output of ntpq's saveconfig command,
which requires authentication and that a directory for such saved
configuration files has been specified in ntp.conf with "saveconfigdir".
$ man 5 ntp.conf
...
Configuration Commands
...
*pool* For type s addresses, this command mobilizes a persistent
client mode association with a number of remote servers. In
this mode the local clock can synchronized to the remote server,
but the remote server can never be synchronized to the local
clock.
...
Options:
...
*preempt* Says the association can be preempted.
...
This manual page was AutoGen‐erated from the ntp.conf option definitions.
4.2.8p18 25 May 2024 ntp.conf(5man)
although the older:
https://www.ntp.org/documentation/4.2.8-series/confopt/#server-commands
says:
"Server Commands and Options
Last update: March 23, 2023 21:05 UTC (6ad51a76f)
...
Server Commands
...
pool
For type s addresses (only) this command mobilizes a preemptable pool
client
mode association for the DNS name specified. "
...
Server Command Options
...
preempt
Specifies the association as preemptable rather than the default
persistent.
This option is ignored with the broadcast command and is most useful with
the
manycastclient and pool commands."
Despite the timestamps you quoted, the web version is likely newer.
Autogen is run against the documentation source files with every release,
so that timestamp reflects the release date, not the last update of the documentation source files (.html in this case).
Since the overhaul of the www.ntp.org website a few years back, that documentation sadly is maintained in two places, and there's no process to ensure they stay in sync. The web version is considered the more
authoritative source, and is maintained in .md (Markdown) published only
via the converted HTML on the website. It started as a copy of the documentation from the source tarballs' /html directory, but after
conversion to Markdown and subsequent improvements, those changes have generally not been made to the HTML version distributed with the source.
I'm partly to blame because I find writing documentation tedious enough
without having to update it in two places, and I've been kept quite busy
with coding work and haven't wanted to take the time to correct
documentation that no longer reflects the reality of the code. In theory
one day I will have time to dedicate to that, but I welcome anyone who
enjoys documentation work or at least really wants accurate NTP
documentation to please volunteer to help out.
My question is why would a preemptable server, acquired using
"pool ...", continue to be polled after it has stopped
responding, i.e., the reach has gone to 0? It is a
misunderstanding on my part or is there an bug in the code?
Or a doc bug?
A doc bug and an off-by-one bug in the preemption logic.
Cheers,
Dave Hart
<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Nov 21, 2024 at 4:56 PM Brian Inglis <<a href="
mailto:
Brian.Inglis@systematicsw.ab.ca">
Brian.Inglis@systematicsw.ab.ca</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 2024-11-20 14:32, Roger wrote:<br>
> On Wed, 20 Nov 2024 19:53:00 -0000 (UTC), Brian Inglis wrote: </blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
>> Maybe add "iburst preempt" options and drop "poll 11" or perhaps change to<br>
>> "maxpoll 11" or higher, unless you have very good reasons to require a longer<br>
>> interval than the default maximum, instead of adaptive polling based on the error.<br>
> <br>
> Well, the documentation (confopt) tells me that the pool command<br>
> "mobilizes a preemptable pool client mode association for the<br> > DNS name specified." Why would adding "preempt" change anything?<br>
It *may* be required and can never hurt:<br></blockquote><div><br></div><div><div class="gmail_default" style=""><span style="font-family:"trebuchet ms",sans-serif">In fact it won't change anything. The only options propagated from the &
quot;</span><font face="monospace">pool</font><font face="trebuchet ms, sans-serif">" directive in ntp.conf (and thereby set on the prototype pool association listed with refid </font><font face="monospace">POOL</font><font face="trebuchet ms, sans-
serif"> in the peers billboard) to the resulting pool server associations are "</font><font face="monospace">iburst</font><font face="trebuchet ms, sans-serif">" and "</font><font face="monospace">noselect</font><font face="trebuchet ms,
sans-serif">". See POOL_FLAG_PMASK in source code file ntp_proto.c.</font></div><br></div><div><div class="gmail_default" style=""><span style="font-family:"trebuchet ms",sans-serif">The preemptible option is forced on for pool servers,
so they are preemptible with or without that option. However, that option doesn't do much in 4.2.8 as the code intended to preempt useless servers has an off-by-one error that's corrected in my test 3792 release, so preemption only happens in
the unusual case where there are more than 2 times as many pool or manycast client associations as "</span><font face="monospace">tos maxclock</font><font face="trebuchet ms, sans-serif">" which defaults to 10. Arguably this could be fixed in
the stable 4.2.8 branch but it would be a substantial change in behavior without any configuration change that might break existing setups that depend on the off-by-one error.</font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-
serif"><br></font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif">As an aside, using "</font><font face="monospace">preempt</font><font face="trebuchet ms, sans-serif">" on a non-pool non-manycastclient
association (basically, configured via "</font><font face="monospace">server</font><font face="trebuchet ms, sans-serif">" or "</font><font face="monospace">peer</font><font face="trebuchet ms, sans-serif">") seems quixotic to me, as
it allows the association to be removed but nothing is done to replace it. I have a difficult time imagining where that might be useful. It may have been useful in the pre-2009 implementation of "</font><font face="monospace">pool</font><font
face="trebuchet ms, sans-serif">" which I'm having a hard time remembering because I thought it was primitive and needed improvement, as it did all its work at startup and never changed the servers selected once up and running. I re-
implemented it to the current iteration, but didn't catch that the preemption was suffering the aforementioned off-by-one error, or it wasn't back then.</font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif"><br></
font></div><div class="gmail_default" style=""><font face="trebuchet ms, sans-serif">If you're wondering why I mentioned "</font><font face="monospace">manycastclient</font><font face="trebuchet ms, sans-serif">", it shares much of the
implementation with "</font><font face="monospace">pool</font><font face="trebuchet ms, sans-serif">". They use different approaches to finding servers, but the rest of the code is common. Both are intended to be automatic server discovery
schemes that discard, or preempt, servers which haven't been useful for 10 poll intervals so that another server can be solicited to replace it.</font></div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-
left:1px solid rgb(204,204,204);padding-left:1ex">
$ grep 'pool.*preempt' ~/src/time/ntp/ntp-4.2.8p18/ntpd/<a href="
http://complete.conf.in" rel="noreferrer" target="_blank">complete.conf.in</a><br>
pool <a href="
http://2.ubuntu.pool.ntp.org" rel="noreferrer" target="_blank">2.ubuntu.pool.ntp.org</a>. iburst preempt<br></blockquote><div><br></div><div><div class="gmail_default" style=""><font face="monospace"><a href="
http://complete.conf.in">
complete.conf.in</a></font><span style="font-family:"trebuchet ms",sans-serif"> is part of the "make check" tests and is not intended to suggest useful configurations. Rather it's used both to ensure every keyword in the
configuration file parser is covered, and to ensure a configuration can successfully round-trip through ntpd's reading and applying the configuration and exporting the configuration via the </span><font face="monospace">--saveconfigquit</font><font
face="trebuchet ms, sans-serif"> command-line option added specifically for that developer test to catch changes which break that functionality. It's </font>no coincidence<font face="trebuchet ms, sans-serif"> </font>it is<font face="trebuchet ms,
sans-serif"> ordered exactly the same as the output of </font><font face="monospace">ntpq</font><font face="trebuchet ms, sans-serif">'s </font><font face="monospace">saveconfig</font><font face="trebuchet ms, sans-serif"> command, which requires
authentication and that a directory for such saved configuration files has been specified in ntp.conf with "</font><font face="monospace">saveconfigdir</font><font face="trebuchet ms, sans-serif">".</font></div></div><div> </div><blockquote
class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
$ man 5 ntp.conf<br>
...<br>
Configuration Commands<br>
...<br>
*pool* For type s addresses, this command mobilizes a persistent<br>
client mode association with a number of remote servers. In<br>
this mode the local clock can synchronized to the remote server,<br>
but the remote server can never be synchronized to the local<br>
clock.<br>
...<br>
Options:<br>
...<br>
*preempt* Says the association can be preempted.<br>
...<br>
This manual page was AutoGen‐erated from the ntp.conf option definitions.<br>
4.2.8p18 25 May 2024 ntp.conf(5man)<br>
although the older:<br>
<a href="
https://www.ntp.org/documentation/4.2.8-series/confopt/#server-commands" rel="noreferrer" target="_blank">
https://www.ntp.org/documentation/4.2.8-series/confopt/#server-commands</a><br>
says:<br>
"Server Commands and Options<br>
Last update: March 23, 2023 21:05 UTC (6ad51a76f)<br>
...<br>
Server Commands<br>
...<br>
pool<br>
For type s addresses (only) this command mobilizes a preemptable pool client <br>
mode association for the DNS name specified. "<br>
...<br>
Server Command Options<br>
...<br>
preempt<br>
Specifies the association as preemptable rather than the default persistent. <br>
This option is ignored with the broadcast command and is most useful with the <br>
manycastclient and pool commands."<br></blockquote><div><br></div><div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif">Despite the timestamps you quoted, the web version is likely newer. Autogen is run against
the documentation source files with every release, so that timestamp reflects the release date, not the last update of the documentation source files (.html in this case).</div><div class="gmail_default" style="font-family:"trebuchet ms",sans-
serif"><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif">Since the overhaul of the <a href="
http://www.ntp.org">www.ntp.org</a> website a few years back, that documentation sadly is maintained in two places, and
there's no process to ensure they stay in sync. The web version is considered the more authoritative source, and is maintained in .md (Markdown) published only via the converted HTML on the website. It started as a copy of the documentation from
the source tarballs' /html directory, but after conversion to Markdown and subsequent improvements, those changes have generally not been made to the HTML version distributed with the source. I'm partly to blame because I find writing
documentation tedious enough without having to update it in two places, and I've been kept quite busy with coding work and haven't wanted to take the time to correct documentation that no longer reflects the reality of the code. In theory one
day I will have time to dedicate to that, but I welcome anyone who enjoys documentation work or at least really wants accurate NTP documentation to please volunteer to help out.</div></div><div> </div><blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> My question is why would a preemptable server, acquired using<br>
> "pool ...", continue to be polled after it has stopped<br>
> responding, i.e., the reach has gone to 0? It is a<br>
> misunderstanding on my part or is there an bug in the code?<br>
Or a doc bug?<br></blockquote><div><br></div><div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif">A doc bug and an off-by-one bug in the preemption logic.</div></div><div class="gmail_default" style="font-family:"
trebuchet ms",sans-serif"><br></div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif">Cheers,</div><div class="gmail_default" style="font-family:"trebuchet ms",sans-serif">Dave Hart</div></div></div>
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)