POSIX ruling on up-to-date vs. identical timestamps

[Eric Blake]

Make folks:
You may want to check out http://austingroupbugs.net/view.php?id=857 and
add comments and/or change GNU make behavior accordingly. There, the
argument is made that HP-UX make behavior is nicer than GNU's current
behavior when two files have identical timestamps: HP-UX considers the
file as out-of-date, while GNU make considers it up-to-date. A strict
reading of POSIX can argue that GNU's behavior was required, but this
reading has been called into question.

GNU's behavior is an optimization that avoids needless churn on file
systems with course timestamps (well, FAT still exists, but these days,
MOST file systems have sub-second resolution, so it is harder to get
files with identical timestamps without actually touch'ing them that
way) - but it risks leaving a tree in an incomplete state. HP-UX
behavior guarantees the rules are run, even if they were not strictly
necessary, but has the nice property that the tree is never left in an
incomplete state due to unfortunate timing on a file system with course
timestamps.

The POSIX recommendation was therefore that GNU should change its
behavior to act like HP-UX, and consider identical timestamps as
out-of-date, because the standard will be fixed to allow HP-UX behavior.

Autoconf folks:
The section of the autoconf manual that discusses this should probably
be modernized, particularly if changes to POSIX and/or GNU make result
from this discussion.
https://www.gnu.org/software/autoconf/manual/autoconf.html#Timestamps-and-Make

[Ray Donnelly]

Post by Eric Blake
You may want to check out http://austingroupbugs.net/view.php?id=857 and
add comments and/or change GNU make behavior accordingly. There, the
argument is made that HP-UX make behavior is nicer than GNU's current
behavior when two files have identical timestamps: HP-UX considers the
file as out-of-date, while GNU make considers it up-to-date. A strict
reading of POSIX can argue that GNU's behavior was required, but this
reading has been called into question.
GNU's behavior is an optimization that avoids needless churn on file
systems with course timestamps (well, FAT still exists, but these days,
MOST file systems have sub-second resolution, so it is harder to get
files with identical timestamps without actually touch'ing them that
way) - but it risks leaving a tree in an incomplete state. HP-UX
behavior guarantees the rules are run, even if they were not strictly
necessary, but has the nice property that the tree is never left in an
incomplete state due to unfortunate timing on a file system with course
timestamps.
The POSIX recommendation was therefore that GNU should change its
behavior to act like HP-UX, and consider identical timestamps as
out-of-date, because the standard will be fixed to allow HP-UX behavior.
The section of the autoconf manual that discusses this should probably
be modernized, particularly if changes to POSIX and/or GNU make result
from this discussion.
https://www.gnu.org/software/autoconf/manual/autoconf.html#Timestamps-and-Make

Some projects that want to have deterministic builds by using the
gitian builder [1] use libfaketime to set the timestamps to a known
constant, and gnu make trips up on this, occasionally causing broken
builds and othertimes autoconf declares suck a build system "insane".
There was a bug in libfaketime so that the nanosecond field wasn't
cleared and this allowed you to avoid both these bugs if the planets
were aligned correctly (which has since been fixed). I agree that
these issues should be fixed in make and autoconf.

[1] https://gitian.org/

Post by Eric Blake
--
Eric Blake eblake redhat com [phone]
Libvirt virtualization library http://libvirt.org
_______________________________________________
Bug-make mailing list
https://lists.gnu.org/mailman/listinfo/bug-make

[Ray Donnelly]

Post by Ray Donnelly

Post by Eric Blake
You may want to check out http://austingroupbugs.net/view.php?id=857 and
add comments and/or change GNU make behavior accordingly. There, the
argument is made that HP-UX make behavior is nicer than GNU's current
behavior when two files have identical timestamps: HP-UX considers the
file as out-of-date, while GNU make considers it up-to-date. A strict
reading of POSIX can argue that GNU's behavior was required, but this
reading has been called into question.
GNU's behavior is an optimization that avoids needless churn on file
systems with course timestamps (well, FAT still exists, but these days,
MOST file systems have sub-second resolution, so it is harder to get
files with identical timestamps without actually touch'ing them that
way) - but it risks leaving a tree in an incomplete state. HP-UX
behavior guarantees the rules are run, even if they were not strictly
necessary, but has the nice property that the tree is never left in an
incomplete state due to unfortunate timing on a file system with course
timestamps.
The POSIX recommendation was therefore that GNU should change its
behavior to act like HP-UX, and consider identical timestamps as
out-of-date, because the standard will be fixed to allow HP-UX behavior.
The section of the autoconf manual that discusses this should probably
be modernized, particularly if changes to POSIX and/or GNU make result
from this discussion.
https://www.gnu.org/software/autoconf/manual/autoconf.html#Timestamps-and-Make

Some projects that want to have deterministic builds by using the
gitian builder [1] use libfaketime to set the timestamps to a known
constant, and gnu make trips up on this, occasionally causing broken
builds and othertimes autoconf declares suck a build system "insane".

Freudian slip? I meant "such", not "suck".

Post by Ray Donnelly
There was a bug in libfaketime so that the nanosecond field wasn't
cleared and this allowed you to avoid both these bugs if the planets
were aligned correctly (which has since been fixed). I agree that
these issues should be fixed in make and autoconf.
[1] https://gitian.org/

Post by Eric Blake
--
Eric Blake eblake redhat com [phone]
Libvirt virtualization library http://libvirt.org
_______________________________________________
Bug-make mailing list
https://lists.gnu.org/mailman/listinfo/bug-make

[Paul Eggert]

Post by Ray Donnelly
There was a bug in libfaketime so that the nanosecond field wasn't
cleared

That sounds like it's a different issue. If a program botches the
nanosecond component of timestamps, it shouldn't matter whether 'make'
uses the traditional/GNU or the HP-UX approach; either way, you can get
into trouble.

If I'm misunderstanding the libfaketime issue, could you please explain
it with an example?

[Paul Eggert]

Post by Eric Blake
You may want to check out http://austingroupbugs.net/view.php?id=857 and
add comments and/or change GNU make behavior accordingly.

Let's leave GNU 'make' alone. Its behavior is better for rules like this:

copy: original
cp -p original copy

I've added a comment to the Austin Group page.

[David Boyce]

The obvious compromise would be to change the behavior only in the
presence of the ".POSIX:" special target.

Post by Paul Eggert

Post by Eric Blake
You may want to check out http://austingroupbugs.net/view.php?id=857 and
add comments and/or change GNU make behavior accordingly.

copy: original
cp -p original copy
I've added a comment to the Austin Group page.
_______________________________________________
Bug-make mailing list
https://lists.gnu.org/mailman/listinfo/bug-make

[Ray Donnelly]

Post by David Boyce
The obvious compromise would be to change the behavior only in the
presence of the ".POSIX:" special target.

Sounds pragmatic; the repeatable builds people would probably like a
solution that doesn't require changing Makefiles though, either an
env. var or a command line option?

Post by David Boyce

Post by Paul Eggert

Post by Eric Blake
You may want to check out http://austingroupbugs.net/view.php?id=857 and
add comments and/or change GNU make behavior accordingly.

copy: original
cp -p original copy
I've added a comment to the Austin Group page.
_______________________________________________
Bug-make mailing list
https://lists.gnu.org/mailman/listinfo/bug-make

_______________________________________________
Bug-make mailing list
https://lists.gnu.org/mailman/listinfo/bug-make

[David Boyce]

Post by Ray Donnelly

Post by David Boyce
The obvious compromise would be to change the behavior only in the
presence of the ".POSIX:" special target.

Sounds pragmatic; the repeatable builds people would probably like a
solution that doesn't require changing Makefiles though, either an
env. var or a command line option?

The command line is always possible for a special target given Paul's
clever --eval option:

make --eval .POSIX: ...

And for an EV approach it should be possible to add that to an
exported MAKEFLAGS.

[Paul Eggert]

Post by David Boyce
The obvious compromise would be to change the behavior only in the
presence of the ".POSIX:" special target.

We should limit ".POSIX" to what POSIX requires. Even if the ruling
stands POSIX won't require the HP-UX behavior, so ".POSIX" should be
independent of this issue.

It'd be OK to introduce a new special target to enable the HP-UX
behavior. .EQUAL_TIMES_ARE_OUT-OF-DATE, say. We could document the new
target next to .LOW_RESOLUTION_TIME, since they're related issues. The
new target could act like .LOW_RESOLUTION_TIME, except that it imposes
HP-UX rather than low-res behavior.

[Paul Smith]

Post by Paul Eggert

Post by David Boyce
The obvious compromise would be to change the behavior only in the
presence of the ".POSIX:" special target.

We should limit ".POSIX" to what POSIX requires. Even if the ruling
stands POSIX won't require the HP-UX behavior, so ".POSIX" should be
independent of this issue.

I pretty much agree with everything Paul says in this thread.

First, I can't remember getting bug reports on GNU make that the current
way it handles identical timestamps is a problem. I'm not saying that
such a thing has never happened (my memory is not what it was for one
thing) but certainly there have not been enough complaints to make this
a known issue. And since this is just the kind of seemingly small
change that will end up annoying and frustrating many people, I'm not
excited about it.

Similarly, unless POSIX mandates this change in behavior I'm not that
excited about having the .POSIX target imply this change either: that
seems to be mixing up too many obscure differences in a single flag.

Post by Paul Eggert
It'd be OK to introduce a new special target to enable the HP-UX
behavior. .EQUAL_TIMES_ARE_OUT-OF-DATE, say. We could document the new
target next to .LOW_RESOLUTION_TIME, since they're related issues. The
new target could act like .LOW_RESOLUTION_TIME, except that it imposes
HP-UX rather than low-res behavior.

I think something like this may be the way to go, but it might need to
be more comprehensive than this even. Consider the Savannah bug
https://savannah.gnu.org/bugs/index.php?40056 which complains about
builds where some of the files live on filesystems that do support hires
timestamps and some files do not. Despite the interesting review of the
10th grade concept of significant digits (*rolleyes*) I don't
particularly care for the solution provided there: assuming that a
subsecond value of 0 always means there is no subsecond resolution seems
to me to be problematic.

However, it needs to be considered carefully.

[David Boyce]

Post by Paul Smith
I pretty much agree with everything Paul says in this thread.

And I tend to agree with David right down the line :-) But I guess two
Pauls beat a David.

[Paul Eggert]

It needs to be considered carefully.

How about having GNU 'make' do what GNU 'cp -u' does?

The idea is to infer filesystem timestamp resolution by looking at every
file timestamp that crosses your desk. When you see a file timestamp
whose tv_nsec is nonzero modulo 1000000, for example, you know that its
filesystem's resolution is finer-grained than 1 millisecond. When
computation starts, you are conservative and assume that filesystems
have 1-second resolution, but as computation goes on you gain more
information about each filesystem and can become less and less conservative.

It's a hack, but the heuristic doesn't require any system calls (and for
what it's worth we don't get bug reports about it...).

This idea is implemented by Gnulib's 'utimecmp' module, which GNU Make
is of course welcome to steal.

[Paul Smith]

Post by Paul Eggert

It needs to be considered carefully.

How about having GNU 'make' do what GNU 'cp -u' does?
The idea is to infer filesystem timestamp resolution by looking at every
file timestamp that crosses your desk. When you see a file timestamp
whose tv_nsec is nonzero modulo 1000000, for example, you know that its
filesystem's resolution is finer-grained than 1 millisecond. When
computation starts, you are conservative and assume that filesystems
have 1-second resolution, but as computation goes on you gain more
information about each filesystem and can become less and less conservative.

It concerns me that this could cause the build to be non-deterministic
in certain pathological situations, but it could be managed I'm sure.

The immediate issue is that today, make doesn't track "filesystems". It
has no idea which files live on which filesystems, so we can't really
keep track of the resolution on a per-filesystem basis.

Of course the ability to track filesystems could be added without too
much effort. It's trivial to determine the filesystem in POSIX via the
device ID available from stat(), of course, but I'm not sure what
facilities are available on the other ports. Although, I don't think
either Windows or VMS support high-resolution timestamps right now (I
don't know why Windows doesn't, because NTFS does support millisecond
resolution timestamps I believe) so perhaps this is irrelevant.

[Eli Zaretskii]

Date: Tue, 26 Aug 2014 10:52:32 -0400
Of course the ability to track filesystems could be added without too
much effort. It's trivial to determine the filesystem in POSIX via the
device ID available from stat(), of course, but I'm not sure what
facilities are available on the other ports.

It's trivial on Windows as well.

Although, I don't think either Windows or VMS support
high-resolution timestamps right now

Correct.

(I don't know why Windows doesn't, because NTFS does support
millisecond resolution timestamps I believe)

Because no one wrote the code, of course.

The main problem is that this requires to write a replacement 'stat'
(not rocket science).

[Paul Smith]

Post by Eli Zaretskii

(I don't know why Windows doesn't, because NTFS does support
millisecond resolution timestamps I believe)

Because no one wrote the code, of course.

Ah, the oldest reason in free software :-).

Post by Eli Zaretskii
The main problem is that this requires to write a replacement 'stat'
(not rocket science).

Can't we just #define stat(_p,_b) _stat(_p,_b)? Not sure if that's
sufficient: I'm not overly familiar with the limitations on the POSIX
emulation functions in Windows.

[Keith Marshall]

Post by Paul Smith
Can't we just #define stat(_p,_b) _stat(_p,_b)? Not sure if that's
sufficient: I'm not overly familiar with the limitations on the POSIX
emulation functions in Windows.

That's effectively what MinGW does anyway, (although it does it through
an import library mapping from the undecorated POSIX name, to the
decorated name exported by MSVCRT.DLL). The upshot is that, when you
call stat() in a Windows program compiled by MinGW's GCC, you actually
call the function provided by Microsoft themselves, with whatever
limitations that imposes.

[Eli Zaretskii]

Date: Tue, 26 Aug 2014 11:18:35 -0400

Post by Eli Zaretskii
The main problem is that this requires to write a replacement 'stat'
(not rocket science).

Can't we just #define stat(_p,_b) _stat(_p,_b)? Not sure if that's
sufficient: I'm not overly familiar with the limitations on the POSIX
emulation functions in Windows.

_stat *is* the problem: the time-related members of the structure it
returns report time only at 1-sec resolution.

We need a replacement that supports sub-second resolution in
additional members, like on some Posix platforms. Not a big deal, but
Someone(TM) needs to sit down and write the code.

[Paul Eggert]

Post by Paul Smith
It's trivial to determine the filesystem in POSIX via the
device ID available from stat()

Yes, that's what the Gnulib utimecmp module does: the idea is that after
every stat-like operation you look at the file's time stamps to infer
more information about the containing file system's resolution.

As far as Windows goes, NTFS file systems have 100 ns resolution, and
FAT file systems are the joker as they have a 2-second resolution for
last-modified time.

While we're on the topic, perhaps I should warn you that in old buggy
Linux kernels file time stamps could spontaneously be truncated to file
system resolution. According to
<https://gcc.gnu.org/ml/gcc/2004-04/msg00152.html> this bug was
introduced into the Linux kernel in 2002; I don't recall when it was
fixed in Linux, but a few years later POSIX prohibited the buggy behavior.

[Eli Zaretskii]

Date: Tue, 26 Aug 2014 08:25:38 -0700
As far as Windows goes, NTFS file systems have 100 ns resolution, and
FAT file systems are the joker as they have a 2-second resolution for
last-modified time.

That's true, but FAT filesystems are hardly important these days.

[Keith Marshall]

Post by Eli Zaretskii

Date: Tue, 26 Aug 2014 08:25:38 -0700
As far as Windows goes, NTFS file systems have 100 ns resolution, and
FAT file systems are the joker as they have a 2-second resolution for
last-modified time.

That's true, but FAT filesystems are hardly important these days.

Except insofar as they tend to be prevalent on removable media devices,
such as USB flash drives; woe betide anyone who happens to store their
working files on one of those.

[Eli Zaretskii]

Date: Tue, 26 Aug 2014 18:30:12 +0100

Post by Eli Zaretskii
FAT filesystems are hardly important these days.

Except insofar as they tend to be prevalent on removable media devices,
such as USB flash drives; woe betide anyone who happens to store their
working files on one of those.

Indeed, let alone builds software on such volumes.

[Bob Friesenhahn]

Post by Eric Blake
The POSIX recommendation was therefore that GNU should change its
behavior to act like HP-UX, and consider identical timestamps as
out-of-date, because the standard will be fixed to allow HP-UX behavior.

A change like this may result in some builds which never complete (or
take much longer) because the timestamp rules require that something
be rebuilt (tail chasing). This is particularly a problem for
recursive builds which may enter the same directories multiple times.

Bob