Autoconf does not like "-D_FORTIFY_SOURCE=2 -O2"

[Anatol Pomozov]

Hi,

Linux Arch distributive recently added following compilation flags to
CPPFLAGS: "-D_FORTIFY_SOURCE=2 -O2". Unfortunately it breaks autoconf based
projects such as gdb, gcc, ...


The issue is that autoconf compiles some programs to find whether system
has headers. And to compile it uses only preprocessor defines from CPPFLAGS
(-D_FORTIFY_SOURCE=2 in case of Arch). recent versions of glibc produces a
warning when it compiles apps with _FORTIFY_SOURCE but without -O2.

Here is a simple application

#include <stdlib.h>
int main(void) { return 0; }

that compiles fine in Ubuntu LTS (glibc 2.15) but produces a warning on
Arch (glibc 2.17)

$ gcc -D_FORTIFY_SOURCE=2 a.c
In file included from /usr/include/stdlib.h:24:0,
from a.c:1:
/usr/include/features.h:330:4: warning: #warning _FORTIFY_SOURCE requires
compiling with optimization (-O) [-Wcpp]
# warning _FORTIFY_SOURCE requires compiling with optimization (-O)

The warning itself was introduced in this glibc commit:
http://sourceware.org/git/?p=glibc.git;a=commit;f=include/features.h;h=05c2c9618f583ea4acd69b3fe5ae2a2922dd2ddc


And it seems autoconf does not like the warning, it treats it as "there is
a problem with header, so header does not exist on this system". And it is
wrong - header exists and it is fine, it just does not like the flags set.

It sounds like an issue in autoconf, it should handle the warning
correctly, or leave -O2 if _FORTIFY_SOURCE is present, or remove
_FORTIFY_SOURCE. In any case autoconf should find the stdlib header even
if _FORTIFY_SOURCE is present in CPPFLAGS.



Here is additional info from Arch developers:
http://permalink.gmane.org/gmane.linux.arch.general/48295
https://mailman.archlinux.org/pipermail/arch-dev-public/2013-April/024776.html

compilation issue in RedHad's crash tool:
https://www.redhat.com/archives/crash-utility/2013-April/msg00015.html

[Paul Eggert]

Post by Anatol Pomozov
Linux Arch distributive recently added following compilation flags to
CPPFLAGS: "-D_FORTIFY_SOURCE=2 -O2". Unfortunately it breaks autoconf based
projects such as gdb, gcc, ...
The issue is that autoconf compiles some programs to find whether system
has headers. And to compile it uses only preprocessor defines from CPPFLAGS
(-D_FORTIFY_SOURCE=2 in case of Arch). recent versions of glibc produces a
warning when it compiles apps with _FORTIFY_SOURCE but without -O2.

Sorry, I don't understand. If CPPFLAGS="-D_FORTIFY_SOURCE=2 -O2",
and if 'configure' uses preprocessor defines from CPPFLAGS,
then 'configure' should be using -D_FORTIFY_SOURCE=2 -O2.
But you seem to be implying that 'configure' is using -D_FORTIFY_SOURCE=2
without -O2. How can this be?

Post by Anatol Pomozov
recent versions of glibc produces a
warning when it compiles apps with _FORTIFY_SOURCE but without -O2

That's a real problem, which will break lots of things.
Fix that, and your Autoconf issue will go away.

[Allan McRae]

Post by Paul Eggert

Post by Anatol Pomozov
Linux Arch distributive recently added following compilation flags to
CPPFLAGS: "-D_FORTIFY_SOURCE=2 -O2". Unfortunately it breaks autoconf based
projects such as gdb, gcc, ...
The issue is that autoconf compiles some programs to find whether system
has headers. And to compile it uses only preprocessor defines from CPPFLAGS
(-D_FORTIFY_SOURCE=2 in case of Arch). recent versions of glibc produces a
warning when it compiles apps with _FORTIFY_SOURCE but without -O2.

Sorry, I don't understand. If CPPFLAGS="-D_FORTIFY_SOURCE=2 -O2",
and if 'configure' uses preprocessor defines from CPPFLAGS,
then 'configure' should be using -D_FORTIFY_SOURCE=2 -O2.
But you seem to be implying that 'configure' is using -D_FORTIFY_SOURCE=2
without -O2. How can this be?

Our distribution packages are compiled with:

CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector
--param=ssp-buffer-size=4"

So when both CPPFLAGS and CFLAGS are passed there is no issue.

I have not looked into this in great detail yet, but it seems some
configure scripts have this in them:

ac_cpp='$CPP $CPPFLAGS'

and this is used to test for header presence. Note I said some
configure scripts - I have not figured out why others do not use
CPPFLAGS in this test.

Post by Paul Eggert

Post by Anatol Pomozov
recent versions of glibc produces a
warning when it compiles apps with _FORTIFY_SOURCE but without -O2

That's a real problem, which will break lots of things.
Fix that, and your Autoconf issue will go away.

I don't agree that the glibc warning is an issue. Unless you use
-Werror (bad) or check for any warning rather than a specific one.

Allan

[Zack Weinberg]

Post by Allan McRae
CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector
--param=ssp-buffer-size=4"
So when both CPPFLAGS and CFLAGS are passed there is no issue.

I think the quick fix from your end is to move -D_FORTIFY_SOURCE to
CFLAGS (and presumably also CXXFLAGS). Despite being a -D switch its
*effect* is much more like an -m or -f switch, so I don't think this
is even technically wrong, and it'll have the results you need.

Post by Allan McRae
I have not looked into this in great detail yet, but it seems some
ac_cpp='$CPP $CPPFLAGS'
and this is used to test for header presence. Note I said some
configure scripts - I have not figured out why others do not use
CPPFLAGS in this test.

The low-hanging fruit here is AC_CHECK_HEADER, which currently by
default does a "double check" for the presence of a header, one of
them using only CPPFLAGS. Some extra-diligent projects have switched
over to doing only a compile check. I have patches mostly written
that will make the default be compile-only. However, anyone that
deliberately uses AC_PREPROC_IFELSE or similar will still have the
problem.

Post by Allan McRae

Post by Paul Eggert

Post by Anatol Pomozov
recent versions of glibc produces a
warning when it compiles apps with _FORTIFY_SOURCE but without -O2

That's a real problem, which will break lots of things.
Fix that, and your Autoconf issue will go away.

I don't agree that the glibc warning is an issue. Unless you use
-Werror (bad) or check for any warning rather than a specific one.

Autoconf can't check for specific warnings because it has no way of
knowing how the compiler in use will react to any given (potential)
problem.

I note that Debian has patched this warning out of their (just now
appeared in unstable) glibc 2.17. See
http://anonscm.debian.org/viewvc/pkg-glibc/glibc-package/trunk/debian/patches/any/local-revert-bz13979.diff?revision=5553&view=markup

zw

[Paul Eggert]

Post by Zack Weinberg
I think the quick fix from your end is to move -D_FORTIFY_SOURCE to
CFLAGS (and presumably also CXXFLAGS).

Another possibility is to append -O2 to CPPFLAGS. The point is that
-O2 should always be used if -D_FORTIFY_SOURCE is.

Post by Zack Weinberg
I note that Debian has patched this warning out of their (just now
appeared in unstable) glibc 2.17.

Sounds like a win to me. Maybe I should file a glibc bug report
upstream....

[Zack Weinberg]

Post by Paul Eggert

Post by Zack Weinberg
I note that Debian has patched this warning out of their (just now
appeared in unstable) glibc 2.17.

Sounds like a win to me. Maybe I should file a glibc bug report
upstream....

If you do, you should indicate it as a regression from their bug 13979.
http://sourceware.org/bugzilla/show_bug.cgi?id=13979

[Ralf Corsepius]

Post by Zack Weinberg

Post by Allan McRae
CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector
--param=ssp-buffer-size=4"
So when both CPPFLAGS and CFLAGS are passed there is no issue.

I think the quick fix from your end is to move -D_FORTIFY_SOURCE to
CFLAGS (and presumably also CXXFLAGS).

FWIW: This is what Fedora and RH are doing for years.

Ralf

[Anatol Pomozov]

Hi

Post by Ralf Corsepius

Post by Zack Weinberg

Post by Allan McRae
CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector
--param=ssp-buffer-size=4"
So when both CPPFLAGS and CFLAGS are passed there is no issue.

I think the quick fix from your end is to move -D_FORTIFY_SOURCE to
CFLAGS (and presumably also CXXFLAGS).

FWIW: This is what Fedora and RH are doing for years.

And I believe it is how Arch worked before. Previously flags were

CFLAGS="-march=i686 -mtune=generic -O2 -pipe -fstack-protector
--param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2"
CXXFLAGS="-march=i686 -mtune=generic -O2 -pipe -fstack-protector
--param=ssp-buffer-size=4 -D_FORTIFY_SOURCE=2"

and now they are

CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=i686 -mtune=generic -O2 -pipe -fstack-protector
--param=ssp-buffer-size=4"
CXXFLAGS="-march=i686 -mtune=generic -O2 -pipe -fstack-protector
--param=ssp-buffer-size=4"

Allan can provide more information about why -D_FORTIFY_SOURCE=2 was moved
to preprocessor flags.


I also have another (probably naive) question. Why autoconf uses CPPFLAGS
(and not CPPFLAGS+CXXFLAGS) for headers discovery? Is it because it passes
the program through preprocessor only and does not care whether programs
compiles? If my statement true then seems the solution for Arch is either
a) move _FORTIFY_SOURCE back to CFLAGS, or b) add -O2 to CPPFLAGS, yes -O2
is not a preprocessor flag, but other flag (_FORTIFY_SOURCE) requires it.

Or c) autoconf should use CFLAGS for header preprocessing (almost sure that
autoconf people would not like it).

[Paul Eggert]

Post by Anatol Pomozov
Why autoconf uses CPPFLAGS
(and not CPPFLAGS+CXXFLAGS) for headers discovery?

It's a long story, but basically autoconf used to invoke
just the preprocessor to test for header existence, partly
on the grounds of making 'configure' go faster. That turns
out to have problems, so it now tries to compile as well,
optionally, using a heuristic that I don't recall right now.
I expect that the goal is that it'll use the compiler exclusivly,
but doing that will require some developer effort, and nobody
has had the time and inclination to carry that through.

[Allan McRae]

Post by Paul Eggert

Post by Anatol Pomozov
Why autoconf uses CPPFLAGS
(and not CPPFLAGS+CXXFLAGS) for headers discovery?

It's a long story, but basically autoconf used to invoke
just the preprocessor to test for header existence, partly
on the grounds of making 'configure' go faster. That turns
out to have problems, so it now tries to compile as well,
optionally, using a heuristic that I don't recall right now.
I expect that the goal is that it'll use the compiler exclusivly,
but doing that will require some developer effort, and nobody
has had the time and inclination to carry that through.

So there are multiple options of how to fix this. This simple "fixes"
are patching glibc and gcc, but these run against my distributions
patching policy. Also, adding -O2 to CPPFLAGS or moving
-D_FORTIFY_SOURCE to CFALGS just does not seem correct either - CFLAGS
and CPPFLAGS are separate for a reason.

I believe autoconf uses "CPP CPPFLAGS" to detect headers mainly because
of "-I" flags needing to be considered. Would an acceptable solution at
the autoconf level be to split the CPPFLAGS into -I flags and others
(-D, -U) and just use the -I ones in the header test?

If that is fine in principle, I can attempt to implement it.

Allan

[Eric Blake]

Post by Allan McRae
I believe autoconf uses "CPP CPPFLAGS" to detect headers mainly because
of "-I" flags needing to be considered. Would an acceptable solution at
the autoconf level be to split the CPPFLAGS into -I flags and others
(-D, -U) and just use the -I ones in the header test?
If that is fine in principle, I can attempt to implement it.

But that won't help you instantly fix the hundreds of existing packages
whose configure script was generated with an earlier version of
autoconf. While I am open to making future autoconf have saner behavior
by default, you MUST consider the fact that your distro's choice of
CFLAGS vs. CPPFLAGS is the real problem here ("if it hurts, don't do
it"), rather than waiting for a patched autoconf to trickle down into
fixed configure scripts on boatloads of packages.

[Allan McRae]

Post by Eric Blake

Post by Allan McRae
I believe autoconf uses "CPP CPPFLAGS" to detect headers mainly because
of "-I" flags needing to be considered. Would an acceptable solution at
the autoconf level be to split the CPPFLAGS into -I flags and others
(-D, -U) and just use the -I ones in the header test?
If that is fine in principle, I can attempt to implement it.

But that won't help you instantly fix the hundreds of existing packages
whose configure script was generated with an earlier version of
autoconf. While I am open to making future autoconf have saner behavior
by default, you MUST consider the fact that your distro's choice of
CFLAGS vs. CPPFLAGS is the real problem here ("if it hurts, don't do
it"), rather than waiting for a patched autoconf to trickle down into
fixed configure scripts on boatloads of packages.

I'm not worried about my distros choice of CPPFLAGS/CFLAGS. The flags
are being defined where they should be defined and working around it in
the relatively few software packages that this causes issues is a single
sed in the build script. Hiding the issue only delays it actually being
fixed at the source.


Saying that, it is really only an issue in a few places... I have
looked into why and it seems to be due to a choice of which
ac_fn_c_check_header_* functions is used. Looking in binutils (where
all configure files are generated with autoconf 2.64), the configure
files that fail use "ac_fn_c_check_header_preproc" while the ones that
succeed use "ac_fn_c_check_header_mongrel" or
"ac_fn_c_check_header_compile".

How is it determined which of those is used? From what I can tell
looking at the latest binutils source the working and failing
(libiberty) configure.ac files all use AC_CHECK_HEADERS.

Allan

[Paul Eggert]

Post by Allan McRae
Would an acceptable solution at
the autoconf level be to split the CPPFLAGS into -I flags and others
(-D, -U) and just use the -I ones in the header test?

I don't think so, no. -D and -U can affect whether cpp works.

[Zack Weinberg]

Post by Allan McRae
So there are multiple options of how to fix this. This simple "fixes"
are patching glibc and gcc, but these run against my distributions
patching policy. Also, adding -O2 to CPPFLAGS or moving
-D_FORTIFY_SOURCE to CFALGS just does not seem correct either - CFLAGS
and CPPFLAGS are separate for a reason.

In this case, I really do think the correct fix is to move
-D_FORTIFY_SOURCE to CFLAGS/CXXFLAGS. It is *semantically* a compiler
option (turn on fortification); that it happens to have the form of a
command-line #define which changes the contents of the system headers
is an implementation detail.

It also has the advantage of being a fix which you can implement
entirely in your build framework, and which does not require you to
regenerate hundreds if not thousands of configure scripts with a
patched autoconf.

(That said, I've never been clear myself on why CFLAGS and CPPFLAGS
*are* separate, except possibly the now-long-obsolete historical
reason that some traditional preprocessors didn't accept arbitrary
compiler options.)

zw

[Paul Eggert]

Post by Zack Weinberg
(That said, I've never been clear myself on why CFLAGS and CPPFLAGS
*are* separate, except possibly the now-long-obsolete historical
reason that some traditional preprocessors didn't accept arbitrary
compiler options.)

I think that's basically it, yes. It used to be that some compilers
didn't have an -E option, so that one had to invoke the preprocessors
directly, and they rejected options like -O2. Nowadays I expect
this problem is no longer relevant.

[Mike Frysinger]

Post by Zack Weinberg
(That said, I've never been clear myself on why CFLAGS and CPPFLAGS
*are* separate, except possibly the now-long-obsolete historical
reason that some traditional preprocessors didn't accept arbitrary
compiler options.)

because there are C-specific flags (CFLAGS) and C++-specific flags (CXXFLAGS) and
Fortran-specific flags (FFLAGS) and .... but they all use the same preprocessor
(CPPFLAGS)
-mike

[Mike Frysinger]

Post by Paul Eggert

Post by Anatol Pomozov
recent versions of glibc produces a
warning when it compiles apps with _FORTIFY_SOURCE but without -O2

That's a real problem, which will break lots of things.

i complained when the change was made, but other devs feel it's the right
thing to do

Gentoo has patched it out too, although you can easily tweak the behavior in
gcc to work with either behavior (which is what at least Gentoo & Ubuntu do
rather than manually putting _FORTIFY_SOURCE into CPPFLAGS).
-mike