Discussion:
Recursive AC_CONFIG_COMMANDS_PRE?
Nick Bowler
2013-01-24 04:16:38 UTC
Permalink
Hello Autoconfers,

I recently ran into an issue where the actions I had configured with
AC_CONFIG_COMMANDS_PRE from a macro were not actually being output to
the configure script. When I investigated, I noticed that the macro in
question was itself invoked inside an AC_CONFIG_COMMANDS_PRE command;
but as there was quite a bit of indirection involved this fact was not
immediately obvious.

Essentially the input looked like this:

% cat >configure.ac <<'EOF'
AC_INIT([test], [0])

dnl FOO uses AC_CONFIG_COMMANDS_PRE as part of its (internal)
dnl implementation.
AC_DEFUN_ONCE([FOO],
[AC_CONFIG_COMMANDS_PRE([m4_fatal([executed!])])])

dnl BAR is designed to be callable from an AC_CONFIG_COMMANDS_PRE block,
dnl but it has a dependency on FOO.
AC_DEFUN_ONCE([BAR], [AC_REQUIRE([FOO])])

dnl Calling BAR as designed ends up breaking FOO's usage of
dnl AC_CONFIG_COMMANDS_PRE.
AC_CONFIG_COMMANDS_PRE([BAR])

AC_OUTPUT
EOF

Currently, Autoconf accepts the above configure script as it does not
execute the m4_fatal call. This happens mainly due to the simplicity
of AC_CONFIG_COMMANDS_PRE: each invocation appends the argument to one
big macro which is then expanded once near the end. Any calls to
AC_CONFIG_COMMANDS_PRE that occur during this expansion still append
to that macro, but since it has already been expanded at this point
those don't go anywhere useful.

There doesn't seem to be an obvious way for FOO to test whether it has
been invoked indirectly from an AC_CONFIG_COMMANDS_PRE block and adapt
its behaviour accordingly, and I'd kind of like the scenario to work.

One possibility that comes to mind is that AC_CONFIG_COMMANDS_PRE
could be redefined to expand to its arguments verbatim just before
expanding AC_OUTPUT_COMMANDS_PRE. While this would work in my case,
I don't really like this option. I think it's reasonable to expect
that AC_CONFIG_COMMANDS_PRE commands are actually executed later
(specifically, that they are expanded sometime *after* the body of a
macro calling AC_CONFIG_COMMANDS_PRE).

A more complicated option would be a sort of "recursive" expansion of
AC_OUTPUT_COMMANDS_PRE, such that invocations of AC_CONFIG_COMMANDS_PRE
during the expansion are collected as usual, then *those* commands are
expanded after the current expansion of AC_OUTPUT_COMMANDS_PRE, and so
on until there are no more commands. I modified lib/autoconf/status.m4
to do just that, by defining the following helper macro:

# AC_OUTPUT_COMMANDS_REC(MACRO)
# -----------------------------
# Until the definition of MACRO is empty, repeatedly expand MACRO
# in a context where it has been redefined to the empty string.
m4_define([AC_OUTPUT_COMMANDS_REC], [m4_ifnblank(m4_defn([$1]),
[m4_define([$1], [m4_define([$1])]m4_defn([$1]))
m4_indir([$1])dnl
AC_OUTPUT_COMMANDS_REC([$1])])])

and replacing the call to AC_OUTPUT_COMMANDS_PRE() with
AC_OUTPUT_COMMANDS_REC([AC_OUTPUT_COMMANDS_PRE]). The same change could
be done for AC_CONFIG_COMMANDS_POST. This seems to work just fine,
although we could imagine some (crazy!) configure scripts for which this
change makes them non-terminating...

Any thoughts?

Thanks,
Nick
Eric Blake
2013-01-24 13:24:55 UTC
Permalink
Post by Nick Bowler
Hello Autoconfers,
I recently ran into an issue where the actions I had configured with
AC_CONFIG_COMMANDS_PRE from a macro were not actually being output to
the configure script. When I investigated, I noticed that the macro in
question was itself invoked inside an AC_CONFIG_COMMANDS_PRE command;
but as there was quite a bit of indirection involved this fact was not
immediately obvious.
Currently, Autoconf accepts the above configure script as it does not
execute the m4_fatal call. This happens mainly due to the simplicity
of AC_CONFIG_COMMANDS_PRE: each invocation appends the argument to one
big macro which is then expanded once near the end. Any calls to
AC_CONFIG_COMMANDS_PRE that occur during this expansion still append
to that macro, but since it has already been expanded at this point
those don't go anywhere useful.
Nice analysis.
Post by Nick Bowler
A more complicated option would be a sort of "recursive" expansion of
AC_OUTPUT_COMMANDS_PRE, such that invocations of AC_CONFIG_COMMANDS_PRE
during the expansion are collected as usual, then *those* commands are
expanded after the current expansion of AC_OUTPUT_COMMANDS_PRE, and so
on until there are no more commands. I modified lib/autoconf/status.m4
# AC_OUTPUT_COMMANDS_REC(MACRO)
# -----------------------------
# Until the definition of MACRO is empty, repeatedly expand MACRO
# in a context where it has been redefined to the empty string.
m4_define([AC_OUTPUT_COMMANDS_REC], [m4_ifnblank(m4_defn([$1]),
[m4_define([$1], [m4_define([$1])]m4_defn([$1]))
m4_indir([$1])dnl
AC_OUTPUT_COMMANDS_REC([$1])])])
Cute. Might be worth moving into the m4sugar 'm4_' namespace instead of
the 'AC_' namespace; and as written it adds spurious whitespace into the
output file, but it looks like a reasonable approach.
Post by Nick Bowler
and replacing the call to AC_OUTPUT_COMMANDS_PRE() with
AC_OUTPUT_COMMANDS_REC([AC_OUTPUT_COMMANDS_PRE]). The same change could
be done for AC_CONFIG_COMMANDS_POST. This seems to work just fine,
although we could imagine some (crazy!) configure scripts for which this
change makes them non-terminating...
It's already possible to write configure.ac that causes an infinite loop
[or fill the disk or exhaust memory], and autoconf is not in the
business of solving the "halting problem". So we don't really have to
worry about such buggy input.
Post by Nick Bowler
Any thoughts?
I think making AC_OUTPUT_COMMANDS_PRE behave more like atexit(3) (where
additional clauses are requested during the execution of an existing
clause) makes sense. Now to turn your suggestion into an actual patch...
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
Nick Bowler
2013-02-17 17:20:31 UTC
Permalink
Hi Eric, sorry for the delay.
[...]
Post by Eric Blake
Post by Nick Bowler
A more complicated option would be a sort of "recursive"
expansion of AC_OUTPUT_COMMANDS_PRE, such that invocations of
AC_CONFIG_COMMANDS_PRE during the expansion are collected as usual,
then *those* commands are expanded after the current expansion of
AC_OUTPUT_COMMANDS_PRE, and so on until there are no more commands.
I modified lib/autoconf/status.m4 to do just that, by defining the
# AC_OUTPUT_COMMANDS_REC(MACRO)
# -----------------------------
# Until the definition of MACRO is empty, repeatedly expand MACRO
# in a context where it has been redefined to the empty string.
m4_define([AC_OUTPUT_COMMANDS_REC], [m4_ifnblank(m4_defn([$1]),
[m4_define([$1], [m4_define([$1])]m4_defn([$1]))
m4_indir([$1])dnl
AC_OUTPUT_COMMANDS_REC([$1])])])
Cute. Might be worth moving into the m4sugar 'm4_' namespace instead of
the 'AC_' namespace; and as written it adds spurious whitespace into the
output file, but it looks like a reasonable approach.
Indeed, it seems like one of the dnls vanished while I was revising. I
presume that's the only spurious whitespace problem. Easily fixed.
Post by Eric Blake
Post by Nick Bowler
This seems to work just fine, although we could imagine some
(crazy!) configure scripts for which this change makes them
non-terminating...
It's already possible to write configure.ac that causes an infinite
loop [or fill the disk or exhaust memory], and autoconf is not in the
business of solving the "halting problem". So we don't really have to
worry about such buggy input.
I agree that the case is not worth worrying about. Just doing my due
diligence to consider potential backwards incompatibilities.
Post by Eric Blake
I think making AC_OUTPUT_COMMANDS_PRE behave more like atexit(3) (where
additional clauses are requested during the execution of an existing
clause) makes sense. Now to turn your suggestion into an actual patch...
I finally managed to get a round tuit. Will follow up with a patch...
Nick Bowler
2013-02-17 17:22:58 UTC
Permalink
Currently, if a command registered with AC_CONFIG_COMMANDS_PRE itself
calls AC_CONFIG_COMMANDS_PRE, such commands will never be executed.
This occurs because each invocation of AC_CONFIG_COMMANDS_PRE appends
the argument to one big macro (AC_OUTPUT_COMMANDS_PRE) which is then
expanded once near the end of the configure output.

To solve this, we can keep track of the command registrations that occur
during expansion of AC_OUTPUT_COMMANDS_PRE, then after the original
registrations have been handled, output *those* commands, and so on
until there are no more commands. This is similar to how atexit
works in the C library (although atexit uses a last-in first-out
ordering, whereas this implementation is first-in first-out).

* lib/m4sugar/m4sugar.m4 (m4_output_rec): New macro to help perform
this command expansion loop.
* lib/autoconf/status.m4 (AC_OUTPUT): Use m4_output_rec to expand
AC_OUTPUT_COMMANDS_PRE and AC_OUTPUT_COMMANDS_POST
* tests/base.at (AC_CONFIG_COMMANDS_PRE/POST: nested invocation): New
test.
---
lib/autoconf/status.m4 | 4 ++--
lib/m4sugar/m4sugar.m4 | 21 +++++++++++++++++++++
tests/base.at | 21 +++++++++++++++++++++
3 files changed, 44 insertions(+), 2 deletions(-)

diff --git a/lib/autoconf/status.m4 b/lib/autoconf/status.m4
index d125843..9b6e13c 100644
--- a/lib/autoconf/status.m4
+++ b/lib/autoconf/status.m4
@@ -1268,7 +1268,7 @@ test "x$exec_prefix" = xNONE && exec_prefix='${prefix}'
m4_ifdef([_AC_SEEN_CONFIG(HEADERS)], [DEFS=-DHAVE_CONFIG_H], [AC_OUTPUT_MAKE_DEFS()])

dnl Commands to run before creating config.status.
-AC_OUTPUT_COMMANDS_PRE()dnl
+m4_output_rec([AC_OUTPUT_COMMANDS_PRE])dnl

: "${CONFIG_STATUS=./config.status}"
ac_write_fail=0
@@ -1281,7 +1281,7 @@ test $ac_write_fail = 0 ||
AC_MSG_ERROR([write failure creating $CONFIG_STATUS])

dnl Commands to run after config.status was created
-AC_OUTPUT_COMMANDS_POST()dnl
+m4_output_rec([AC_OUTPUT_COMMANDS_POST])dnl

# configure is writing to config.log, and then calls config.status.
# config.status does its own redirection, appending to config.log.
diff --git a/lib/m4sugar/m4sugar.m4 b/lib/m4sugar/m4sugar.m4
index b732abc..f2e04c3 100644
--- a/lib/m4sugar/m4sugar.m4
+++ b/lib/m4sugar/m4sugar.m4
@@ -1305,6 +1305,27 @@ m4_define([_m4_map_args_w],
[m4_substr([$1], [$2], m4_eval(m4_len([$1]) - [$2] - [$3]))])


+# _m4_output_rec(MACRO)
+# ---------------------
+# Internal recursive part of m4_output_rec. This clobbers the definition of
+# MACRO so it should not be called directly.
+m4_define([_m4_output_rec], [m4_ifnblank(m4_defn([$1]),
+[m4_define([$1], [m4_define([$1])]m4_defn([$1]))dnl
+m4_indir([$1])dnl
+_m4_output_rec([$1])])])
+
+
+# m4_output_rec(MACRO)
+# --------------------
+# Until the definition of MACRO is empty, repeatedly expand MACRO in a
+# context where it has been redefined to the empty string. The original
+# definition of MACRO is restored after all expansion is complete.
+m4_define([m4_output_rec],
+[m4_pushdef([$1], m4_defn([$1]))dnl
+_m4_output_rec([$1])dnl
+m4_popdef([$1])])
+
+
# m4_stack_foreach(MACRO, FUNC)
# m4_stack_foreach_lifo(MACRO, FUNC)
# ----------------------------------
diff --git a/tests/base.at b/tests/base.at
index 87a0417..35bfd48 100644
--- a/tests/base.at
+++ b/tests/base.at
@@ -767,3 +767,24 @@ FOO
]])

AT_CLEANUP
+
+
+# -------------------------------------- #
+# Nested AC_CONFIG_COMMANDS_{PRE,POST}. #
+# -------------------------------------- #
+
+AT_SETUP([AC_CONFIG_COMMANDS_PRE/POST: nested invocation])
+AT_KEYWORDS([AC@&***@_CONFIG_COMMANDS_PRE AC@&***@_CONFIG_COMMANDS_POST
+AC@&***@_OUTPUT m4@&***@_output_rec])
+
+AT_DATA([configure.ac],
+[[AC_INIT([test], [1])
+AC_CONFIG_COMMANDS_PRE([AC_CONFIG_COMMANDS_PRE([m4_define([okpre])])])
+AC_CONFIG_COMMANDS_POST([AC_CONFIG_COMMANDS_POST([m4_define([okpost])])])
+AC_OUTPUT
+m4@&***@_ifndef([okpre], [m4][_fatal([dropped nested AC_CONFIG_COMMANDS_PRE])])
+m4@&***@_ifndef([okpost], [m4][_fatal([dropped nested AC_CONFIG_COMMANDS_POST])])
+]])
+AT_CHECK_AUTOCONF([], [], [], [stderr])
+
+AT_CLEANUP
--
1.7.8.6
Loading...