config.status and /bin/sh on Solaris

[Dimitrios Apostolou]

Hello list,

on my Solaris 10, for some unknown reason, /sometimes/ config.status is
generated with "#! /bin/sh" shebang, instead of bash. When this happens,
the configure script prints ugly messages but *succeeds*, but then gmake
fails miserably because of malformed Makefile. Here are the message
config.status prints, when it runs under /bin/sh:

$ ./config.status
./config.status: print: not found
./config.status: print: not found
./config.status: print: not found
[ ... lots of the same messages ... ]
config.status: creating Makefile
config.status: executing depfiles commands
config.status: executing libtool commands


I am setting CONFIG_SHELL=/usr/xpg4/bin/sh to work-around this. However I
am trying to fix this at the same time, since Solaris /bin/sh should work,
as it's a bourne shell.

I'm trying to fix the generated code in config.status to work with the
printf statement from /bin/sh (instead if inexistent print). I can see
that first print is tried, then printf, then echo. Besides the generated
files, where is the relevant source file in autoconf source? Relevant
snippet:

if test -z "$BASH_VERSION$ZSH_VERSION" \
&& (test "X`print -r -- $as_echo`" = "X$as_echo") 2>/dev/null; then
as_echo='print -r --'
as_echo_n='print -rn --'
elif (test "X`printf %s $as_echo`" = "X$as_echo") 2>/dev/null; then
as_echo='printf %s\n'
as_echo_n='printf %s'
else
if test "X`(/usr/ucb/echo -n -n $as_echo) 2>/dev/null`" = "X-n
$as_echo"; then


Thanks,
Dimitris

[Zack Weinberg]

Post by Dimitrios Apostolou
on my Solaris 10, for some unknown reason, /sometimes/ config.status is
generated with "#! /bin/sh" shebang, instead of bash.

You should figure out why *this* is happening and fix it. Solaris 10
/bin/sh is *extremely* old (behavior frozen circa 1992 - yes,
seriously) and it is likely that the problem you've noticed is only
the tip of the iceberg.

config.status on Solaris 10 should be generated with #!
/usr/xpg4/bin/sh and should also set the PATH to prioritize
/usr/xpg4/bin over {/usr,}/bin.

zw

[Dimitrios Apostolou]

Post by Zack Weinberg

Post by Dimitrios Apostolou
on my Solaris 10, for some unknown reason, /sometimes/ config.status is
generated with "#! /bin/sh" shebang, instead of bash.

You should figure out why *this* is happening and fix it. Solaris 10
/bin/sh is *extremely* old (behavior frozen circa 1992 - yes,
seriously) and it is likely that the problem you've noticed is only
the tip of the iceberg.
config.status on Solaris 10 should be generated with #!
/usr/xpg4/bin/sh and should also set the PATH to prioritize
/usr/xpg4/bin over {/usr,}/bin.

Thanks for suggesting this. Indeed my PATH has /usr/xpg4/bin in front, but
config.status is sometimes generated with /bin/bash, and some others with
/bin/sh (investigating the conditions on that). I've never seen
/usr/xpg4/bin/sh been chosen.

Where is the source for this shebang generation? Maybe I can make some
sense out of it.

Dimitris

[Eric Blake]

Post by Dimitrios Apostolou
Where is the source for this shebang generation? Maybe I can make some
sense out of it.

The #! line in config.status is _supposed_ to be the value of
CONFIG_SHELL (whether inherited from the environment because you pre-set
it, or because it was the value of the best candidate shell found during
the initial startup of configure where it re-exec's itself into the best
shell found). config.log may or may not be useful at determining this,
because finding the right candidate shell is very early in the startup.
But a trace of manually running '/path/to/shell -x ./configure' may help
see what was probed (warning, the trace may be huge, so don't go blindly
posting megabytes of stuff to this list).

[Dimitrios Apostolou]

Post by Eric Blake

Post by Dimitrios Apostolou
Where is the source for this shebang generation? Maybe I can make some
sense out of it.

The #! line in config.status is _supposed_ to be the value of
CONFIG_SHELL (whether inherited from the environment because you pre-set
it, or because it was the value of the best candidate shell found during
the initial startup of configure where it re-exec's itself into the best
shell found).

So I've been tracing the execution of "configure", but I don't
CONFIG_SHELL to be set, and the script reaches a point that the following
happens:

SHELL=${CONFIG_SHELL-/bin/sh}
export SHELL

So SHELL is now /bin/sh and config.status is generated as such:

cat >$CONFIG_STATUS <<_ASEOF || as_write_fail=1
#! $SHELL
# Generated by $as_me.

Post by Eric Blake
But a trace of manually running '/path/to/shell -x ./configure' may help
see what was probed (warning, the trace may be huge, so don't go blindly
posting megabytes of stuff to this list).

Trying to mimic my build environment, here is the command that always
reproduces this behaviour (the CSW part of the path has no shell, just
other utilities):

PATH=/opt/csw/bin:/usr/xpg4/bin:/usr/bin /usr/xpg4/bin/sh -x ./configure >trace2 2>&1


I'm still examining the logs, I'll send an update if I get closer to a
solution. Thanks!

Dimitris

[Zack Weinberg]

So I've been tracing the execution of "configure", but I don't CONFIG_SHELL
SHELL=${CONFIG_SHELL-/bin/sh}
export SHELL

It now occurs to me to ask exactly which version of Autoconf generated
this configure script.

There *should* be code immediately above the
SHELL=${CONFIG_SHELL-/bin/sh} line that *should* have detected that
/usr/xpg4/bin/sh is the right shell to use on this system. It's
possible that its tests are not stringent enough. Do you see a
variable named "as_required" being set in your logs, before this
point? If so, please tell us what it's being set to. If not, please
tell us it's not there.

zw

[Dimitrios Apostolou]

Post by Zack Weinberg

So I've been tracing the execution of "configure", but I don't CONFIG_SHELL
SHELL=${CONFIG_SHELL-/bin/sh}
export SHELL

It now occurs to me to ask exactly which version of Autoconf generated
this configure script.

At the top of configure:

#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for hello 1.0.

Post by Zack Weinberg
There *should* be code immediately above the
SHELL=${CONFIG_SHELL-/bin/sh} line that *should* have detected that
/usr/xpg4/bin/sh is the right shell to use on this system. It's
possible that its tests are not stringent enough. Do you see a
variable named "as_required" being set in your logs, before this
point? If so, please tell us what it's being set to. If not, please
tell us it's not there.

+ as_required=as_fn_return () { (exit $1); }
as_fn_success () { as_fn_return 0; }
as_fn_failure () { as_fn_return 1; }
as_fn_ret_success () { return 0; }
as_fn_ret_failure () { return 1; }

exitcode=0
as_fn_success || { exitcode=1; echo as_fn_success failed.; }
as_fn_failure && { exitcode=1; echo as_fn_failure succeeded.; }
as_fn_ret_success || { exitcode=1; echo as_fn_ret_success failed.; }
as_fn_ret_failure && { exitcode=1; echo as_fn_ret_failure succeeded.; }
if ( set x; as_fn_ret_success y && test x = "$1" ); then :

else
exitcode=1; echo positional parameters were not saved.
fi
test x$exitcode = x0 || exit 1
test -x / || exit 1
+ as_suggested= as_lineno_1=
[ ... ]


Dimitris

[Dimitrios Apostolou]

Post by Dimitrios Apostolou
+ as_required=as_fn_return () { (exit $1); }
as_fn_success () { as_fn_return 0; }
as_fn_failure () { as_fn_return 1; }
as_fn_ret_success () { return 0; }
as_fn_ret_failure () { return 1; }
exitcode=0
as_fn_success || { exitcode=1; echo as_fn_success failed.; }
as_fn_failure && { exitcode=1; echo as_fn_failure succeeded.; }
as_fn_ret_success || { exitcode=1; echo as_fn_ret_success failed.; }
as_fn_ret_failure && { exitcode=1; echo as_fn_ret_failure succeeded.; }
else
exitcode=1; echo positional parameters were not saved.
fi
test x$exitcode = x0 || exit 1
test -x / || exit 1
+ as_suggested= as_lineno_1=
[ ... ]

So the configure script goes like this:

if test x$as_have_required = xyes && (eval "$as_suggested") 2>/dev/null;
then :
else
[ ... Almost 100 lines never executed ... ]
fi
fi
SHELL=${CONFIG_SHELL-/bin/sh}
export SHELL



as_have_required is "yes".
as_suggested is set in multiple consecutive statements:


+ as_suggested= as_lineno_1=
+ as_suggested= as_lineno_1=195
+ as_suggested= as_lineno_1=195 as_lineno_1a=$LINENO
as_lineno_2=
+ as_suggested= as_lineno_1=195 as_lineno_1a=$LINENO
as_lineno_2=196
+ as_suggested= as_lineno_1=195 as_lineno_1a=$LINENO
as_lineno_2=196 as_lineno_2a=$LINENO
eval 'test "x$as_lineno_1'$as_run'" != "x$as_lineno_2'$as_run'" &&
test "x`expr $as_lineno_1'$as_run' + 1`" = "x$as_lineno_2'$as_run'"' ||
exit 1

test -n "${ZSH_VERSION+set}${BASH_VERSION+set}" || (

ECHO='\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'
ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO
ECHO=$ECHO$ECHO$ECHO$ECHO$ECHO$ECHO
PATH=/empty FPATH=/empty; export PATH FPATH
test "X`printf %s $ECHO`" = "X$ECHO" \
|| test "X`print -r -- $ECHO`" = "X$ECHO" ) || exit 1
test $(( 1 + 1 )) = 2 || exit 1



Dimitris

[Bob Friesenhahn]

I have been using Solaris 10 daily for Autoconf related development
since 2005 and have never seen configure select /bin/sh. One reason
is that the system normally has /usr/bin/bash installed.

Are you missing /usr/bin/bash on your system?

Bob

[Dimitrios Apostolou]

I have been using Solaris 10 daily for Autoconf related development since
2005 and have never seen configure select /bin/sh. One reason is that the
system normally has /usr/bin/bash installed.
Are you missing /usr/bin/bash on your system?

Hi, I have it, the installation is a full-sized one so it comes with all
utilities. Can you try running the configure script with this environment:

PATH=/opt/csw/bin:/usr/xpg4/bin:/usr/bin /usr/xpg4/bin/sh ./configure

Then what's the shebang line on top of config.status?

Thanks!

[Bob Friesenhahn]

Post by Dimitrios Apostolou

I have been using Solaris 10 daily for Autoconf related development since
2005 and have never seen configure select /bin/sh. One reason is that the
system normally has /usr/bin/bash installed.
Are you missing /usr/bin/bash on your system?

Hi, I have it, the installation is a full-sized one so it comes with all
PATH=/opt/csw/bin:/usr/xpg4/bin:/usr/bin /usr/xpg4/bin/sh ./configure
Then what's the shebang line on top of config.status?

I get the same problem you do. The config.status script starts with
"/bin/sh" and it fails to execute properly.

If I don't specifically execute configure using /usr/xpg4/bin/sh then
the config.status script starts with "/bin/bash" and everything is
fine.

Bob

[Dimitrios Apostolou]

Post by Bob Friesenhahn

Post by Dimitrios Apostolou

I have been using Solaris 10 daily for Autoconf related development since
2005 and have never seen configure select /bin/sh. One reason is that the
system normally has /usr/bin/bash installed.
Are you missing /usr/bin/bash on your system?

Hi, I have it, the installation is a full-sized one so it comes with all
PATH=/opt/csw/bin:/usr/xpg4/bin:/usr/bin /usr/xpg4/bin/sh ./configure
Then what's the shebang line on top of config.status?

I get the same problem you do. The config.status script starts with
"/bin/sh" and it fails to execute properly.
If I don't specifically execute configure using /usr/xpg4/bin/sh then the
config.status script starts with "/bin/bash" and everything is fine.

Thanks for verifying! I'm now getting a similar (same cause it seems)
error on a different software compilation, this time on Solaris 11. The
common denominator is the tweaked PATH, putting /usr/xpg4/bin before
/usr/bin causes havoc. I'll investigate more and report back.

It seems tweaking the PATH in my build environment was not the best idea.
:-)


Dimitris

[Bob Friesenhahn]

Thanks for verifying! I'm now getting a similar (same cause it seems) error
on a different software compilation, this time on Solaris 11. The common
denominator is the tweaked PATH, putting /usr/xpg4/bin before /usr/bin causes
havoc. I'll investigate more and report back.

I typically have this order in my PATH:

/usr/xpg6/bin:/usr/xpg4/bin:/usr/bin

so I think that your claim is not correct. Solaris and other systems
will invoke scripts starting with /bin/sh using /bin/sh regarding of
the PATH. You had me test by force-overriding the shell used.

Bob

[Bob Friesenhahn]

What does 'echo $SHELL' show for your login session?

Bob

[Dimitrios Apostolou]

Post by Bob Friesenhahn
What does 'echo $SHELL' show for your login session?

It's /usr/bin/bash for this system (but I'm having /similar/, maybe
related issues on a Solaris 11 with /bin/sh as login shell). I don't think
the login shell is relevant, but the PATH tweak is the common denominator.


Dimitris

P.S. I'm writing a report of the other issue on a separate thread.

[Dimitrios Apostolou]

Post by Dimitrios Apostolou
if test x$as_have_required = xyes && (eval "$as_suggested") 2>/dev/null;
else
[ ... Almost 100 lines never executed ... ]
fi
fi
SHELL=${CONFIG_SHELL-/bin/sh}
export SHELL

Digging deeper into this, I believe this is the relevant code, file
m4sh.m4, macro _AS_DETECT_BETTER_SHELL (with slight indentation edits by
myself while struggling to follow logic):

[if test "x$CONFIG_SHELL" = x; then
as_bourne_compatible="AS_ESCAPE(_m4_expand([_AS_BOURNE_COMPATIBLE]))"
_AS_DETECT_EXPAND([as_required], [_AS_DETECT_REQUIRED_BODY])
_AS_DETECT_EXPAND([as_suggested], [_AS_DETECT_SUGGESTED_BODY])
AS_IF([_AS_RUN(["$as_required"])],
[as_have_required=yes],
[as_have_required=no])

AS_IF([test x$as_have_required = xyes && _AS_RUN(["$as_suggested"])],
[],
[ ... long snippet trying various shells in various directories
and setting CONFIG_SHELL accordingly ...
]
)
fi
SHELL=${CONFIG_SHELL-/bin/sh}
export SHELL


In my case, /usr/xpg4/bin/sh is a capable-enough shell, thus
as_have_required is *yes* and _AS_RUN(["$as_suggested"]) *succeeds*, thus
the long snippet that tries all shells, is skipped!

By reading this, it seems that the intent is to set SHELL to the
*currently running shell interpreter*, since it passed all tests. But that
one is not /bin/sh, since the script was manually executed with "sh
./configure" and a modified PATH.

A slight improvement would be to change the last line to search for the
first "sh" in PATH:

SHELL=${CONFIG_SHELL-`command -v sh`}

The best solution would be to detect the currently running interpreter.
But how?

Thanks,
Dimitris

[Zack Weinberg]

Post by Dimitrios Apostolou
In my case, /usr/xpg4/bin/sh is a capable-enough shell, thus
as_have_required is *yes* and _AS_RUN(["$as_suggested"]) *succeeds*, thus
the long snippet that tries all shells, is skipped!
By reading this, it seems that the intent is to set SHELL to the *currently
running shell interpreter*, since it passed all tests. But that one is not
/bin/sh, since the script was manually executed with "sh ./configure" and a
modified PATH.

Yes, that's the intent.

Post by Dimitrios Apostolou
The best solution would be to detect the currently running interpreter. But
how?

Inconveniently, I'm not sure there's a reliable way to detect it. $0
is going to be set to "configure". Bash has a $BASH special variable
that expands to the full pathname of the shell interpreter, but that
is actually evidence for the _lack_ of a portable way to detect it.
Similarly, /proc/$$/exe is a Linuxism. Anyone else got an idea?

zw

[Jeffrey Walton]

So I've been tracing the execution of "configure", but I don't CONFIG_SHELL
SHELL=${CONFIG_SHELL-/bin/sh}
export SHELL

That could fail on some of the BSDs, too. Some of the BSDs place some
of the shells in locations like /usr/local/bin.

Jeff