Discussion:
configure[xxxx]: : is not an identifer
Kevin R. Bulgrien
2018-06-07 21:30:25 UTC
Permalink
----- Original Message -----
Sent: Thursday, June 7, 2018 3:13:18 PM
Subject: Re: configure[xxxx]: : is not an identifer
I find myself trying to build newer versions of GNU toolchain
components on a very
old system, namely OpenServer 5.0.7. I know that supporting old
systems like this
is difficult, so I do recognize a need to do the heavy lifting on
my own, however,
if anyone could offer some help, it would be great.
I've run into the following error all too frequently, and, to avoid
the error, so
far, have just backed off to an older release of whatever is being
built. For
--prefix=${HOME}
./configure[1845]: : is not an identifier
Can you run under '/bin/sh -x ./configure' (or whatever shell
configure
actually picks in place of /bin/sh) to get a more verbose log right
before the failure message of what the script was attempting?
Yes, I did this after posting the original e-mail, and I got a result
that showed me the last command that ran was the ac_subst_vars
assignment:

...
#ifdef HAVE_UNISTD_H
# include <unistd.h>
#endif
+ gl_use_threads_default=
+ ac_func_list=
+ ac_header_list=
+ gl_getopt_required=POSIX
+ gl_getopt_required=POSIX
+ ac_subst_vars=M4tests_LTLIBOBJS
...
HAVE_POSIX_SPAWN
GNULIB_POSIX_SPAWNATTR_DESTROY
GNULIB_POSIX_SPAWNATTR_SETSIGMASK
GNULIB_POSIX_SPAWNATTR_GETSIGMASK
GNULIB_POSIX_SPAWNATTR_SETSIGDEFAULT
GNULIB_POSIX_SPAWNATTR_GETSIGDEFAULT
GNULIB_POSIX_SPAWNATTR_SETSCHEDPOLICY
GNU
+ ./configure[1845]: : is not an identifier

This is about 4K into the value being assigned.

I found that if I broke up this value by assigning it to different
variables (so as not to reduce variable store usage), the script
runs "fine". I.e., I break up the ac_subst_vars along the lines
of:

ac_subst_vars='...
...'
foo='...
...'
bar='...
...'

Where the size of each piece is similar (~4K). When this is done,
the configure script runs "fine", though it is certainly damaged.

I unintentionally over-rode the #!/bin/sh in the script with:

$ bash -x ./configure -C --prefix=${HOME}

This resolved the problem, and ./configure ran fine with the
unmodified configure script.

The system /bin/sh does not report version, but can be loosely
identified in the OpenServer ecosystem:

$ ls -l /bin/sh
lrwxrwxrwx 1 root root 30 May 5 2008 /bin/sh -> /opt/K/SCO/Unix/5.0.7Hw/bin/sh

I do have BASH built for this system:

$ bash --version
GNU bash, version 4.3.30(1)-release (i586-pc-sco3.2v5.0)
I supposed that perhaps in this case, [1845] might be a line
number.
Or maybe a process id?
Based on using /bin/sh -x to invoke the configure script, it is clear
that [1845] is a line number.

It seems clear the OpenServer 5.0.7 system /bin/sh shell is inadequate in
some way, but it also seems clear that it is not a simple matter of
variable length. If, using vim, I create my own script with #!/bin/sh, I
can copy the ac_subst_vars assignment into it and it works just fine, and,
besides, I can create variables with much larger values in this shell.

#!/bin/sh
ac_subst_vars='...
...
...'
echo "${ac_subst_vars}" | wc -c

./test.sh
19222

Any thought as to whether I should I just move along and recognize I
need to use BASH in these cases, or is this an indication of
something to address somewhere in the tool chain that does
not detect this problem?
--
Kevin R. Bulgrien, Network/Software Engineer
Computer Systems Design, Inc dba Systems Design
Kevin R. Bulgrien
2018-06-07 23:12:21 UTC
Permalink
----- Original Message -----
Sent: Thursday, June 7, 2018 4:54:34 PM
Subject: Re: configure[xxxx]: : is not an identifer
Post by Kevin R. Bulgrien
Can you run under '/bin/sh -x ./configure' (or whatever shell
configure
actually picks in place of /bin/sh) to get a more verbose log right
before the failure message of what the script was attempting?
Yes, I did this after posting the original e-mail, and I got a result
that showed me the last command that ran was the ac_subst_vars
...
+ ./configure[1845]: : is not an identifier
This is about 4K into the value being assigned.
Then it sounds like we've found a legitimate shell bug in your
/bin/sh.
It would be nice to fine-tune the characteristics of the bug; is it
a='8k wall of text'
a='4k...'
b='4k...'
a="$a$b"
works? Or is it that a shell variable in general cannot hold more
than
4k? Does the size of the shell script matter (as in the bug is
present
if the assignment also coincides with some other large boundary, such
as
a 64k page size)?
Post by Kevin R. Bulgrien
I found that if I broke up this value by assigning it to different
variables (so as not to reduce variable store usage), the script
runs "fine". I.e., I break up the ac_subst_vars along the lines
ac_subst_vars='...
...'
foo='...
...'
bar='...
...'
Where the size of each piece is similar (~4K). When this is done,
the configure script runs "fine", though it is certainly damaged.
Well, yeah, if you break it up to get past tickling the overlarge
assignment bug, but then don't reassemble it, the resulting
substitutions are incomplete and the end product would be damaged.
As
long as variables can hold more than 4k (even if we have to build it
via
smaller steps), that can be worked around.
I found that using the last displayed value shown in the error message
became unreliable near the end of the value. I ended up splitting
the variable assignment into pieces of about 200 lines each to
get it to work. This correlates to splitting the assignment into
six pieces:

$ diff -U1 configure.orig configure
--- configure.orig 2018-06-06 05:41:59.000000000 -0500
+++ configure 2018-06-06 06:22:33.000000000 -0500
@@ -629,3 +629,3 @@
gl_getopt_required=POSIX
-ac_subst_vars='M4tests_LTLIBOBJS
+ac_subst_vars1='M4tests_LTLIBOBJS
M4tests_LIBOBJS
@@ -838,4 +838,4 @@
GNULIB_POSIX_SPAWNATTR_SETSIGDEFAULT
-GNULIB_POSIX_SPAWNATTR_GETSIGDEFAULT
-GNULIB_POSIX_SPAWNATTR_SETSCHEDPOLICY
+GNULIB_POSIX_SPAWNATTR_GETSIGDEFAULT'
+ac_subst_vars2='GNULIB_POSIX_SPAWNATTR_SETSCHEDPOLICY
GNULIB_POSIX_SPAWNATTR_GETSCHEDPOLICY
@@ -1038,4 +1038,4 @@
REPLACE_TANHF
-REPLACE_TANF
-REPLACE_SQRTL
+REPLACE_TANF'
+ac_subst_vars3='REPLACE_SQRTL
REPLACE_SQRTF
@@ -1239,4 +1239,4 @@
GNULIB_LOG1P
-GNULIB_LOG10L
-GNULIB_LOG10F
+GNULIB_LOG10L'
+ac_subst_vars4='GNULIB_LOG10F
GNULIB_LOG10
@@ -1439,4 +1439,4 @@
UNISTD_H_HAVE_WINSOCK2_H_AND_USE_SOCKETS
-UNISTD_H_HAVE_WINSOCK2_H
-REPLACE_WRITE
+UNISTD_H_HAVE_WINSOCK2_H'
+ac_subst_vars5='REPLACE_WRITE
REPLACE_USLEEP
@@ -1639,4 +1639,4 @@
GNULIB_CALLOC_POSIX
-GNULIB_ATOLL
-GNULIB__EXIT
+GNULIB_ATOLL'
+ac_subst_vars6='GNULIB__EXIT
LTLIBCSTACK
@@ -1844,2 +1844,10 @@
SHELL'
+echo $ac_subst_vars1 | wc -c >krb
+echo $ac_subst_vars2 | wc -c >>krb
+echo $ac_subst_vars3 | wc -c >>krb
+echo $ac_subst_vars4 | wc -c >>krb
+echo $ac_subst_vars5 | wc -c >>krb
+echo $ac_subst_vars6 | wc -c >>krb
+ac_subst_vars="$ac_subst_vars1\n$ac_subst_vars2\n$ac_subst_vars3\n$ac_subst_vars4\n$ac_subst_vars5\n$ac_subst_vars6"
+echo $ac_subst_vars | wc -c >>krb
ac_subst_files=''

$ cat krb
4030
3609
2745
3136
3162
2540
19222

This shows the whole constructed as a sum of the parts, with the
final result matching the original 19222 bytes assigned (as reported
in a prior post by copying the assignment to a standalone script).

With this modification, the script runs fine under this /bin/sh.

If 4K is a factor in this bug, it is subtle. It has the appearance
that the shell error message does not reliably show the points where
the variable has to be split the further into the value you get, though
I suspect the error messages occur near 4K boundaries.

I first broke the value by splitting just before the last value
reported by the crash, then re-ran the modified script to find the
next split point, until I had 5 pieces. This would be about right
to split a 19220 byte value into 4K pieces, but, at this point, I
found that the script bombed other places and moving the 5th split
around wasn't working, so I just ended up splitting by line count
into 6 pieces to rid myself of the error.
Post by Kevin R. Bulgrien
$ bash -x ./configure -C --prefix=${HOME}
This resolved the problem, and ./configure ran fine with the
unmodified configure script.
CONFIG_SHELL=/path/to/bash ./configure CONFIG_SHELL=/path/to/bash
-C...
(yes, this is the one case where you have to pass CONFIG_SHELL both
in
the environment and on the command line; it's a bit of a
boot-strapping
dilemma for forcing the configure script to choose a given shell
rather
than its normal behavior of choosing a best shell).
Thanks for this tip. I'll be glad to know this for the future.
Post by Kevin R. Bulgrien
The system /bin/sh does not report version, but can be loosely
$ ls -l /bin/sh
lrwxrwxrwx 1 root root 30 May 5 2008 /bin/sh ->
/opt/K/SCO/Unix/5.0.7Hw/bin/sh
Old, but the real question is if it is still part of a supported
distro
(in which case it would be really nice to work around it) or if the
vendor has long since dropped support (in which case it's probably
not
worth spending time on something even the vendor doesn't care about)
Well, to my knowledge, fresh updates to the OpenServer 5 product have
not been issued in years. Based on the ftp server, I'd say the last
maintenance rollup was issued in 2013, and a shell update was issued
in late 2014 to address a number of shell vulnerabilities.
Does bash (or any other shell flavors) come pre-installed, or does a
pristine system come with only the broken /bin/sh? (Again, the
bootstrapping question, of whether it is merely sufficient to detect
the
/bin/sh bug and avoid it altogether because we should always be able
to
find something else, or whether we should try hard to work in spite
of
the /bin/sh bug because bash won't always be found). Obviously, it's
worked for you to install bash, and if nothing else, we can always
recommend that a user with a broken /bin/sh install bash, but that
feels
like a fallback solution (and may not work if bash itself can't be
self-built because it has a configure script that also tickles the
bug).
Being new to this environment, I'm not sure what comes installed by
default when installing from media, but, on this system, that 2014
shell update was packaged in an extshells-1.4.1Ab package:

SOLUTION: This supplement addresses the 'shellshock' security
vulnerabilities defined by the following CVEs:

CVE-2014-6271
CVE-2014-6277
CVE-2014-6278
CVE-2014-7169
CVE-2014-7186
CVE-2014-7187

These vulnerabilities could allow a regular bash user to gain
privileges through a crafted environment variable.

This supplement is intended for installation on OpenServer releases:

SCO OpenServer Release 5.0.7 with Maintenance Pack 5
OpenServer 507V

The extshells-1.4.1Ab package contains these shells:
GNU Bourne-Again Shell (bash) 4.3.30
The Z-Shell (zsh) 4.2.6
Enhanced C-Shell (tcsh) 6.14.00
Korn Shell 93r

Only the bash shell was updated in this package.
bash-4.3.30 corrects the 'shellshock' vulnerability.

I do believe that BASH is not installed by default as a number of
customer systems do not have it installed. That said, I do
believe that when the vendor's Extended Shells are installed,
bash is available.

On this system I have the following vendor-supplied volumes along
with others probably not relevant to this issue:

Extended Shells (ver 1.4.1Ab)
GNU Development Tools (ver 5.0.7Kj)
GNU Development Tools (ver 5.0.7g)
SCO OpenServer Enterprise System (ver 5.0.7Hw)
Post by Kevin R. Bulgrien
$ bash --version
GNU bash, version 4.3.30(1)-release (i586-pc-sco3.2v5.0)
I supposed that perhaps in this case, [1845] might be a line
number.
Or maybe a process id?
Based on using /bin/sh -x to invoke the configure script, it is clear
that [1845] is a line number.
It seems clear the OpenServer 5.0.7 system /bin/sh shell is
inadequate in
some way, but it also seems clear that it is not a simple matter of
variable length. If, using vim, I create my own script with
#!/bin/sh, I
can copy the ac_subst_vars assignment into it and it works just fine, and,
besides, I can create variables with much larger values in this shell.
#!/bin/sh
ac_subst_vars='...
...
...'
echo "${ac_subst_vars}" | wc -c
./test.sh
19222
Oh well, that's going to make characterizing the bug even harder :(
You've identified the size of the assignment statement, and the line
it
starts on; can you also identify the text near various 4k points in
the
generated configure file lie, to see if something syntactically
interesting is happening if the shell treats two separate parts of a
line in an interesting manner? (For example, splitting 'ab=cd' into
'a'
and 'b=cd' has different semantic than splitting into 'ab' and '=cd';
and the error message about an empty variable name being an invalid
assignment might be a case of splitting right before the =).
As to syntactically interesting constructs, I'd say not. There is
nothing present in the value list that matches [^a-zA-Z0-9_] other
than linefeeds separating each value. Everything looks like a plain
and simple alpha-numeric values with underscores embedded.
Post by Kevin R. Bulgrien
Any thought as to whether I should I just move along and recognize I
need to use BASH in these cases, or is this an indication of
something to address somewhere in the tool chain that does
not detect this problem?
Knowing that bash is a workaround is good; so now it's a question of
whether it's also worth trying to spend effort on making autoconf
smarter to avoid this problem with /bin/sh for other users of your
platform, or whether your situation is niche enough that we just let
you live with the workaround.
In summary, it is not terribly difficult to get this bash installed
via a vendor distribution file. It is not necessary to build from
source. I can't really speak to how many people still might use
the platform, but I can't imagine a lot of folk attempt to build
packages on it. I have found myself supporting an effort to conjure
up support for more secure network protocols as some significant
protocol security mitigations are about to go into effect.

Though the purist part of me would like to see a fix, I also fully
realize how archaic this system is, so perhaps this thread being on
the net may be a reasonable enough way to point other people to a
reasonable workaround without draining resources from things that
do get a bigger audience.
--
Kevin R. Bulgrien, Network/Software Engineer
Computer Systems Design, Inc dba Systems Design
Kevin R. Bulgrien
2018-06-08 02:19:59 UTC
Permalink
----- Original Message -----
Sent: Thursday, June 7, 2018 8:47:55 PM
Subject: Re: configure[xxxx]: : is not an identifer
This looks like a known problem that we decided in 2006 not to fix
because even back then hardly anybody used OpenServer. For that old
discussion thread (which suggests some possible workarounds), please
https://lists.gnu.org/archive/html/autoconf-patches/2006-06/msg00036.html
The thread even ends suggesting that people could just use bash instead:

https://lists.gnu.org/archive/html/autoconf-patches/2006-06/msg00111.html

Awesome catch. Thanks. Apparently I need to sharpen my search skills.
--
Kevin R. Bulgrien, Network/Software Engineer
Computer Systems Design, Inc dba Systems Design
Loading...