cl.exe and system types

[Sébastien Hinderer]

Dear all,

I am writing an autoconf script for a project which is supposed to
compile under both Unix and Windows.

Under Windows, the project will be built under Cygwin but it should be
possible to use both MingW and cl.exe to compile it.

For the MingW case, giving thee right --host= option works just fine.
For the cl.exe case, though, I am not sure exactly what to do. Indeed
when I run

./configure CC=cl

Then configure detects the Microsoft C compiler properly. However the
host and target types will be cygwin as the build type. I am wondering
whether I should adjust host and target in case the C compiler is
detected to be cl.exe?

Are there known examples of projects that handle this situation properly
and where I could take inspiration from?

Many thanks in advance for any help / pointers!

Sébastien.

[Bob Friesenhahn]

Post by Sébastien Hinderer
Dear all,
I am writing an autoconf script for a project which is supposed to
compile under both Unix and Windows.
Under Windows, the project will be built under Cygwin but it should be
possible to use both MingW and cl.exe to compile it.

I don't know that MSVC is supported by Autotools. It would be nice if
it was. Have you seen a statement that this is supported?

Due to this shortcoming, many projects are supporting cmake, and often
abandoning an Autotools-based build which otherwise worked just fine.

Bob

[Bob Friesenhahn]

Post by Sébastien Hinderer
Under Windows, the project will be built under Cygwin but it should be
possible to use both MingW and cl.exe to compile it.

I don't know that MSVC is supported by Autotools. It would be nice if it
was. Have you seen a statement that this is supported?

I should mention that I do recall that there is (historically) at
least one add-on wrapper program on top of MSVC which allows MSVC to
be used to some extent with Autotools. This is not GNU-supported
software.

Bob

[Sébastien Hinderer]

Dear Bob,

Many thanks for your responses.

I don't know that MSVC is supported by Autotools. It would be nice if it
was. Have you seen a statement that this is supported?

Well, here is what AC_PROG_CC outputs when running
$ ./configure CC=cl
checking for i686-pc-windows-gcc... cl
checking whether the C compiler works... yes
checking for C compiler default output file name... conftest.exe
checking for suffix of executables... .exe
checking whether we are cross compiling... no
checking for suffix of object files... obj
checking whether we are using the GNU C compiler... no
checking whether cl accepts -g... yes
checking for cl option to accept ISO C89... none needed
checking how to run the C preprocessor... cl -E

From that, I inferred that cl is supported. Am I wrong here?

Also, in autoconf's documentaiotn, in section 5.10.3 (C compiler
characteristics) where the AC_PROG_CC macro is documented, one finds the
following example:

AC_PROG_CC([gcc cl cc])

From that, too, I inderred it was okay to use cl.exe in conjunction with
an autoconf script. Am I wrong here?

Thanks a lot in advance for any help in clarifying this!

Sébastien.

[Bob Friesenhahn]

Post by Sébastien Hinderer

I don't know that MSVC is supported by Autotools. It would be nice if it
was. Have you seen a statement that this is supported?

Well, here is what AC_PROG_CC outputs when running
$ ./configure CC=cl
checking for i686-pc-windows-gcc... cl
checking whether the C compiler works... yes
checking for C compiler default output file name... conftest.exe
checking for suffix of executables... .exe
checking whether we are cross compiling... no
checking for suffix of object files... obj
checking whether we are using the GNU C compiler... no
checking whether cl accepts -g... yes
checking for cl option to accept ISO C89... none needed
checking how to run the C preprocessor... cl -E
From that, I inferred that cl is supported. Am I wrong here?

I don't know. The pace of Visual Studio development is much faster
than Autotools development in recent years. It is possible that MSVC
has done something to make things easier.

Are you using Cygwin for your shell environment?

Bob

[Sébastien Hinderer]

Post by Bob Friesenhahn
Are you using Cygwin for your shell environment?

Yes, absolutely. Do you think this makes things better, or worse? :)

Sébastien.

[Earnie]

Post by Sébastien Hinderer

Post by Bob Friesenhahn
Are you using Cygwin for your shell environment?

Yes, absolutely. Do you think this makes things better, or worse? :)

Well MSYS2 might work better because it caters to the Windows
applications but perhaps not.

You need to declare the --build environment and the --host environment
so that configure isn't trying to find it. You'll not only need to set
CC but also LD and maybe other such variables.

The values for --build and --host need to be the same and can be
whatever you desire but something sensible like x86_64-pc-msvc might
work well.

[Bob Friesenhahn]

Post by Sébastien Hinderer

Post by Bob Friesenhahn
Are you using Cygwin for your shell environment?

Yes, absolutely. Do you think this makes things better, or worse? :)

It does mean that you are effectively cross-compiling so you would
need to execute the configure script with options for cross-compiling.

Cygwin creates binaries for a Unix-like API implemented by a Cygwin
DLL.

Bob

[Sébastien Hinderer]

Dear Earnie and Bob,

Many thanks for your replies.

Post by Earnie

Post by Sébastien Hinderer

Post by Bob Friesenhahn
Are you using Cygwin for your shell environment?

Yes, absolutely. Do you think this makes things better, or worse? :)

Well MSYS2 might work better because it caters to the Windows
applications but perhaps not.

Okay. Well I think I'll try to make things work under Cygwin first,
because that's the build environment the project currently uses. Only if
I am not able to make things work will I start considering another
build environment, I think.

Post by Earnie
You need to declare the --build environment and the --host environment
so that configure isn't trying to find it.

Thanks for mentionning this, because that's a point I am unclear about.

Post by Earnie
You'll not only need to set
CC but also LD and maybe other such variables.

Sure, I am prepared to do that if they can not be automatically
detected.

Post by Earnie
The values for --build and --host need to be the same and can be
whatever you desire but something sensible like x86_64-pc-msvc might
work well.

So here it seems Bob and you disagree, because as far as I understand
his mail quoted below he says that build and host should be different. I
tend to think they should be different, too, because although the
resulting binaries will be usable under Cygwin, they will not be cygwin
applications but rather native Windows applications. So in my
understanding even if the built binaries can be run in the build
environemnt, this is a case of cross-compiling.

What I do not understand, though, is why it is necessary to specify
--build. I know the manual says that when one specifies --host then one
has to specify --build, too, for historical reasons, but the manual also
says that this will be fixed in the next version. Moreover, I did try to
run configure with --host and no --build and that seemed to work,
independently of whether it was a cross-compilation scenario or not. So
I came to wonder how up-to-date the manual actually is.

Oh and also, x86_64-pc-msvc is not recognized. I am wondering whether
there is a list somewhere of the valid parameters for --build and
--host? I tried running config.sub and config.guess with -h but that
didn't help.

Post by Earnie

Post by Sébastien Hinderer

Post by Bob Friesenhahn
Are you using Cygwin for your shell environment?

Yes, absolutely. Do you think this makes things better, or worse? :)

It does mean that you are effectively cross-compiling so you would need to
execute the configure script with options for cross-compiling.

Yeah that's my understanding, too, with the restriction mentionned
above, i.e. I am not sure it is still necessary to specify --build when
specifying --host since my experiments tend to show a normal behaviour in
the case where --host is specified and --build is omitted.

Any input welcome because I am really in an unknown territory, here.

Many thanks in advance,

Sébastien.

[Nick Bowler]

Hi Sébastien,

Post by Sébastien Hinderer
What I do not understand, though, is why it is necessary to specify
--build. I know the manual says that when one specifies --host then one
has to specify --build, too, for historical reasons, but the manual also
says that this will be fixed in the next version. Moreover, I did try to
run configure with --host and no --build and that seemed to work,
independently of whether it was a cross-compilation scenario or not. So
I came to wonder how up-to-date the manual actually is.

When you specify just --host (and not --build), configure autodetects
cross compiliation by attempting to run a compiled program. If the
program runs, configure assumes you are not cross compiling. If the
program does not run for any reason, configure assumes you are cross
compiling.

The problem is this heuristic tends to be wrong as often as it is right,
so the recommendation now is to always specify --build and --host, which
disables this autodetection (configure enters cross compilation mode if
and only if the build and host triplets are different).

The comment that "This will be fixed in the future" appears to have been
added in 2002. Clearly we've not made it to the future yet. :)

Cheers,
Nick

[Earnie]

Post by Nick Bowler
Hi Sébastien,

Post by Sébastien Hinderer
What I do not understand, though, is why it is necessary to specify
--build. I know the manual says that when one specifies --host then one
has to specify --build, too, for historical reasons, but the manual also
says that this will be fixed in the next version. Moreover, I did try to
run configure with --host and no --build and that seemed to work,
independently of whether it was a cross-compilation scenario or not. So
I came to wonder how up-to-date the manual actually is.

When you specify just --host (and not --build), configure autodetects
cross compiliation by attempting to run a compiled program. If the
program runs, configure assumes you are not cross compiling. If the
program does not run for any reason, configure assumes you are cross
compiling.

Yes, exactly. And MSVC isn't setup for autotools style cross compiling
so you need to specify both to stop the auto-detection process.

Post by Nick Bowler
The problem is this heuristic tends to be wrong as often as it is right,
so the recommendation now is to always specify --build and --host, which
disables this autodetection (configure enters cross compilation mode if
and only if the build and host triplets are different).
The comment that "This will be fixed in the future" appears to have been
added in 2002. Clearly we've not made it to the future yet. :)

Should it ever get here? Maybe the fix is to remove the comment. What
we have has worked for many years with no one bugged about it enough to
supply a patch to do otherwise.

[Sébastien Hinderer]

Dear Nick,

Post by Nick Bowler
Hi Sébastien,

Post by Sébastien Hinderer
What I do not understand, though, is why it is necessary to specify
--build. I know the manual says that when one specifies --host then one
has to specify --build, too, for historical reasons, but the manual also
says that this will be fixed in the next version. Moreover, I did try to
run configure with --host and no --build and that seemed to work,
independently of whether it was a cross-compilation scenario or not. So
I came to wonder how up-to-date the manual actually is.

When you specify just --host (and not --build), configure autodetects
cross compiliation by attempting to run a compiled program. If the
program runs, configure assumes you are not cross compiling. If the
program does not run for any reason, configure assumes you are cross
compiling.
The problem is this heuristic tends to be wrong as often as it is right,
so the recommendation now is to always specify --build and --host, which
disables this autodetection (configure enters cross compilation mode if
and only if the build and host triplets are different).

I had no idea! Thanks! It's much clearer now. I thought that when --build
is not specified then it is determined by the AC_CANONICAL_BUILD
macro.

One question I still have is about the possible values for the arguments
to --build & co. Is there a specification or an exhaustive list
somewhere? I know the general syntax but I'd like to figure out what
would be the best values for our Windows builds, given that the build
environment will probably be cygwin-related (for that I can use the result
of AC_CANONICAL_BUILD) but the host may be something different given
that we produce three kinds of binaries: cygwin ones, those compiled
with MingW and those compiled for msvc. I'd need to make sure which host
to use for these three categories (for Cygwin binaries it will be
identical to --build, that's not a big deal), and this in both 32 and 64
bits mode.

Can anybody help with this particular point, please?

Post by Nick Bowler
The comment that "This will be fixed in the future" appears to have been
added in 2002. Clearly we've not made it to the future yet. :)

Well if there is no obviously better solution, it may indeed be worth
just removing this comment, as has been suggested.

Also, I really found your explanation enlightening and I think it
would have helped me to have it in the
manual so perhaps it could
be added.

Bet wishes and thanks a lot again,

Sébastien.

[Earnie]

Post by Sébastien Hinderer
One question I still have is about the possible values for the arguments
to --build & co. Is there a specification or an exhaustive list
somewhere? I know the general syntax but I'd like to figure out what
would be the best values for our Windows builds, given that the build
environment will probably be cygwin-related (for that I can use the result
of AC_CANONICAL_BUILD) but the host may be something different given
that we produce three kinds of binaries: cygwin ones, those compiled
with MingW and those compiled for msvc. I'd need to make sure which host
to use for these three categories (for Cygwin binaries it will be
identical to --build, that's not a big deal), and this in both 32 and 64
bits mode.

It's going to depend on what you want. If you're using the crosstools
for MinGW-w64 then you would specify --host as {i686,x86_64}-w64-mingw32
and --build as {i686,x86_64}-pc-cygwin. If you're using MSVC, there is
no crosstool chain of software so you specify a fictitious --host and
--build to be the same such as {i686,x86_64}-pc-msvc and point the
environment variables to the binaries. IIRC, The 64bit version of
Cygwin has a cross build for the 32bit version[1] so the --host would be
i686-pc-cygwin and config.guess can figure out the --build or you can
specify it as x86_64-pc-cygwin.

[1]
https://cygwin.com/cgi-bin2/package-grep.cgi?grep=i686-pc-cygwin&arch=x86_64

[Keith Marshall]

If you're using MSVC, there is no crosstool chain of software so you
specify a fictitious --host and --build to be the same such as
{i686,x86_64}-pc-msvc and point the environment variables to the
binaries.

But the OP seems to want to use AC_CANONICAL_*. IIRC, these demand that
the --build and --host, (and --target, if applicable), must be known to
config.sub; that kind of precludes just inventing an arbitrary name.

For --build, the output from config.guess should always be suitable; for
--host, it may be okay to affix an arbitrary suffix to mingw32, such as
(untested [*]) --host=mingw32cl CC=cl.exe, (and any other explicit tool
identification assignments which may be required).

You could possibly also use AS_IF, or AS_CASE, in configure.ac or a user
written macro included by configure.ac, to test $host_alias for your
specific modified host identifier, and make a conditional assignment of
CC=cl.exe and so on, within the generated configure script itself.

[*] Note that I've explicitly avoided "mingw32msvc" as a host identifier
in this example, because some GNU/Linux distributors have adopted that,
(inappropriately, IMO), to designate their mingw32 cross-compiler tool
chains.

[Peter Rosin]

Post by Sébastien Hinderer
Dear all,
I am writing an autoconf script for a project which is supposed to
compile under both Unix and Windows.
Under Windows, the project will be built under Cygwin but it should be
possible to use both MingW and cl.exe to compile it.
For the MingW case, giving thee right --host= option works just fine.
For the cl.exe case, though, I am not sure exactly what to do. Indeed
when I run
./configure CC=cl
Then configure detects the Microsoft C compiler properly. However the
host and target types will be cygwin as the build type. I am wondering
whether I should adjust host and target in case the C compiler is
detected to be cl.exe?
Are there known examples of projects that handle this situation properly
and where I could take inspiration from?
Many thanks in advance for any help / pointers!

If you add Automake to the mix, it can provide help with its "compile"
wrapper script which solves some issues. You need to add AM_PROG_CC_C_O
to your configure.ac to bring that in.

If you want to build DLLs, Libtool can do that using cl.exe

If you want to build plain old static archives, you use lib.exe for that
instead of falling back to MinGW ar. IIRC, you need to say AR=lib for that
though, and again, you need a wrapper from Automake (ar-lib, use AM_PROG_AR
to bring that in).

Libtool also knows how to use dumpbin instead of the MinGW nm, I think
you need to say NM="dumpbin -symbols" for that (or something like that).

So, the pieces are there, it's just not obvious how to put them together.
However, it's been a long time since I bothered. Something might have
died...

And of course, the source has to adapt to a lot of annoying crap.

Cheers,
Peter

[Sébastien Hinderer]

Dear Peter,

Many thanks for all the helpful informationn you provide!

I don't think we will be able to introduce automake and libtool at this
stage (there are even developers in the community who think we should
replace make by a language-specific build system), but it is still
very useful to know what's possible so let me thank you again for the hints
you have provided.

Best wishes,

Sébastien.

[Sébastien Hinderer]

Dear Earnie and Keith,

Many thanks for your support, warmly appreciated.

Post by Earnie
It's going to depend on what you want. If you're using the crosstools
for MinGW-w64 then you would specify --host as {i686,x86_64}-w64-mingw32
and --build as {i686,x86_64}-pc-cygwin. If you're using MSVC, there is
no crosstool chain of software so you specify a fictitious --host and
--build to be the same such as {i686,x86_64}-pc-msvc and point the
environment variables to the binaries. IIRC, The 64bit version of
Cygwin has a cross build for the 32bit version[1] so the --host would be
i686-pc-cygwin and config.guess can figure out the --build or you can
specify it as x86_64-pc-cygwin.
[1]
https://cygwin.com/cgi-bin2/package-grep.cgi?grep=i686-pc-cygwin&arch=x86_64
But the OP seems to want to use AC_CANONICAL_*. IIRC, these demand that
the --build and --host, (and --target, if applicable), must be known to
config.sub; that kind of precludes just inventing an arbitrary name.

yeah that's what I observe here indeed, I could'nt use "msvc" in the
argument of --host, for instance.

Post by Earnie
For --build, the output from config.guess should always be suitable; for
--host, it may be okay to affix an arbitrary suffix to mingw32, such as
(untested [*]) --host=mingw32cl CC=cl.exe, (and any other explicit tool
identification assignments which may be required).

But why would you include a "mingw" string in something that does not
use mingw at all? Isn't that a bit confising?

Post by Earnie
You could possibly also use AS_IF, or AS_CASE, in configure.ac or a user
written macro included by configure.ac, to test $host_alias for your
specific modified host identifier, and make a conditional assignment of
CC=cl.exe and so on, within the generated configure script itself.

Ah I quite like this idea, thanks! Let's see whether it's doable! :)

Post by Earnie
[*] Note that I've explicitly avoided "mingw32msvc" as a host identifier
in this example, because some GNU/Linux distributors have adopted that,
(inappropriately, IMO), to designate their mingw32 cross-compiler tool
chains.

The choice looks unfortunate to me, too.

Okay so as far as msvc is concerned, on 32bits I am able to run
configure with:

./configure --build=i686-pc-cygwin --host=i686-pc-windows CC=cl

(what matters here is --build and --host), the assignment to CC could
indeed go away if it becomes host-based.

For the 64 bits architecture, it seems the followingworks:

./configure --build=x86_64-unknown-cygwin --host=x86_64-pc-windows CC=cl

Do you guys think I can proceed with these values for build and host?

Thanks!

Sébastien.

[Keith Marshall]

Post by Sébastien Hinderer

Post by Keith Marshall
For --build, the output from config.guess should always be
suitable; for --host, it may be okay to affix an arbitrary
suffix to mingw32, such as (untested [*]) --host=mingw32cl
CC=cl.exe, (and any other explicit tool identification
assignments which may be required).

But why would you include a "mingw" string in something that does
not use mingw at all?

Because it's an OS name already known to config.sub, and because it's
an OS designator which is specific to native MS-Windows software
builds, (but see below). Also, "MinGW" is a contraction of
"Minimalist GNU for Windows", and you are using at least one GNU tool
(autoconf) in your build process.

Post by Sébastien Hinderer
Isn't that a bit confising?

Perhaps not to anyone who understands the "MinGW" concept, but again,
see below.

Post by Sébastien Hinderer

Post by Keith Marshall
You could possibly also use AS_IF, or AS_CASE, in configure.ac or
a user written macro included by configure.ac, to test
$host_alias for your specific modified host identifier, and make
a conditional assignment of CC=cl.exe and so on, within the
generated configure script itself.

Ah I quite like this idea, thanks! Let's see whether it's doable! :)

It is:

$ cat configure.ac
AC_INIT
AC_CANONICAL_SYSTEM

AS_CASE([$host_alias],[*mingw32cl],[CC=cl.exe])

AC_PROG_CC

$ autoconf
$ ./configure --build=`./config.guess` --host=i386-pc-mingw32cl
checking build system type... x86_64-unknown-linux-gnu
checking host system type... i386-pc-mingw32cl
checking target system type... i386-pc-mingw32cl
checking for i386-pc-mingw32cl-gcc... cl.exe
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... yes
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cl.exe accepts -g... yes
checking for cl.exe option to accept ISO C89... none needed

Note that I've run this on GNU/Linux, so cl.exe doesn't really exist;
I've kludged it as a hard link to gcc.

Post by Sébastien Hinderer

Post by Keith Marshall
[*] Note that I've explicitly avoided "mingw32msvc" as a host
identifier in this example, because some GNU/Linux distributors
have adopted that, (inappropriately, IMO), to designate their
mingw32 cross-compiler tool chains.

The choice looks unfortunate to me, too.

It's a really terrible idea. I believe that their idea is to indicate
that the generated code depends on MSVCRT.DLL, (which is distributed
with Windows itself -- not with MSVC); this would be credible, if they
called it "mingw32msvcrt", but they don't! "mingw32msvc" implies a
dependency on MSVC -- the Microsoft Visual-C compiler -- which is 100%
bullshit; there is no such dependency.

Post by Sébastien Hinderer
Okay so as far as msvc is concerned, on 32bits I am able to run
./configure --build=i686-pc-cygwin --host=i686-pc-windows CC=cl
(what matters here is --build and --host), the assignment to CC
could indeed go away if it becomes host-based.

Okay. I wasn't aware that config.sub already understands "windows",
(and indeed, it even understands an arbitrary suffix appended to
"windows", provided there's no hyphen within it). If you are happy to
interpret "--host=windows", (or any canonical triplet which ends with
"-windows"), to implicitly require cl.exe, then you could adapt:

AS_CASE([$host_alias],[*windows | *mingw*cl],[CC=cl.exe])

or otherwise, you could qualify it:

AS_CASE([$host_alias],[*windowsmsvc | *mingw*cl],[CC=cl.exe])

Post by Sébastien Hinderer
./configure --build=x86_64-unknown-cygwin --host=x86_64-pc-windows CC=cl

This should also work, with any of the preceding adaptations.

Post by Sébastien Hinderer
Do you guys think I can proceed with these values for build and host?

I don't see why not; you've nothing to lose by trying.

- --
Regards,
Keith.

Public key available from keys.gnupg.net
Key fingerprint: C19E C018 1547 DE50 E1D4 8F53 C0AD 36C6 347E 5A3F

[Sébastien Hinderer]

Post by Keith Marshall

Post by Sébastien Hinderer
But why would you include a "mingw" string in something that does
not use mingw at all?

Because it's an OS name already known to config.sub, and because it's
an OS designator which is specific to native MS-Windows software
builds, (but see below). Also, "MinGW" is a contraction of
"Minimalist GNU for Windows", and you are using at least one GNU tool
(autoconf) in your build process.

True. If I wanted to nitpick I'd say that since it's used only during
the build it would be okay to have it in the description of the build
environment but not in the one of the host environment, but if it has to
descibe the build *process* rather than the build environment, then you
are certainly right.

Post by Keith Marshall

Post by Sébastien Hinderer
Ah I quite like this idea, thanks! Let's see whether it's doable! :)

Yep I had no doubt actually, thanks for having provided the example
though!!

Post by Keith Marshall
Note that I've run this on GNU/Linux, so cl.exe doesn't really exist;
I've kludged it as a hard link to gcc.

Okay thanks for telling, the results looked a bit odd (e.g. name of
default output file).

Post by Keith Marshall
Okay. I wasn't aware that config.sub already understands "windows",
(and indeed, it even understands an arbitrary suffix appended to
"windows", provided there's no hyphen within it).

Am I correct that you know all this merely by reading config.sub, or is
there another documentation I am not aware of?

Post by Keith Marshall
If you are happy to
interpret "--host=windows", (or any canonical triplet which ends with
AS_CASE([$host_alias],[*windows | *mingw*cl],[CC=cl.exe])
AS_CASE([$host_alias],[*windowsmsvc | *mingw*cl],[CC=cl.exe])

Yeah, I think I am happy with this solution. At least as a starting
point. Why would you keep the two cases by the way? Wouldn't one be
enough?

I mean, am I correct that the mingw*cl could be removed?

Post by Keith Marshall

Post by Sébastien Hinderer
./configure --build=x86_64-unknown-cygwin --host=x86_64-pc-windows CC=cl

This should also work, with any of the preceding adaptations.

Post by Sébastien Hinderer
Do you guys think I can proceed with these values for build and host?

I don't see why not; you've nothing to lose by trying.

Indeed. Many thanks for having helped with this!

Best wishes,

Sébastien.