Using autotest with Emacs’ compilation mode

[Nikolai Weibull]

Hi!

A question to all you users of Autoconf and Emacs: How are you using
autotest with Emacs’ compilation mode?

The output of an autotest testsuite isn’t really suited for Emacs’
compilation mode, as the paths to tests are, at least when using a
separate build directory, not correct.

Also, how do you check the logs of failed tests effectively when a test fails?

I would like to handle all of this via compilation mode, but it seems
that one really needs to set up one’s own set of commands for doing
this instead, as autotest’s output doesn’t seem friendly to
compilation mode.

[Nikolai Weibull]

Hi!

Since there hasn’t been any responses yet, let me fill in the blanks,
as I see them.

Post by Nikolai Weibull
A question to all you users of Autoconf and Emacs: How are you using
autotest with Emacs’ compilation mode?
The output of an autotest testsuite isn’t really suited for Emacs’
compilation mode, as the paths to tests are, at least when using a
separate build directory, not correct.

Here’s a, most likely incomplete, list of issues with the testsuite
regarding paths:

1. The -C option doesn’t have a corresponding -w and --print-directory
(and --no-print-directory) option to output the directory it’s
changing to, which means that paths can’t be tracked by Emacs.

2. The path stored in $at_check_line_file is always relative to the
location of testsuite. It should be relative to $at_dir.

3. The path stored in $at_setup_line is always relative to the
location of testsuite. It should be relative to $at_dir.

4. The path stored in $at_group_log to the command being executed
should be relative to the location of $at_group_log. This is probably
hard to fix using relative directories, as the path is also output to
the terminal in verbose mode (via $at_tee_pipe) and in this case the
path should be relative to $at_srcdir (as it is today).

5. It would be useful if there was a way to include the path to the
log file in the output, perhaps enabled via an option, so that Emacs’
compilation mode could pick it up for next-error.

Post by Nikolai Weibull
Also, how do you check the logs of failed tests effectively when a test fails?

Here’s a list of issues with the log files that open up in compilation mode:

1. The log sets up compilation mode (via “-*- compilation -*-”), but
it doesn’t set default-directory, meaning that relative paths are
relative to the log instead of to the directory where compilation took
place.

2. The log doesn’t set compile-command, which could theoretically be
set to “run” to re-run the test. It might be necessary to set
compile-directory as well.

3. Executing recompile (g) from one of these log files will replace
the content of the log with the output of the compilation, as Emacs
will reuse the buffer. I’m not sure how you’d get Emacs to (re-)use
the “*compilation*” buffer, as compilation-buffer-name-function can’t
safely be set via a file-local variable, so this might have to be
fixed in Emacs.

[Nikolai Weibull]

Hi!

I don’t want to be a nag, but does anyone have any input on this? Why
has the testsuite output been designed this way?

Post by Nikolai Weibull
Since there hasn’t been any responses yet, let me fill in the blanks,
as I see them.

Post by Nikolai Weibull
A question to all you users of Autoconf and Emacs: How are you using
autotest with Emacs’ compilation mode?
The output of an autotest testsuite isn’t really suited for Emacs’
compilation mode, as the paths to tests are, at least when using a
separate build directory, not correct.

Here’s a, most likely incomplete, list of issues with the testsuite
1. The -C option doesn’t have a corresponding -w and --print-directory
(and --no-print-directory) option to output the directory it’s
changing to, which means that paths can’t be tracked by Emacs.
2. The path stored in $at_check_line_file is always relative to the
location of testsuite. It should be relative to $at_dir.
3. The path stored in $at_setup_line is always relative to the
location of testsuite. It should be relative to $at_dir.
4. The path stored in $at_group_log to the command being executed
should be relative to the location of $at_group_log. This is probably
hard to fix using relative directories, as the path is also output to
the terminal in verbose mode (via $at_tee_pipe) and in this case the
path should be relative to $at_srcdir (as it is today).
5. It would be useful if there was a way to include the path to the
log file in the output, perhaps enabled via an option, so that Emacs’
compilation mode could pick it up for next-error.

Post by Nikolai Weibull
Also, how do you check the logs of failed tests effectively when a test fails?

1. The log sets up compilation mode (via “-*- compilation -*-”), but
it doesn’t set default-directory, meaning that relative paths are
relative to the log instead of to the directory where compilation took
place.
2. The log doesn’t set compile-command, which could theoretically be
set to “run” to re-run the test. It might be necessary to set
compile-directory as well.
3. Executing recompile (g) from one of these log files will replace
the content of the log with the output of the compilation, as Emacs
will reuse the buffer. I’m not sure how you’d get Emacs to (re-)use
the “*compilation*” buffer, as compilation-buffer-name-function can’t
safely be set via a file-local variable, so this might have to be
fixed in Emacs.

[Eric Blake]

Hi!
I don’t want to be a nag, but does anyone have any input on this? Why
has the testsuite output been designed this way?

You're more than welcome to submit patches. The shortcomings in the
testsuite are likely due to being written by people that aren't familiar
with using emacs' compilation mode, so we didn't know any better - so
having patches on hand rather than just descriptions of what is wrong
will make it easier to fix.

Since there hasn’t been any responses yet, let me fill in the blanks,
as I see them.

Post by Nikolai Weibull
A question to all you users of Autoconf and Emacs: How are you using
autotest with Emacs’ compilation mode?
The output of an autotest testsuite isn’t really suited for Emacs’
compilation mode, as the paths to tests are, at least when using a
separate build directory, not correct.

Here’s a, most likely incomplete, list of issues with the testsuite
1. The -C option doesn’t have a corresponding -w and --print-directory
(and --no-print-directory) option to output the directory it’s
changing to, which means that paths can’t be tracked by Emacs.

Seems like it would be a useful addition.

2. The path stored in $at_check_line_file is always relative to the
location of testsuite. It should be relative to $at_dir.
3. The path stored in $at_setup_line is always relative to the
location of testsuite. It should be relative to $at_dir.

Seems like those would be sane bug fixes.

4. The path stored in $at_group_log to the command being executed
should be relative to the location of $at_group_log. This is probably
hard to fix using relative directories, as the path is also output to
the terminal in verbose mode (via $at_tee_pipe) and in this case the
path should be relative to $at_srcdir (as it is today).
5. It would be useful if there was a way to include the path to the
log file in the output, perhaps enabled via an option, so that Emacs’
compilation mode could pick it up for next-error.

May take more code to write, but seems like it would be okay if we have
a patch to evaluate.

Post by Nikolai Weibull
Also, how do you check the logs of failed tests effectively when a test fails?

1. The log sets up compilation mode (via “-*- compilation -*-”), but
it doesn’t set default-directory, meaning that relative paths are
relative to the log instead of to the directory where compilation took
place.
2. The log doesn’t set compile-command, which could theoretically be
set to “run” to re-run the test. It might be necessary to set
compile-directory as well.

How best to fix that? By adding more comment text into the log file?

3. Executing recompile (g) from one of these log files will replace
the content of the log with the output of the compilation, as Emacs
will reuse the buffer. I’m not sure how you’d get Emacs to (re-)use
the “*compilation*” buffer, as compilation-buffer-name-function can’t
safely be set via a file-local variable, so this might have to be
fixed in Emacs.