meaning autotest exit code 13

[djien]

I generate test.at to a testsuite script using autom4te (v.2.68) and execute
this test. It fails with a return code 13.
Please inform me whether there is a list of return-codes which explains this
error code.

Test is simple: it compares output Hello, world

Thank you for your effort


t.at
#AT_PACKAGE_STRING
#AT_PACKAGE_BUGREPORT
m4_define([AT_PACKAGE_STRING], [hello])
m4_define([AT_PACKAGE_BUGREPORT], [gnuprog2-***@sourceforge.org])
AT_INIT([hello])
AT_SETUP([hello])
AT_TESTED(hello)
AT_CHECK([hello],['Hello, world'])
AT_CLEANUP

t.log
## ---------------------- ##
## Detailed failed tests. ##
## ---------------------- ##

# -*- compilation -*-
1. testsuite.at:6: testing hello ...
++ set +x
/testsuite.at:8: hello
++ hello
--- /dev/null 2013-03-05 22:33:26.850202646 +0100
+++
/media/wd40/TODOs/work/work_20130306/4a_autotest/t.dir/at-groups/1/stdout
2013-03-06 09:36:49.099438123 +0100
@@ -0,0 +1 @@
+Hello, world
/testsuite.at:8: exit code was 13, expected Hello, world
1. testsuite.at:6: 1. hello (testsuite.at:6): FAILED (testsuite.at:8)



--
View this message in context: http://gnu-autoconf.7623.n7.nabble.com/meaning-autotest-exit-code-13-tp18573.html
Sent from the Gnu - Autoconf - General mailing list archive at Nabble.com.

[Eric Blake]

Post by djien
I generate test.at to a testsuite script using autom4te (v.2.68) and execute
this test. It fails with a return code 13.
Please inform me whether there is a list of return-codes which explains this
error code.
Test is simple: it compares output Hello, world
Thank you for your effort
t.at
#AT_PACKAGE_STRING
#AT_PACKAGE_BUGREPORT
m4_define([AT_PACKAGE_STRING], [hello])
AT_INIT([hello])
AT_SETUP([hello])
AT_TESTED(hello)
AT_CHECK([hello],['Hello, world'])

This is a usage error. This line says to execute 'hello', and expect $?
(which will always be numeric) to contain the string 'Hello, world'.
Perhaps we can enhance the AT_CHECK macro to sanity check the second
argument (omitted, literal 'ignore', a numeric value, or a shell
substitution); but remember that it is impossible to sanity check that
all shell substitutions would eventually resolve into a numeric value.

Post by djien
@@ -0,0 +1 @@
+Hello, world
/testsuite.at:8: exit code was 13, expected Hello, world
1. testsuite.at:6: 1. hello (testsuite.at:6): FAILED (testsuite.at:8)

Meanwhile, this says that when the testsuite executed 'hello', the exit
status was 13. But autotest did not write the 'hello' program, so
that's something your package presumably provided. The testsuite itself
exited with status 1 (a test failed), not status 13. I have no idea
what hello's status 13 means.

I suspect you meant to test that 'hello' prints 'Hello, world\n' to
stdout, and exits with status 0; if so, write:

AT_CHECK([hello], [], [[Hello, world
]])

Or, if it is normal for 'hello' to have non-zero status, change the
second argument to [13] to match an exact status, or to [ignore] to pass
the test in spite of the exit status.

[djien]

Thank you Eric for the explanation. Let me explain what i have done:

step 1: the hello.c code
include <stdio.h>

main( int argc, char *argv[] )
{
printf( "Hello, world\n" );
}

step2: compile gcc hello.c -o hello

step3: t17.at
# Process with autom4te to create an -*- Autotest -*- test suite.

m4_define([AT_PACKAGE_STRING], [GNU Programming])
m4_define([AT_PACKAGE_BUGREPORT], [gnuprog2-***@sourceforge.org])

# AT_PACKAGE_STRING([Hello],[v.1.0])
# AT_PACKAGE_BUGREPORT([***@gnu.org])
AT_INIT([hello])
AT_SETUP([hello])
AT_TESTED([hello])
AT_CHECK([hello],[],[[Hello, world
]])
AT_CLEANUP()

step 4: execute hello
$ ./hello
Hello, world
$

step 5:
autom4te -l autotest t17.at -o t17_hello

step 6: t17_hello script
update PATH with the current working dir

## ------------- ##
## Actual tests. ##
## ------------- ##
#AT_START_1
..
{ set +x
$as_echo "$at_srcdir/t17.at:11: hello"
at_fn_check_prepare_trace "t17.at:11"
( $at_check_trace; hello
) >>"$at_stdout" 2>>"$at_stderr"
at_status=$? at_failed=false
$at_check_filter
at_fn_diff_devnull "$at_stderr" || at_failed=:
echo >>"$at_stdout"; $as_echo "Hello, world
" | \
$at_diff - "$at_stdout" || at_failed=:
at_fn_check_status 0 $at_status "$at_srcdir/t17.at:11"
$at_failed && at_fn_log_failure
$at_traceon; }

..
#AT_STOP_1

step 7: $ ./t17_hello -x -v

step 8: t17_hello.log

# -*- compilation -*-
1. t17.at:9: testing hello ...
++ set +x
/t17.at:11: hello
++ hello
/t17.at:11: exit code was 13, expected 0
1. t17.at:9: 1. hello (t17.at:9): FAILED (t17.at:11)

step 9: modift t17.at
AT_CHECK([hello],[],[["Hello, world
"]])

step 10: and do the same steps as above:
t17_hello script:

echo >>"$at_stdout"; $as_echo "\"Hello, world
\"" | \

step 10: t17_hello.log
1. t18.at:9: testing hello ...
++ set +x
/t18.at:11: hello
++ hello
--- - 2013-03-07 14:01:44.798854464 +0100
+++
/media/wd40/TODOs/work/work_20130306/autotest/trial2b/t18_hello.dir/at-groups/1/stdout2013-03-07
14:01:44.791793357 +0100
@@ -1,2 +1,2 @@
-"Hello, world
-"
+Hello, world
+
/t18.at:11: exit code was 13, expected 0
1. t18.at:9: FAILED (t18.at:11)

Please inform me where the error is.
Thank you





--
View this message in context: http://gnu-autoconf.7623.n7.nabble.com/meaning-autotest-exit-code-13-tp18573p18576.html
Sent from the Gnu - Autoconf - General mailing list archive at Nabble.com.

[Eric Blake]

Post by djien
step 1: the hello.c code
include <stdio.h>
main( int argc, char *argv[] )
{
printf( "Hello, world\n" );
}

Where's your return statement? You forgot one, so your C compiler
assumed that you wanted the return value of printf() to become your
return value of your program. You printed 13 bytes, so printf()
returned 13, hence, your 'hello' exits with status 13, with your choice
of compilers (not all compilers would do the same, though, because your
program uses unspecified behavior). Fix your C program to avoid
unspecified behavior if you want a reliable exit status from 'hello'.

Post by djien
/t17.at:11: hello
++ hello
/t17.at:11: exit code was 13, expected 0
1. t17.at:9: 1. hello (t17.at:9): FAILED (t17.at:11)

Your testsuite is telling you that 'hello' failed to meet the
expectations you had in the testsuite of it exiting with status 0.

Post by djien
step 9: modift t17.at
AT_CHECK([hello],[],[["Hello, world
"]])

This modification says that you want your 'hello' program to output
double quotes as part of its literal output - but that's not what your C
program did.

Post by djien
@@ -1,2 +1,2 @@
-"Hello, world
-"
+Hello, world
+

So now the testsuite is telling you that your 'hello' program produced
different output than your expectation,

Post by djien
/t18.at:11: exit code was 13, expected 0
1. t18.at:9: FAILED (t18.at:11)

in addition to still having the wrong exit status for your expectation.

Post by djien
Please inform me where the error is.

In your .c file, or in your testsuite expectations.

[djien]

Thank you Eric. Return code in the sourcecode: main() is mandatory for this
example.



--
View this message in context: http://gnu-autoconf.7623.n7.nabble.com/meaning-autotest-exit-code-13-tp18573p18578.html
Sent from the Gnu - Autoconf - General mailing list archive at Nabble.com.