Documentation

Some documentation is in the "Testsuites" section in GNU Compiler Collection (GCC) Internals and at the Installing GCC: Testing webpage.

Getting Started

Here are the steps to get started with the gcc testsuite:

Download it from your favorite GNU mirror:

http://ftp.gnu.org/pub/gnu/gcc/gcc-4.2.1/gcc-testsuite-4.2.1.tar.bz2

Extract it.

Install DejaGnu

Install the testing software called DejaGnu -- available at pkgsrc/devel/dejagnu or your favorite GNU mirror. It is tcl-expect based tool.

The docs are online at http://www.gnu.org/software/dejagnu/manual/ (and manpage should be installed for "runtest").

runtest is the tool to use.

runtest basic syntax:

runtest --tool gcc --srcdir ./testsuite test/testfile.exp

Without "test/testfile.exp" it will run all tests for gcc*

--srcdir sets directory for finding tests under instead of cwd

--tool_exec to choose tool to test

--tool_opts to choose options for that tool

-v -v makes logging very verbose.

Available Tests

To see a list of tests, go to your extracted gcc-testsuite and go into gcc-4.2.1/gcc/testsuite.

find gcc* | grep .exp$

Try it

mkdir ~/pcc-testing

cd ~/pcc-testing

runtest --tool_exec pcc  --srcdir ~/tmp/gcc-4.2.1/gcc/testsuite \
        --tool gcc gcc.c-torture/compile/compile.exp

This will result in placing gcc.sum and gcc.log in your current directory.

Example Output

         === gcc Summary ===

# of expected passes            3688
# of unexpected failures        1478
# of unresolved testcases       24
# of unsupported tests          70

See the gcc.log for the details and errors and then look under the gcc-testsuite to see the source.

cpp Tests

runtest --tool_exec pcc  --srcdir ~/tmp/gcc-4.2.1/gcc/testsuite \
        --tool gcc gcc.dg/cpp/trad/trad.exp

(But I don't understand how the testing framework works for that.)