Documentation
Some documentation is in the "Testsuites" section in GNU Compiler Collection (GCC) Internals and at the Installing GCC: Testing webpage.
Getting Started
Here are the steps to get started with the gcc testsuite:
Download it from your favorite GNU mirror:
http://ftp.gnu.org/pub/gnu/gcc/gcc-4.2.1/gcc-testsuite-4.2.1.tar.bz2
Extract it.
Install DejaGnu
Install the testing software called DejaGnu -- available at pkgsrc/devel/dejagnu or your favorite GNU mirror. It is tcl-expect based tool.
The docs are online at http://www.gnu.org/software/dejagnu/manual/ (and manpage should be installed for "runtest").
runtest is the tool to use.
runtest basic syntax:
runtest --tool gcc --srcdir ./testsuite test/testfile.exp
Without "test/testfile.exp" it will run all tests for gcc*
--srcdir sets directory for finding tests under instead of cwd
--tool_exec to choose tool to test
--tool_opts to choose options for that tool
-v -v makes logging very verbose.
Available Tests
To see a list of tests, go to your extracted gcc-testsuite and go into gcc-4.2.1/gcc/testsuite.
find gcc* | grep .exp$
Try it
mkdir ~/pcc-testing
cd ~/pcc-testing
runtest --tool_exec pcc --srcdir ~/tmp/gcc-4.2.1/gcc/testsuite \
--tool gcc gcc.c-torture/compile/compile.exp
This will result in placing gcc.sum and gcc.log in your current directory.
Example Output
=== gcc Summary ===
# of expected passes 3688
# of unexpected failures 1478
# of unresolved testcases 24
# of unsupported tests 70
See the gcc.log for the details and errors and then look under the gcc-testsuite to see the source.
cpp Tests
runtest --tool_exec pcc --srcdir ~/tmp/gcc-4.2.1/gcc/testsuite \
--tool gcc gcc.dg/cpp/trad/trad.exp
(But I don't understand how the testing framework works for that.)