====== Reftests ======
A //reftest// is a test that compares the visual output of one file (the testcase) with the output of another file (the reference). Unlike the standard [[test:selftest|self-describing tests]], reftests can be scripted to run and report results automatically.
A test can be both a self-describing test and a reftest at the same time.
Here is an example of a reftest that is also a self-describing test: **[[http://test.csswg.org/source/contributors/apple/incoming/css-2d-transforms/rotate-90deg-001.xhtml|TEST]] & [[http://test.csswg.org/source/contributors/apple/incoming/css-2d-transforms/reftest/rotate-90deg-001-ref.xhtml|REF]]**
===== Components of a Reftest =====
A reftest has three parts:
; test file : The test file. This must follow the [[test:css2.1:format|CSS test format guidelines]].
; reference file : This is a different, usually simpler, file that results in the same rendering as the test.
; reftest comparison : A line in a reftest "manifest" file that says which two files are to be compared.
==== The Reftest Test ====
The test file uses the technology to be tested. This file must follow the [[test:css2.1:format|CSS test format guidelines]].
In addition to matching a reftest reference, the test may also function as an self-describing test. This is preferred because having the description lets the tester check that the test and the reference are not //both// being rendered incorrectly and triggering a false pass, and because designing it for an obvious fail makes it easier to find what went wrong when the reftest does fail.
==== The Reftest Reference ====
The reference file uses a different method to produce the same rendering as the test file. Multiple tests can (and often do) share the same reference file.
References should be named after the earliest test that uses them in the ''test-topic'' series they belong to, and have ''-ref'' appended before the file extension. Depending on the test suite, they may be placed in the ''reftest'' subfolder of the main test directory or directly in the main test directory.
For example, if two self-describing tests ''list-style-type-003.xht'' and ''list-style-type-004.xht'' share the same reference, that reference could be named ''reftest/list-style-type-003-ref.xht''.
=== Simplified Reference Format ===
Like the format for the test file, the reftest reference format is also XHTML in UTF-8 with bitmap images in PNG format. Unlike the format for the test file, there is no metadata except for the [[test:css2.1:format#credits|author credits]].
CSS Reftest Reference
CONTENT OF REFERENCE
=== Common References ===
There are several common references, such as those used for parsing and selectors tests. Their names begin with ''ref-'' so they can be easily found in the ''reftest'' directory. Email public-css-testsuite@w3.org if you would like to add to the common references collection.
==== The Reftest Manifest ====
The reftest manifest is a plain text file that lists test-reference pairs for comparison. Each line starts with ''=='' to indicate equality. (Mozilla has a more sophisticated manifest format: we are not using its advanced features here at W3C.) This is followed by a space, the relative path to the test file, another space, and the relative path to the reference.
White space followed by ''#'' indicates the start of a comment that runs until the end of the line. A line starting with ''#'' is also a comment.
The reftest manifest should be named ''reftest.list'' and placed in the ''reftest'' subfolder of the main test directory.
Here is an example of a reftest manifest.
# reftest.list snippet
== ../test-topic-000.xht test-topic-000-ref.xht
== ../test-topic-001.xht test-topic-001-ref.xht
== ../test-topic-002.xht test-topic-000-ref.xht # note same reference as test 000