Implementation Report Template

The CSSWG has adopted a tab-separated file format for its implementation reports. It consists of a header with comments noting the UA and the test suite version and the first line of the data gives the column header names. Comments may be added to the report by prefixing the line with a pound sign (#).

Data columns

The three tab-separated data columns are:

testname
The first column gives the relative path from the root of the test suite to the test. This is auto-generated by the build scripts when generating the template.
result
The second column gives a result code. The template generates a ? for this field, which must be filled in by the tester. Valid result codes are:
code Meaning
pass The test passes
fail The test fails
uncertain The tester is uncertain whether the test passes or fails
invalid The tester believes the test is invalid.
na The test is not applicable: the test is for a different device or media type, or the feature is not implemented
comment
The third column is for per-result comments to be stored in the results database. (Comments prepended with a hash mark (#) may be discarded.)

Only the subset of tests actually tested need be reported.

The header consists of three comments denoting the UA and OS version, the UA string (if applicable), and the test suite version.

Example of the template header:

# UA version OS version
# UA string (if applicable)
# http://test.csswg.org/suites/css2.1/DATESTAMP/

Example of a completed report header:

# Firefox 4.0b5 Ubuntu Linux 10.10
# Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13
# http://test.csswg.org/suites/css2.1/20101210/

DATESTAMP might not be an actual date, for example https://test.csswg.org/suites/css-color-4_dev/nightly-unstable/html/lch-007.htm DATESTAMP=nightly-unstable

However, the upload form does not seem to use this information for the report table headers - you must first use the “Change” link where it says “You are about run tests for the following user agent” to set the user agent to, for example “PrinceXML” before uploading the test report.

 
test/implementation-report.txt · Last modified: 2021/02/18 05:17 by chrisl
Recent changes RSS feed Valid XHTML 1.0 Valid CSS Driven by DokuWiki