Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
spec:teststatus [2012/05/03 14:15]
dschulze created
spec:teststatus [2014/12/09 15:48] (current)
Line 2: Line 2:
  
 ===== Prolog ===== ===== Prolog =====
-Testing is an important part of writing specifications. It helps verifying the validity of specified features. But tests can also be used as examples for developers and serve implementers as verification tool of their own implementation. Therefore tests can be seen as the second, practical part of the specification,​ beside the written part. For this reasons every part of a specification must be tested. In general the intention is: as much tests as possible. See [[test:​|Testing]] for more details about writing tests. 
  
-The preferred way to write tests for the CSS WG, SVG WG and the FX task force are the so called ​[[test:​reftest|Writing Reftests]].+Testing is an important part of writing specifications. It helps verifying the validity of specified features. But tests can also be used as examples for developers and serve implementers as verification tool of their own implementation. Therefore tests can be seen as the second, practical part of the specification,​ beside the written part. For this reasons every part of a specification must be tested. In general the intention is: every testable assertion in a specification must have at least one test. See [[test:​|Testing]] for more details about writing tests. 
 + 
 +Tests for the CSS WG, SVG WG and the FX task force must be written to the [[test:​format|test format guidelines]]. Where the tested features can't be easily verified in script, the preferred form of test is a [[test:​reftest|reftest]].
  
 ===== Introduction ===== ===== Introduction =====
 +
 To support developers as well as specification authors, specifications should have detailed information about the testing status on each specific feature. To support developers as well as specification authors, specifications should have detailed information about the testing status on each specific feature.
  
Line 13: Line 15:
 For authors a rarely tested section indicates that there might be specification issues that are not solved yet. It can also mean that this section needs more clarification. For authors a rarely tested section indicates that there might be specification issues that are not solved yet. It can also mean that this section needs more clarification.
  
-For a better transparency ​these information should be prompted on the specification itself. And even better: To every assertable part of the specification. This article describes how to do that in the combination with the testing system ​[[http://​test.csswg.org/​shepherd/​|Shepherd]] for tests submitted to the CSS test suite.+For a better transparency ​this information should be prompted on the specification itself. And even better: To every assertable part of the specification. This article describes how to do that in the combination with the testing system ​infrastructure used by the CSS working group.
  
-===== Adding ​assertions ​to a specification ===== +===== Identifying ​assertions ​within ​a specification ===== 
-The first step is to add assertions ​into specificationAt the moment this is done by adding a unique identifier ​to every headline on the text with the '​id'​ attribute. It is good practice to use the same name on the id as for the title of the section. Example:+ 
 +Currently tests are mapped to the most relevant section of the specification. ​The CSS testing systems will soon be able to recognize testable ​assertions ​within ​narrower scope 
 + 
 +The first step is to ensure that all section headings within ​the specification have unique identifiers set to their '​id'​ attribute. It is good practice to use the same name on the id as for the title of the section. Example:
  
 <​code>​ <​code>​
Line 25: Line 30:
 </​code>​ </​code>​
  
-Ids helps [[http://​test.csswg.org/​shepherd/​|Shepherd]] to recognize ​assertions ​in a specification ​and map submitted tests to this assertion.+Second, all testable ​assertions ​within the specification should also have unique '​id'​ attribute on their smallest containing element. The paragraph or even sentence level is generally suitable. Non-testable areas of the specification ​must have one of the following class attributes: '​example',​ '​note',​ '​doc',​ '​no-toc',​ '​informative'​.
  
-===== Write tests ===== +===== Binding ​tests to the specification ​=====
-Obviously [[http://​test.csswg.org/​shepherd/​|Shepherd]] needs tests for the mapping to assertions. If not done so far, it is the best time to write tests :-).+
  
-===== Creating ​testing harness ===== +When writing the tests for specification, ​the test must contain one or more [[test:​format#​links|links]] ​to the area(s) ​of the specification ​being testedThe fragment identifier of the link should be the '​id'​ of the heading or containing element of the assertion(s) under test.
-Even if tests can be committed to the CSS test suite. It is a requirement to create a so called ​test harness per specification ​to embed information about the testing status. This test harness is a collection ​of all submitted tests to the CSS test suite per specification. ​All tests get sorted by the tested sections and subsections.+
  
-This test harness must be created ​by e.g Peter Linss (W3C). The testing harness ​will be synchronized with the CSS test suite once a dayExamples for existing ​test harnesses can be found here: [[http://​test.csswg.org/​harness/]].+===== Creating a test suite ===== 
 + 
 +Test suites are currently ​created ​manuallyThis will soon be a fully automated process. 
 + 
 +Once a sufficient number of tests have been submitted, [[mailto:​peter.linss@hp.com|email Peter]] to have the test suite generatedAt that point, the test suite will be built daily and available for testing in the [[http://​test.csswg.org/​harness|test harness]].
  
 ===== Embedding testing status into the specification ===== ===== Embedding testing status into the specification =====
-Once the test harness was created, the testing status information can be embedded into the specification. This is done by adding the following script to the header of the specification:​+ 
 +Once the test suite is created, the testing status information can be embedded into the specification. This is done by adding the following script to the header of the specification:​
  
 <​code>​ <​code>​
-<script src='​http://​test.csswg.org/​harness/​annotate.js#<​specification>' type='​text/​javascript'​ defer></​script>​+<script src='​http://​test.csswg.org/​harness/​annotate.js#<​test-suite>' type='​text/​javascript'​ defer></​script>​
 </​code>​ </​code>​
  
-Note that '<specification>' must be replaced by the name of the specification ​in the test harness. This is e.g '​CSS3-TRANSFORMS_DEV'​ for CSS3 Transforms.+Note that '<test-suite>' must be replaced by the identifier used by the testing harness for the test suite. This can be found in the URL of the testing pages within ​the test harness. This is e.g '​CSS3-TRANSFORMS_DEV'​ for CSS3 Transforms.
  
-After uploading the changes, the script ​should ​run immediately. If you don't see any testing information on the specification,​ that either means you used the wrong name for the specification, no tests were found, or the file name of the specification is unknown. For the last case ask Peter Linss for advice.+The script ​will run every time the specification in viewed and will inject test information directly into the specification. Every section heading that has tests will have an annotation containing links to the test in the harness and the current result state. If you don't see any testing information on the specification,​ that either means you used the wrong name for the test suite, no tests were found, or the test suite has not been set up yet. For the last case [[mailto:​peter.linss@hp.com|email ​Peter]] for advice.
  
 
spec/teststatus.txt · Last modified: 2014/12/09 15:48 (external edit)
Recent changes RSS feed Valid XHTML 1.0 Valid CSS Driven by DokuWiki