Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
Last revisionBoth sides next revision
test:css2.1:review [2012/06/16 11:26] – [Review Checklist] fantasaitest:review [2013/11/11 19:21] – added deprecation note & redirect to testthewebforward.org rhauck
Line 1: Line 1:
 +<html>
 +<strong>
 +<div style="color: red; font-size: 20px; border: 2px solid red; padding: 10px; line-height: 1.5; text-align: center;">
 +This page has been deprecated and is no longer being maintained. 
 +<br>For the current review policy and procedure, see: 
 +<br><a href="http://testthewebforward.org/docs/review-process.html">http://testthewebforward.org/docs/review-process.html</a>
 +<br><a href="http://testthewebforward.org/docs/review-checklist.html">http://testthewebforward.org/docs/review-checklist.html</a>
 +</strong>
 +</div>
 +</html>
 +
 ====== Reviewing Tests ====== ====== Reviewing Tests ======
  
Line 5: Line 16:
 Anyone with the ability to read and understand the [[http://www.w3.org/Style/CSS/Specs/|spec]] and a thorough understanding of the [[http://www.w3.org/Style/CSS/Test/guidelines.html|test suite guidelines]] can review additions and changes to the Approved collection. However a reviewer cannot review his/her own tests and changes. Anyone with the ability to read and understand the [[http://www.w3.org/Style/CSS/Specs/|spec]] and a thorough understanding of the [[http://www.w3.org/Style/CSS/Test/guidelines.html|test suite guidelines]] can review additions and changes to the Approved collection. However a reviewer cannot review his/her own tests and changes.
  
-Tests pending approval are linked from the [[test:css2.1:pending|pending review]] page.+Tests pending approval are linked from the [[.:css2.1:pending|pending review]] page.
  
 ===== Process ==== ===== Process ====
  
-  - The reviewer must check each test submitted for approval for conformance to the test suite [[test:format|format]] and [[http://www.w3.org/Style/CSS/Test/guidelines.html|guidelines]] and for correctness with respect to the [[http://www.w3.org/TR/CSS21/|specification]]. For proposed changes to existing approved tests, the reviewer must check the changes for correctness. +  - The reviewer must check each test submitted for approval for conformance to the test suite [[format|format]] and [[http://www.w3.org/Style/CSS/Test/guidelines.html|guidelines]] and for correctness with respect to the [[http://www.w3.org/TR/CSS21/|specification]]. For proposed changes to existing approved tests, the reviewer must check the changes for correctness. 
-  - If the test does not pass review, the reviewer must tell the submitter what is wrong with the test and what steps should be taken to correct any problems.  Typically the submitter is responsible for fixing the tests, but the reviewer may make the changes directly and then explain to the submitter what they were (possibly by pointing to the change log) and why they were necessary. If these were not metadata-only changes and they are sufficient for the reviewer to consider the test acceptable, then the reviewer should [[test:format#reviewer|add himself as reviewer-author]], and the submitter (or, in the case of abandoned tests, another reviewer) needs to review and accept these changes.+  - If the test does not pass review, the reviewer must tell the submitter what is wrong with the test and what steps should be taken to correct any problems.  Typically the submitter is responsible for fixing the tests, but the reviewer may make the changes directly and then explain to the submitter what they were (possibly by pointing to the change log) and why they were necessary. If these were not metadata-only changes and they are sufficient for the reviewer to consider the test acceptable, then the reviewer should [[format#reviewer|add himself as reviewer-author]], and the submitter (or, in the case of abandoned tests, another reviewer) needs to review and accept these changes.
        * New tests and changes that have been declined by one reviewer cannot be accepted by someone else until the problems are corrected or have been demonstrated to be invalid. In case of a conflict, an Owner or Peer's opinion carries more weight. In all cases the CSS Working Group has final say in whether a test is valid, and may be consulted if deemed necessary.        * New tests and changes that have been declined by one reviewer cannot be accepted by someone else until the problems are corrected or have been demonstrated to be invalid. In case of a conflict, an Owner or Peer's opinion carries more weight. In all cases the CSS Working Group has final say in whether a test is valid, and may be consulted if deemed necessary.
-  - Once test/changes pass review, the reviewer must [[test:format#reviewer|note their acceptance in the test]]. The test is now "Accepted". At this point an Owner or Peer needs to approve the test. +  - Once test/changes pass review, the reviewer must [[format#reviewer|note their acceptance in the test]]. The test is now "Accepted". At this point an Owner or Peer needs to approve the test. 
-  - The [[test:oversight|Owner or Peer]] can either approve the reviewer's judgement if the reviewer is known to be competent in this area, or review the test himself. Once the Owner or Peer is satisfied the test is "Approved". (If the initial reviewer was an Owner or Peer then this step is automatic).+  - The [[oversight|Owner or Peer]] can either approve the reviewer's judgement if the reviewer is known to be competent in this area, or review the test himself. Once the Owner or Peer is satisfied the test is "Approved". (If the initial reviewer was an Owner or Peer then this step is automatic).
   - New tests and changes that have been accepted and approved can be moved to the ''approved'' directory by anyone with write access, and that person is not required to perform any further review. The checkin comment should indicate the approver by either "r=approver" if the approver performed a review or "r=reviewer rs=approver" if the approver rubber-stamped the previous reviewer's judgement.   - New tests and changes that have been accepted and approved can be moved to the ''approved'' directory by anyone with write access, and that person is not required to perform any further review. The checkin comment should indicate the approver by either "r=approver" if the approver performed a review or "r=reviewer rs=approver" if the approver rubber-stamped the previous reviewer's judgement.
  
Line 22: Line 33:
 Changes to the build scripts must be reviewed by a test suite Owner. Changes to the build scripts must be reviewed by a test suite Owner.
  
-To request review, submit proposed tests and changes as described in the [[test:css2.1:contribute|contribution guidelines]].+To request review, submit proposed tests and changes as described in the [[.:css2.1:contribute|contribution guidelines]].
  
 If these requirements change, notice will be sent to public-css-testsuite of the changes. If these requirements change, notice will be sent to public-css-testsuite of the changes.
Line 28: Line 39:
 ===== Review Checklist ===== ===== Review Checklist =====
  
-When reviewing a test, you're responsible for making sure the test matches the [[test:format|test format]] and [[http://www.w3.org/Style/CSS/Test/guidelines.html|testing guidelines]]. In addition to all the format and validity requirements in those documents, make sure you also check the following:+When reviewing a test, you're responsible for making sure the test matches the [[format|test format]] and [[http://www.w3.org/Style/CSS/Test/guidelines.html|testing guidelines]]. In addition to all the format and validity requirements in those documents, make sure you also check the following:
  
   * That the test is testing what it thinks it's testing. (I've run across many tests that think they're testing something, but are actually testing something else.)   * That the test is testing what it thinks it's testing. (I've run across many tests that think they're testing something, but are actually testing something else.)
Line 37: Line 48:
   * That the test is as cross-platform as reasonably possible, working across different devices, screen resolutions, paper sizes, etc. If there are limitations (e.g. the test will only work on 96dpi devices, or screens wider than 200 pixels) that these are documented in the instructions.   * That the test is as cross-platform as reasonably possible, working across different devices, screen resolutions, paper sizes, etc. If there are limitations (e.g. the test will only work on 96dpi devices, or screens wider than 200 pixels) that these are documented in the instructions.
   * That the spec backs up the expected behavior in the test. (I've run into a number of tests that make assumptions I could've sworn were in the spec, but aren't there when I go and check. Since this often means the spec forgot to handle something, you should send a message to [[http://lists.w3.org/Archives/Public/www-style/|www-style]] about it.)   * That the spec backs up the expected behavior in the test. (I've run into a number of tests that make assumptions I could've sworn were in the spec, but aren't there when I go and check. Since this often means the spec forgot to handle something, you should send a message to [[http://lists.w3.org/Archives/Public/www-style/|www-style]] about it.)
-  * That the test is a [[test:reftest|reftest]] (unless there's a very good reason why the test can not use a reference), preferably self-describing+  * That the test is a [[reftest|reftest]] (unless there's a very good reason why the test can not use a reference), preferably self-describing.
- +
-A [[test:css2.1:review-checklist|more detailed review checklist]] is available.+
  
 +A [[.:css2.1:review-checklist|more detailed review checklist]] is available.
 
test/review.txt · Last modified: 2014/12/09 15:48 by 127.0.0.1
Recent changes RSS feed Valid XHTML 1.0 Valid CSS Driven by DokuWiki