This shows you the differences between two versions of the page.
— | test:format [2012/04/16 13:53] – [Test Assertions] fantasai | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Test Format ====== | ||
+ | This page describes the standard test format for CSSWG [[test: | ||
+ | |||
+ | The recommended structure for CSS tests is a [[selftest|self-describing]] [[reftest|reftest]]. If it's possible for a test to be a [[reftest|reftest]], | ||
+ | |||
+ | ===== Design Requirements ===== | ||
+ | |||
+ | The following are requested of all tests submitted to the CSSWG, both for [[test: | ||
+ | |||
+ | ; Short | ||
+ | : Tests should be very short and certainly not require scrolling on even the most modest of screens, unless the test is specifically for scrolling or paginating behaviour. This enables them to be run more easily on various testing platforms. | ||
+ | ; Valid | ||
+ | : Unless specifically testing error-recovery features, the tests should all be valid. Inconsistencies in e.g. HTML parsing shouldn' | ||
+ | ; Cross-platform | ||
+ | : Tests should be as cross-platform as reasonably possible, working across different devices, screen resolutions, | ||
+ | ; Red Means Failure | ||
+ | : Don't use the color red other than to indicate a failure. (Exception: testing for support of red) Since green-with-no-red is often used to indicate success, it's best to also avoid green unless using the presence of red to indicate failures. | ||
+ | |||
+ | ===== Acceptable Test Formats ===== | ||
+ | |||
+ | The preferred submission format for CSSWG tests is XHTML in UTF-8. HTML is also acceptable; however note that the test source will be parsed and reserialized. | ||
+ | |||
+ | Images must be in PNG format. | ||
+ | |||
+ | A set of scripts will generate the various formats (XHTML, HTML, XHTML for Printers) from this source version. | ||
+ | |||
+ | Tests must be //valid// XHTML, so please [[http:// | ||
+ | |||
+ | ===== Template for New Tests ===== | ||
+ | |||
+ | A template for new tests follows. Copy and paste the code below into a new file or save {{test: | ||
+ | |||
+ | <code html> | ||
+ | < | ||
+ | <html xmlns=" | ||
+ | < | ||
+ | < | ||
+ | <link rel=" | ||
+ | <link rel=" | ||
+ | <meta name=" | ||
+ | <meta name=" | ||
+ | <style type=" | ||
+ | CSS FOR TEST | ||
+ | ]]></ | ||
+ | </ | ||
+ | < | ||
+ | CONTENT OF TEST | ||
+ | </ | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | |||
+ | ===== Template Details ===== | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ==== Title element ==== | ||
+ | |||
+ | <code html> | ||
+ | < | ||
+ | </ | ||
+ | |||
+ | The title appears in the generated index, so make sure it is // | ||
+ | |||
+ | <note example> | ||
+ | Bad example: | ||
+ | <code html> | ||
+ | < | ||
+ | </ | ||
+ | We have 100+ tests on " | ||
+ | |||
+ | Good example: | ||
+ | <code html> | ||
+ | < | ||
+ | </ | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | ==== Credits ==== | ||
+ | |||
+ | <code html> | ||
+ | <link rel=" | ||
+ | </ | ||
+ | |||
+ | Credits provide a way to identify the person or organization that created the test and/or holds copyright in the test. This is useful for reviewing purposes and for asking questions about the individual test. A test can have multiple author credits if necessary. | ||
+ | |||
+ | <note example> | ||
+ | Example 1: | ||
+ | <code html> | ||
+ | <link rel=" | ||
+ | </ | ||
+ | Example 2: | ||
+ | <code html> | ||
+ | <link rel=" | ||
+ | </ | ||
+ | Example 3: | ||
+ | <code html> | ||
+ | <link rel=" | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | ==== Reviewer ==== | ||
+ | |||
+ | <code html> | ||
+ | <link rel=" | ||
+ | </ | ||
+ | |||
+ | If a test has passed review, then the reviewer should note this by adding his or her name as a reviewer, along with the date of the review. A test can have multiple reviewers if necessary. A reviewer must be a person, not an organization. | ||
+ | |||
+ | <note example> | ||
+ | Example 1: | ||
+ | <code html> | ||
+ | <link rel=" | ||
+ | </ | ||
+ | Example 2: | ||
+ | <code html> | ||
+ | <link rel=" | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | If a test would pass review with some (non-metadata) changes and the reviewer chooses to make these changes, then the reviewer should add his or her name as a reviewer-author, | ||
+ | |||
+ | <note example> | ||
+ | Example of a fully-reviewed test: | ||
+ | <code html> | ||
+ | <link rel=" | ||
+ | <link rel=" | ||
+ | <link rel=" | ||
+ | </ | ||
+ | This test was written by Bert Bos, then reviewed by Boris Zbarsky, who made some corrections before deeming it acceptable. Those corrections were then reviewed and accepted by Bert Bos. | ||
+ | </ | ||
+ | |||
+ | ==== Specification Links ==== | ||
+ | |||
+ | <code html> | ||
+ | <link rel=" | ||
+ | </ | ||
+ | |||
+ | The specification link elements provide a way to align test with information in the CSS 2.1 specification. | ||
+ | |||
+ | * Links should link to relevant sections within the CSS 2.1 Specification | ||
+ | * Use the anchors from the CSS 2.1 Specification [[http:// | ||
+ | * A test can have multiple specification links | ||
+ | * Always list the primary section that is being tested as the first item in the list of specification links | ||
+ | * Order the list from the most used/ | ||
+ | * There is no need to list common incidental features like the color green if it is being used to validate the test unless the case is specifically testing the color green | ||
+ | * If the test is part of multiple test suites, link to the relevant sections of each spec. | ||
+ | |||
+ | <note example> | ||
+ | Example 1: | ||
+ | <code html> | ||
+ | <link rel=" | ||
+ | </ | ||
+ | |||
+ | Example 2: | ||
+ | <code html> | ||
+ | <link rel=" | ||
+ | <link rel=" | ||
+ | <link rel=" | ||
+ | <link rel=" | ||
+ | </ | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | ==== Reference Links ==== | ||
+ | |||
+ | <code html> | ||
+ | <link rel=" | ||
+ | <link rel=" | ||
+ | </ | ||
+ | |||
+ | The reference link elements are used in [[.: | ||
+ | |||
+ | * '' | ||
+ | * Multiple '' | ||
+ | * If a test requires multiple '' | ||
+ | * '' | ||
+ | * Reference files may be dedicated reference files, images, or other tests | ||
+ | |||
+ | <note example> | ||
+ | Example 1: | ||
+ | <code html> | ||
+ | <link rel=" | ||
+ | </ | ||
+ | |||
+ | Example 2: | ||
+ | <code html> | ||
+ | <link rel=" | ||
+ | <link rel=" | ||
+ | <link rel=" | ||
+ | <link rel=" | ||
+ | </ | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | ==== Requirement Flags ==== | ||
+ | |||
+ | <code html> | ||
+ | <meta name=" | ||
+ | </ | ||
+ | |||
+ | Flags document the test’s prerequisites, | ||
+ | |||
+ | All flags documented elsewhere are deprecated except the following: | ||
+ | ^ Token ^ Description | ||
+ | | ahem | Requires the [[http:// | ||
+ | | animated | ||
+ | | combo | Test, which must have an unsuffixed filename number, is strictly the union of all the suffixed tests with the same name and number. (See File name format, below.) | | ||
+ | | dom | Requires support for JavaScript and the Document Object Model (DOM) | | ||
+ | | font | Requires a specific font to be installed. (Details must be provided and/or the font linked to in the test description) | | ||
+ | | history | ||
+ | | http | Requires HTTP headers | | ||
+ | | HTMLonly | ||
+ | | image | Requires support for bitmap graphics and the graphic to load | | ||
+ | | interact | ||
+ | | invalid | ||
+ | | namespace | ||
+ | | nonHTML | ||
+ | | may | Behavior tested is // | ||
+ | | paged | Only valid for paged media | | ||
+ | | should | ||
+ | | scroll | ||
+ | | svg | Requires support for vector graphics (SVG) | | ||
+ | | userstyle | ||
+ | | 32bit | Assumes a 32-bit integer as the minimum (-2147483648) or maximum (2147483647) value | | ||
+ | | 96dpi | Assumes 96dpi display | | ||
+ | <note example> | ||
+ | Example 1 (one token applies): | ||
+ | <code html> | ||
+ | <meta name=" | ||
+ | </ | ||
+ | |||
+ | Example 2 (multiple tokens apply): | ||
+ | <code html> | ||
+ | <meta name=" | ||
+ | </ | ||
+ | |||
+ | Example 3 (no tokens apply): | ||
+ | <code html> | ||
+ | <meta name=" | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||
+ | ==== Test Assertions ==== | ||
+ | |||
+ | <code html> | ||
+ | <meta name=" | ||
+ | </ | ||
+ | |||
+ | This element should contain a complete detailed statement expressing what specifically the test is attempting to prove. If the assertion is only valid in certain cases, those conditions should be described in the statement. | ||
+ | |||
+ | The assertion should //not// be: | ||
+ | * A copy of the title text | ||
+ | * A copy of the test verification instructions | ||
+ | * A duplicate of another assertion in the test suite | ||
+ | * A line or reference from the CSS specification unless that line is a complete assertion when taken out of context. | ||
+ | |||
+ | The test assertion is **optional**. It helps the reviewer understand the goal of the test so that he or she can make sure it is being tested correctly. Also, in case a problem is found with the test later, the testing method (e.g. using ' | ||
+ | |||
+ | < | ||
+ | Examples of good test assertions: | ||
+ | * "A background image with no intrinsic size covers the entire padding box." | ||
+ | * " | ||
+ | * "If ' | ||
+ | * " | ||
+ | </ | ||
+ | |||
+ | ==== Style Element (embedded styles) ==== | ||
+ | |||
+ | <code html> | ||
+ | <style type=" | ||
+ | CSS FOR TEST | ||
+ | ]]></ | ||
+ | </ | ||
+ | |||
+ | When creating styles primarily use ID or Class selectors. Inline styles should not be used unless the case is specifically testing this scenario. | ||
+ | |||
+ | |||
+ | ==== Script type (embedded script) ==== | ||
+ | |||
+ | <code html> | ||
+ | <script type=" | ||
+ | ... Javascript code here ... | ||
+ | ]]></ | ||
+ | </ | ||
+ | |||
+ | Some testcases require support for javascript and the Document Object Model (DOM). | ||
+ | |||
+ | Although type=" | ||
+ | |||
+ | |||
+ | ==== '' | ||
+ | |||
+ | <code html> | ||
+ | < | ||
+ | | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | * When creating content use : ''< | ||
+ | * Beware! ''< | ||
+ | * Use other elements only to test the interaction with those elements' | ||
+ | * Test '' | ||
+ | * Do not use the '' | ||
+ | * Avoid making assumptions that are not in the [[http:// | ||
+ | * It's helpful to people trying to understand the test if you use meaningful class and ID names and escape invisible characters like no-break-spaces. | ||
+ | * Indent nicely! | ||
+ | |||
+ | Guidelines on designing the content of the test can be found in the [[http:// | ||
+ | |||
+ | ===== File name format ===== | ||
+ | |||
+ | The new file name format is '' | ||
+ | ''###'' | ||
+ | |||
+ | The file name is no longer restricted to 31 characters, but please try to keep them short. | ||
+ | |||
+ | **test-topic**\\ | ||
+ | A short identifier that describes the test. The '' | ||
+ | conjunctions, | ||
+ | concise as possible. | ||
+ | |||
+ | <note example> | ||
+ | Examples:\\ | ||
+ | '' | ||
+ | border-solid-### | ||
+ | float-clear-### | ||
+ | </ | ||
+ | |||
+ | **###**\\ | ||
+ | This is a zero-filled number used to keep the file names unique when files have the same test-topic name. | ||
+ | |||
+ | Note: The number format is limited to 999 cases. If you go over this number it is recommended that you reevaluate your test-topic name. | ||
+ | |||
+ | <note example> | ||
+ | For example, in the case of margin-collapsing there are multiple cases so each case could have the same test-topic but different numbers.\\ | ||
+ | '' | ||
+ | margin-collapsing-002.xht\\ | ||
+ | margin-collapsing-003.xht'' | ||
+ | </ | ||
+ | |||
+ | There may also be a letter affixed after the number, which can be used to indicate variants of a test. | ||
+ | |||
+ | <note example> | ||
+ | For example, '' | ||
+ | </ | ||
+ | |||
+ | If tests using both the unsuffixed number and the suffixed number exist, the suffixed tests must be subsets of the unsuffixed test. | ||
+ | |||
+ | <note example> | ||
+ | For example, if '' | ||
+ | </ | ||
+ | |||
+ | If the unsuffixed test is strictly the union of the suffixed tests, i.e. covers all aspects of the suffixed tests (such that a user agent passing the unsuffixed test will, by design, pass all the suffixed tests), then the unsuffixed test should be marked with the '' | ||
+ | |||
+ | <note example> | ||
+ | If '' | ||
+ | </ | ||
+ | |||
+ | **ext**\\ | ||
+ | The file extension or format of the file, usually '' | ||
+ | |||
+ | ===== Support files ===== | ||
+ | |||
+ | A number of standard images are provided in the [[http:// | ||
+ | * 1x1 color swatches | ||
+ | * 15x15 color swatches | ||
+ | * 15x15 bordered color swatches | ||
+ | * assorted rulers and red/green grids | ||
+ | * a cat | ||
+ | * a 4-part picture | ||
+ | |||
+ | Additional generic files can be added as necessary, and should have a descriptive file name. Test-specific files should be named after the test (or test-topic if they are shared across several tests within a series). If possible tests should not rely on transparency in images, as they are converted to JPEG (which does not support transparency) for the xhtml1print version. | ||
+ | |||
+ | |||
+ | ===== User style sheets ===== | ||
+ | |||
+ | Some test may require special user style sheets to be applied in order for the case to be verified. | ||
+ | |||
+ | In order for proper indications and prerequisite to be displayed every user style sheet should contain the following rules. | ||
+ | |||
+ | <code css> | ||
+ | # | ||
+ | { | ||
+ | /* Used by the harness to display and indication there is a user style sheet applied */ | ||
+ | display: block!important; | ||
+ | } | ||
+ | </ | ||
+ | The rule ''# | ||
+ | |||
+ | A harness should identify test that need a user style sheet by looking at their flags meta tag. It then should display appropriate messages indicating if a style sheet is applied or if a style sheet should not be applied. | ||
+ | <note example> | ||
+ | Harness style sheet rules: | ||
+ | <code css> | ||
+ | #userstyle | ||
+ | { | ||
+ | color: green; | ||
+ | display: none; | ||
+ | } | ||
+ | # | ||
+ | { | ||
+ | color: red; | ||
+ | display: none; | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | Harness '' | ||
+ | <code html> | ||
+ | <p id=" | ||
+ | </ | ||
+ | |||
+ | Harness '' | ||
+ | <code html> | ||
+ | <p id=" | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | |||
+ | Within the test case it is recommended that the case itself indicate the necessary user style sheet that is required. | ||
+ | <note example> | ||
+ | Examples: | ||
+ | <code css> | ||
+ | #cascade /* ID name should match user style sheet file name */ | ||
+ | { | ||
+ | /* Used by the test to hide the prerequisite */ | ||
+ | display: none; | ||
+ | } | ||
+ | </ | ||
+ | The rule ''# | ||
+ | |||
+ | |||
+ | Examples: | ||
+ | <code html> | ||
+ | <p id=" | ||
+ | </ | ||
+ | The '' | ||
+ | </ | ||
+ | |||
+ | Please flag test that require user style sheets with the '' | ||
+ | |||
+ | ===== HTTP headers ===== | ||
+ | |||
+ | Some tests may require special HTTP headers. These should be configured in a '' | ||
+ | |||
+ | <Files ~ " | ||
+ | AddLanguage fr .xht .xhtml .xml .html .htm | ||
+ | </ | ||
+ | |||
+ | |||
+ | Please flag tests that require HTTP interaction with the '' |