A testsuite has been set up, derived from the examples in the draft specification. The testsuite contains 10 end-to-end tests. A build target, `make check`, has been constructed that renders each test with the `sem-md.lua` filter enabled, and compares the result against expected output using diff. 7 of the 10 tests currently fail. These test failures are examined closer below, and summarized at the end of the chapter. ## Wrongly removes detached annotation The test `block_scope` contains this in Semantic Markdown: ```markdown You can give me a ring. {=<#manu>} You can give me a ring {=<#manu>}. ``` Expected output as Markdown is this: ```markdown You can give me a ring. You can give me a ring {=<#manu>}. ``` Output from the test command `pandoc --from commonmark --to commonmark --lua-filter sem-md/sem-md.lua --wrap preserve < test/block_scope.md` is instead this: ```markdown You can give me a ring. You can give me a ring . ``` A more concise view is the following unified diff, where minus-prefixed lines are expected output differing from plus-prefixed actual output: ```diff --- test/block_scope.md.md +++ /dev/stdin @@ -1,3 +1,3 @@ You can give me a ring. -You can give me a ring {=<#manu>}. +You can give me a ring . ``` The filter wrongly removes a non-annotation: Only when text enclosed with braces are either immediately followed by an explicit inline enclosure (linked, emphasized, strongly emphasized, underlined or bracketed) or is the very first or very last inline content of a block, should it be considered an annotation. This is a violation of the Semantic Markdown specification, and also one of the rules defined for this project that "rendering of syntactically incorrect [...] annotations [...] **should** include the annotation as regular text" (see @sec-spirit). ## Fails to remove image annotation The test `ex_paragraph` contains this in Semantic Markdown: ```markdown My name is [Manu Sporny]{:name} and you can give me a ring via [1-800-555-0199]{:telephone}. ![](http://manu.sporny.org/images/manu.png){:image} My favorite animal is the [Liger]{ov:preferredAnimal}. {=<#manu> .:Person} {schema}: @default {ov}: http://open.vocab.org/terms/ ``` Unified diff between expected and actual output: ```diff --- test/ex_paragraph.md.md +++ /dev/stdin @@ -2,5 +2,5 @@ Manu Sporny and you can give me a ring via 1-800-555-0199. -![](http://manu.sporny.org/images/manu.png) +![](http://manu.sporny.org/images/manu.png){:image} My favorite animal is the Liger. ``` The annotation, `{:image}`, for the inline linked image is wrongly kept. This is a violation of the Semantic Markdown specification, and also one of the rules defined for this project that "A rendering of syntactically correct annotations [...] **must** be functionally equal to the text authored without annotations" (see @sec-spirit). ## Fails to remove annotation-only block {#sec-test-annotation-block} The test `block-siblings_scope_list` contains this in Semantic Markdown: ```markdown # People ## {.schema:Person} ## Manu Sporny My name is Manu Sporny. ## Thomas Francart My name is Thomas Francart. # Animals ``` Unified diff between expected and actual output: ```diff --- test/block-siblings_scope.md.md +++ /dev/stdin @@ -1,5 +1,7 @@ # People +## + ## Manu Sporny My name is Manu Sporny. ``` A `Header` block containing only an annotation has had the annotation removed inline, but the block -- which after removal is virtually empty (contains only a space) -- is kept. Tests `block-siblings_scope_list` and `ex_title_and_list` fail due to similar issue but for other block elemet types. These test failures reveal violations of the Semantic Markdown specification, but parts of the specifications that was not planned to be implemented with this project, because introducing logic that adds or removes blocks (not only inlines) arguably is too large a deviation of core Markdown. This type of error is therefore a bug in the testsuite rather than in the codebase. ## Text after link definition annotation wrongly removed The test `link_definition_scope` contains this: ```markdown My name is [Manu Sporny] and you can give me a ring via [1-800-555-0199]. [Manu Sporny]: {schema:name} [1-800-555-0199]: tel:+1-800-555-0199 "make a phone call to Manu Sporny" {schema:telephone} ``` Unified diff between expected and actual output: ```diff --- test/link_definition_scope.md.md +++ /dev/stdin @@ -1,4 +1,4 @@ My name is Manu Sporny and you can give me a ring via -[1-800-555-0199](tel:+1-800-555-0199 "make a phone call to Manu Sporny"). +[1-800-555-0199](tel:+1-800-555-0199 "make a phone call to Manu Sporny") ``` A link annotation from a link definition is correctly removed, but trailing text (a dot) is wrongly removed as well. ## Fails to remove annotation and repeats word before non-annotated The test `span_scope` contains this: ```markdown My name is [Manu Sporny]{schema:name}. My name is **Manu [Sporny]**{schema:name}. My name is [Manu Sporny] {schema:name}. My name is Manu Sporny{schema:name}. ``` Unified diff between expected and actual output: ```diff --- test/span_scope.md.md +++ /dev/stdin @@ -1,4 +1,4 @@ My name is Manu Sporny. -My name is **Manu [Sporny]**. -My name is [Manu Sporny] {schema:name}. -My name is Manu Sporny{schema:name}. +My name is **Manu \[Sporny\]**{schema:name}. +My name is \[Manu Sporny\] . +My name is Manu SpornySporny{schema:name} ``` This reveals 3 issues, one of which have already been covered. In second line, an annotation for emphasised text is wrongly kept. It also (for third line) reveals the issue covered previously of wrongly removing detached annotation. In fourth line, a non-annotation is correctly kept, but the word before that is wrongly repeated. ## Summary of test failures The small testsuite containing 10 tests currently exposes 6 types of test failures, which in closer inspection reveals 7 bugs in the code and (arguably, see discussion at @sec-experimental-test) 1 bug in the testsuite.