A testsuite has been set up,
derived from the examples in the draft specification.
The testsuite contains 10 end-to-end tests.
A build target, make check, has been constructed
that renders each test with the sem-md.lua filter enabled,
and compares the result against expected output using diff.
7 of the 10 tests currently fail.
These test failures are examined closer below,
and summarized at the end of the chapter.
Wrongly removes detached annotation
The test block_scope contains this in Semantic Markdown:
You can give me a ring. {=<#manu>}
You can give me a ring {=<#manu>}.
Expected output as Markdown is this:
You can give me a ring.
You can give me a ring {=<#manu>}.
Output from the test command
pandoc --from commonmark --to commonmark --lua-filter sem-md/sem-md.lua --wrap preserve < test/block_scope.md
is instead this:
You can give me a ring.
You can give me a ring .
A more concise view is the following unified diff,
where minus-prefixed lines are expected output
differing from plus-prefixed actual output:
--- test/block_scope.md.md
+++ /dev/stdin
@@ -1,3 +1,3 @@
You can give me a ring.
-You can give me a ring {=<#manu>}.
+You can give me a ring .
The filter wrongly removes a non-annotation:
Only when text enclosed with braces
are either immediately followed by an explicit inline enclosure
(linked, emphasized, strongly emphasized, underlined or bracketed)
or is the very first or very last inline content of a block,
should it be considered an annotation.
This is a violation of the Semantic Markdown specification,
and also one of the rules defined for this project
that "rendering of syntactically incorrect [...] annotations
[...] should include the annotation as regular text"
(see @sec-spirit).
Fails to remove image annotation
The test ex_paragraph contains this in Semantic Markdown:
My name is
[Manu Sporny]{:name}
and you can give me a ring via
[1-800-555-0199]{:telephone}.
{:image}
My favorite animal is the [Liger]{ov:preferredAnimal}.
{=<#manu> .:Person}
{schema}: @default
{ov}: http://open.vocab.org/terms/
Unified diff between expected and actual output:
--- test/ex_paragraph.md.md
+++ /dev/stdin
@@ -2,5 +2,5 @@
Manu Sporny
and you can give me a ring via
1-800-555-0199.
-
+{:image}
My favorite animal is the Liger.
The annotation, {:image}, for the inline linked image is wrongly kept.
This is a violation of the Semantic Markdown specification,
and also one of the rules defined for this project
that "A rendering of syntactically correct annotations [...]
must be functionally equal to the text authored without annotations"
(see @sec-spirit).
Fails to remove annotation-only block {#sec-test-annotation-block}
The test block-siblings_scope_list contains this in Semantic Markdown:
# People
## {.schema:Person}
## Manu Sporny
My name is Manu Sporny.
## Thomas Francart
My name is Thomas Francart.
# Animals
Unified diff between expected and actual output:
--- test/block-siblings_scope.md.md
+++ /dev/stdin
@@ -1,5 +1,7 @@
# People
+##
+
## Manu Sporny
My name is Manu Sporny.
A Header block containing only an annotation
has had the annotation removed inline,
but the block -- which after removal is virtually empty
(contains only a space) --
is kept.
Tests block-siblings_scope_list and ex_title_and_list
fail due to similar issue but for other block elemet types.
These test failures reveal
violations of the Semantic Markdown specification,
but parts of the specifications that was not planned
to be implemented with this project,
because introducing logic that adds or removes blocks
(not only inlines) arguably is too large a deviation
of core Markdown.
This type of error is therefore a bug in the testsuite
rather than in the codebase.
Text after link definition annotation wrongly removed
The test link_definition_scope contains this:
My name is
[Manu Sporny]
and you can give me a ring via
[1-800-555-0199].
[Manu Sporny]: {schema:name}
[1-800-555-0199]: tel:+1-800-555-0199
"make a phone call to Manu Sporny"
{schema:telephone}
Unified diff between expected and actual output:
--- test/link_definition_scope.md.md
+++ /dev/stdin
@@ -1,4 +1,4 @@
My name is
Manu Sporny
and you can give me a ring via
-[1-800-555-0199](tel:+1-800-555-0199 "make a phone call to Manu Sporny").
+[1-800-555-0199](tel:+1-800-555-0199 "make a phone call to Manu Sporny")
A link annotation from a link definition is correctly removed,
but trailing text (a dot) is wrongly removed as well.
Fails to remove annotation and repeats word before non-annotated
The test span_scope contains this:
My name is [Manu Sporny]{schema:name}.
My name is **Manu [Sporny]**{schema:name}.
My name is [Manu Sporny] {schema:name}.
My name is Manu Sporny{schema:name}.
Unified diff between expected and actual output:
--- test/span_scope.md.md
+++ /dev/stdin
@@ -1,4 +1,4 @@
My name is Manu Sporny.
-My name is **Manu [Sporny]**.
-My name is [Manu Sporny] {schema:name}.
-My name is Manu Sporny{schema:name}.
+My name is **Manu \[Sporny\]**{schema:name}.
+My name is \[Manu Sporny\] .
+My name is Manu SpornySporny{schema:name}
This reveals 3 issues,
one of which have already been covered.
In second line,
an annotation for emphasised text is wrongly kept.
It also (for third line) reveals the issue covered previously
of wrongly removing detached annotation.
In fourth line,
a non-annotation is correctly kept,
but the word before that is wrongly repeated.
Summary of test failures
The small testsuite containing 10 tests
currently exposes 6 types of test failures,
which in closer inspection reveals 7 bugs in the code
and (arguably, see discussion at @sec-experimental-test)
1 bug in the testsuite.
