Preliminary spec changes for new tab handling.

Instead of having a preprocessing step where tabs are
converted to spaces, we now handle tabs directly in
the parser.

This allows tabs to be retained in code blocks and code
spans.

This commit adds some general language to the effect that,
for purposes of determining block structure, tabs are to
be treated just like equivalent spaces.

Much more work is needed on the spec regarding spaces and tabs,
but this change (and corresponding changes to the implementations)
is a start.

1 files changed, 38 insertions, 7 deletions

diff --git a/spec.txt b/spec.txt
index f6ea793..0c42aae 100644
--- a/spec.txt
+++ b/spec.txt
@@ -249,15 +249,24 @@ A [punctuation character](@punctuation-character) is an [ASCII
 punctuation character] or anything in
 the unicode classes `Pc`, `Pd`, `Pe`, `Pf`, `Pi`, `Po`, or `Ps`.
 
-## Preprocessing
+## Tabs
 
-Tabs in lines are immediately expanded to [spaces][space], with a tab
-stop of 4 characters:
+Tabs in lines are not expanded to [spaces][space].  However,
+in contexts where indentation is significant for the
+document's structure, tabs behave as if they were replaced
+by spaces with a tab stop of 4 characters.
 
 .
 →foo→baz→→bim
 .
-<pre><code>foo baz     bim
+<pre><code>foo→baz→→bim
+</code></pre>
+.
+
+.
+  →foo→baz→→bim
+.
+<pre><code>foo→baz→→bim
 </code></pre>
 .
 
@@ -265,11 +274,33 @@ stop of 4 characters:
     a→a
     ὐ→a
 .
-<pre><code>a   a
-ὐ   a
+<pre><code>a→a
+ὐ→a
 </code></pre>
 .
 
+.
+  - foo
+
+→bar
+.
+<ul>
+<li>
+<p>foo</p>
+<p>bar</p>
+</li>
+</ul>
+.
+
+.
+>→foo→bar
+.
+<blockquote>
+<p>foo→bar</p>
+</blockquote>
+.
+
+
 ## Insecure characters
 
 For security reasons, the Unicode character `U+0000` must be replaced
@@ -4251,7 +4282,7 @@ backslashes:
 .
 \→\A\a\ \3\φ\«
 .
-<p>\   \A\a\ \3\φ\«</p>
+<p>\→\A\a\ \3\φ\«</p>
 .
 
 Escaped characters are treated as regular characters and do