aboutsummaryrefslogtreecommitdiff
path: root/test/spec_tests.py
AgeCommit message (Collapse)Author
2016-08-08Test suite: Don't mess up on examples with 32 `s.John MacFarlane
Closes #423.
2016-06-02spec_tests.py: exit code is sum of failures and errors.John MacFarlane
2016-06-02spec_tests.py: specify newline when opening file.John MacFarlane
2016-01-10New format for spec tests, new lua formatter for specs.John MacFarlane
The format for the spec examples has changed from . markdown . html . to ```````````````````````````````` example markdown . html ```````````````````````````````` One advantage of this is that `spec.txt` becomes a valid Markdown file. `tests/spec_test.py` has been changed to use the new format. The old `tools/makespec.py` has been replaced by a lua program, `tools/make_spec.lua`, which uses the `lcmark` rock (and indirectly libcmark). It can generate html, latex, and commonmark versions of the spec. Pandoc is no longer needed for the latex/PDF version. And, since the new program uses the cmark API and operates directly on the parse tree, we avoid certain bad results we got with the regex replacements done by the python script.
2015-07-10Fixed small bug in spec test runner.John MacFarlane
2015-07-08spec_test.py: write to stdout instead of using 'print' throughout.John MacFarlane
Combining direct stdout buffer writes and print doesn't work well; results are out of order.
2015-07-04spec_tests.py: force utf-8 on test failure output.John MacFarlane
See #345.
2015-07-03Force utf-8 output for `spec_test.py --dump-tests`.John MacFarlane
Closes #344.
2015-07-03spec_tests.py: Properly handle visible tab → in test expected output.John MacFarlane
2015-01-24Added spec tests.John MacFarlane
2015-01-24Removed implementation-specific material from repository.John MacFarlane
The C and JS implementations are being split off into different repositories. This repository will just have the spec itself.
2014-12-31Fix spec tests on WindowsNick Wellnhofer
2014-12-29Updated tests to use python3.John MacFarlane
2014-12-24Use python2 for man, test scripts.John MacFarlane
Closes #251.
2014-12-23Made spec_tests.py more DRY.John MacFarlane
Factored test selection logic out of do_tests. Eliminated do_tests, putting the (now very short) code into main. Improves 1ba4822 @cirosantilli
2014-12-22spec_tests.py: make --dump-tests sensitive to --pattern, --number.John MacFarlane
Closes #249 in a slightly different way.
2014-12-22Merge pull request #248 from cirosantilli/short-optionsJohn MacFarlane
spec_tests add short options for common parameters
2014-12-22Merge pull request #247 from cirosantilli/run-single-test-by-numberJohn MacFarlane
Add --number option to run a single test by its id
2014-12-22Slight improvement to #250.John MacFarlane
Indicate which is expected and which is actual output.
2014-12-21Don't raise exception on invalid UTF-8 outputCiro Santilli
This could be reproduced with: ./test/spec_tests.py --program kramdown --pattern Enti with Kramdown version 1.5.0. The main problem is that it would stop future tests from being carried out. After this commit it is just counted as a failure. The error message printed is of the form: Unicode error: 'utf8' codec can't decode byte 0xfc in position 8: invalid start byte '<p># \xd3\x92 \xcf\xa0 \xef\xbf\xbd</p>\n' '<p># \xd3\x92 \xcf\xa0 \xfc\x85\xb8\xb0\xa9\xb8</p>\n'
2014-12-21spec_tests add short options for common parametersCiro Santilli
2014-12-21Add --number option to run a single test by its idCiro Santilli
Do not check input validity, e.g. input `>` number of tests, because other inputs don't check it either, e.g. `--pattern`. If given, the summary line will be like: 1 passed, 0 failed, 0 errored, 509 skipped
2014-12-01make the diff output workCarsten Brandt
2014-12-01fixed python scriptCarsten Brandt
all these variables are not defined and have to be extracted from `test`. Running the script this way resulted in error: ``` python cmm/test/spec_tests.py --spec cmm/spec.txt --program "./bin/markdown --flavor=common" Traceback (most recent call last): File "cmm/test/spec_tests.py", line 137, in <module> if do_tests(cmark, tests, args.pattern, args.normalize): File "cmm/test/spec_tests.py", line 114, in do_tests result = do_test(test, normalize) File "cmm/test/spec_tests.py", line 47, in do_test print_test_header(headertext, example_number,start_line,end_line) NameError: global name 'headertext' is not defined ```
2014-11-29More code cleanup in spec_tests.py.John MacFarlane
2014-11-29Reorganized code in spec_tests.py.John MacFarlane
2014-11-29test: Factored out normalization code into normalize.py.John MacFarlane
2014-11-29spec_tests.py: if dump_args, don't try to load dynamic library.John MacFarlane
2014-11-29Factored out cmark.py from test programs.John MacFarlane
2014-11-29Moved testing programs to test/.John MacFarlane
Added test/CMakeLists.txt.