annotate Markdown/markdown2.py @ 7:0f2b740536fb draft

Uploaded
author saskia-hiltemann
date Mon, 21 Aug 2017 09:16:07 -0400
parents a4813532bbc6
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1 #!/usr/bin/env python
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2 # Copyright (c) 2012 Trent Mick.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
3 # Copyright (c) 2007-2008 ActiveState Corp.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
4 # License: MIT (http://www.opensource.org/licenses/mit-license.php)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
5
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
6 from __future__ import generators
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
7
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
8 r"""A fast and complete Python implementation of Markdown.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
9
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
10 [from http://daringfireball.net/projects/markdown/]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
11 > Markdown is a text-to-HTML filter; it translates an easy-to-read /
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
12 > easy-to-write structured text format into HTML. Markdown's text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
13 > format is most similar to that of plain text email, and supports
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
14 > features such as headers, *emphasis*, code blocks, blockquotes, and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
15 > links.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
16 >
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
17 > Markdown's syntax is designed not as a generic markup language, but
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
18 > specifically to serve as a front-end to (X)HTML. You can use span-level
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
19 > HTML tags anywhere in a Markdown document, and you can use block level
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
20 > HTML tags (like <div> and <table> as well).
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
21
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
22 Module usage:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
23
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
24 >>> import markdown2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
25 >>> markdown2.markdown("*boo!*") # or use `html = markdown_path(PATH)`
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
26 u'<p><em>boo!</em></p>\n'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
27
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
28 >>> markdowner = Markdown()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
29 >>> markdowner.convert("*boo!*")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
30 u'<p><em>boo!</em></p>\n'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
31 >>> markdowner.convert("**boom!**")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
32 u'<p><strong>boom!</strong></p>\n'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
33
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
34 This implementation of Markdown implements the full "core" syntax plus a
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
35 number of extras (e.g., code syntax coloring, footnotes) as described on
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
36 <https://github.com/trentm/python-markdown2/wiki/Extras>.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
37 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
38
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
39 cmdln_desc = """A fast and complete Python implementation of Markdown, a
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
40 text-to-HTML conversion tool for web writers.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
41
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
42 Supported extra syntax options (see -x|--extras option below and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
43 see <https://github.com/trentm/python-markdown2/wiki/Extras> for details):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
44
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
45 * code-friendly: Disable _ and __ for em and strong.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
46 * cuddled-lists: Allow lists to be cuddled to the preceding paragraph.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
47 * fenced-code-blocks: Allows a code block to not have to be indented
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
48 by fencing it with '```' on a line before and after. Based on
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
49 <http://github.github.com/github-flavored-markdown/> with support for
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
50 syntax highlighting.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
51 * footnotes: Support footnotes as in use on daringfireball.net and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
52 implemented in other Markdown processors (tho not in Markdown.pl v1.0.1).
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
53 * header-ids: Adds "id" attributes to headers. The id value is a slug of
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
54 the header text.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
55 * html-classes: Takes a dict mapping html tag names (lowercase) to a
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
56 string to use for a "class" tag attribute. Currently only supports
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
57 "pre" and "code" tags. Add an issue if you require this for other tags.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
58 * markdown-in-html: Allow the use of `markdown="1"` in a block HTML tag to
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
59 have markdown processing be done on its contents. Similar to
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
60 <http://michelf.com/projects/php-markdown/extra/#markdown-attr> but with
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
61 some limitations.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
62 * metadata: Extract metadata from a leading '---'-fenced block.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
63 See <https://github.com/trentm/python-markdown2/issues/77> for details.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
64 * nofollow: Add `rel="nofollow"` to add `<a>` tags with an href. See
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
65 <http://en.wikipedia.org/wiki/Nofollow>.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
66 * pyshell: Treats unindented Python interactive shell sessions as <code>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
67 blocks.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
68 * link-patterns: Auto-link given regex patterns in text (e.g. bug number
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
69 references, revision number references).
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
70 * smarty-pants: Replaces ' and " with curly quotation marks or curly
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
71 apostrophes. Replaces --, ---, ..., and . . . with en dashes, em dashes,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
72 and ellipses.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
73 * toc: The returned HTML string gets a new "toc_html" attribute which is
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
74 a Table of Contents for the document. (experimental)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
75 * xml: Passes one-liner processing instructions and namespaced XML tags.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
76 * tables: Tables using the same format as GFM
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
77 <https://help.github.com/articles/github-flavored-markdown#tables> and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
78 PHP-Markdown Extra <https://michelf.ca/projects/php-markdown/extra/#table>.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
79 * wiki-tables: Google Code Wiki-style tables. See
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
80 <http://code.google.com/p/support/wiki/WikiSyntax#Tables>.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
81 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
82
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
83 # Dev Notes:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
84 # - Python's regex syntax doesn't have '\z', so I'm using '\Z'. I'm
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
85 # not yet sure if there implications with this. Compare 'pydoc sre'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
86 # and 'perldoc perlre'.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
87
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
88 __version_info__ = (2, 3, 1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
89 __version__ = '.'.join(map(str, __version_info__))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
90 __author__ = "Trent Mick"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
91
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
92 import os
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
93 import sys
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
94 from pprint import pprint, pformat
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
95 import re
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
96 import logging
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
97 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
98 from hashlib import md5
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
99 except ImportError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
100 from md5 import md5
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
101 import optparse
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
102 from random import random, randint
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
103 import codecs
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
104
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
105
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
106 #---- Python version compat
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
107
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
108 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
109 from urllib.parse import quote # python3
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
110 except ImportError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
111 from urllib import quote # python2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
112
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
113 if sys.version_info[:2] < (2,4):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
114 from sets import Set as set
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
115 def reversed(sequence):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
116 for i in sequence[::-1]:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
117 yield i
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
118
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
119 # Use `bytes` for byte strings and `unicode` for unicode strings (str in Py3).
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
120 if sys.version_info[0] <= 2:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
121 py3 = False
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
122 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
123 bytes
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
124 except NameError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
125 bytes = str
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
126 base_string_type = basestring
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
127 elif sys.version_info[0] >= 3:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
128 py3 = True
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
129 unicode = str
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
130 base_string_type = str
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
131
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
132
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
133
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
134 #---- globals
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
135
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
136 DEBUG = False
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
137 log = logging.getLogger("markdown")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
138
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
139 DEFAULT_TAB_WIDTH = 4
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
140
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
141
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
142 SECRET_SALT = bytes(randint(0, 1000000))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
143 def _hash_text(s):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
144 return 'md5-' + md5(SECRET_SALT + s.encode("utf-8")).hexdigest()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
145
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
146 # Table of hash values for escaped characters:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
147 g_escape_table = dict([(ch, _hash_text(ch))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
148 for ch in '\\`*_{}[]()>#+-.!'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
149
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
150
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
151
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
152 #---- exceptions
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
153
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
154 class MarkdownError(Exception):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
155 pass
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
156
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
157
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
158
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
159 #---- public api
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
160
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
161 def markdown_path(path, encoding="utf-8",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
162 html4tags=False, tab_width=DEFAULT_TAB_WIDTH,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
163 safe_mode=None, extras=None, link_patterns=None,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
164 use_file_vars=False):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
165 fp = codecs.open(path, 'r', encoding)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
166 text = fp.read()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
167 fp.close()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
168 return Markdown(html4tags=html4tags, tab_width=tab_width,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
169 safe_mode=safe_mode, extras=extras,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
170 link_patterns=link_patterns,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
171 use_file_vars=use_file_vars).convert(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
172
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
173 def markdown(text, html4tags=False, tab_width=DEFAULT_TAB_WIDTH,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
174 safe_mode=None, extras=None, link_patterns=None,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
175 use_file_vars=False):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
176 return Markdown(html4tags=html4tags, tab_width=tab_width,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
177 safe_mode=safe_mode, extras=extras,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
178 link_patterns=link_patterns,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
179 use_file_vars=use_file_vars).convert(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
180
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
181 class Markdown(object):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
182 # The dict of "extras" to enable in processing -- a mapping of
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
183 # extra name to argument for the extra. Most extras do not have an
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
184 # argument, in which case the value is None.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
185 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
186 # This can be set via (a) subclassing and (b) the constructor
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
187 # "extras" argument.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
188 extras = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
189
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
190 urls = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
191 titles = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
192 html_blocks = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
193 html_spans = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
194 html_removed_text = "[HTML_REMOVED]" # for compat with markdown.py
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
195
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
196 # Used to track when we're inside an ordered or unordered list
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
197 # (see _ProcessListItems() for details):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
198 list_level = 0
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
199
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
200 _ws_only_line_re = re.compile(r"^[ \t]+$", re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
201
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
202 def __init__(self, html4tags=False, tab_width=4, safe_mode=None,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
203 extras=None, link_patterns=None, use_file_vars=False):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
204 if html4tags:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
205 self.empty_element_suffix = ">"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
206 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
207 self.empty_element_suffix = " />"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
208 self.tab_width = tab_width
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
209
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
210 # For compatibility with earlier markdown2.py and with
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
211 # markdown.py's safe_mode being a boolean,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
212 # safe_mode == True -> "replace"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
213 if safe_mode is True:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
214 self.safe_mode = "replace"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
215 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
216 self.safe_mode = safe_mode
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
217
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
218 # Massaging and building the "extras" info.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
219 if self.extras is None:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
220 self.extras = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
221 elif not isinstance(self.extras, dict):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
222 self.extras = dict([(e, None) for e in self.extras])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
223 if extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
224 if not isinstance(extras, dict):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
225 extras = dict([(e, None) for e in extras])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
226 self.extras.update(extras)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
227 assert isinstance(self.extras, dict)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
228 if "toc" in self.extras and not "header-ids" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
229 self.extras["header-ids"] = None # "toc" implies "header-ids"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
230 self._instance_extras = self.extras.copy()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
231
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
232 self.link_patterns = link_patterns
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
233 self.use_file_vars = use_file_vars
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
234 self._outdent_re = re.compile(r'^(\t|[ ]{1,%d})' % tab_width, re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
235
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
236 self._escape_table = g_escape_table.copy()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
237 if "smarty-pants" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
238 self._escape_table['"'] = _hash_text('"')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
239 self._escape_table["'"] = _hash_text("'")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
240
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
241 def reset(self):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
242 self.urls = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
243 self.titles = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
244 self.html_blocks = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
245 self.html_spans = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
246 self.list_level = 0
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
247 self.extras = self._instance_extras.copy()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
248 if "footnotes" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
249 self.footnotes = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
250 self.footnote_ids = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
251 if "header-ids" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
252 self._count_from_header_id = {} # no `defaultdict` in Python 2.4
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
253 if "metadata" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
254 self.metadata = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
255
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
256 # Per <https://developer.mozilla.org/en-US/docs/HTML/Element/a> "rel"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
257 # should only be used in <a> tags with an "href" attribute.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
258 _a_nofollow = re.compile(r"<(a)([^>]*href=)", re.IGNORECASE)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
259
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
260 def convert(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
261 """Convert the given text."""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
262 # Main function. The order in which other subs are called here is
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
263 # essential. Link and image substitutions need to happen before
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
264 # _EscapeSpecialChars(), so that any *'s or _'s in the <a>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
265 # and <img> tags get encoded.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
266
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
267 # Clear the global hashes. If we don't clear these, you get conflicts
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
268 # from other articles when generating a page which contains more than
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
269 # one article (e.g. an index page that shows the N most recent
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
270 # articles):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
271 self.reset()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
272
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
273 if not isinstance(text, unicode):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
274 #TODO: perhaps shouldn't presume UTF-8 for string input?
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
275 text = unicode(text, 'utf-8')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
276
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
277 if self.use_file_vars:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
278 # Look for emacs-style file variable hints.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
279 emacs_vars = self._get_emacs_vars(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
280 if "markdown-extras" in emacs_vars:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
281 splitter = re.compile("[ ,]+")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
282 for e in splitter.split(emacs_vars["markdown-extras"]):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
283 if '=' in e:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
284 ename, earg = e.split('=', 1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
285 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
286 earg = int(earg)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
287 except ValueError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
288 pass
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
289 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
290 ename, earg = e, None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
291 self.extras[ename] = earg
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
292
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
293 # Standardize line endings:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
294 text = re.sub("\r\n|\r", "\n", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
295
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
296 # Make sure $text ends with a couple of newlines:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
297 text += "\n\n"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
298
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
299 # Convert all tabs to spaces.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
300 text = self._detab(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
301
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
302 # Strip any lines consisting only of spaces and tabs.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
303 # This makes subsequent regexen easier to write, because we can
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
304 # match consecutive blank lines with /\n+/ instead of something
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
305 # contorted like /[ \t]*\n+/ .
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
306 text = self._ws_only_line_re.sub("", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
307
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
308 # strip metadata from head and extract
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
309 if "metadata" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
310 text = self._extract_metadata(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
311
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
312 text = self.preprocess(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
313
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
314 if "fenced-code-blocks" in self.extras and not self.safe_mode:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
315 text = self._do_fenced_code_blocks(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
316
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
317 if self.safe_mode:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
318 text = self._hash_html_spans(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
319
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
320 # Turn block-level HTML blocks into hash entries
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
321 text = self._hash_html_blocks(text, raw=True)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
322
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
323 if "fenced-code-blocks" in self.extras and self.safe_mode:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
324 text = self._do_fenced_code_blocks(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
325
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
326 # Strip link definitions, store in hashes.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
327 if "footnotes" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
328 # Must do footnotes first because an unlucky footnote defn
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
329 # looks like a link defn:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
330 # [^4]: this "looks like a link defn"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
331 text = self._strip_footnote_definitions(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
332 text = self._strip_link_definitions(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
333
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
334 text = self._run_block_gamut(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
335
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
336 if "footnotes" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
337 text = self._add_footnotes(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
338
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
339 text = self.postprocess(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
340
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
341 text = self._unescape_special_chars(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
342
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
343 if self.safe_mode:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
344 text = self._unhash_html_spans(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
345
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
346 if "nofollow" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
347 text = self._a_nofollow.sub(r'<\1 rel="nofollow"\2', text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
348
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
349 text += "\n"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
350
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
351 rv = UnicodeWithAttrs(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
352 if "toc" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
353 rv._toc = self._toc
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
354 if "metadata" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
355 rv.metadata = self.metadata
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
356 return rv
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
357
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
358 def postprocess(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
359 """A hook for subclasses to do some postprocessing of the html, if
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
360 desired. This is called before unescaping of special chars and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
361 unhashing of raw HTML spans.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
362 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
363 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
364
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
365 def preprocess(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
366 """A hook for subclasses to do some preprocessing of the Markdown, if
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
367 desired. This is called after basic formatting of the text, but prior
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
368 to any extras, safe mode, etc. processing.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
369 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
370 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
371
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
372 # Is metadata if the content starts with '---'-fenced `key: value`
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
373 # pairs. E.g. (indented for presentation):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
374 # ---
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
375 # foo: bar
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
376 # another-var: blah blah
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
377 # ---
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
378 _metadata_pat = re.compile("""^---[ \t]*\n((?:[ \t]*[^ \t:]+[ \t]*:[^\n]*\n)+)---[ \t]*\n""")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
379
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
380 def _extract_metadata(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
381 # fast test
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
382 if not text.startswith("---"):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
383 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
384 match = self._metadata_pat.match(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
385 if not match:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
386 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
387
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
388 tail = text[len(match.group(0)):]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
389 metadata_str = match.group(1).strip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
390 for line in metadata_str.split('\n'):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
391 key, value = line.split(':', 1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
392 self.metadata[key.strip()] = value.strip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
393
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
394 return tail
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
395
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
396
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
397 _emacs_oneliner_vars_pat = re.compile(r"-\*-\s*([^\r\n]*?)\s*-\*-", re.UNICODE)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
398 # This regular expression is intended to match blocks like this:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
399 # PREFIX Local Variables: SUFFIX
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
400 # PREFIX mode: Tcl SUFFIX
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
401 # PREFIX End: SUFFIX
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
402 # Some notes:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
403 # - "[ \t]" is used instead of "\s" to specifically exclude newlines
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
404 # - "(\r\n|\n|\r)" is used instead of "$" because the sre engine does
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
405 # not like anything other than Unix-style line terminators.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
406 _emacs_local_vars_pat = re.compile(r"""^
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
407 (?P<prefix>(?:[^\r\n|\n|\r])*?)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
408 [\ \t]*Local\ Variables:[\ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
409 (?P<suffix>.*?)(?:\r\n|\n|\r)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
410 (?P<content>.*?\1End:)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
411 """, re.IGNORECASE | re.MULTILINE | re.DOTALL | re.VERBOSE)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
412
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
413 def _get_emacs_vars(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
414 """Return a dictionary of emacs-style local variables.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
415
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
416 Parsing is done loosely according to this spec (and according to
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
417 some in-practice deviations from this):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
418 http://www.gnu.org/software/emacs/manual/html_node/emacs/Specifying-File-Variables.html#Specifying-File-Variables
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
419 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
420 emacs_vars = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
421 SIZE = pow(2, 13) # 8kB
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
422
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
423 # Search near the start for a '-*-'-style one-liner of variables.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
424 head = text[:SIZE]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
425 if "-*-" in head:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
426 match = self._emacs_oneliner_vars_pat.search(head)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
427 if match:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
428 emacs_vars_str = match.group(1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
429 assert '\n' not in emacs_vars_str
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
430 emacs_var_strs = [s.strip() for s in emacs_vars_str.split(';')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
431 if s.strip()]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
432 if len(emacs_var_strs) == 1 and ':' not in emacs_var_strs[0]:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
433 # While not in the spec, this form is allowed by emacs:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
434 # -*- Tcl -*-
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
435 # where the implied "variable" is "mode". This form
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
436 # is only allowed if there are no other variables.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
437 emacs_vars["mode"] = emacs_var_strs[0].strip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
438 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
439 for emacs_var_str in emacs_var_strs:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
440 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
441 variable, value = emacs_var_str.strip().split(':', 1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
442 except ValueError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
443 log.debug("emacs variables error: malformed -*- "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
444 "line: %r", emacs_var_str)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
445 continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
446 # Lowercase the variable name because Emacs allows "Mode"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
447 # or "mode" or "MoDe", etc.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
448 emacs_vars[variable.lower()] = value.strip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
449
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
450 tail = text[-SIZE:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
451 if "Local Variables" in tail:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
452 match = self._emacs_local_vars_pat.search(tail)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
453 if match:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
454 prefix = match.group("prefix")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
455 suffix = match.group("suffix")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
456 lines = match.group("content").splitlines(0)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
457 #print "prefix=%r, suffix=%r, content=%r, lines: %s"\
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
458 # % (prefix, suffix, match.group("content"), lines)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
459
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
460 # Validate the Local Variables block: proper prefix and suffix
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
461 # usage.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
462 for i, line in enumerate(lines):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
463 if not line.startswith(prefix):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
464 log.debug("emacs variables error: line '%s' "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
465 "does not use proper prefix '%s'"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
466 % (line, prefix))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
467 return {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
468 # Don't validate suffix on last line. Emacs doesn't care,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
469 # neither should we.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
470 if i != len(lines)-1 and not line.endswith(suffix):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
471 log.debug("emacs variables error: line '%s' "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
472 "does not use proper suffix '%s'"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
473 % (line, suffix))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
474 return {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
475
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
476 # Parse out one emacs var per line.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
477 continued_for = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
478 for line in lines[:-1]: # no var on the last line ("PREFIX End:")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
479 if prefix: line = line[len(prefix):] # strip prefix
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
480 if suffix: line = line[:-len(suffix)] # strip suffix
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
481 line = line.strip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
482 if continued_for:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
483 variable = continued_for
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
484 if line.endswith('\\'):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
485 line = line[:-1].rstrip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
486 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
487 continued_for = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
488 emacs_vars[variable] += ' ' + line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
489 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
490 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
491 variable, value = line.split(':', 1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
492 except ValueError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
493 log.debug("local variables error: missing colon "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
494 "in local variables entry: '%s'" % line)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
495 continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
496 # Do NOT lowercase the variable name, because Emacs only
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
497 # allows "mode" (and not "Mode", "MoDe", etc.) in this block.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
498 value = value.strip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
499 if value.endswith('\\'):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
500 value = value[:-1].rstrip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
501 continued_for = variable
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
502 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
503 continued_for = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
504 emacs_vars[variable] = value
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
505
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
506 # Unquote values.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
507 for var, val in list(emacs_vars.items()):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
508 if len(val) > 1 and (val.startswith('"') and val.endswith('"')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
509 or val.startswith('"') and val.endswith('"')):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
510 emacs_vars[var] = val[1:-1]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
511
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
512 return emacs_vars
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
513
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
514 # Cribbed from a post by Bart Lateur:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
515 # <http://www.nntp.perl.org/group/perl.macperl.anyperl/154>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
516 _detab_re = re.compile(r'(.*?)\t', re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
517 def _detab_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
518 g1 = match.group(1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
519 return g1 + (' ' * (self.tab_width - len(g1) % self.tab_width))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
520 def _detab(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
521 r"""Remove (leading?) tabs from a file.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
522
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
523 >>> m = Markdown()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
524 >>> m._detab("\tfoo")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
525 ' foo'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
526 >>> m._detab(" \tfoo")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
527 ' foo'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
528 >>> m._detab("\t foo")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
529 ' foo'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
530 >>> m._detab(" foo")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
531 ' foo'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
532 >>> m._detab(" foo\n\tbar\tblam")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
533 ' foo\n bar blam'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
534 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
535 if '\t' not in text:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
536 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
537 return self._detab_re.subn(self._detab_sub, text)[0]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
538
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
539 # I broke out the html5 tags here and add them to _block_tags_a and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
540 # _block_tags_b. This way html5 tags are easy to keep track of.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
541 _html5tags = '|article|aside|header|hgroup|footer|nav|section|figure|figcaption'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
542
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
543 _block_tags_a = 'p|div|h[1-6]|blockquote|pre|table|dl|ol|ul|script|noscript|form|fieldset|iframe|math|ins|del'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
544 _block_tags_a += _html5tags
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
545
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
546 _strict_tag_block_re = re.compile(r"""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
547 ( # save in \1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
548 ^ # start of line (with re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
549 <(%s) # start tag = \2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
550 \b # word break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
551 (.*\n)*? # any number of lines, minimally matching
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
552 </\2> # the matching end tag
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
553 [ \t]* # trailing spaces/tabs
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
554 (?=\n+|\Z) # followed by a newline or end of document
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
555 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
556 """ % _block_tags_a,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
557 re.X | re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
558
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
559 _block_tags_b = 'p|div|h[1-6]|blockquote|pre|table|dl|ol|ul|script|noscript|form|fieldset|iframe|math'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
560 _block_tags_b += _html5tags
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
561
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
562 _liberal_tag_block_re = re.compile(r"""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
563 ( # save in \1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
564 ^ # start of line (with re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
565 <(%s) # start tag = \2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
566 \b # word break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
567 (.*\n)*? # any number of lines, minimally matching
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
568 .*</\2> # the matching end tag
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
569 [ \t]* # trailing spaces/tabs
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
570 (?=\n+|\Z) # followed by a newline or end of document
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
571 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
572 """ % _block_tags_b,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
573 re.X | re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
574
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
575 _html_markdown_attr_re = re.compile(
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
576 r'''\s+markdown=("1"|'1')''')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
577 def _hash_html_block_sub(self, match, raw=False):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
578 html = match.group(1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
579 if raw and self.safe_mode:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
580 html = self._sanitize_html(html)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
581 elif 'markdown-in-html' in self.extras and 'markdown=' in html:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
582 first_line = html.split('\n', 1)[0]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
583 m = self._html_markdown_attr_re.search(first_line)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
584 if m:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
585 lines = html.split('\n')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
586 middle = '\n'.join(lines[1:-1])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
587 last_line = lines[-1]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
588 first_line = first_line[:m.start()] + first_line[m.end():]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
589 f_key = _hash_text(first_line)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
590 self.html_blocks[f_key] = first_line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
591 l_key = _hash_text(last_line)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
592 self.html_blocks[l_key] = last_line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
593 return ''.join(["\n\n", f_key,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
594 "\n\n", middle, "\n\n",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
595 l_key, "\n\n"])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
596 key = _hash_text(html)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
597 self.html_blocks[key] = html
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
598 return "\n\n" + key + "\n\n"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
599
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
600 def _hash_html_blocks(self, text, raw=False):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
601 """Hashify HTML blocks
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
602
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
603 We only want to do this for block-level HTML tags, such as headers,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
604 lists, and tables. That's because we still want to wrap <p>s around
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
605 "paragraphs" that are wrapped in non-block-level tags, such as anchors,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
606 phrase emphasis, and spans. The list of tags we're looking for is
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
607 hard-coded.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
608
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
609 @param raw {boolean} indicates if these are raw HTML blocks in
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
610 the original source. It makes a difference in "safe" mode.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
611 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
612 if '<' not in text:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
613 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
614
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
615 # Pass `raw` value into our calls to self._hash_html_block_sub.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
616 hash_html_block_sub = _curry(self._hash_html_block_sub, raw=raw)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
617
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
618 # First, look for nested blocks, e.g.:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
619 # <div>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
620 # <div>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
621 # tags for inner block must be indented.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
622 # </div>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
623 # </div>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
624 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
625 # The outermost tags must start at the left margin for this to match, and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
626 # the inner nested divs must be indented.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
627 # We need to do this before the next, more liberal match, because the next
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
628 # match will start at the first `<div>` and stop at the first `</div>`.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
629 text = self._strict_tag_block_re.sub(hash_html_block_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
630
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
631 # Now match more liberally, simply from `\n<tag>` to `</tag>\n`
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
632 text = self._liberal_tag_block_re.sub(hash_html_block_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
633
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
634 # Special case just for <hr />. It was easier to make a special
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
635 # case than to make the other regex more complicated.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
636 if "<hr" in text:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
637 _hr_tag_re = _hr_tag_re_from_tab_width(self.tab_width)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
638 text = _hr_tag_re.sub(hash_html_block_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
639
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
640 # Special case for standalone HTML comments:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
641 if "<!--" in text:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
642 start = 0
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
643 while True:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
644 # Delimiters for next comment block.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
645 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
646 start_idx = text.index("<!--", start)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
647 except ValueError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
648 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
649 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
650 end_idx = text.index("-->", start_idx) + 3
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
651 except ValueError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
652 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
653
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
654 # Start position for next comment block search.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
655 start = end_idx
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
656
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
657 # Validate whitespace before comment.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
658 if start_idx:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
659 # - Up to `tab_width - 1` spaces before start_idx.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
660 for i in range(self.tab_width - 1):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
661 if text[start_idx - 1] != ' ':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
662 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
663 start_idx -= 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
664 if start_idx == 0:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
665 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
666 # - Must be preceded by 2 newlines or hit the start of
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
667 # the document.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
668 if start_idx == 0:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
669 pass
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
670 elif start_idx == 1 and text[0] == '\n':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
671 start_idx = 0 # to match minute detail of Markdown.pl regex
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
672 elif text[start_idx-2:start_idx] == '\n\n':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
673 pass
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
674 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
675 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
676
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
677 # Validate whitespace after comment.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
678 # - Any number of spaces and tabs.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
679 while end_idx < len(text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
680 if text[end_idx] not in ' \t':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
681 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
682 end_idx += 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
683 # - Must be following by 2 newlines or hit end of text.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
684 if text[end_idx:end_idx+2] not in ('', '\n', '\n\n'):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
685 continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
686
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
687 # Escape and hash (must match `_hash_html_block_sub`).
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
688 html = text[start_idx:end_idx]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
689 if raw and self.safe_mode:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
690 html = self._sanitize_html(html)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
691 key = _hash_text(html)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
692 self.html_blocks[key] = html
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
693 text = text[:start_idx] + "\n\n" + key + "\n\n" + text[end_idx:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
694
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
695 if "xml" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
696 # Treat XML processing instructions and namespaced one-liner
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
697 # tags as if they were block HTML tags. E.g., if standalone
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
698 # (i.e. are their own paragraph), the following do not get
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
699 # wrapped in a <p> tag:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
700 # <?foo bar?>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
701 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
702 # <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="chapter_1.md"/>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
703 _xml_oneliner_re = _xml_oneliner_re_from_tab_width(self.tab_width)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
704 text = _xml_oneliner_re.sub(hash_html_block_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
705
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
706 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
707
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
708 def _strip_link_definitions(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
709 # Strips link definitions from text, stores the URLs and titles in
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
710 # hash references.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
711 less_than_tab = self.tab_width - 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
712
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
713 # Link defs are in the form:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
714 # [id]: url "optional title"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
715 _link_def_re = re.compile(r"""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
716 ^[ ]{0,%d}\[(.+)\]: # id = \1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
717 [ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
718 \n? # maybe *one* newline
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
719 [ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
720 <?(.+?)>? # url = \2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
721 [ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
722 (?:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
723 \n? # maybe one newline
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
724 [ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
725 (?<=\s) # lookbehind for whitespace
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
726 ['"(]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
727 ([^\n]*) # title = \3
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
728 ['")]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
729 [ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
730 )? # title is optional
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
731 (?:\n+|\Z)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
732 """ % less_than_tab, re.X | re.M | re.U)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
733 return _link_def_re.sub(self._extract_link_def_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
734
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
735 def _extract_link_def_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
736 id, url, title = match.groups()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
737 key = id.lower() # Link IDs are case-insensitive
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
738 self.urls[key] = self._encode_amps_and_angles(url)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
739 if title:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
740 self.titles[key] = title
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
741 return ""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
742
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
743 def _extract_footnote_def_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
744 id, text = match.groups()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
745 text = _dedent(text, skip_first_line=not text.startswith('\n')).strip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
746 normed_id = re.sub(r'\W', '-', id)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
747 # Ensure footnote text ends with a couple newlines (for some
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
748 # block gamut matches).
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
749 self.footnotes[normed_id] = text + "\n\n"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
750 return ""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
751
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
752 def _strip_footnote_definitions(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
753 """A footnote definition looks like this:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
754
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
755 [^note-id]: Text of the note.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
756
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
757 May include one or more indented paragraphs.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
758
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
759 Where,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
760 - The 'note-id' can be pretty much anything, though typically it
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
761 is the number of the footnote.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
762 - The first paragraph may start on the next line, like so:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
763
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
764 [^note-id]:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
765 Text of the note.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
766 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
767 less_than_tab = self.tab_width - 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
768 footnote_def_re = re.compile(r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
769 ^[ ]{0,%d}\[\^(.+)\]: # id = \1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
770 [ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
771 ( # footnote text = \2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
772 # First line need not start with the spaces.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
773 (?:\s*.*\n+)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
774 (?:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
775 (?:[ ]{%d} | \t) # Subsequent lines must be indented.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
776 .*\n+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
777 )*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
778 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
779 # Lookahead for non-space at line-start, or end of doc.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
780 (?:(?=^[ ]{0,%d}\S)|\Z)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
781 ''' % (less_than_tab, self.tab_width, self.tab_width),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
782 re.X | re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
783 return footnote_def_re.sub(self._extract_footnote_def_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
784
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
785 _hr_re = re.compile(r'^[ ]{0,3}([-_*][ ]{0,2}){3,}$', re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
786
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
787 def _run_block_gamut(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
788 # These are all the transformations that form block-level
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
789 # tags like paragraphs, headers, and list items.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
790
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
791 if "fenced-code-blocks" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
792 text = self._do_fenced_code_blocks(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
793
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
794 text = self._do_headers(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
795
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
796 # Do Horizontal Rules:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
797 # On the number of spaces in horizontal rules: The spec is fuzzy: "If
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
798 # you wish, you may use spaces between the hyphens or asterisks."
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
799 # Markdown.pl 1.0.1's hr regexes limit the number of spaces between the
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
800 # hr chars to one or two. We'll reproduce that limit here.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
801 hr = "\n<hr"+self.empty_element_suffix+"\n"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
802 text = re.sub(self._hr_re, hr, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
803
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
804 text = self._do_lists(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
805
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
806 if "pyshell" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
807 text = self._prepare_pyshell_blocks(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
808 if "wiki-tables" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
809 text = self._do_wiki_tables(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
810 if "tables" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
811 text = self._do_tables(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
812
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
813 text = self._do_code_blocks(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
814
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
815 text = self._do_block_quotes(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
816
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
817 # We already ran _HashHTMLBlocks() before, in Markdown(), but that
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
818 # was to escape raw HTML in the original Markdown source. This time,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
819 # we're escaping the markup we've just created, so that we don't wrap
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
820 # <p> tags around block-level tags.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
821 text = self._hash_html_blocks(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
822
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
823 text = self._form_paragraphs(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
824
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
825 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
826
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
827 def _pyshell_block_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
828 lines = match.group(0).splitlines(0)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
829 _dedentlines(lines)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
830 indent = ' ' * self.tab_width
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
831 s = ('\n' # separate from possible cuddled paragraph
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
832 + indent + ('\n'+indent).join(lines)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
833 + '\n\n')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
834 return s
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
835
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
836 def _prepare_pyshell_blocks(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
837 """Ensure that Python interactive shell sessions are put in
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
838 code blocks -- even if not properly indented.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
839 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
840 if ">>>" not in text:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
841 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
842
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
843 less_than_tab = self.tab_width - 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
844 _pyshell_block_re = re.compile(r"""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
845 ^([ ]{0,%d})>>>[ ].*\n # first line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
846 ^(\1.*\S+.*\n)* # any number of subsequent lines
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
847 ^\n # ends with a blank line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
848 """ % less_than_tab, re.M | re.X)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
849
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
850 return _pyshell_block_re.sub(self._pyshell_block_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
851
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
852 def _table_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
853 head, underline, body = match.groups()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
854
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
855 # Determine aligns for columns.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
856 cols = [cell.strip() for cell in underline.strip('| \t\n').split('|')]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
857 align_from_col_idx = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
858 for col_idx, col in enumerate(cols):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
859 if col[0] == ':' and col[-1] == ':':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
860 align_from_col_idx[col_idx] = ' align="center"'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
861 elif col[0] == ':':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
862 align_from_col_idx[col_idx] = ' align="left"'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
863 elif col[-1] == ':':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
864 align_from_col_idx[col_idx] = ' align="right"'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
865
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
866 # thead
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
867 hlines = ['<table>', '<thead>', '<tr>']
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
868 cols = [cell.strip() for cell in head.strip('| \t\n').split('|')]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
869 for col_idx, col in enumerate(cols):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
870 hlines.append(' <th%s>%s</th>' % (
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
871 align_from_col_idx.get(col_idx, ''),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
872 self._run_span_gamut(col)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
873 ))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
874 hlines.append('</tr>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
875 hlines.append('</thead>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
876
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
877 # tbody
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
878 hlines.append('<tbody>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
879 for line in body.strip('\n').split('\n'):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
880 hlines.append('<tr>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
881 cols = [cell.strip() for cell in line.strip('| \t\n').split('|')]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
882 for col_idx, col in enumerate(cols):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
883 hlines.append(' <td%s>%s</td>' % (
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
884 align_from_col_idx.get(col_idx, ''),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
885 self._run_span_gamut(col)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
886 ))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
887 hlines.append('</tr>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
888 hlines.append('</tbody>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
889 hlines.append('</table>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
890
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
891 return '\n'.join(hlines) + '\n'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
892
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
893 def _do_tables(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
894 """Copying PHP-Markdown and GFM table syntax. Some regex borrowed from
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
895 https://github.com/michelf/php-markdown/blob/lib/Michelf/Markdown.php#L2538
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
896 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
897 less_than_tab = self.tab_width - 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
898 table_re = re.compile(r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
899 (?:(?<=\n\n)|\A\n?) # leading blank line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
900
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
901 ^[ ]{0,%d} # allowed whitespace
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
902 (.*[|].*) \n # $1: header row (at least one pipe)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
903
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
904 ^[ ]{0,%d} # allowed whitespace
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
905 ( # $2: underline row
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
906 # underline row with leading bar
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
907 (?: \|\ *:?-+:?\ * )+ \|? \n
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
908 |
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
909 # or, underline row without leading bar
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
910 (?: \ *:?-+:?\ *\| )+ (?: \ *:?-+:?\ * )? \n
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
911 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
912
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
913 ( # $3: data rows
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
914 (?:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
915 ^[ ]{0,%d}(?!\ ) # ensure line begins with 0 to less_than_tab spaces
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
916 .*\|.* \n
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
917 )+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
918 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
919 ''' % (less_than_tab, less_than_tab, less_than_tab), re.M | re.X)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
920 return table_re.sub(self._table_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
921
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
922 def _wiki_table_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
923 ttext = match.group(0).strip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
924 #print 'wiki table: %r' % match.group(0)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
925 rows = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
926 for line in ttext.splitlines(0):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
927 line = line.strip()[2:-2].strip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
928 row = [c.strip() for c in re.split(r'(?<!\\)\|\|', line)]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
929 rows.append(row)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
930 #pprint(rows)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
931 hlines = ['<table>', '<tbody>']
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
932 for row in rows:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
933 hrow = ['<tr>']
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
934 for cell in row:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
935 hrow.append('<td>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
936 hrow.append(self._run_span_gamut(cell))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
937 hrow.append('</td>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
938 hrow.append('</tr>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
939 hlines.append(''.join(hrow))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
940 hlines += ['</tbody>', '</table>']
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
941 return '\n'.join(hlines) + '\n'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
942
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
943 def _do_wiki_tables(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
944 # Optimization.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
945 if "||" not in text:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
946 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
947
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
948 less_than_tab = self.tab_width - 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
949 wiki_table_re = re.compile(r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
950 (?:(?<=\n\n)|\A\n?) # leading blank line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
951 ^([ ]{0,%d})\|\|.+?\|\|[ ]*\n # first line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
952 (^\1\|\|.+?\|\|\n)* # any number of subsequent lines
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
953 ''' % less_than_tab, re.M | re.X)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
954 return wiki_table_re.sub(self._wiki_table_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
955
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
956 def _run_span_gamut(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
957 # These are all the transformations that occur *within* block-level
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
958 # tags like paragraphs, headers, and list items.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
959
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
960 text = self._do_code_spans(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
961
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
962 text = self._escape_special_chars(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
963
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
964 # Process anchor and image tags.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
965 text = self._do_links(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
966
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
967 # Make links out of things like `<http://example.com/>`
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
968 # Must come after _do_links(), because you can use < and >
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
969 # delimiters in inline links like [this](<url>).
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
970 text = self._do_auto_links(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
971
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
972 if "link-patterns" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
973 text = self._do_link_patterns(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
974
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
975 text = self._encode_amps_and_angles(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
976
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
977 text = self._do_italics_and_bold(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
978
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
979 if "smarty-pants" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
980 text = self._do_smart_punctuation(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
981
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
982 # Do hard breaks:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
983 if "break-on-newline" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
984 text = re.sub(r" *\n", "<br%s\n" % self.empty_element_suffix, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
985 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
986 text = re.sub(r" {2,}\n", " <br%s\n" % self.empty_element_suffix, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
987
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
988 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
989
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
990 # "Sorta" because auto-links are identified as "tag" tokens.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
991 _sorta_html_tokenize_re = re.compile(r"""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
992 (
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
993 # tag
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
994 </?
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
995 (?:\w+) # tag name
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
996 (?:\s+(?:[\w-]+:)?[\w-]+=(?:".*?"|'.*?'))* # attributes
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
997 \s*/?>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
998 |
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
999 # auto-link (e.g., <http://www.activestate.com/>)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1000 <\w+[^>]*>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1001 |
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1002 <!--.*?--> # comment
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1003 |
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1004 <\?.*?\?> # processing instruction
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1005 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1006 """, re.X)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1007
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1008 def _escape_special_chars(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1009 # Python markdown note: the HTML tokenization here differs from
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1010 # that in Markdown.pl, hence the behaviour for subtle cases can
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1011 # differ (I believe the tokenizer here does a better job because
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1012 # it isn't susceptible to unmatched '<' and '>' in HTML tags).
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1013 # Note, however, that '>' is not allowed in an auto-link URL
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1014 # here.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1015 escaped = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1016 is_html_markup = False
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1017 for token in self._sorta_html_tokenize_re.split(text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1018 if is_html_markup:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1019 # Within tags/HTML-comments/auto-links, encode * and _
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1020 # so they don't conflict with their use in Markdown for
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1021 # italics and strong. We're replacing each such
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1022 # character with its corresponding MD5 checksum value;
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1023 # this is likely overkill, but it should prevent us from
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1024 # colliding with the escape values by accident.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1025 escaped.append(token.replace('*', self._escape_table['*'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1026 .replace('_', self._escape_table['_']))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1027 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1028 escaped.append(self._encode_backslash_escapes(token))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1029 is_html_markup = not is_html_markup
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1030 return ''.join(escaped)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1031
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1032 def _hash_html_spans(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1033 # Used for safe_mode.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1034
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1035 def _is_auto_link(s):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1036 if ':' in s and self._auto_link_re.match(s):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1037 return True
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1038 elif '@' in s and self._auto_email_link_re.match(s):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1039 return True
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1040 return False
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1041
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1042 tokens = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1043 is_html_markup = False
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1044 for token in self._sorta_html_tokenize_re.split(text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1045 if is_html_markup and not _is_auto_link(token):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1046 sanitized = self._sanitize_html(token)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1047 key = _hash_text(sanitized)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1048 self.html_spans[key] = sanitized
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1049 tokens.append(key)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1050 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1051 tokens.append(token)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1052 is_html_markup = not is_html_markup
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1053 return ''.join(tokens)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1054
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1055 def _unhash_html_spans(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1056 for key, sanitized in list(self.html_spans.items()):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1057 text = text.replace(key, sanitized)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1058 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1059
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1060 def _sanitize_html(self, s):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1061 if self.safe_mode == "replace":
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1062 return self.html_removed_text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1063 elif self.safe_mode == "escape":
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1064 replacements = [
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1065 ('&', '&amp;'),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1066 ('<', '&lt;'),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1067 ('>', '&gt;'),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1068 ]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1069 for before, after in replacements:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1070 s = s.replace(before, after)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1071 return s
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1072 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1073 raise MarkdownError("invalid value for 'safe_mode': %r (must be "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1074 "'escape' or 'replace')" % self.safe_mode)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1075
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1076 _inline_link_title = re.compile(r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1077 ( # \1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1078 [ \t]+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1079 (['"]) # quote char = \2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1080 (?P<title>.*?)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1081 \2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1082 )? # title is optional
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1083 \)$
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1084 ''', re.X | re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1085 _tail_of_reference_link_re = re.compile(r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1086 # Match tail of: [text][id]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1087 [ ]? # one optional space
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1088 (?:\n[ ]*)? # one optional newline followed by spaces
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1089 \[
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1090 (?P<id>.*?)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1091 \]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1092 ''', re.X | re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1093
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1094 _whitespace = re.compile(r'\s*')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1095
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1096 _strip_anglebrackets = re.compile(r'<(.*)>.*')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1097
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1098 def _find_non_whitespace(self, text, start):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1099 """Returns the index of the first non-whitespace character in text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1100 after (and including) start
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1101 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1102 match = self._whitespace.match(text, start)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1103 return match.end()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1104
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1105 def _find_balanced(self, text, start, open_c, close_c):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1106 """Returns the index where the open_c and close_c characters balance
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1107 out - the same number of open_c and close_c are encountered - or the
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1108 end of string if it's reached before the balance point is found.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1109 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1110 i = start
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1111 l = len(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1112 count = 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1113 while count > 0 and i < l:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1114 if text[i] == open_c:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1115 count += 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1116 elif text[i] == close_c:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1117 count -= 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1118 i += 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1119 return i
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1120
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1121 def _extract_url_and_title(self, text, start):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1122 """Extracts the url and (optional) title from the tail of a link"""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1123 # text[start] equals the opening parenthesis
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1124 idx = self._find_non_whitespace(text, start+1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1125 if idx == len(text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1126 return None, None, None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1127 end_idx = idx
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1128 has_anglebrackets = text[idx] == "<"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1129 if has_anglebrackets:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1130 end_idx = self._find_balanced(text, end_idx+1, "<", ">")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1131 end_idx = self._find_balanced(text, end_idx, "(", ")")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1132 match = self._inline_link_title.search(text, idx, end_idx)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1133 if not match:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1134 return None, None, None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1135 url, title = text[idx:match.start()], match.group("title")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1136 if has_anglebrackets:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1137 url = self._strip_anglebrackets.sub(r'\1', url)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1138 return url, title, end_idx
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1139
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1140 def _do_links(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1141 """Turn Markdown link shortcuts into XHTML <a> and <img> tags.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1142
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1143 This is a combination of Markdown.pl's _DoAnchors() and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1144 _DoImages(). They are done together because that simplified the
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1145 approach. It was necessary to use a different approach than
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1146 Markdown.pl because of the lack of atomic matching support in
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1147 Python's regex engine used in $g_nested_brackets.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1148 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1149 MAX_LINK_TEXT_SENTINEL = 3000 # markdown2 issue 24
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1150
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1151 # `anchor_allowed_pos` is used to support img links inside
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1152 # anchors, but not anchors inside anchors. An anchor's start
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1153 # pos must be `>= anchor_allowed_pos`.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1154 anchor_allowed_pos = 0
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1155
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1156 curr_pos = 0
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1157 while True: # Handle the next link.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1158 # The next '[' is the start of:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1159 # - an inline anchor: [text](url "title")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1160 # - a reference anchor: [text][id]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1161 # - an inline img: ![text](url "title")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1162 # - a reference img: ![text][id]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1163 # - a footnote ref: [^id]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1164 # (Only if 'footnotes' extra enabled)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1165 # - a footnote defn: [^id]: ...
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1166 # (Only if 'footnotes' extra enabled) These have already
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1167 # been stripped in _strip_footnote_definitions() so no
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1168 # need to watch for them.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1169 # - a link definition: [id]: url "title"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1170 # These have already been stripped in
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1171 # _strip_link_definitions() so no need to watch for them.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1172 # - not markup: [...anything else...
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1173 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1174 start_idx = text.index('[', curr_pos)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1175 except ValueError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1176 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1177 text_length = len(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1178
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1179 # Find the matching closing ']'.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1180 # Markdown.pl allows *matching* brackets in link text so we
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1181 # will here too. Markdown.pl *doesn't* currently allow
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1182 # matching brackets in img alt text -- we'll differ in that
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1183 # regard.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1184 bracket_depth = 0
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1185 for p in range(start_idx+1, min(start_idx+MAX_LINK_TEXT_SENTINEL,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1186 text_length)):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1187 ch = text[p]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1188 if ch == ']':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1189 bracket_depth -= 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1190 if bracket_depth < 0:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1191 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1192 elif ch == '[':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1193 bracket_depth += 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1194 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1195 # Closing bracket not found within sentinel length.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1196 # This isn't markup.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1197 curr_pos = start_idx + 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1198 continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1199 link_text = text[start_idx+1:p]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1200
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1201 # Possibly a footnote ref?
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1202 if "footnotes" in self.extras and link_text.startswith("^"):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1203 normed_id = re.sub(r'\W', '-', link_text[1:])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1204 if normed_id in self.footnotes:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1205 self.footnote_ids.append(normed_id)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1206 result = '<sup class="footnote-ref" id="fnref-%s">' \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1207 '<a href="#fn-%s">%s</a></sup>' \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1208 % (normed_id, normed_id, len(self.footnote_ids))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1209 text = text[:start_idx] + result + text[p+1:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1210 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1211 # This id isn't defined, leave the markup alone.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1212 curr_pos = p+1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1213 continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1214
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1215 # Now determine what this is by the remainder.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1216 p += 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1217 if p == text_length:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1218 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1219
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1220 # Inline anchor or img?
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1221 if text[p] == '(': # attempt at perf improvement
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1222 url, title, url_end_idx = self._extract_url_and_title(text, p)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1223 if url is not None:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1224 # Handle an inline anchor or img.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1225 is_img = start_idx > 0 and text[start_idx-1] == "!"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1226 if is_img:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1227 start_idx -= 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1228
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1229 # We've got to encode these to avoid conflicting
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1230 # with italics/bold.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1231 url = url.replace('*', self._escape_table['*']) \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1232 .replace('_', self._escape_table['_'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1233 if title:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1234 title_str = ' title="%s"' % (
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1235 _xml_escape_attr(title)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1236 .replace('*', self._escape_table['*'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1237 .replace('_', self._escape_table['_']))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1238 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1239 title_str = ''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1240 if is_img:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1241 img_class_str = self._html_class_str_from_tag("img")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1242 result = '<img src="%s" alt="%s"%s%s%s' \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1243 % (url.replace('"', '&quot;'),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1244 _xml_escape_attr(link_text),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1245 title_str, img_class_str, self.empty_element_suffix)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1246 if "smarty-pants" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1247 result = result.replace('"', self._escape_table['"'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1248 curr_pos = start_idx + len(result)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1249 text = text[:start_idx] + result + text[url_end_idx:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1250 elif start_idx >= anchor_allowed_pos:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1251 result_head = '<a href="%s"%s>' % (url, title_str)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1252 result = '%s%s</a>' % (result_head, link_text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1253 if "smarty-pants" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1254 result = result.replace('"', self._escape_table['"'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1255 # <img> allowed from curr_pos on, <a> from
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1256 # anchor_allowed_pos on.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1257 curr_pos = start_idx + len(result_head)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1258 anchor_allowed_pos = start_idx + len(result)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1259 text = text[:start_idx] + result + text[url_end_idx:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1260 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1261 # Anchor not allowed here.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1262 curr_pos = start_idx + 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1263 continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1264
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1265 # Reference anchor or img?
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1266 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1267 match = self._tail_of_reference_link_re.match(text, p)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1268 if match:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1269 # Handle a reference-style anchor or img.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1270 is_img = start_idx > 0 and text[start_idx-1] == "!"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1271 if is_img:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1272 start_idx -= 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1273 link_id = match.group("id").lower()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1274 if not link_id:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1275 link_id = link_text.lower() # for links like [this][]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1276 if link_id in self.urls:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1277 url = self.urls[link_id]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1278 # We've got to encode these to avoid conflicting
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1279 # with italics/bold.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1280 url = url.replace('*', self._escape_table['*']) \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1281 .replace('_', self._escape_table['_'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1282 title = self.titles.get(link_id)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1283 if title:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1284 before = title
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1285 title = _xml_escape_attr(title) \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1286 .replace('*', self._escape_table['*']) \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1287 .replace('_', self._escape_table['_'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1288 title_str = ' title="%s"' % title
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1289 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1290 title_str = ''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1291 if is_img:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1292 img_class_str = self._html_class_str_from_tag("img")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1293 result = '<img src="%s" alt="%s"%s%s%s' \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1294 % (url.replace('"', '&quot;'),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1295 link_text.replace('"', '&quot;'),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1296 title_str, img_class_str, self.empty_element_suffix)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1297 if "smarty-pants" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1298 result = result.replace('"', self._escape_table['"'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1299 curr_pos = start_idx + len(result)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1300 text = text[:start_idx] + result + text[match.end():]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1301 elif start_idx >= anchor_allowed_pos:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1302 result = '<a href="%s"%s>%s</a>' \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1303 % (url, title_str, link_text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1304 result_head = '<a href="%s"%s>' % (url, title_str)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1305 result = '%s%s</a>' % (result_head, link_text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1306 if "smarty-pants" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1307 result = result.replace('"', self._escape_table['"'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1308 # <img> allowed from curr_pos on, <a> from
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1309 # anchor_allowed_pos on.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1310 curr_pos = start_idx + len(result_head)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1311 anchor_allowed_pos = start_idx + len(result)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1312 text = text[:start_idx] + result + text[match.end():]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1313 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1314 # Anchor not allowed here.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1315 curr_pos = start_idx + 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1316 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1317 # This id isn't defined, leave the markup alone.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1318 curr_pos = match.end()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1319 continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1320
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1321 # Otherwise, it isn't markup.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1322 curr_pos = start_idx + 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1323
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1324 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1325
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1326 def header_id_from_text(self, text, prefix, n):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1327 """Generate a header id attribute value from the given header
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1328 HTML content.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1329
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1330 This is only called if the "header-ids" extra is enabled.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1331 Subclasses may override this for different header ids.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1332
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1333 @param text {str} The text of the header tag
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1334 @param prefix {str} The requested prefix for header ids. This is the
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1335 value of the "header-ids" extra key, if any. Otherwise, None.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1336 @param n {int} The <hN> tag number, i.e. `1` for an <h1> tag.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1337 @returns {str} The value for the header tag's "id" attribute. Return
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1338 None to not have an id attribute and to exclude this header from
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1339 the TOC (if the "toc" extra is specified).
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1340 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1341 header_id = _slugify(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1342 if prefix and isinstance(prefix, base_string_type):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1343 header_id = prefix + '-' + header_id
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1344 if header_id in self._count_from_header_id:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1345 self._count_from_header_id[header_id] += 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1346 header_id += '-%s' % self._count_from_header_id[header_id]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1347 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1348 self._count_from_header_id[header_id] = 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1349 return header_id
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1350
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1351 _toc = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1352 def _toc_add_entry(self, level, id, name):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1353 if self._toc is None:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1354 self._toc = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1355 self._toc.append((level, id, self._unescape_special_chars(name)))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1356
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1357 _h_re_base = r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1358 (^(.+)[ \t]*\n(=+|-+)[ \t]*\n+)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1359 |
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1360 (^(\#{1,6}) # \1 = string of #'s
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1361 [ \t]%s
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1362 (.+?) # \2 = Header text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1363 [ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1364 (?<!\\) # ensure not an escaped trailing '#'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1365 \#* # optional closing #'s (not counted)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1366 \n+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1367 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1368 '''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1369
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1370 _h_re = re.compile(_h_re_base % '*', re.X | re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1371 _h_re_tag_friendly = re.compile(_h_re_base % '+', re.X | re.M)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1372
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1373 def _h_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1374 if match.group(1) is not None:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1375 # Setext header
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1376 n = {"=": 1, "-": 2}[match.group(3)[0]]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1377 header_group = match.group(2)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1378 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1379 # atx header
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1380 n = len(match.group(5))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1381 header_group = match.group(6)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1382
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1383 demote_headers = self.extras.get("demote-headers")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1384 if demote_headers:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1385 n = min(n + demote_headers, 6)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1386 header_id_attr = ""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1387 if "header-ids" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1388 header_id = self.header_id_from_text(header_group,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1389 self.extras["header-ids"], n)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1390 if header_id:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1391 header_id_attr = ' id="%s"' % header_id
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1392 html = self._run_span_gamut(header_group)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1393 if "toc" in self.extras and header_id:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1394 self._toc_add_entry(n, header_id, html)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1395 return "<h%d%s>%s</h%d>\n\n" % (n, header_id_attr, html, n)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1396
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1397 def _do_headers(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1398 # Setext-style headers:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1399 # Header 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1400 # ========
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1401 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1402 # Header 2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1403 # --------
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1404
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1405 # atx-style headers:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1406 # # Header 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1407 # ## Header 2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1408 # ## Header 2 with closing hashes ##
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1409 # ...
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1410 # ###### Header 6
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1411
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1412 if 'tag-friendly' in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1413 return self._h_re_tag_friendly.sub(self._h_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1414 return self._h_re.sub(self._h_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1415
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1416 _marker_ul_chars = '*+-'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1417 _marker_any = r'(?:[%s]|\d+\.)' % _marker_ul_chars
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1418 _marker_ul = '(?:[%s])' % _marker_ul_chars
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1419 _marker_ol = r'(?:\d+\.)'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1420
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1421 def _list_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1422 lst = match.group(1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1423 lst_type = match.group(3) in self._marker_ul_chars and "ul" or "ol"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1424 result = self._process_list_items(lst)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1425 if self.list_level:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1426 return "<%s>\n%s</%s>\n" % (lst_type, result, lst_type)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1427 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1428 return "<%s>\n%s</%s>\n\n" % (lst_type, result, lst_type)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1429
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1430 def _do_lists(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1431 # Form HTML ordered (numbered) and unordered (bulleted) lists.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1432
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1433 # Iterate over each *non-overlapping* list match.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1434 pos = 0
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1435 while True:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1436 # Find the *first* hit for either list style (ul or ol). We
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1437 # match ul and ol separately to avoid adjacent lists of different
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1438 # types running into each other (see issue #16).
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1439 hits = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1440 for marker_pat in (self._marker_ul, self._marker_ol):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1441 less_than_tab = self.tab_width - 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1442 whole_list = r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1443 ( # \1 = whole list
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1444 ( # \2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1445 [ ]{0,%d}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1446 (%s) # \3 = first list item marker
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1447 [ \t]+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1448 (?!\ *\3\ ) # '- - - ...' isn't a list. See 'not_quite_a_list' test case.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1449 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1450 (?:.+?)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1451 ( # \4
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1452 \Z
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1453 |
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1454 \n{2,}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1455 (?=\S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1456 (?! # Negative lookahead for another list item marker
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1457 [ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1458 %s[ \t]+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1459 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1460 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1461 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1462 ''' % (less_than_tab, marker_pat, marker_pat)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1463 if self.list_level: # sub-list
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1464 list_re = re.compile("^"+whole_list, re.X | re.M | re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1465 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1466 list_re = re.compile(r"(?:(?<=\n\n)|\A\n?)"+whole_list,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1467 re.X | re.M | re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1468 match = list_re.search(text, pos)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1469 if match:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1470 hits.append((match.start(), match))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1471 if not hits:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1472 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1473 hits.sort()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1474 match = hits[0][1]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1475 start, end = match.span()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1476 middle = self._list_sub(match)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1477 text = text[:start] + middle + text[end:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1478 pos = start + len(middle) # start pos for next attempted match
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1479
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1480 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1481
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1482 _list_item_re = re.compile(r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1483 (\n)? # leading line = \1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1484 (^[ \t]*) # leading whitespace = \2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1485 (?P<marker>%s) [ \t]+ # list marker = \3
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1486 ((?:.+?) # list item text = \4
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1487 (\n{1,2})) # eols = \5
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1488 (?= \n* (\Z | \2 (?P<next_marker>%s) [ \t]+))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1489 ''' % (_marker_any, _marker_any),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1490 re.M | re.X | re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1491
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1492 _last_li_endswith_two_eols = False
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1493 def _list_item_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1494 item = match.group(4)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1495 leading_line = match.group(1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1496 leading_space = match.group(2)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1497 if leading_line or "\n\n" in item or self._last_li_endswith_two_eols:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1498 item = self._run_block_gamut(self._outdent(item))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1499 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1500 # Recursion for sub-lists:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1501 item = self._do_lists(self._outdent(item))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1502 if item.endswith('\n'):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1503 item = item[:-1]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1504 item = self._run_span_gamut(item)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1505 self._last_li_endswith_two_eols = (len(match.group(5)) == 2)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1506 return "<li>%s</li>\n" % item
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1507
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1508 def _process_list_items(self, list_str):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1509 # Process the contents of a single ordered or unordered list,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1510 # splitting it into individual list items.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1511
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1512 # The $g_list_level global keeps track of when we're inside a list.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1513 # Each time we enter a list, we increment it; when we leave a list,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1514 # we decrement. If it's zero, we're not in a list anymore.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1515 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1516 # We do this because when we're not inside a list, we want to treat
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1517 # something like this:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1518 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1519 # I recommend upgrading to version
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1520 # 8. Oops, now this line is treated
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1521 # as a sub-list.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1522 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1523 # As a single paragraph, despite the fact that the second line starts
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1524 # with a digit-period-space sequence.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1525 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1526 # Whereas when we're inside a list (or sub-list), that line will be
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1527 # treated as the start of a sub-list. What a kludge, huh? This is
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1528 # an aspect of Markdown's syntax that's hard to parse perfectly
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1529 # without resorting to mind-reading. Perhaps the solution is to
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1530 # change the syntax rules such that sub-lists must start with a
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1531 # starting cardinal number; e.g. "1." or "a.".
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1532 self.list_level += 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1533 self._last_li_endswith_two_eols = False
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1534 list_str = list_str.rstrip('\n') + '\n'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1535 list_str = self._list_item_re.sub(self._list_item_sub, list_str)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1536 self.list_level -= 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1537 return list_str
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1538
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1539 def _get_pygments_lexer(self, lexer_name):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1540 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1541 from pygments import lexers, util
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1542 except ImportError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1543 return None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1544 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1545 return lexers.get_lexer_by_name(lexer_name)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1546 except util.ClassNotFound:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1547 return None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1548
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1549 def _color_with_pygments(self, codeblock, lexer, **formatter_opts):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1550 import pygments
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1551 import pygments.formatters
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1552
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1553 class HtmlCodeFormatter(pygments.formatters.HtmlFormatter):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1554 def _wrap_code(self, inner):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1555 """A function for use in a Pygments Formatter which
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1556 wraps in <code> tags.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1557 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1558 yield 0, "<code>"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1559 for tup in inner:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1560 yield tup
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1561 yield 0, "</code>"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1562
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1563 def wrap(self, source, outfile):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1564 """Return the source with a code, pre, and div."""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1565 return self._wrap_div(self._wrap_pre(self._wrap_code(source)))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1566
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1567 formatter_opts.setdefault("cssclass", "codehilite")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1568 formatter = HtmlCodeFormatter(**formatter_opts)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1569 return pygments.highlight(codeblock, lexer, formatter)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1570
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1571 def _code_block_sub(self, match, is_fenced_code_block=False):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1572 lexer_name = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1573 if is_fenced_code_block:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1574 lexer_name = match.group(1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1575 if lexer_name:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1576 formatter_opts = self.extras['fenced-code-blocks'] or {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1577 codeblock = match.group(2)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1578 codeblock = codeblock[:-1] # drop one trailing newline
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1579 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1580 codeblock = match.group(1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1581 codeblock = self._outdent(codeblock)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1582 codeblock = self._detab(codeblock)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1583 codeblock = codeblock.lstrip('\n') # trim leading newlines
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1584 codeblock = codeblock.rstrip() # trim trailing whitespace
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1585
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1586 # Note: "code-color" extra is DEPRECATED.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1587 if "code-color" in self.extras and codeblock.startswith(":::"):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1588 lexer_name, rest = codeblock.split('\n', 1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1589 lexer_name = lexer_name[3:].strip()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1590 codeblock = rest.lstrip("\n") # Remove lexer declaration line.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1591 formatter_opts = self.extras['code-color'] or {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1592
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1593 if lexer_name:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1594 def unhash_code( codeblock ):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1595 for key, sanitized in list(self.html_spans.items()):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1596 codeblock = codeblock.replace(key, sanitized)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1597 replacements = [
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1598 ("&amp;", "&"),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1599 ("&lt;", "<"),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1600 ("&gt;", ">")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1601 ]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1602 for old, new in replacements:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1603 codeblock = codeblock.replace(old, new)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1604 return codeblock
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1605 lexer = self._get_pygments_lexer(lexer_name)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1606 if lexer:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1607 codeblock = unhash_code( codeblock )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1608 colored = self._color_with_pygments(codeblock, lexer,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1609 **formatter_opts)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1610 return "\n\n%s\n\n" % colored
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1611
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1612 codeblock = self._encode_code(codeblock)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1613 pre_class_str = self._html_class_str_from_tag("pre")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1614 code_class_str = self._html_class_str_from_tag("code")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1615 return "\n\n<pre%s><code%s>%s\n</code></pre>\n\n" % (
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1616 pre_class_str, code_class_str, codeblock)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1617
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1618 def _html_class_str_from_tag(self, tag):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1619 """Get the appropriate ' class="..."' string (note the leading
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1620 space), if any, for the given tag.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1621 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1622 if "html-classes" not in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1623 return ""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1624 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1625 html_classes_from_tag = self.extras["html-classes"]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1626 except TypeError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1627 return ""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1628 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1629 if tag in html_classes_from_tag:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1630 return ' class="%s"' % html_classes_from_tag[tag]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1631 return ""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1632
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1633 def _do_code_blocks(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1634 """Process Markdown `<pre><code>` blocks."""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1635 code_block_re = re.compile(r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1636 (?:\n\n|\A\n?)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1637 ( # $1 = the code block -- one or more lines, starting with a space/tab
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1638 (?:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1639 (?:[ ]{%d} | \t) # Lines must start with a tab or a tab-width of spaces
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1640 .*\n+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1641 )+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1642 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1643 ((?=^[ ]{0,%d}\S)|\Z) # Lookahead for non-space at line-start, or end of doc
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1644 # Lookahead to make sure this block isn't already in a code block.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1645 # Needed when syntax highlighting is being used.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1646 (?![^<]*\</code\>)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1647 ''' % (self.tab_width, self.tab_width),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1648 re.M | re.X)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1649 return code_block_re.sub(self._code_block_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1650
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1651 _fenced_code_block_re = re.compile(r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1652 (?:\n\n|\A\n?)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1653 ^```([\w+-]+)?[ \t]*\n # opening fence, $1 = optional lang
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1654 (.*?) # $2 = code block content
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1655 ^```[ \t]*\n # closing fence
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1656 ''', re.M | re.X | re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1657
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1658 def _fenced_code_block_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1659 return self._code_block_sub(match, is_fenced_code_block=True);
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1660
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1661 def _do_fenced_code_blocks(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1662 """Process ```-fenced unindented code blocks ('fenced-code-blocks' extra)."""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1663 return self._fenced_code_block_re.sub(self._fenced_code_block_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1664
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1665 # Rules for a code span:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1666 # - backslash escapes are not interpreted in a code span
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1667 # - to include one or or a run of more backticks the delimiters must
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1668 # be a longer run of backticks
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1669 # - cannot start or end a code span with a backtick; pad with a
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1670 # space and that space will be removed in the emitted HTML
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1671 # See `test/tm-cases/escapes.text` for a number of edge-case
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1672 # examples.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1673 _code_span_re = re.compile(r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1674 (?<!\\)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1675 (`+) # \1 = Opening run of `
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1676 (?!`) # See Note A test/tm-cases/escapes.text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1677 (.+?) # \2 = The code block
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1678 (?<!`)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1679 \1 # Matching closer
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1680 (?!`)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1681 ''', re.X | re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1682
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1683 def _code_span_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1684 c = match.group(2).strip(" \t")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1685 c = self._encode_code(c)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1686 return "<code>%s</code>" % c
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1687
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1688 def _do_code_spans(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1689 # * Backtick quotes are used for <code></code> spans.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1690 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1691 # * You can use multiple backticks as the delimiters if you want to
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1692 # include literal backticks in the code span. So, this input:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1693 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1694 # Just type ``foo `bar` baz`` at the prompt.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1695 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1696 # Will translate to:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1697 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1698 # <p>Just type <code>foo `bar` baz</code> at the prompt.</p>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1699 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1700 # There's no arbitrary limit to the number of backticks you
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1701 # can use as delimters. If you need three consecutive backticks
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1702 # in your code, use four for delimiters, etc.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1703 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1704 # * You can use spaces to get literal backticks at the edges:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1705 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1706 # ... type `` `bar` `` ...
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1707 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1708 # Turns to:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1709 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1710 # ... type <code>`bar`</code> ...
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1711 return self._code_span_re.sub(self._code_span_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1712
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1713 def _encode_code(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1714 """Encode/escape certain characters inside Markdown code runs.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1715 The point is that in code, these characters are literals,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1716 and lose their special Markdown meanings.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1717 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1718 replacements = [
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1719 # Encode all ampersands; HTML entities are not
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1720 # entities within a Markdown code span.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1721 ('&', '&amp;'),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1722 # Do the angle bracket song and dance:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1723 ('<', '&lt;'),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1724 ('>', '&gt;'),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1725 ]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1726 for before, after in replacements:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1727 text = text.replace(before, after)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1728 hashed = _hash_text(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1729 self._escape_table[text] = hashed
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1730 return hashed
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1731
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1732 _strong_re = re.compile(r"(\*\*|__)(?=\S)(.+?[*_]*)(?<=\S)\1", re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1733 _em_re = re.compile(r"(\*|_)(?=\S)(.+?)(?<=\S)\1", re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1734 _code_friendly_strong_re = re.compile(r"\*\*(?=\S)(.+?[*_]*)(?<=\S)\*\*", re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1735 _code_friendly_em_re = re.compile(r"\*(?=\S)(.+?)(?<=\S)\*", re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1736 def _do_italics_and_bold(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1737 # <strong> must go first:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1738 if "code-friendly" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1739 text = self._code_friendly_strong_re.sub(r"<strong>\1</strong>", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1740 text = self._code_friendly_em_re.sub(r"<em>\1</em>", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1741 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1742 text = self._strong_re.sub(r"<strong>\2</strong>", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1743 text = self._em_re.sub(r"<em>\2</em>", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1744 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1745
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1746 # "smarty-pants" extra: Very liberal in interpreting a single prime as an
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1747 # apostrophe; e.g. ignores the fact that "round", "bout", "twer", and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1748 # "twixt" can be written without an initial apostrophe. This is fine because
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1749 # using scare quotes (single quotation marks) is rare.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1750 _apostrophe_year_re = re.compile(r"'(\d\d)(?=(\s|,|;|\.|\?|!|$))")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1751 _contractions = ["tis", "twas", "twer", "neath", "o", "n",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1752 "round", "bout", "twixt", "nuff", "fraid", "sup"]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1753 def _do_smart_contractions(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1754 text = self._apostrophe_year_re.sub(r"&#8217;\1", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1755 for c in self._contractions:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1756 text = text.replace("'%s" % c, "&#8217;%s" % c)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1757 text = text.replace("'%s" % c.capitalize(),
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1758 "&#8217;%s" % c.capitalize())
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1759 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1760
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1761 # Substitute double-quotes before single-quotes.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1762 _opening_single_quote_re = re.compile(r"(?<!\S)'(?=\S)")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1763 _opening_double_quote_re = re.compile(r'(?<!\S)"(?=\S)')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1764 _closing_single_quote_re = re.compile(r"(?<=\S)'")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1765 _closing_double_quote_re = re.compile(r'(?<=\S)"(?=(\s|,|;|\.|\?|!|$))')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1766 def _do_smart_punctuation(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1767 """Fancifies 'single quotes', "double quotes", and apostrophes.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1768 Converts --, ---, and ... into en dashes, em dashes, and ellipses.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1769
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1770 Inspiration is: <http://daringfireball.net/projects/smartypants/>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1771 See "test/tm-cases/smarty_pants.text" for a full discussion of the
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1772 support here and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1773 <http://code.google.com/p/python-markdown2/issues/detail?id=42> for a
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1774 discussion of some diversion from the original SmartyPants.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1775 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1776 if "'" in text: # guard for perf
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1777 text = self._do_smart_contractions(text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1778 text = self._opening_single_quote_re.sub("&#8216;", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1779 text = self._closing_single_quote_re.sub("&#8217;", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1780
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1781 if '"' in text: # guard for perf
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1782 text = self._opening_double_quote_re.sub("&#8220;", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1783 text = self._closing_double_quote_re.sub("&#8221;", text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1784
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1785 text = text.replace("---", "&#8212;")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1786 text = text.replace("--", "&#8211;")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1787 text = text.replace("...", "&#8230;")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1788 text = text.replace(" . . . ", "&#8230;")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1789 text = text.replace(". . .", "&#8230;")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1790 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1791
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1792 _block_quote_re = re.compile(r'''
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1793 ( # Wrap whole match in \1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1794 (
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1795 ^[ \t]*>[ \t]? # '>' at the start of a line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1796 .+\n # rest of the first line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1797 (.+\n)* # subsequent consecutive lines
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1798 \n* # blanks
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1799 )+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1800 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1801 ''', re.M | re.X)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1802 _bq_one_level_re = re.compile('^[ \t]*>[ \t]?', re.M);
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1803
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1804 _html_pre_block_re = re.compile(r'(\s*<pre>.+?</pre>)', re.S)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1805 def _dedent_two_spaces_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1806 return re.sub(r'(?m)^ ', '', match.group(1))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1807
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1808 def _block_quote_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1809 bq = match.group(1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1810 bq = self._bq_one_level_re.sub('', bq) # trim one level of quoting
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1811 bq = self._ws_only_line_re.sub('', bq) # trim whitespace-only lines
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1812 bq = self._run_block_gamut(bq) # recurse
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1813
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1814 bq = re.sub('(?m)^', ' ', bq)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1815 # These leading spaces screw with <pre> content, so we need to fix that:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1816 bq = self._html_pre_block_re.sub(self._dedent_two_spaces_sub, bq)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1817
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1818 return "<blockquote>\n%s\n</blockquote>\n\n" % bq
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1819
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1820 def _do_block_quotes(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1821 if '>' not in text:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1822 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1823 return self._block_quote_re.sub(self._block_quote_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1824
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1825 def _form_paragraphs(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1826 # Strip leading and trailing lines:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1827 text = text.strip('\n')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1828
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1829 # Wrap <p> tags.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1830 grafs = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1831 for i, graf in enumerate(re.split(r"\n{2,}", text)):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1832 if graf in self.html_blocks:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1833 # Unhashify HTML blocks
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1834 grafs.append(self.html_blocks[graf])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1835 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1836 cuddled_list = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1837 if "cuddled-lists" in self.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1838 # Need to put back trailing '\n' for `_list_item_re`
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1839 # match at the end of the paragraph.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1840 li = self._list_item_re.search(graf + '\n')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1841 # Two of the same list marker in this paragraph: a likely
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1842 # candidate for a list cuddled to preceding paragraph
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1843 # text (issue 33). Note the `[-1]` is a quick way to
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1844 # consider numeric bullets (e.g. "1." and "2.") to be
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1845 # equal.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1846 if (li and len(li.group(2)) <= 3 and li.group("next_marker")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1847 and li.group("marker")[-1] == li.group("next_marker")[-1]):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1848 start = li.start()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1849 cuddled_list = self._do_lists(graf[start:]).rstrip("\n")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1850 assert cuddled_list.startswith("<ul>") or cuddled_list.startswith("<ol>")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1851 graf = graf[:start]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1852
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1853 # Wrap <p> tags.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1854 graf = self._run_span_gamut(graf)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1855 grafs.append("<p>" + graf.lstrip(" \t") + "</p>")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1856
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1857 if cuddled_list:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1858 grafs.append(cuddled_list)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1859
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1860 return "\n\n".join(grafs)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1861
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1862 def _add_footnotes(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1863 if self.footnotes:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1864 footer = [
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1865 '<div class="footnotes">',
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1866 '<hr' + self.empty_element_suffix,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1867 '<ol>',
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1868 ]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1869 for i, id in enumerate(self.footnote_ids):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1870 if i != 0:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1871 footer.append('')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1872 footer.append('<li id="fn-%s">' % id)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1873 footer.append(self._run_block_gamut(self.footnotes[id]))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1874 backlink = ('<a href="#fnref-%s" '
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1875 'class="footnoteBackLink" '
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1876 'title="Jump back to footnote %d in the text.">'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1877 '&#8617;</a>' % (id, i+1))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1878 if footer[-1].endswith("</p>"):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1879 footer[-1] = footer[-1][:-len("</p>")] \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1880 + '&#160;' + backlink + "</p>"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1881 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1882 footer.append("\n<p>%s</p>" % backlink)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1883 footer.append('</li>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1884 footer.append('</ol>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1885 footer.append('</div>')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1886 return text + '\n\n' + '\n'.join(footer)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1887 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1888 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1889
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1890 # Ampersand-encoding based entirely on Nat Irons's Amputator MT plugin:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1891 # http://bumppo.net/projects/amputator/
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1892 _ampersand_re = re.compile(r'&(?!#?[xX]?(?:[0-9a-fA-F]+|\w+);)')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1893 _naked_lt_re = re.compile(r'<(?![a-z/?\$!])', re.I)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1894 _naked_gt_re = re.compile(r'''(?<![a-z0-9?!/'"-])>''', re.I)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1895
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1896 def _encode_amps_and_angles(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1897 # Smart processing for ampersands and angle brackets that need
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1898 # to be encoded.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1899 text = self._ampersand_re.sub('&amp;', text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1900
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1901 # Encode naked <'s
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1902 text = self._naked_lt_re.sub('&lt;', text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1903
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1904 # Encode naked >'s
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1905 # Note: Other markdown implementations (e.g. Markdown.pl, PHP
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1906 # Markdown) don't do this.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1907 text = self._naked_gt_re.sub('&gt;', text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1908 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1909
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1910 def _encode_backslash_escapes(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1911 for ch, escape in list(self._escape_table.items()):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1912 text = text.replace("\\"+ch, escape)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1913 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1914
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1915 _auto_link_re = re.compile(r'<((https?|ftp):[^\'">\s]+)>', re.I)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1916 def _auto_link_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1917 g1 = match.group(1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1918 return '<a href="%s">%s</a>' % (g1, g1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1919
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1920 _auto_email_link_re = re.compile(r"""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1921 <
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1922 (?:mailto:)?
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1923 (
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1924 [-.\w]+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1925 \@
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1926 [-\w]+(\.[-\w]+)*\.[a-z]+
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1927 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1928 >
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1929 """, re.I | re.X | re.U)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1930 def _auto_email_link_sub(self, match):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1931 return self._encode_email_address(
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1932 self._unescape_special_chars(match.group(1)))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1933
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1934 def _do_auto_links(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1935 text = self._auto_link_re.sub(self._auto_link_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1936 text = self._auto_email_link_re.sub(self._auto_email_link_sub, text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1937 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1938
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1939 def _encode_email_address(self, addr):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1940 # Input: an email address, e.g. "foo@example.com"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1941 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1942 # Output: the email address as a mailto link, with each character
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1943 # of the address encoded as either a decimal or hex entity, in
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1944 # the hopes of foiling most address harvesting spam bots. E.g.:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1945 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1946 # <a href="&#x6D;&#97;&#105;&#108;&#x74;&#111;:&#102;&#111;&#111;&#64;&#101;
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1947 # x&#x61;&#109;&#x70;&#108;&#x65;&#x2E;&#99;&#111;&#109;">&#102;&#111;&#111;
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1948 # &#64;&#101;x&#x61;&#109;&#x70;&#108;&#x65;&#x2E;&#99;&#111;&#109;</a>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1949 #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1950 # Based on a filter by Matthew Wickline, posted to the BBEdit-Talk
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1951 # mailing list: <http://tinyurl.com/yu7ue>
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1952 chars = [_xml_encode_email_char_at_random(ch)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1953 for ch in "mailto:" + addr]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1954 # Strip the mailto: from the visible part.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1955 addr = '<a href="%s">%s</a>' \
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1956 % (''.join(chars), ''.join(chars[7:]))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1957 return addr
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1958
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1959 def _do_link_patterns(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1960 """Caveat emptor: there isn't much guarding against link
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1961 patterns being formed inside other standard Markdown links, e.g.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1962 inside a [link def][like this].
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1963
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1964 Dev Notes: *Could* consider prefixing regexes with a negative
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1965 lookbehind assertion to attempt to guard against this.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1966 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1967 link_from_hash = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1968 for regex, repl in self.link_patterns:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1969 replacements = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1970 for match in regex.finditer(text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1971 if hasattr(repl, "__call__"):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1972 href = repl(match)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1973 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1974 href = match.expand(repl)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1975 replacements.append((match.span(), href))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1976 for (start, end), href in reversed(replacements):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1977 escaped_href = (
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1978 href.replace('"', '&quot;') # b/c of attr quote
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1979 # To avoid markdown <em> and <strong>:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1980 .replace('*', self._escape_table['*'])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1981 .replace('_', self._escape_table['_']))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1982 link = '<a href="%s">%s</a>' % (escaped_href, text[start:end])
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1983 hash = _hash_text(link)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1984 link_from_hash[hash] = link
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1985 text = text[:start] + hash + text[end:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1986 for hash, link in list(link_from_hash.items()):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1987 text = text.replace(hash, link)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1988 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1989
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1990 def _unescape_special_chars(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1991 # Swap back in all the special characters we've hidden.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1992 for ch, hash in list(self._escape_table.items()):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1993 text = text.replace(hash, ch)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1994 return text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1995
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1996 def _outdent(self, text):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1997 # Remove one level of line-leading tabs or spaces
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1998 return self._outdent_re.sub('', text)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
1999
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2000
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2001 class MarkdownWithExtras(Markdown):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2002 """A markdowner class that enables most extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2003
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2004 - footnotes
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2005 - code-color (only has effect if 'pygments' Python module on path)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2006
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2007 These are not included:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2008 - pyshell (specific to Python-related documenting)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2009 - code-friendly (because it *disables* part of the syntax)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2010 - link-patterns (because you need to specify some actual
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2011 link-patterns anyway)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2012 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2013 extras = ["footnotes", "code-color"]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2014
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2015
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2016 #---- internal support functions
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2017
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2018 class UnicodeWithAttrs(unicode):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2019 """A subclass of unicode used for the return value of conversion to
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2020 possibly attach some attributes. E.g. the "toc_html" attribute when
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2021 the "toc" extra is used.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2022 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2023 metadata = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2024 _toc = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2025 def toc_html(self):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2026 """Return the HTML for the current TOC.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2027
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2028 This expects the `_toc` attribute to have been set on this instance.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2029 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2030 if self._toc is None:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2031 return None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2032
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2033 def indent():
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2034 return ' ' * (len(h_stack) - 1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2035 lines = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2036 h_stack = [0] # stack of header-level numbers
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2037 for level, id, name in self._toc:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2038 if level > h_stack[-1]:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2039 lines.append("%s<ul>" % indent())
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2040 h_stack.append(level)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2041 elif level == h_stack[-1]:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2042 lines[-1] += "</li>"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2043 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2044 while level < h_stack[-1]:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2045 h_stack.pop()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2046 if not lines[-1].endswith("</li>"):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2047 lines[-1] += "</li>"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2048 lines.append("%s</ul></li>" % indent())
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2049 lines.append('%s<li><a href="#%s">%s</a>' % (
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2050 indent(), id, name))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2051 while len(h_stack) > 1:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2052 h_stack.pop()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2053 if not lines[-1].endswith("</li>"):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2054 lines[-1] += "</li>"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2055 lines.append("%s</ul>" % indent())
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2056 return '\n'.join(lines) + '\n'
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2057 toc_html = property(toc_html)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2058
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2059 ## {{{ http://code.activestate.com/recipes/577257/ (r1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2060 _slugify_strip_re = re.compile(r'[^\w\s-]')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2061 _slugify_hyphenate_re = re.compile(r'[-\s]+')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2062 def _slugify(value):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2063 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2064 Normalizes string, converts to lowercase, removes non-alpha characters,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2065 and converts spaces to hyphens.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2066
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2067 From Django's "django/template/defaultfilters.py".
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2068 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2069 import unicodedata
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2070 value = unicodedata.normalize('NFKD', value).encode('ascii', 'ignore').decode()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2071 value = _slugify_strip_re.sub('', value).strip().lower()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2072 return _slugify_hyphenate_re.sub('-', value)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2073 ## end of http://code.activestate.com/recipes/577257/ }}}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2074
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2075
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2076 # From http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/52549
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2077 def _curry(*args, **kwargs):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2078 function, args = args[0], args[1:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2079 def result(*rest, **kwrest):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2080 combined = kwargs.copy()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2081 combined.update(kwrest)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2082 return function(*args + rest, **combined)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2083 return result
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2084
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2085 # Recipe: regex_from_encoded_pattern (1.0)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2086 def _regex_from_encoded_pattern(s):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2087 """'foo' -> re.compile(re.escape('foo'))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2088 '/foo/' -> re.compile('foo')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2089 '/foo/i' -> re.compile('foo', re.I)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2090 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2091 if s.startswith('/') and s.rfind('/') != 0:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2092 # Parse it: /PATTERN/FLAGS
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2093 idx = s.rfind('/')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2094 pattern, flags_str = s[1:idx], s[idx+1:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2095 flag_from_char = {
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2096 "i": re.IGNORECASE,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2097 "l": re.LOCALE,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2098 "s": re.DOTALL,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2099 "m": re.MULTILINE,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2100 "u": re.UNICODE,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2101 }
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2102 flags = 0
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2103 for char in flags_str:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2104 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2105 flags |= flag_from_char[char]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2106 except KeyError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2107 raise ValueError("unsupported regex flag: '%s' in '%s' "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2108 "(must be one of '%s')"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2109 % (char, s, ''.join(list(flag_from_char.keys()))))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2110 return re.compile(s[1:idx], flags)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2111 else: # not an encoded regex
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2112 return re.compile(re.escape(s))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2113
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2114 # Recipe: dedent (0.1.2)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2115 def _dedentlines(lines, tabsize=8, skip_first_line=False):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2116 """_dedentlines(lines, tabsize=8, skip_first_line=False) -> dedented lines
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2117
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2118 "lines" is a list of lines to dedent.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2119 "tabsize" is the tab width to use for indent width calculations.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2120 "skip_first_line" is a boolean indicating if the first line should
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2121 be skipped for calculating the indent width and for dedenting.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2122 This is sometimes useful for docstrings and similar.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2123
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2124 Same as dedent() except operates on a sequence of lines. Note: the
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2125 lines list is modified **in-place**.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2126 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2127 DEBUG = False
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2128 if DEBUG:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2129 print("dedent: dedent(..., tabsize=%d, skip_first_line=%r)"\
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2130 % (tabsize, skip_first_line))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2131 indents = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2132 margin = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2133 for i, line in enumerate(lines):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2134 if i == 0 and skip_first_line: continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2135 indent = 0
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2136 for ch in line:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2137 if ch == ' ':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2138 indent += 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2139 elif ch == '\t':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2140 indent += tabsize - (indent % tabsize)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2141 elif ch in '\r\n':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2142 continue # skip all-whitespace lines
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2143 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2144 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2145 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2146 continue # skip all-whitespace lines
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2147 if DEBUG: print("dedent: indent=%d: %r" % (indent, line))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2148 if margin is None:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2149 margin = indent
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2150 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2151 margin = min(margin, indent)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2152 if DEBUG: print("dedent: margin=%r" % margin)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2153
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2154 if margin is not None and margin > 0:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2155 for i, line in enumerate(lines):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2156 if i == 0 and skip_first_line: continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2157 removed = 0
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2158 for j, ch in enumerate(line):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2159 if ch == ' ':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2160 removed += 1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2161 elif ch == '\t':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2162 removed += tabsize - (removed % tabsize)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2163 elif ch in '\r\n':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2164 if DEBUG: print("dedent: %r: EOL -> strip up to EOL" % line)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2165 lines[i] = lines[i][j:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2166 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2167 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2168 raise ValueError("unexpected non-whitespace char %r in "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2169 "line %r while removing %d-space margin"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2170 % (ch, line, margin))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2171 if DEBUG:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2172 print("dedent: %r: %r -> removed %d/%d"\
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2173 % (line, ch, removed, margin))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2174 if removed == margin:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2175 lines[i] = lines[i][j+1:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2176 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2177 elif removed > margin:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2178 lines[i] = ' '*(removed-margin) + lines[i][j+1:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2179 break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2180 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2181 if removed:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2182 lines[i] = lines[i][removed:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2183 return lines
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2184
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2185 def _dedent(text, tabsize=8, skip_first_line=False):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2186 """_dedent(text, tabsize=8, skip_first_line=False) -> dedented text
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2187
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2188 "text" is the text to dedent.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2189 "tabsize" is the tab width to use for indent width calculations.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2190 "skip_first_line" is a boolean indicating if the first line should
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2191 be skipped for calculating the indent width and for dedenting.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2192 This is sometimes useful for docstrings and similar.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2193
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2194 textwrap.dedent(s), but don't expand tabs to spaces
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2195 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2196 lines = text.splitlines(1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2197 _dedentlines(lines, tabsize=tabsize, skip_first_line=skip_first_line)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2198 return ''.join(lines)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2199
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2200
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2201 class _memoized(object):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2202 """Decorator that caches a function's return value each time it is called.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2203 If called later with the same arguments, the cached value is returned, and
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2204 not re-evaluated.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2205
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2206 http://wiki.python.org/moin/PythonDecoratorLibrary
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2207 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2208 def __init__(self, func):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2209 self.func = func
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2210 self.cache = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2211 def __call__(self, *args):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2212 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2213 return self.cache[args]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2214 except KeyError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2215 self.cache[args] = value = self.func(*args)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2216 return value
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2217 except TypeError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2218 # uncachable -- for instance, passing a list as an argument.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2219 # Better to not cache than to blow up entirely.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2220 return self.func(*args)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2221 def __repr__(self):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2222 """Return the function's docstring."""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2223 return self.func.__doc__
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2224
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2225
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2226 def _xml_oneliner_re_from_tab_width(tab_width):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2227 """Standalone XML processing instruction regex."""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2228 return re.compile(r"""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2229 (?:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2230 (?<=\n\n) # Starting after a blank line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2231 | # or
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2232 \A\n? # the beginning of the doc
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2233 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2234 ( # save in $1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2235 [ ]{0,%d}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2236 (?:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2237 <\?\w+\b\s+.*?\?> # XML processing instruction
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2238 |
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2239 <\w+:\w+\b\s+.*?/> # namespaced single tag
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2240 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2241 [ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2242 (?=\n{2,}|\Z) # followed by a blank line or end of document
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2243 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2244 """ % (tab_width - 1), re.X)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2245 _xml_oneliner_re_from_tab_width = _memoized(_xml_oneliner_re_from_tab_width)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2246
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2247 def _hr_tag_re_from_tab_width(tab_width):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2248 return re.compile(r"""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2249 (?:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2250 (?<=\n\n) # Starting after a blank line
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2251 | # or
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2252 \A\n? # the beginning of the doc
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2253 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2254 ( # save in \1
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2255 [ ]{0,%d}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2256 <(hr) # start tag = \2
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2257 \b # word break
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2258 ([^<>])*? #
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2259 /?> # the matching end tag
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2260 [ \t]*
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2261 (?=\n{2,}|\Z) # followed by a blank line or end of document
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2262 )
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2263 """ % (tab_width - 1), re.X)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2264 _hr_tag_re_from_tab_width = _memoized(_hr_tag_re_from_tab_width)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2265
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2266
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2267 def _xml_escape_attr(attr, skip_single_quote=True):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2268 """Escape the given string for use in an HTML/XML tag attribute.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2269
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2270 By default this doesn't bother with escaping `'` to `&#39;`, presuming that
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2271 the tag attribute is surrounded by double quotes.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2272 """
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2273 escaped = (attr
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2274 .replace('&', '&amp;')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2275 .replace('"', '&quot;')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2276 .replace('<', '&lt;')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2277 .replace('>', '&gt;'))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2278 if not skip_single_quote:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2279 escaped = escaped.replace("'", "&#39;")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2280 return escaped
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2281
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2282
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2283 def _xml_encode_email_char_at_random(ch):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2284 r = random()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2285 # Roughly 10% raw, 45% hex, 45% dec.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2286 # '@' *must* be encoded. I [John Gruber] insist.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2287 # Issue 26: '_' must be encoded.
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2288 if r > 0.9 and ch not in "@_":
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2289 return ch
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2290 elif r < 0.45:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2291 # The [1:] is to drop leading '0': 0x63 -> x63
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2292 return '&#%s;' % hex(ord(ch))[1:]
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2293 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2294 return '&#%s;' % ord(ch)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2295
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2296
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2297
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2298 #---- mainline
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2299
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2300 class _NoReflowFormatter(optparse.IndentedHelpFormatter):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2301 """An optparse formatter that does NOT reflow the description."""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2302 def format_description(self, description):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2303 return description or ""
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2304
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2305 def _test():
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2306 import doctest
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2307 doctest.testmod()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2308
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2309 def main(argv=None):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2310 if argv is None:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2311 argv = sys.argv
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2312 if not logging.root.handlers:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2313 logging.basicConfig()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2314
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2315 usage = "usage: %prog [PATHS...]"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2316 version = "%prog "+__version__
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2317 parser = optparse.OptionParser(prog="markdown2", usage=usage,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2318 version=version, description=cmdln_desc,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2319 formatter=_NoReflowFormatter())
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2320 parser.add_option("-v", "--verbose", dest="log_level",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2321 action="store_const", const=logging.DEBUG,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2322 help="more verbose output")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2323 parser.add_option("--encoding",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2324 help="specify encoding of text content")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2325 parser.add_option("--html4tags", action="store_true", default=False,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2326 help="use HTML 4 style for empty element tags")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2327 parser.add_option("-s", "--safe", metavar="MODE", dest="safe_mode",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2328 help="sanitize literal HTML: 'escape' escapes "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2329 "HTML meta chars, 'replace' replaces with an "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2330 "[HTML_REMOVED] note")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2331 parser.add_option("-x", "--extras", action="append",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2332 help="Turn on specific extra features (not part of "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2333 "the core Markdown spec). See above.")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2334 parser.add_option("--use-file-vars",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2335 help="Look for and use Emacs-style 'markdown-extras' "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2336 "file var to turn on extras. See "
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2337 "<https://github.com/trentm/python-markdown2/wiki/Extras>")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2338 parser.add_option("--link-patterns-file",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2339 help="path to a link pattern file")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2340 parser.add_option("--self-test", action="store_true",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2341 help="run internal self-tests (some doctests)")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2342 parser.add_option("--compare", action="store_true",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2343 help="run against Markdown.pl as well (for testing)")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2344 parser.set_defaults(log_level=logging.INFO, compare=False,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2345 encoding="utf-8", safe_mode=None, use_file_vars=False)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2346 opts, paths = parser.parse_args()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2347 log.setLevel(opts.log_level)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2348
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2349 if opts.self_test:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2350 return _test()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2351
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2352 if opts.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2353 extras = {}
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2354 for s in opts.extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2355 splitter = re.compile("[,;: ]+")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2356 for e in splitter.split(s):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2357 if '=' in e:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2358 ename, earg = e.split('=', 1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2359 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2360 earg = int(earg)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2361 except ValueError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2362 pass
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2363 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2364 ename, earg = e, None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2365 extras[ename] = earg
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2366 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2367 extras = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2368
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2369 if opts.link_patterns_file:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2370 link_patterns = []
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2371 f = open(opts.link_patterns_file)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2372 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2373 for i, line in enumerate(f.readlines()):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2374 if not line.strip(): continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2375 if line.lstrip().startswith("#"): continue
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2376 try:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2377 pat, href = line.rstrip().rsplit(None, 1)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2378 except ValueError:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2379 raise MarkdownError("%s:%d: invalid link pattern line: %r"
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2380 % (opts.link_patterns_file, i+1, line))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2381 link_patterns.append(
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2382 (_regex_from_encoded_pattern(pat), href))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2383 finally:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2384 f.close()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2385 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2386 link_patterns = None
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2387
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2388 from os.path import join, dirname, abspath, exists
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2389 markdown_pl = join(dirname(dirname(abspath(__file__))), "test",
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2390 "Markdown.pl")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2391 if not paths:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2392 paths = ['-']
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2393 for path in paths:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2394 if path == '-':
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2395 text = sys.stdin.read()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2396 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2397 fp = codecs.open(path, 'r', opts.encoding)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2398 text = fp.read()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2399 fp.close()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2400 if opts.compare:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2401 from subprocess import Popen, PIPE
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2402 print("==== Markdown.pl ====")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2403 p = Popen('perl %s' % markdown_pl, shell=True, stdin=PIPE, stdout=PIPE, close_fds=True)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2404 p.stdin.write(text.encode('utf-8'))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2405 p.stdin.close()
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2406 perl_html = p.stdout.read().decode('utf-8')
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2407 if py3:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2408 sys.stdout.write(perl_html)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2409 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2410 sys.stdout.write(perl_html.encode(
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2411 sys.stdout.encoding or "utf-8", 'xmlcharrefreplace'))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2412 print("==== markdown2.py ====")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2413 html = markdown(text,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2414 html4tags=opts.html4tags,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2415 safe_mode=opts.safe_mode,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2416 extras=extras, link_patterns=link_patterns,
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2417 use_file_vars=opts.use_file_vars)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2418 if py3:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2419 sys.stdout.write(html)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2420 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2421 sys.stdout.write(html.encode(
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2422 sys.stdout.encoding or "utf-8", 'xmlcharrefreplace'))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2423 if extras and "toc" in extras:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2424 log.debug("toc_html: " +
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2425 html.toc_html.encode(sys.stdout.encoding or "utf-8", 'xmlcharrefreplace'))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2426 if opts.compare:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2427 test_dir = join(dirname(dirname(abspath(__file__))), "test")
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2428 if exists(join(test_dir, "test_markdown2.py")):
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2429 sys.path.insert(0, test_dir)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2430 from test_markdown2 import norm_html_from_html
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2431 norm_html = norm_html_from_html(html)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2432 norm_perl_html = norm_html_from_html(perl_html)
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2433 else:
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2434 norm_html = html
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2435 norm_perl_html = perl_html
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2436 print("==== match? %r ====" % (norm_perl_html == norm_html))
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2437
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2438
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2439 if __name__ == "__main__":
a4813532bbc6 Added MarkDown support
saskia-hiltemann
parents:
diff changeset
2440 sys.exit( main(sys.argv) )