Outdated egg!

This is an egg for CHICKEN 3, the unsupported old release. You're almost certainly looking for the CHICKEN 4 version of this egg, if it exists.

If it does not exist, there may be equivalent functionality provided by another egg; have a look at the egg index. Otherwise, please consider porting this egg to the current version of CHICKEN.

eggs

Introduction

Streams-based code for converting from the relatively standard and easy to type wiki format to many useful formats.

It currently supports converting to the following formats:

HTML
LaTeX (useful for generating PDF files)
Text (useful for indexing contents without markup, for example for searching).

We are working on implementing support for Texi. OpenDocumentFormat may come after.

This egg provides the parsing functionality used by svnwiki. Using this egg other content management systems could allow its users to upload and modify contents in wiki format and display it in many different formats.

Examples

Converting from wiki format to HTML

The following code receives input from current-input-port in wiki format and converts it to HTML written to current-output-port:

(use stream-ext stream-wiki)

(write-stream (wiki->html (port->stream (current-input-port))))

Note that the resulting page does not include the <html> and <body> tags (to make it easy to include it as part of a page with other stuff).

Converting from wiki format to LaTeX

The following code receives input from current-input-port in wiki format and converts it to LaTeX written to current-output-port.

(use stream-ext stream-wiki)

(write-stream
  (latex-page "english"
              "article"
              (wiki->latex (port->stream (current-input-port)))))

Returning the links in a file

The following code reads information in wiki format from current-input-port and returns a stream with all the local links found. Note that links to absolute URLs are not considered local links and are thus not returned. Each entry in the stream has the form (dst name), where dst is the target of the link and name the text shown in the link.

(use stream-ext stream-wiki)

(wiki-links (port->stream (current-input-port)))

Returning a list of tags in a file

It is possible to include "tags" (in the sense of tagging files with categories) in wiki pages as part of their contents. To do this, the page should include a stream of the form <nowiki>[[tags:tag0 ... tagn]]</nowiki>. This allows users of wiki systems to browse pages by categories (eg. list all pages having tag tag0).

The following code loads a file in wiki format from test.wiki and returns a stream of symbols, one for each tag found.

(use stream-ext stream-wiki)

(wiki-tags (port->stream (open-input-file "test.wiki")))

Authors

This egg is made by Alejandro Forero Cuervo <azul@freaks-unidos.net>.

License

The stream-wiki egg is available under the GNU General Public License.

Requirements

This egg depends on the following:

Converting from wiki to HTML

(wiki->html input [tail] [name] [open] [include] [linktypes] [make-header] [data-output-func] [check-exists?] [extensions] [url-adjust])

Converts from a simple wiki format (which is relatively standard nowadays) into HTML.

input should be a stream of characters (built with the mechanisms provided by SRFI-40 or the stream-ext egg) with the contents in wiki-format; the function returns a stream of characters to be included as part of an HTML page.

Note that the returned stream will be built dynamically as it is read (and so will the input stream be read).

The following are the optional parameteres. Please be aware that they are bound to change in the future, hopefully as the interface is improved or generalized.

tail: Defaults to stream-null. The stream of characters tail will be appended at the end of the parsing.
name: Defaults to "". You should pass the name of the file currently being parsed. The only reason it might be needed is in a TOC entry, passed to the procedure received as the open parameter.
open: Defaults to (constantly stream-null) (for security reasons; we won't allow the wiki contents to open files and include them unless the caller explicitly allows it). This should be a procedure receiving a filename as a string and returning a stream of characters with its associated contents. This is used for TOC entries. If you trust your wiki contents, you could use (compose port->stream open-input-file) here.
include: Defaults to (lambda (name tail) tail). This function is used for files includes into the current file (with an include:file link). It should return the contents that should be shown instead of the include directive, followed by the tail. Currently they are not parsed by stream-wiki but included directly in the output, so if they are wiki contents you might want to call wiki->html on them before returning them.
linktypes: Defaults to (make-hash-table). A list of definitions for types of links, which you can build with load-linktypes.
make-header: Defaults to (make-html-header 1). Function to turn wiki headers into HTML. TODO: Document better.
data-output-func: Defaults to (constantly stream-null). Use that for now. This is used when an extension needs to generate a file, which currently only happens for the math tags. TODO: Document better.
check-exists?: Defaults to (constantly #t). It is a procedure given a pathname. It should check to see if the file exists in the wiki. This is used to know if local links should be rendered in red (meaning the file doesn't exist) or in the regular color.
extensions: Defaults to (make-hash-table). Contains the list of extensions loaded that stream-wiki will use while rendering files. They should be loaded with (load-extensions-from-file).
url-adjust: Defaults to identity. stream-wiki uses this for relative URLs in the wiki. For example, if you want to turn all relative links into absolute links with some specific URL, use this function. The procedure gets passed the link as a stream of characters and is expected to return a stream of characters itself.

Converting from wiki to LaTeX

Converting from wiki to Plain Text

(wiki->text input [human-readable])

Remove all wiki markup from the stream of characters input and produce a corresponding stream of characters for its associated representation as a plain-text file. This is useful, for example, to register the file in a search database (see, for example, the estraier egg).

human-readable specifies whether the resulting file will be optimized to be read by a human (eg. a user reading it over a text-only interface) or a computer (eg. an indexer using the estraier egg).

Analyzing wiki files

Obtaining links

(wiki-links input)

Return a stream with all local (not external) links in the stream of characters input. Each element in the stream is a stream of characters with the destination of the link.

Additional fixes to the texi code
Some tweaks to the latex-driver code

1.10

Imported texi code (from Ivan Raikov)
Some small tweaks and fixes, particularly a bug that would cause stream-wiki to take unearthly amounts of time to parse some strings.3

1.9.2

Fixed bug in the handling of lists.

1.9.1

Small bug fix by Kon.

1.9

Lots of bug fixes.
New handling of lists (see <http://listas.el-directorio.org/pipermail/svnwiki/2007-February/000035.html>).
Better handling of local links. Links to unexistant local files are converted to a canonical form.
Extended the "driver" structure to support the "anchor" object.
Fixed the HTML generation for the anchors that used to wrap the headers.
Better handling for anchors.

1.8

Support for new types of extensions (code-break).
List of symbols that extensions can define is no longer hardcoded.
Allow the caller of wiki->html to pass the object that should be passed as an argument to extensions.
Allow tildes (~) in URLs.
Better handling of tags (as in HTML tags).
If multiple sections have the same name, use numbers to distinguish them in the anchors.
Never emit empty paragraphs (<p></p>).
Better support in the syntax for parameters to tags (as in HTML tags).
Register both internal and external links (previously wiki-links only reported internal links).
Allow the caller to specify for which external links we should use rel=nofollow for the HTML driver (by means of a callback function).
Better handling for definition lists.
Lots of new extensions and improvements to extensions.
Lots of bug fixes.

Other releases

TODO: Convert to the new format.

1.7: Support for new types of extensions. This shouldn't be visible to stream-wiki users that don't use the extensions code advancedly (ie. only svnwiki should view this change). In the future I will strip the extensions code to a separate egg, I think.
1.6: Added wiki->text, small fix in URL handling, adjust URLs in images (using check-exists? and url-adjust, don't strip HTML comments (since some users have reported they need to be able to include HTML comments in their wiki files, eg. for TOL).
1.5: Added url-adjust parameter for local/relative URLs.
1.4: Fixed a typo that was causing crashes during Texi/LaTeX output (again, the <=> string); made extensions see macros; more fixes in URL recognition; many fixes to the extension files.
1.3: Improvements in the recognition of URLs, the generation of TexVC math image files, add support for external extensions, preliminary menus and navigation for texi-driver (by Graham Fawcett), fixed typos that were causing problems with <blockquote> and LaTeX output and with the string <=>.
1.2: Improved the parsing of email addresses and URLs (which in some cases failed to recognized them as such). Parse the result of evaluating linktypes as wiki format recursively (detecting infinite recursion and failing gracefully).
1.1: Lots of improvements, way too many to list here. :-/
1.0: First public release