1. internet-message
    1. Description
    2. Usage
    3. Library Procedures
    4. internet-message-lens: bidirectional parsing
      1. Words, phrases, and unstructured text
      2. Date and time
      3. Addresses
      4. Message identifiers
      5. Body
      6. Header fields and messages
      7. Example
    5. Repository
    6. Requires
    7. Version History
    8. License

internet-message

Description

internet-message is a collection of parser combinators for the grammar defined in RFC 5322 (Internet Message Format).

Usage

The combinator procedures in this library are based on the interface provided by the abnf library.

Each procedure exported by internet-message parser combinator of the form (lambda (cont s) ...), which takes a continuation and input stream. For instance, the message parser combinator can be used to parse messages:

;; A procedure which creates an input stream from a string
(define (string->input-stream s) `(() ,(string->list s)))

;; A procedure used to report parse errors 
(define (err s)
  (print "internet message error on stream: " s)
  (list))

(let* (;; Parser combinator procedure which takes continuation and input stream
         (parse-message (lambda (cont s) ((message) (compose cont car) err s)))
         (my-message "From: John Doe <jdoe@machine.example>\r\nTo: Mary Smith <mary@example.net>\r\nSubject: Saying Hello\r\nDate: Fri, 21 Nov 1997 09:55:06 -0600\r\nMessage-ID: <1234@local.machine.example>\r\n\r\nThis is a message just to say hello.\r\nSo, \r\n\r\n\"Hello\".")
          )        

  (parse-message (lambda (s) (test (apply sprintf "~S -> ~S" p) res s))
                 (string->input-stream inp))
  )
->
(message 
  (fields (From (mailbox-list (mailbox (display-name (" John " "Doe ")) (local-part "jdoe") (domain "machine.example"))))
          (To (mailbox (display-name (" Mary " "Smith ")) (local-part "mary") (domain "example.net")))
          (Subject " Saying Hello") 
	  (Date (day-of-week "Fri") (date "21" "Nov" "1997") (time "09" "55" "06" "-" "06" "00"))
	  (Message-id  (message-id "1234" "local.machine.example")))
  (body "This is a message just to say hello." "So, " "\"Hello\"."))

Library Procedures

The following procedures are exported:

[procedure] fields

This parser will parse an arbitrary number of header fields as defined in the RFC. For each field, an appropriate alist is created. The following fields are recognized:

[procedure] body

This parser will parse a message body as specified by the RFC; that is, any number of text characters, which may be divided into separate lines by CRLF.

[procedure] message

This parser will parse a complete message as defined by the RFC and it will break it down into the separate header fields and the message body.

[procedure] comment

This parser parses comment text, as defined by the RFC. Comments may nest.

internet-message-lens: bidirectional parsing

internet-message-lens builds on abnf's abnf-lens component to add bidirectional transforms to a representative part of the grammar above: besides parsing text into a value, each rule below can also print a value back out as text that the same rule accepts. A From: header built as a NamedMailbox record, for instance, prints as "From: John Doe <jdoe@example.com>\r\n", and parsing that line back reproduces the record.

Every rule below is a bp (bidirectional parser), used with bp-parse and bp-print; the bp record and these two procedures, along with bi-iso, bi-seq, define-bi-rule, define-bi-datatype, and the other combinators the rules below are built from, are documented on the abnf page. A record type declared with define-bi-rule gets a constructor, a predicate, and one accessor per field, the same as define-record-type. A sum type declared with define-bi-datatype gets a constructor per variant and a predicate.

Comments and folding whitespace are canonicalized rather than preserved: dropped when parsing, and printed back as either nothing or a single space. A value built by a program therefore always prints as valid, readable text, and a value obtained by parsing always prints back to text with the same meaning.

Out of scope: the trace fields (Received, Return-Path), the Unicode text variants, and parts.

Words, phrases, and unstructured text

[procedure] (bi-atom) => BP

A single atom (RFC 5322 section 3.2.3), such as "jdoe": one or more letters, digits, or the punctuation marks !#$%&'*+-/=?^_`{|}~, folded to a string.

[procedure] (bi-dot-atom) => BP
[procedure] (bi-dot-atom-text-string) => BP

A dot-separated run of atoms, such as "jane.doe", folded to one string with the dots kept as part of it. bi-dot-atom additionally allows surrounding comments and folding whitespace, the way dot-atom does in the original grammar; bi-dot-atom-text-string does not, the way dot-atom-text does not, and is used where the RFC calls for dot-atom-text directly (a message ID's left and right parts, for instance).

[procedure] (bi-quoted-string) => BP

A quoted string, such as "\"Joe Q. Public\"". The surrounding quotes are dropped from the value and reconstructed on print, so the value itself is just the plain content string, e.g. "Joe Q. Public"; an embedded quote or backslash is escaped automatically on print.

[procedure] (bi-word) => BP

An atom or a quoted string, whichever the input is. On print, a plain Scheme string that is entirely made of atom characters prints unquoted; any other string (containing a space, a quote, ...) prints quoted.

[procedure] (bi-phrase) => BP
[procedure] (bi-display-name) => BP

One or more words, folded to a list of word strings, e.g. ("John" "Doe") for "John Doe". bi-display-name is the same rule under the name the RFC uses for it in an address.

[procedure] (bi-unstructured) => BP

Free text such as a Subject: value, folded to a single string; an embedded line fold prints back as one canonical space.

Date and time

[record] date-spec DAY MONTH YEAR
[procedure] (bi-date-spec) => BP

A calendar date, e.g. the "29" "Aug" "2008" in 29 Aug 2008. Accessors: date-spec-day, date-spec-month, date-spec-year.

[record] time-spec HOUR MINUTE SECOND ZONE-SIGN ZONE-HOUR ZONE-MINUTE
[procedure] (bi-time-spec) => BP

A time of day and zone offset, e.g. 12:21:46 +0200. SECOND is #f when absent, as in 12:21; ZONE-SIGN is the character #\+ or #\-. Accessors: time-spec-hour, time-spec-minute, time-spec-second, time-spec-zone-sign, time-spec-zone-hour, time-spec-zone-minute.

[record] date-time-spec DAY-OF-WEEK DATE TIME
[procedure] (bi-date-time-spec) => BP

A complete Date:-style value, e.g. Fri, 29 Aug 2008 12:21:46 +0200. DAY-OF-WEEK is a bare weekday string such as "Fri", or #f when the input has none -- either way, printing supplies exactly one correctly-placed separator. DATE is a date-spec and TIME a time-spec. Accessors: date-time-spec-day-of-week, date-time-spec-date, date-time-spec-time.

Addresses

[record] addr-spec LOCAL-PART DOMAIN
[procedure] (bi-addr-spec) => BP

A bare e-mail address, e.g. jdoe@machine.example. LOCAL-PART and DOMAIN are plain strings; a quoted local part or a bracketed domain literal is unwrapped the same way a quoted-string is elsewhere in this module. Accessors: addr-spec-local-part, addr-spec-domain.

mailbox -- a datatype, predicate mailbox?<br> <procedure>(bi-mailbox) => BP</procedure><br> <procedure>(bi-mailbox-list) => BP</procedure><br>

One mailbox: either

 (NamedMailbox DISPLAY-NAME ADDRESS)   ; DISPLAY-NAME as in bi-phrase, ADDRESS an addr-spec
 (BareMailbox ADDRESS)                 ; ADDRESS an addr-spec

NamedMailbox prints with a space before the angle brackets, e.g. John Doe <jdoe@machine.example>; BareMailbox prints as the bare address alone. bi-mailbox-list matches one or more, comma-separated, folded to a list of mailbox values.

[record] group DISPLAY-NAME MAILBOXES
[procedure] (bi-group) => BP
[procedure] (bi-group-list) => BP

A named group of mailboxes, e.g. A Group: Ed Jones <c@a.test>, joe@where.test;. MAILBOXES is a (possibly empty) list of mailbox values. bi-group-list is the group's own mailbox-list-or-nothing rule, exported in case it is useful on its own. Accessors: group-display-name, group-mailboxes.

address -- a datatype, predicate address?<br> <procedure>(bi-address) => BP</procedure><br> <procedure>(bi-address-list) => BP</procedure><br> <procedure>(bi-address-list-or-empty) => BP</procedure><br>

A single mailbox or a group:

 (MailboxAddress MAILBOX)   ; MAILBOX a mailbox
 (GroupAddress GROUP)       ; GROUP a group

bi-address-list matches one or more, comma-separated, folded to a list of address values; bi-address-list-or-empty additionally allows an empty list (for Bcc:-style headers, which may carry no addresses at all).

Message identifiers

[record] msg-id ID-LEFT ID-RIGHT
[procedure] (bi-msg-id) => BP
[procedure] (bi-msg-id-list1) => BP

A message identifier, e.g. <1234@local.machine.example>, with the angle brackets dropped, not stored. bi-msg-id-list1 matches one or more, with no separator character required between them (each one self-delimits), folded to a list of msg-id values, and printed with a canonical space between them. Accessors: msg-id-id-left, msg-id-id-right.

Body

[procedure] (bi-body) => BP

A message body, folded to a list of line strings, one per line, with the CRLFs between them dropped, not stored. A run of two or more consecutive CRLFs -- a blank line in the body -- collapses to a single one on print, the same way internet-message.scm's own body grammar collapses it on parse, rather than printing back as an empty line of its own.

Header fields and messages

header-field -- a datatype, predicate header-field?<br> <procedure>(bi-header-field) => BP</procedure><br> <procedure>(bi-fields) => BP</procedure><br>

One header field. Every recognized header is its own variant, its fixed keyword printed with a canonical single space before the value that follows it:

Variant Header Field(s)
FromField From: MAILBOXES, a list of mailbox
SenderField Sender: MAILBOX
ReplyToField Reply-To: ADDRESSES, a list of address
ToField To: ADDRESSES
CcField Cc: ADDRESSES
BccField Bcc: ADDRESSES, possibly empty
MessageIdField Message-ID: ID, a msg-id
InReplyToField In-Reply-To: IDS, a list of msg-id
ReferencesField References: IDS
SubjectField Subject: TEXT, a string
CommentsField Comments: TEXT
KeywordsField Keywords: PHRASES, a list of phrases (each a list of word strings)
DateField Date: DATE, a date-time-spec
ResentDateField Resent-Date: DATE
ResentFromField Resent-From: MAILBOXES
ResentSenderField Resent-Sender: MAILBOX
ResentToField Resent-To: ADDRESSES
ResentCcField Resent-Cc: ADDRESSES
ResentBccField Resent-Bcc: ADDRESSES, possibly empty
ResentMessageIdField Resent-Message-ID: ID
ResentReplyToField Resent-Reply-To: ADDRESSES
OptionalField any other name NAME (the exact header name, case preserved) and TEXT

Unlike internet-message.scm's fields parser, OptionalField keeps the header name as a plain string in its original casing rather than a titlecased symbol, since a printed value has to reparse to the same value, and a case-preserving string does that with no extra work.

bi-fields matches zero or more header fields in a row, folded to a list of header-field values, trying the specific headers above before falling back to OptionalField -- the same order internet-message.scm's own fields parser uses.

[record] message FIELDS BODY
[procedure] (bi-message) => BP

A complete message. FIELDS is a list of header-field values. BODY is #f when the message has no blank line and body at all, or a (possibly empty) list of line strings when it does -- the two are easy to conflate, but not the same thing. Accessors: message-fields, message-body.

Example

 
(import internet-message-lens abnf-lens)

(define (parse-error s) (error "could not parse" s))

(define msg
  (car (car (bp-parse
             bi-message
             (string-append
              "From: John Doe <jdoe@machine.example>\r\n"
              "To: Mary Smith <mary@example.net>\r\n"
              "Subject: Saying Hello\r\n"
              "\r\n"
              "This is a message just to say hello.")
             parse-error))))

(bp-print bi-message msg)
;; => "From: John Doe <jdoe@machine.example>\r\nTo: Mary Smith <mary@example.net>\r\nSubject: Saying Hello\r\n\r\nThis is a message just to say hello."

;; Building a message from scratch works the same way:
(bp-print bi-message
          (make-message
           (list (FromField (list (BareMailbox (make-addr-spec "jdoe" "example.com")))))
           #f))
;; => "From: jdoe@example.com\r\n"

The test suite, tests/lens-run.scm, has one round-trip test group per rule above.

Repository

https://github.com/iraikov/chicken-internet-message

Requires

Version History

License

Based on the Haskell Rfc2822 module by Peter Simons.

       Copyright 2009-2026 Ivan Raikov.


 This program is free software: you can redistribute it and/or
 modify it under the terms of the GNU General Public License as
 published by the Free Software Foundation, either version 3 of the
 License, or (at your option) any later version.

 This program is distributed in the hope that it will be useful, but
 WITHOUT ANY WARRANTY; without even the implied warranty of
 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
 General Public License for more details.

 A full copy of the GPL license can be found at
 <http://www.gnu.org/licenses/>.