Porting CHICKEN 4 code to CHICKEN 5

Porting CHICKEN 4 code to CHICKEN 5

This guide will take you by the hand and explain how to port code from CHICKEN 4 to CHICKEN 5. If your code is an egg, you might want to manually compile the source components at first before finishing up the egg itself.

Please read it through and if you hit any issues not in this document which you think are important to note, add them to this page. It's a wiki after all! With your additions you'll be saving others from the same pain.

If you prefer to first fix the egg's compilation infrastructure, scroll down to the section on egg (re-)packaging, then go back up and take care of your code afterwards.

Begin with the easy part: replacing module imports

When you have some CHICKEN 4 code to port to CHICKEN 5, the best way to get started is by first installing all the eggs you know are needed.

Then, in your code, replace every use and require-extension with import, and then compile the code.

This should give lots of errors about unknown modules. If you are using any of the following modules from CHICKEN 4 core, these have moved to an egg so you should install the corresponding egg before proceeding:

Some other procedures have also been moved into eggs, but we'll deal with those later. Installing these "large" eggs will take care of fixing the bulk of your import issues.

Now, after you've installed the required eggs you can recompile, and you'll typically still get errors about nonexisting core modules (like "chicken", "extras", "posix" and so on). The easiest way to get rid of this is to simply remove those modules from your import forms, and then retrying the compilation.

If your code is wrapped in a module, eventually, you'll be left only with errors about missing imports, and cascade-style errors caused by missing macros (these can be very misleading, so pay attention!). For example, when I was porting the uri-generic egg, I got this list after I had replaced use by import and installed the requisite SRFI eggs:

   Warning: reference to possibly unbound identifier `->string' in:
   Warning:    uri-normalize-case
   Warning:    suggesting: `(import chicken.string)'
   
   Warning: reference to possibly unbound identifier `conc'
   Warning:    suggesting: `(import chicken.string)'
   
   Warning: reference to possibly unbound identifier `reverse-list->string' in:
   Warning:    uri-char-list->string
   Warning:    suggesting: `(import chicken.string)'
   
   Warning: reference to possibly unbound identifier `string-intersperse' in:
   Warning:    join-segments
   Warning:    suggesting: `(import chicken.string)'
   
   Warning: reference to possibly unbound identifier `with-output-to-string' in:
   Warning:    uri->string
   Warning:    suggesting: `(import chicken.port)'
   
   Warning: reference to possibly unbound identifier `fprintf'
   Warning:    suggesting: `(import chicken.format)'
   
   Warning: reference to possibly unbound identifier `out'
   
   Warning: reference to possibly unbound identifier `x'
   
   Warning: reference to possibly unbound identifier `define-record-printer'
   Warning:    suggesting: `(import chicken.base)'
   
   Warning: reference to possibly unbound identifier `URIAuth-port-set!'
   
   Warning: reference to possibly unbound identifier `URIAuth-port' in:
   Warning:    uri-auth->list
   Warning:    uri-auth-equal?
   Warning:    uri-port
   Warning:    update-URIAuth
   
   Warning: reference to possibly unbound identifier `URIAuth-host-set!'
   
   Warning: reference to possibly unbound identifier `URIAuth-host' in:
   Warning:    uri-auth->list
   Warning:    uri-auth-equal?
   Warning:    uri-host
   Warning:    update-URIAuth
   
   Warning: reference to possibly unbound identifier `URIAuth-password-set!'
   
   Warning: reference to possibly unbound identifier `URIAuth-password' in:
   Warning:    uri-auth->list
   Warning:    uri-auth-equal?
   Warning:    uri-password
   Warning:    update-URIAuth
   
   Warning: reference to possibly unbound identifier `URIAuth-username-set!'
   
   Warning: reference to possibly unbound identifier `URIAuth-username' in:
   Warning:    uri-auth->list
   Warning:    uri-auth-equal?
   Warning:    uri-username
   Warning:    update-URIAuth
   
   Warning: reference to possibly unbound identifier `URIAuth?' in:
   Warning:    ucdiff?
   Warning:    uri-auth->list
   
   Warning: reference to possibly unbound identifier `password'
   
   Warning: reference to possibly unbound identifier `username'
   
   Warning: reference to possibly unbound identifier `make-URIAuth' in:
   Warning:    authority
   Warning:    loop
   
   Warning: reference to possibly unbound identifier `<URIAuth>'
   
   Warning: reference to possibly unbound identifier `URI-fragment-set!'
   
   Warning: reference to possibly unbound identifier `URI-fragment' in:
   Warning:    uri->list
   Warning:    uri-equal?
   Warning:    update-uri*
   Warning:    update-URI
   
   Warning: reference to possibly unbound identifier `URI-query-set!'
   
   Warning: reference to possibly unbound identifier `URI-query' in:
   Warning:    uri-relative-to
   Warning:    uri->list
   Warning:    uri-equal?
   Warning:    update-uri*
   Warning:    update-URI
   
   Warning: reference to possibly unbound identifier `URI-path-set!'
   
   Warning: reference to possibly unbound identifier `URI-path' in:
   Warning:    uri-relative-to
   Warning:    uri->list
   Warning:    uri-equal?
   Warning:    update-uri*
   Warning:    update-URI
   
   Warning: reference to possibly unbound identifier `URI-authority-set!'
   
   Warning: reference to possibly unbound identifier `URI-authority' in:
   Warning:    uri-relative-to
   Warning:    uri->list
   Warning:    uri-equal?
   Warning:    loop
   Warning:    uri-password
   Warning:    uri-username
   Warning:    uri-port
   Warning:    uri-host
   Warning:    update-URI
   
   Warning: reference to possibly unbound identifier `URI-scheme-set!'
   
   Warning: reference to possibly unbound identifier `URI-scheme' in:
   Warning:    uri-relative-to
   Warning:    uri->list
   Warning:    uri-equal?
   Warning:    update-uri*
   Warning:    update-URI
   
   Warning: reference to possibly unbound identifier `URI?' in:
   Warning:    uri->list
   Warning:    uri->string
   
   Warning: reference to possibly unbound identifier `path'
   
   Warning: reference to possibly unbound identifier `make-URI' in:
   Warning:    failure29947
   Warning:    relative-ref
   Warning:    uri
   Warning:    make-uri
   Warning:    make-uri*
   Warning:    loop
   
   Warning: reference to possibly unbound identifier `<URI>'
   
   Warning: reference to possibly unbound identifier `define-record-type'
   Warning:    suggesting one of:
   Warning:    (import chicken.base)
   Warning:    (import srfi-9)
   
   Warning: reference to possibly unbound identifier `error'
   Warning:    suggesting one of:
   Warning:    (import chicken.base)
   Warning:    (import srfi-23)
   
   Error: module unresolved: uri-generic

This long list looks pretty intimidating, but it's not that hard to fix, actually. The way to do that is to quickly skim the suggestions. Oftentimes you'll get lots of errors about procedures that don't directly come from missing imports, like in this case (all the URI-blabla procedures being undefined).

If you look closely, you'll notice that all of these follow a certain naming pattern: URI-foo or URI-foo-set!. This points us to the fact that they are record accessors. You'll also notice the suggestion mentioning define-record-type, which is exactly what we need to import to fix all these errors.

Especially when you're using defining macros like you'll see lots of these kinds of errors repeated all over the place where the definitions are used. So, be aware that this can make the output a bit confusing. If you have large macros with lots of keywords. That's why it's usually best to start getting your macro imports in order, then recompiling to get rid of all the excess noise.

In my example of uri-generic, after just adding (import (chicken base)), which I prefer over (import chicken.base), the number of warnings/errors were drastically reduced:

   Warning: reference to possibly unbound identifier `->string' in:
   Warning:    uri-normalize-case
   Warning:    suggesting: `(import chicken.string)'
   
   Warning: reference to possibly unbound identifier `conc'
   Warning:    suggesting: `(import chicken.string)'
   
   Warning: reference to possibly unbound identifier `reverse-list->string' in:
   Warning:    uri-char-list->string
   Warning:    suggesting: `(import chicken.string)'
   
   Warning: reference to possibly unbound identifier `string-intersperse' in:
   Warning:    join-segments
   Warning:    suggesting: `(import chicken.string)'
   
   Warning: reference to possibly unbound identifier `with-output-to-string' in:
   Warning:    uri->string
   Warning:    suggesting: `(import chicken.port)'
   
   Warning: reference to possibly unbound identifier `fprintf'
   Warning:    suggesting: `(import chicken.format)'
   
   Error: module unresolved: uri-generic

After adding the suggested imports, the code compiled cleanly and the tests ran without problems, after I fixed a few trivial imports in the test suite.

So you see, even for medium sized eggs like uri-generic, porting can be a breeze. But beware, there are situations where things can get hairier!

Fixing more missing procedures by installing eggs

After you've fixed your core imports and installed the SRFI eggs, you might notice some problems if you're using particular procedures from old core modules that are no longer available. Some of those have been moved into separate eggs. If this is the case with your code, take a look through the list below.

Some procedures from the "utils" module have been moved into several eggs:

Some procedures from "lolevel" have been moved into an object-evict egg:

object-evict
object-evict-to-location
object-evicted?
object-release
object-unevict
object-size

The following procedures from data-structures have been moved to the queues egg:

list->queue
make-queue
queue?
queue-length
queue->list
queue-add!
queue-empty?
queue-first
queue-last
queue-remove!
queue-push-back!
queue-push-back-list!

And the binary-search procedure from data-structures has also been put into its own egg.

From the posix module, memory mapped I/O has been moved into its own memory-mapped-files egg. Specifically, these procedures have been moved there:

memory-mapped-file?
map-file-to-memory
memory-mapped-file-pointer
unmap-file-from-memory

Also removed from posix are the group information procedures, which have been moved to the posix-groups egg. These are the following:

group-information
initialize-groups
get-groups
set-groups!

Lastly, a few macros from chicken were moved into the miscmacros egg:

eval-when
ensure
select

Random and random-bsd

The random procedure produced very weak random numbers, so we've deleted it. Instead, you can import (chicken random) and use pseudo-random-integer, which should produce random numbers in the range between 0 and the number you supply (exclusive). Its argument must be an exact integer.

If you were using the random-bsd egg (which produced slightly better random numbers), it's probably best to use the built-in random number generator.

If before you would divide by a floating-point number to get a number in the range 0..1, you can now use (pseudo-random-real), which will do the same thing, but in a better way.

If you need to initialize the PRNG to a known state, instead of randomize, you can use set-pseudo-random-seed!, but be aware that the numbers you'll get will be different.

For cryptographic uses or other uses that require "really" random numbers, you can use random-bytes, which returns numbers from the operating system's entropy source.

Fixing more subtle issues

Then, there are some semantic changes. These are of course listed in the NEWS file, but let's take a look at some of the more important ones:

read-string returns `#!eof` instead of `""` at end of file

The title says it all: The read-string procedure now returns #!eof when the end of file is hit, instead of an empty string. This is more consistent and allows one to distinguish between a successful read of zero characters and an exhausted file descriptor.

This is listed first because it's quite tricky and can cause very subtle bugs, so be very careful about this one in particular. I hope you have a good test suite! :)

'...'expected syntax-transformer, but got: #<procedure (? f r c)>

You are probably using a define-syntax call with a procedure as argument. This has been deprecated. You need to wrap this inside a er-macro-transformer or ir-macro-transformer expression.

process, process* and process-execute argument type change

The third argument for these three procedures is now an association list instead of a list of strings.

For example, this CHICKEN 4 code:

(process-execute "/usr/bin/env" '() '("VARIABLE=VALUE"))

Becomes this for CHICKEN 5:

(process-execute "/usr/bin/env" '() '(("VARIABLE" . "VALUE")))

repository-path type change

The (repository-path) parameter, living in the (chicken platform) module, now contains a list of strings instead of a string, representing the multiple repository directories requested by the user.

Change in the code would look like this, for example:

;; Changing this:
(repository-path "awesome-repository")
;; to this
(repository-path '("awesome-repository")) ;; if you want to replace the user choice
;; or
(repository-path (cons "awesome-repository" (repository-path))) ;; if you want to extend it

FFI integers arguments require fixnum or bignum values

Now that support for the full numeric tower, support for floating point values behaving like integers has been removed. The FFI layer does not convert flonum to integers when passing them as arguments to C functions any longer.

;; As an example:

 ;; might come from an external library, like JSON parsing (where every number is floating point)
(define milliseconds-time 1534096420492.0)

;; CHICKEN 4 way
; (seconds->local-time (/ milliseconds-time 1000))

;; CHICKEN 5 way
(seconds->local-time (inexact->exact (round (/ milliseconds-time 1000))))

Linking to external libraries

In CHICKEN 5, the -l flag was removed, you now have to use -L -llib.

For example:

# Something compiled like so:
csc program.scm -lncurses
# Would now be compiled like this:
csc program.scm -L -lncurses

Manually created record instances

In CHICKEN 5 structure tags created by records (as defined with define-record, etc. are by default qualified with the module name. Sometimes code creates these record instances manually (i.e. by using ##sys#make-structure, which must take this qualification into account, e.g.:

(module foo ()
  (import scheme (chicken base))
  (define-record baz)
  (print (eq? (make-baz) (##sys#make-structure 'baz))))   ; ==> prints "#f"

Re-packaging eggs

If your code is an egg, you'll have to rewrite your setup file into the new "declarative" format. There's a short description on how to create eggs for CHICKEN 5 in the manual, as well as a complete reference on the new file format.

Basically, the new way to make an egg involves writing a .egg file. This looks a lot like the old .meta file, with the author, name, license and dependency declarations. Added to this are "components", which make up the egg's sources.

For simple eggs, this is also trivial. Here's uri-generic's old .meta file:

((egg "uri-generic.egg")
 (license "BSD")
 (category web)
 (needs matchable)
 (test-depends test)
 (author "Ivan Raikov, Peter Bex, Seth Alves")
 (synopsis "URI generic syntax (RFC 3986) parsing and manipulation."))

And its .setup file:

(standard-extension 'uri-generic "2.43")

After porting this to CHICKEN 5's .egg file, you get:

((license "BSD")
 (category web)
 (dependencies matchable srfi-1 srfi-13 srfi-14)
 (test-dependencies test)
 (author "Ivan Raikov, Peter Bex, Seth Alves")
 (synopsis "URI generic syntax (RFC 3986) parsing and manipulation.")
 (components (extension uri-generic)))

As you can see, for eggs consisting of a single file, this is very straightforward: just add any missing egg dependencies for the stuff that has moved out of core, remove obsolete forms like (egg ..), replace needs or depends with dependencies, replace test-depends to test-dependencies and add a (components (extension ..)) entry which corresponds to standard-extension from the old .setup file.

For more complex examples of .egg files, just look through the list of available CHICKEN 5 eggs. There are a few in there with multiple source files and even some with custom build scripts.

Porting other author's eggs

If you want to port old eggs someone else made to a new Chicken version, follow the steps above for the porting itself. Other than that, there are two things you should regard.

First, you can find the repositories of Chicken eggs on these lists:

Second, be sure to check back with the original author (their email address is usually in the metadata files of the egg) so they're aware of and okay with you porting their code. And finally, to get the egg listed in the index, announce it on the mailing list.