You are looking at historical revision 34190 of this page. It may differ significantly from its current revision.

Eggs

Extension libraries

Extension libraries (eggs) are extensions to the core functionality provided by the basic CHICKEN system, to be built and installed separately. The mechanism for loading compiled extensions is based on dynamically loadable code and as such is only available on systems on which loading compiled code at runtime is supported. Currently these are most UNIX-compatible platforms that provide the libdl functionality like Linux, Solaris, BSD, Mac OS X and Windows using Cygwin.

On systems where dynamic loading is not available, extensions can still be built and linked as static object files.

Note: eggs may also be normal applications or shell scripts, but are usually libraries.

Extensions are technically nothing but dynamically loadable compiled files with added meta-data that describes dependencies to other eggs, version information and things like the author/maintainer of the egg. Three tools provide an easy to use interface for installing eggs, removing them and querying the current status of installed eggs.

Installing eggs

To install an egg, run the chicken-install program with the egg name as argument. The egg archive is downloaded, its contents extracted and the contained egg description file is analyzed and the appropriate commands executed. This file is an abstract description of the contents of the egg and will be translated by chicken-install into build- and installation scripts for the current platform. After running these scripts (and if all goes well), the egg is installed and will be available like a built-in library. The format and meaning of the file will be described below.

Installation will copy a number of given files into the local egg repository or in the path where the CHICKEN executables are located (in the case of executable programs or scripts). Additionally the list of installed files, and user-defined metadata is stored in the repository.

If no egg name is given on the command-line, then all .egg files in the current directory are processed, including all .egg files in a subdirectory chicken (if such a directory exists), in some arbitrary order, unless the egg descriptions specify dependencies.

Installing eggs that use libraries

Sometimes an egg requires a C library to compile. Compilation can fail when your system has this library in a nonstandard location. Normally the C compiler searches in the default locations /usr and /usr/local, and in the prefix where CHICKEN itself was installed. Sometimes this is not enough, so you'll need to supply chicken-install with some extra hints to the C compiler/linker. Here's an example:

 CSC_OPTIONS='-I/usr/pkg/include/mysql -L/usr/pkg/lib/mysql -L -R/usr/pkg/lib/mysql' chicken-install mysql

This installs the mysql egg with the extra compiler options -I and -L to set the include path and the library search path. The second -L switch passes the -R option directly to the linker, which causes the library path to get hardcoded into the resulting extension file (for systems that do not use ld.so.conf).

The environment variables CHICKEN_C_INCLUDE_PATH and CHICKEN_C_LIBRARY_PATH can also be used to override include- and linker-paths. Each of these variables may contain one or more directory names, separated by : or ; and will be passed using -I and -L to the C compiler.

Creating eggs

An egg can be created by placing its code and some special files in a directory named after the desired name of the egg. For example, if your egg is called foo, create a directory called foo and put the egg code there.

Eggs need an egg description file <egg name>.egg. This file indicates how the egg is to be compiled and provides some information about the egg (author, license, dependencies etc).

Format of the egg description file

An egg description is basically an association list holding information about the components of the egg. An egg may contain multiple components: libraries, programs, Scheme or C include files and arbitrary data files. Dependencies between eggs can be specified as can be dependencies between components of an egg.

A list of valid properties follows.

Global properties

version

[egg property] (version STRING)

Specifies version string for this egg. STRING should have the format <MAJOR>.<MINOR>.<PATCHLEVEL>, where only the <MAJOR> part is mandatory.

Eggs from remote egg servers are automatically versioned - the version is part of the protocol to retrieve the egg and does not have to be specified in the .egg file. Eggs installed from local directories (see below) should explicitly specify a version.

synopsis

[egg property] (synopsis STRING)

Gives a short description of this egg.

author

[egg property] (author STRING)

Names the author or authors of the contained code.

maintainer

[egg property] (maintainer STRING)

Names the maintainer of this code, if different from author(s).

category

[egg property] (category NAME)

Gives the category under which this egg should be contained. See the egg index for a list of currently used categories.

license

[egg property] (license STRING)

Names the license under which this code is available.

dependencies

[egg property] (dependencies EGG ...)

Lists eggs that this egg depends on, and which should be built and installed if they do not already exist in the repository. EGG should be whether a symbol or a list of the form EGGNAME VERSION, where the former means to install the newest available egg with this name and the latter specifies a specific version or higher.

Note that this property has a different meaning depending on whether it appears at toplevel in an egg description or inside a component.

test-dependencies

[egg property] (test-dependencies EGG ...)

Lists eggs that are required for this egg to run the tests (if tests exist.) This only has an effect if the -test option has been given to chicken-install.

build-dependencies

[egg property] (build-dependencies EGG ...)

Lists eggs that are build-time dependencies for this egg, i.e. there are required to build, but not to run the contained code. Currently this is treated identical to dependencies.

foreign-dependencies

[egg property] (foreign-dependencies NAME ...)

Lists external dependencies like native code libraries or system-specific packages and is currently only used for documentation purposes.

platform

[egg property] (platform PLATFORM)

Specifies for which platform this egg is intended. PLATFORM should be a symbol naming the target platform (windows, linux or unix) or a boolean combination of platform values, allowed are (not PLATFORM), (or PLATFORM ...) and (and PLATFORM ...). If the expression can not be satisfied, then installation of this egg will abort.

components

[egg property] (components COMPONENT ...)

Lists components (extensions, programs, include- or data files) that this extension installs. See below for information on how to specify component-specific information.

host

[egg property] (host PROP ...)

Recursively process PROP ..., but only for the host (build) platform, in case this is a "cross-chicken", a CHICKEN installation intended for cross compilation.

target

[egg property] (target PROP ...)

Recursively process PROP ..., but only for the target platform, in case this is a "cross-chicken", a CHICKEN installation intended for cross compilation.

Components
extension

[egg property] (extension NAME PROP ...)

Specifies an extension library component. The properties PROP... are processed recursively and apply only to this component.

data

[egg property] (data NAME PROP ...)

Specifies one or more arbitrary data files.

generated-source-file

[egg property] (generated-source-file NAME PROP ...)

Specifies a file that is generated during the process of building the egg.

c-include

[egg property] (c-include NAME PROP ...)

Specifies one or more C include files.

scheme-include

[egg property] (scheme-include NAME PROP ...)

Specifies one or more Scheme include files.

program

[egg property] (program NAME PROP ...)

Specifies an executable program.

Component properties
host

[egg property] (host PROP ...)

Process PROP ... recursively for the current component, but apply the properties only to the host (build) part, when using a CHICKEN installation intended for cross-compilation.

target

[egg property] (target PROP ...)

Process PROP ... recursively for the current component, but apply the properties only to the target part, when using a CHICKEN installation intended for cross-compilation.

linkage

[egg property] (linkage LINKAGE)

Define whether the component should be linked dynamically or statically. LINKAGE can be static or dynamic. This property only makes sense for extension libraries.

types-file

[egg property] (types-file NAME)

Specifies that a "type-database" file should be generated and installed for this component. This property is only used for extension libraries.

inline-file

[egg property] (inline-file NAME)

Specifies that an "inline" file should be generated and installed for this component. This property is only used for extension libraries.

custom-build

[egg property] (custom-build STRING)

Specifies a custom build operation that should be executed instead of the default build operations. This property is mandatory for components of type generated-source-file. STRING usually contains a shell command and thus may be platform sensitive.

csc-options

[egg property] (csc-options OPTION ...)

Specifies additional compiler options for csc that should be used when building this component. If this property is not given, the default options are used, which are -O2 -d1 for extensions and programs and -O2 -d0 for import libraries.

[egg property] (link-options OPTION ...)

Specifies additional link options for csc that should be used when building this component.

modules

[egg property] (modules [MODULE ...])

Specifies the modules that the extension component generates. If this property is not specified, chicken-install assumes that a single module named after the extension file name will be emitted.

This property can be used to specify that the extension doesn't generate any module (in this case (modules) should be used); or to specify the module names that the extension generates (a list of symbols).

source

[egg property] (source NAME)

Specifies an alternative source file, in case it has a name distinct from the component name. By default the source file for a component is named after the component, with the .scm extension added.

install-name

[egg property] (install-name NAME)

Specifies an alternative installation name of the component, if it differs from the actual component name. This property is most useful if an egg installs an extension and a program of the same name, but needs to distinguish the components during build time.

dependencies

[egg property] (dependencies NAME ...)

Specifies dependencies between components. Note that this use of the dependencies property is different from the property of the same name that is used at the "global" level of the egg description file.

destination

[egg property] (destination NAME)

Specifies an alternative installation destination for the built component and only applies to components of type data, c-include and scheme-include. This property should only be used in extreme cases, as it is recommended to use the default installation locations, which are:

files

[egg property] (files NAME ...)

Specifies source files for this component and only applies to components of type data, c-include and scheme-include.

Examples for extensions

A simple library

The simplest case is a single file that does not export any syntax. For example

;;;; hello.scm

(define (hello name)
  (print "Hello, " name " !") )

We need an .egg file to build and install our nifty extension:

;;;; hello.egg

((author "Me")
 (synopsis "A cool hello-world library")
 (license "GPLv3")
 (components (extension hello)))

After entering

$ chicken-install

at the shell prompt (and in the same directory where the two files exist), the file hello.scm will be compiled into a dynamically loadable library and a statically linkable object. If the compilation succeeds, hello.so and hello.o will be stored in the repository, together with a file named hello.egg-info containing an a-list with metadata (what you stored above in hello.egg plus some additional metadata). If no extension name is given to chicken-install, it will simply process the any files with the .egg extension it can find.

Use it like any other CHICKEN extension:

$ csi -q
#;1> (require-library hello)
; loading /usr/local/lib/chicken/8/hello.so ...
#;2> (hello "me")
Hello, me!
#;3>

An application

Here we create a simple application:

;;;; hello2.scm

(print "Hello, ")
(for-each (lambda (x) (printf "~A " x)) (command-line-arguments))
(print "!")

We also need an egg file:

;;; hello2.egg
((author "Me")
 (synopsis "A cool hello-world application")
 (license "proprietary")
 (components (program hello2)))

To use it, just run chicken-install in the same directory:

$ chicken-install

(Here we omit the extension name)

Now the program hello2 will be installed in the same location as the other CHICKEN tools (like chicken, csi, etc.), which will normally be /usr/local/bin. Note that you need write-permissions for those locations and may have to run chicken-install with administrative rights or use the -sudo option.

The extension can be used from the command line:

$ hello2 one two three
Hello, 
one two three !

De-installation is just as easy - use the chicken-uninstall program to remove one or more extensions from the local repository:

$ chicken-uninstall hello2

A module exporting syntax

The hello module was just a shared library, and not a module.

To create an extension that exports syntax see the chapter on Modules. We will show a simple example here: a module my-lib that exports one macro (prog1) and one procedure (my-sum):

;;; my-lib.scm

(module my-lib
  *
  (import scheme chicken)

(define-syntax prog1
  (syntax-rules ()
    ((_ e1 e2 ...)
     (let ((result e1))
       (begin e2 ...)
       result))))

(define my-sum
  (lambda (numbers)
    (prog1
      (apply + numbers)
      (display "my-sum used one more time!")
      (newline))))

)

The prog1 macro is similar to Common Lisp's prog1: it evaluates a list of forms, but returns the value of the first form.

The egg file:

;;; my-lib.egg

((components (extension my-lib))
 (version 1.0)
 (licence "BSD")
 (author "Me again")
 (synopsis "My own cool libraries"))

Running chicken-install on the same directory will install the extension.

Next, it should be possible to load the library:

$ csi -q
#;1> (import my-lib)
; loading /usr/local/lib/chicken/6/my-lib.import.so ...
; loading /usr/local/lib/chicken/6/scheme.import.so ...
; loading /usr/local/lib/chicken/6/chicken.import.so ...
; loading /usr/local/lib/chicken/6/my-lib.so ...
#;2> (my-sum '(10 20 30))
my-sum used one more time!
60
#;3> (my-sum '(-1 1 0))
my-sum used one more time!
0
#;4> (prog1 (+ 2 2) (print "---"))
---
4

Notes on chicken-install

When running chicken-install with an argument NAME, for which no associated .egg file exists, then it will try to download the extension via HTTP from the CHICKEN code repository at http://code.call-cc.org/svn/chicken-eggs/. Extensions that are required to compile and/or use the requested extension are downloaded and installed automatically.

To query the list of currently installed extensions, use chicken-status. It can list what extensions are installed and what files belong to a particular installed extension.

chicken-install reference

Available options:

-h -help
show this message and exit
-version
show version and exit
-force
don't ask, install even if versions don't match
-k -keep
keep temporary files
-s -sudo
use external command to elevate privileges when installing or removing files
-no-install-deps
do not install dependencies
-r -retrieve
only retrieve egg into current directory, don't install (giving -r more than once implies -recursive)
-recursive
if -retrieve is given, retrieve also dependencies
-dry-run
do not build or install, just print the locations of the generated build + install scripts
-list-versions
list available version for an extension (HTTP transport only)
-n -no-install
do not install, only build the egg.
-purge
remove cached files for given eggs (or purge cache completely)
-host
when cross-compiling, compile egg for host only
-target
when cross-compiling, compile egg for target only
-test
run included test-cases, if available
-u -update-db
update export database
-repository
print path to egg repository
-override FILENAME
override versions for installed eggs with information given in FILENAME, which can be generated by -scan or by the -list option of the chicken-status program
-v -verbose
be verbose

chicken-install recognizes the SUDO, http_proxy and proxy_auth environment variables, if set.

chicken-uninstall reference

-h -help
show usage information and exit
-version
show version and exit
-force
don't ask, delete whatever matches
-match
treat egg-names as glob patterns
-s -sudo
use external command to elevate privileges for deleting files
-host
when cross-compiling, remove eggs for host system only
-target
when cross-compiling, remove eggs for target system only

chicken-status reference

-h -help
show usage information and exit
-version
show version and exit
-f -files
list installed files
-match
treat egg-names as glob patterns
-host
when cross-compiling, show eggs for host system only
-target
when cross-compiling, show eggs for target system only
-list
list installed egg version in format suitable for chicken-install -override
-c -components
list installed components

Security

When eggs are downloaded and installed one is executing code from potentially compromised systems. This applies also when chicken-install executes system tests for required eggs. As the code has been retrieved over the network effectively untrusted code is going to be evaluated. When chicken-install is run as root the whole system is at the mercy of the build instructions (note that this is also the case every time you install software via sudo make install, so this is not specific to the CHICKEN egg mechanism).

Security-conscious users should never run chicken-install as root. A simple remedy is to keep the repository inside a user's home directory (see the section "Changing repository location" below). Alternatively obtain write/execute access to the default location of the repository (usually /usr/local/lib/chicken) to avoid running as root. chicken-install also provides a -sudo option to perform the last installation steps as root user, but do building and other .setup script processing as normal. A third solution is to override VARDIR when building the system (for example by passing "VARDIR=/foo/bar" on the make command line, or by modifying config.make. Eggs will then be installed in $(VARDIR)/chicken/8.

Changing the repository location

When CHICKEN is installed a repository for eggs is created and initialized in a default location (usually something like /usr/local/lib/chicken/6/). It is possible to keep an egg repository in another location. This can be configured at build-time by passing VARDIR=<directory> to make(3) or by modifying the config.make configuration file. If you want to override this location after chicken is installed, you can create a repository directory, set the CHICKEN_INSTALL_REPOSITORY and/or CHICKEN_REPOSITORY_PATH environment variables and copy all files from the default repository into the new one.

Note that your binary version can differ from the one given in the examples here, if your chicken version is older or newer than the one used in these examples. Check your default location for the correct binary-version number.

CHICKEN_REPOSITORY_PATH is the place where eggs are to be loaded from for all chicken-based programs. CHICKEN_INSTALL_REPOSITORY is the place where eggs will be installed and which the egg-related tools like chicken-install, chicken-uninstall and chicken-status consult and update.

Static linking

Static linking of extensions and programs is fully supported and should be transparent to the user. Every extension will by default be compiled into a dynamically loadable and a statically linkable entity. By passing -static on the csc command-line, eggs that are available in static form will be linked instead of the dynamically loadable version. Use the linkage egg description property to select in what modes a component should be built.

To identify the necessary object files during linking of extensions, csc creates so-called "link files" and installs them along the statically compiled object file in the local egg repository. These link files specify what objects should be linked when an application is using a static variant of an extension.

Locations

For experimentation or in-house builds of software it may be useful to have private egg repositories in addition to the official CHICKEN egg repository. This can be accomplished by defining so-called "locations", directories that contain egg source-code and description files and which should be scanned before trying to retrieve an egg via the network.

The file <PREFIX>/share/chicken/setup.defaults holds various parameters that define where eggs should be downloaded, together with more obscure options, and can be used to customize the sources where eggs will be retrieved from. Adding a line of the form

(location "<PATH>")

will add <PATH> as an additional egg source, where <PATH> should be a directory in the local filesystem that contains any number of eggs, one directory for each, including the source code and the .egg files for each egg.

Locations are searched before trying to retrieve from the network. Any number of locations may be defined.

The egg cache

Eggs are downloaded and built in the so called "egg cache", an intermediate location used for storing already downloaded source code and for providing a temporary location for building the eggs before they are installed.

By default the cache is located in the directory .chicken-install.cache is the user's home directory ($HOME on UNIX, or %USERPROFILE% on Windows. If the respective environment variable is not set, then /tmp or /Temp is used.

Built extensions and programs remain in the cache, to avoid rebuilding already compiled code and multiple downloads of eggs in case the installation of an egg fails - the dependencies will be cached after the first download and re-download is not needed.

chicken-install tries to take extreme care to avoid stale binaries, but should you be in doubt, simply delete the directory, or run chicken-install -purge to clear the cache or parts of it.

You can override the location of the cache by setting the CHICKEN_EGG_CACHE environment variable.

Egg installation in detail

Retrieval

First the egg names given on the command line (or, if no arguments are given, all eggs identified by .egg files in the current directory, including any in a subdirectory named chicken) must be retrieved, either from a local source or from the official egg repository. Should the egg exist in the egg cache we check whether this cached version is out of date. A cached egg is considered out of date, if a) it is locally available and all cached files belonging to the egg do not have newer timestamps than the local files, or b) if it is a remotely retrieved egg and no newer versions exist on the remote egg server and the last time the cache was filled from the remote location is not later than one hour. Additionally, if any changes in certain environment variables that may influence the compilation of an egg, or if the CHICKEN version changed, then retrieval of the egg sources is enforced in any case.

If the egg is in the current directory, or in a "location" (as described above), the files are copied into the cache. If the egg is remotely available, then it is retrieved via HTTP from one of the egg servers defined in setup.defaults.

Once the egg sources are retrieved and stored in the cache, their .egg files are loaded and validated. After this any egg dependencies are resolved and located in the cache, triggering a recursive retrieval, if necessary.

Preparation

Unless the -retrieve option was given, the eggs intended to be built and installed are now scheduled for compilation. The egg information from the .egg files is processed and translated into build and install scripts for the current platform - if this CHICKEN was configured for cross compilation, and no separate host- or target-build was selected, two sets of build/install scripts will be generated, one for the host system and one for the target.

Building and installation

Unless -dry-run was given on the command-line, the build- and install scripts are now executed, ordered by the dependency relationships between the full set of eggs that are scheduled for compilation. If the -test option was given and a file named run.scm exists in the tests subdirectory of the egg sources, then this script is executed. Should it terminate with an error or a non-zero exit code, the installation is still performed and chicken-install does not abort. Only after all scheduled eggs have been installed, chicken-install will terminate with a non-zero exit code.

Note that the build process attempts to minimize re-building of already compiled files in the cache, using the chicken-do program. See the manual page for chicken-do for more details.

From the egg-information in the .egg file, the set of files installed for a particular egg are added to the egg-information and stored together with the build-artifacts produced by the build scripts.


Previous: Interface to external functions and variables

Next: Bugs and limitations