Releasing your egg

  1. Releasing your egg
    1. Note for Subversion users
    2. Creating a release-info file
      1. Meta-file distribution
      2. files-list
    3. Publishing your egg
    4. Instructions for popular code hosting methods and VCSes
      1. Chicken subversion eggs repository
      2. Bitbucket (mercurial)
        1. Location of release-info file
        2. Making releases
      3. Gitorious (git)
        1. Location of release-info file
        2. Making releases
      4. Github (git)
        1. Location of release-info file
        2. Making releases
      5. Google Code (mercurial, subversion)
        1. Location of release-info file
        2. Making releases
      6. hgweb (mercurial) - aka "hg serve"
        1. Location of release-info file
        2. Making releases
      7. Fossil's default web UI
        1. Location of release-info file
        2. Making releases
      8. Launchpad (bazaar)
        1. Location of release-info file
        2. Making releases
    5. Helper scripts
      1. Subversion
      2. Mercurial
      3. Git
      4. Bazaar
    6. Moving an egg

Let's say you've been reading the eggs tutorial and you are almost done writing your first egg. You've tested that it works locally, and now you probably want to make it available to others. If you would like people to be able to install your egg using chicken-install YOUR-EGG, there are a few steps you need to take.

Note for Subversion users

If you host your egg on code.call-cc.org or you host your own, you don't have to do any of the steps below. Instead, you can use the release generating script at call-cc.org. It does all the steps below automatically.

The URI for an automatically generated release-info for egg YOUR-EGG is http://code.call-cc.org/release-info?egg=YOUR-EGG

This URI can be put in the egg-locations file as described under "Publishing your egg".

If you want more control over the process, please read on.

Creating a release-info file

First, you must create a so-called "release-info" file and make it available on a well-known location via HTTP (discussed below). This is a file which describes where the released versions of your egg can be found.

After everything's set up, releasing a new version of your egg is simply a matter of tagging your release with your VCS of choice and adding a line to this file. See below for some tools to automate even this small step.

It looks something like this:

(repo git "git://example.com/{egg-name}.git")   ; optional

(uri targz "http://example.com/{egg-name}/releases/{egg-name}-{egg-release}.tar.gz")
(release "0.1")
(release "0.2")
(release "1.0")

This example describes where to find the canonical repository and an URI pattern which describes how to find release tarballs. It then goes on to declare three official releases; 0.1, 0.2 and 1.0.

The patterns in the URI enclosed in curly braces are seen as substitution patterns to be replaced by values. egg-name expands to the current egg's name (which is already known when the release-info file is being fetched) and egg-release is replaced by the string in each release declaration.

The supported types for uri are currently targz, tarbz2, zip, meta-file and files-list. The first three are all archives expected to extract to a directory with the egg sources in it (zip files are allowed to expand directly into the files, but this is not recommended). The latter two are special and deserves some more explanation; see the next section for that.

Currently repo is not used by automated tools, but is intended to make it easy for users to find the repository. In preparation for when it will be used in the future, it's a good idea to use consistent names, so please use the short name of the tool. Generally this is how it is invoked on the commandline: svn, mtn, git, bzr, hg, darcs, fossil etc.

Meta-file distribution

Sometimes it is impractical to create archives containing your egg's file contents. The egg releasing system has been designed to require as few manual steps as possible, so it is easy for people to release early and often.

A core ideal of the release system is to make it possible to release simply by tagging your code in a VCS. Some code hosting sites automatically make archives available for each tagged version, thereby making the release available immediately when the code is pushed to the server. For those that don't, the meta-file distribution file type is a way to release by tagging without having to manually make a tarball, as long as there is a way to directly obtain the raw sources of a specific release version via HTTP.

The idea is that each released egg always must have a meta-file to describe its dependencies, its author and license and so on. This meta-file can be used to provide a listing of all the files that are part of an egg. The system that manages egg releases automatically will download all these files and put them in a directory. When someone requests your egg with chicken-install, these files are all served up. Just add a files entry to your meta-file, which lists the files (these are resolved relatively to the meta-file itself).

((files "uri-common.setup" "uri-common.release-info" "uri-common.meta"
        "uri-common.scm" "tests/run.scm")
 (license "BSD")
 (category web)
 (depends (uri-generic 2.3) defstruct matchable)
 (test-depends test)
 (author "Peter Bex")
 (synopsis "Parser for common URI schemes"))

This is a real-world example of the uri-common egg. It lists all the files which it consists of, and these will be downloaded by chicken-install. The disadvantage of this approach is that if you forget a file, your egg is uninstallable, so if you can please use the archive distribution types instead. Another disadvantage is that every time you add, remove or rename a file you need to remember to change the meta-file.

The uri entry in uri-common's release-info file looked like this before there was an autogenerated one:

(uri meta-file "http://anonymous@code.call-cc.org/svn/chicken-eggs/release/4/{egg-name}/tags/{egg-release}/{egg-name}.meta")

Because this egg uses subversion and each release has a corresponding tag, it can simply point to the metafile in the right subdirectory under tags and it will simply work. For other version systems this may require some more messing around.

files-list

This is an extremely stupid format containing a manifest of files in an egg. It's mostly useful when automatically generating lists of files. It starts with a base URI and then lists all the files in the egg. Here's the files-list for version 1.0 of the uri-generic egg:

 http://anonymous:@code.call-cc.org/svn/chicken-eggs/release/4/uri-generic/tags/1.0
 tests/run.scm
 uri-generic.meta
 uri-generic.scm
 uri-generic.setup

The trailing slash after the URI is required.

It's not recommended you use this manually, as this format is subject to change and only really intended to be used as a communication system between pseudo-meta-egg-info and henrietta-cache.

However, if you really need to use this, you can replace the meta-file uri type in the example above by files-list, like the auto-generated release-info file for uri-generic, which starts like this:

 (uri files-list "http://code.call-cc.org/files-list?egg={egg-name};release={egg-release}")
 (release "1.0")
 (release "1.1")

Publishing your egg

After creating the release-info file, you need to make it known to the chicken-install infrastructure that an egg with the given name has a release info file at the location where you published it. This step finally makes it possible for people to say chicken-install YOUR-EGG and have it install your egg, or use (depends YOUR-EGG) in meta-files of their eggs.

Currently this is done by adding a line to the egg-locations file in Subversion, which can be found at https://code.call-cc.org/svn/chicken-eggs/release/4/egg-locations. For security reasons, editing the egg-locations file requires special permissions. So, to have an entry for your egg in that file, send a message to the chicken-users mailing list announcing your egg and requesting the inclusion of its location to egg-locations.

To keep maintenance to a minimum, it's best to add a release-info location which will never change; only the release-info file itself should be changed when you make a new release. This can most easily be accomplished by pointing to the "raw file" view of your trunk/tip/head/master branch in your canonical repository's web interface. That way, this file is kept under version control alongside all the other files in your egg.

If you do have an svn account and don't want to checkout the whole egg repository to be able to just edit this file, you can work around svn's limitations like this:

$ cd /tmp
$ svn co https://code.call-cc.org/svn/chicken-eggs/release/4 --depth empty
$ cd 4
$ svn update egg-locations

If you prefer, you can just publish the file separately via HTTP somewhere and keep overwriting it. If your code hoster doesn't provide an easy way to point to a raw view on a moving "latest version" pointer via HTTP, you could instead opt to store just the release-info files of your eggs in the centralised Chicken eggs subversion repository. Just contact one of the project maintainers or write to the chicken-users mailinglist.

Now we've explained the basic idea, here's an overview of how to figure out the correct URIs and how to automate some steps away.

Chicken subversion eggs repository

The traditional way to distribute eggs has not changed much. We hope to get rid of the manual steps by writing post-commit hooks but for now it is necessary to maintain a meta-file listing all the files in your egg and to create a release-info file containing all your tags and the following repo and uri entries:

(repo svn "http://anonymous@code.call-cc.org/svn/chicken-eggs/release/4/{egg-name}")
(uri meta-file "http://anonymous@code.call-cc.org/svn/chicken-eggs/release/4/{egg-name}/tags/{egg-release}/{egg-name}.meta")}}

You can copy these verbatim. For existing eggs we've already done the work for you.

Bitbucket (mercurial)

Location of release-info file

You can use the code browser to figure out the path to your release-info file. First, copy the link that says "raw". This URI is almost correct but will contain a revision ID hash, so it is pinned to whatever revision is currently the tip. However, you can replace it with the string "tip" and it will still work, and when you visit it again after making changes it will have picked up those changes. Example:

Clicking "spiffy.release-info" and then copying the "raw" link on https://bitbucket.org/sjamaan/spiffy/src gives us https://bitbucket.org/sjamaan/spiffy/raw/d370bdd7ecf8/spiffy.release-info

Just change it to https://bitbucket.org/sjamaan/spiffy/raw/tip/spiffy.release-info and you have the latest tip. This link should be registered in the egg-locations list for your egg and it will automatically be able to fetch any new releases as they are tagged and added to the release-info file.

Making releases

The bitbucket code browser also offers a "get source" link that allows you to fetch the files in that revision as a tarball. The same trick as with the raw files works here; just replace the link's revision ID hash with a symbolic name. You can use tags and bookmarks as symbolic names.

So to make a new release, just tag your release with a well-defined name. If you tag your eggs with the version as tag or bookmark name, you can use the following release-info file. Just don't forget to substitute your bitbucket username!

(repo hg "https://bitbucket.org/YOUR-BITBUCKET-USERNAME/{egg-name}")
(uri targz "https://bitbucket.org/YOUR-BITBUCKET-USERNAME/{egg-name}/get/{egg-release}.tar.gz")
(release "0.1")

Gitorious (git)

Location of release-info file

Use the code browser (the "Source tree" button) to browse to your master branch's release-info file and copy the "Raw blob data" link. For example, for the ssql egg it would look like:

https://gitorious.org/chicken-eggs/ssql/raw/master:ssql.release-info

Making releases

Gitorious makes tags available for download as tarballs. To find the location, go to the source tree browser and switch to a tag in the right-hand panel. You should now get a button in the same panel that says "Download TAG-NAME as tar.gz".

So to make a new release, just tag your release with a well-defined name. If you tag your eggs with the version as tag name, you can use the following release-info file. Just don't forget to substitute your Gitorious project name!

(repo git "git@gitorious.org:PROJECT-NAME/{egg-name}.git")
(uri targz "https://gitorious.org/PROJECT-NAME/{egg-name}/archive-tarball/{egg-release}")
(release "0.1")

Note that there is a Chicken Eggs project on Gitorious already. If you want you can ask its owner to add your egg's repository to it.

Github (git)

Location of release-info file

Use the code browser ("source" tab) to browse to your release-info file and copy the "raw" link. This link should work as-is, as long as you do this while looking at the latest revision of the file in the master branch. For example, for the Kalaha egg it would look like:

https://raw.github.com/DerGuteMoritz/kalaha/master/kalaha.release-info

Making releases

Github makes tags available for download as tarballs. To find the location, click the big "Downloads" button/link in the code browser. It will pop up a selection dialog where you can choose between tarball and "zipball". It will also offer downloads for each tag, but those are zipballs only. However, you can copy the link and replace "zipball" in the URL by "tarball", or you can first click "switch tags" and then open the "Downloads" dialog and it will offer you download links for both tar and zip for that specific tag.

So to make a new release, just tag your release with a well-defined name. If you tag your eggs with the version as tag name, you can use the following release-info file. Just don't forget to substitute your Github username!

(repo git "git://github.com/YOUR-GITHUB-USERNAME/{egg-name}.git")
(uri targz "https://codeload.github.com/YOUR-GITHUB-USERNAME/{egg-name}/tar.gz/{egg-release}")
(release "0.1")

Google Code (mercurial, subversion)

The location of files in Google code is mostly similar for subversion and mercurial, so we're discussing them in one go.

Location of release-info file

On the "source" tab, click "browse" and visit the release-info file. Ensure that you didn't click any revision numbers, otherwise it will remember the specific revision you clicked, even if that was the latest. Look for the link that says "raw" and copy it. It should look something like http://my-egg.googlecode.com/hg/my-egg.release-info for Mercurial or http://my-egg.googlecode.com/svn/trunk/my-egg.release-info for Subversion.

Making releases

Google Code currently does not have a way to automatically serve up tagged revisions as tarballs. They have also shut down the ability to host custom downloads, so right now the only way to host eggs on Google Code is by listing all the files in your egg in its meta-file. See the meta-file section at the start of this wiki page to figure out how to set up a files section in your meta-file.

When you have the list set up in your meta-file, you can then point to it from your release-info file.

For Mercurial, you can base your release-info file off the following snippet:

(repo hg "https://my-egg.googlecode.com/hg/")
(uri meta-file "http://my-egg.googlecode.com/hg/{egg-name}.meta?r={egg-release}")
(release "0.1")

For Subversion, you can use this snippet:

(repo svn "http://my-egg.googlecode.com/svn/")
(uri meta-file "http://my-egg.googlecode.com/svn/tags/{egg-release}/{egg-name}.meta")
(release "0.1")

hgweb (mercurial) - aka "hg serve"

The built-in web interface for mercurial can be used to serve up eggs, usually invoked from hgweb and hgwebdir on a "real" server.

Location of release-info file

Use the browser to navigate to the latest revision of the release-info file and click the "raw" link on the left. Then replace the commit ID hash with "tip". Example:

http://example.com/my-egg/raw-file/tip/my-egg.release-info

Making releases

Since this interface doesn't have a way to serve up tarballs, you must either create your own tarballs and publish them elsewhere or use the manual "meta-file" way.

See the meta-file section at the start of this wiki page to figure out how to set up a files section in your meta-file. The URI to your meta file is similar to the release-info file, except this time you can replace "tip" with the release's tagname. If you use tagnames that are identical to the release version, you can base the release-info file on the following example:

(repo hg "http://example.com/{egg-name}")
(uri meta-file "http://example.com/{egg-name}/raw-file/{egg-release}/{egg-name}.meta")
(release "0.1")

This assumes hgwebdir is running on the domain's root URI and the egg is available in a repo which has the same name as your egg.

Fossil's default web UI

This is the web UI you get when running "fossil serve" or when running fossil as a CGI script.

There is one tricky part: by default, this interface disallows almost all outside access; you must log in, even if you just want to browse anonymously. To allow everybody to download release zipfiles or tarballs without login, add the "z" permission for the user "nobody" in the Admin / Users section of the web UI or with the "fossil user capabilities" command.

Location of release-info file

Fossil's web UI allows serving of documents directly from the repository under URLs of the form "http://example.com/my-egg/doc/{version}/{file}". If the files have a ".txt" or ".wiki" extension, they are rendered into HTML, but anything else is served as is.

For example you can store the release-info file in the repository and always access the latest version of the default branch at a URL like "http://example.com/my-egg/doc/trunk/my-egg.release-info".

Making releases

Fossil can serve tarballs or zipfiles of repository snapshots. A release-info file using zipfile packaging may look like this:

(repo fossil "http://example.com/{egg-name}")
(uri zip "http://example.com/{egg-name}/zip/{egg-name}.zip?uuid=v{egg-release}")
(release "1.0.0")

For tarball packaging, the equivalent uri entry should look like this:

(uri targz "http://example.com/{egg-name}/tarball/{egg-name}.tar.gz?uuid=v{egg-release}")

The releases are referenced using symbolic tags of the form "v{egg-release}" here. You can add these tags to the changesets corresponding to released versions of the egg using the "fossil tag add" command.

Launchpad (bazaar)

Unfortunately, Launchpad doesn't currently seem very suitable for developing eggs in a streamlined way. Here's a description of one possible way to integrate with their way of doing release management though.

Location of release-info file

You can browse to the release-info file using Loggerhead (the web interface to bazaar), but there doesn't seem to be a reliable way to construct a raw download URI for "the latest version of" a file.

A workaround is to create a release series just for the release-info, and make an (unused) milestone for that. Then you can upload a release-info file. The URI that's listed directly on the downloads page is stable though (something like http://launchpad.net/my-project/main/release-info/+download/my-egg.release-info if you named the milestone "release-info" in the "main" series) This link redirects to an unstable URI, so don't paste that one!

When you make a new release you can delete the file and upload a new one manually to the same pseudo-"release".

Making releases

If you do it through their release management, you just create release tarballs which you manually upload into the release milestone.

The only automation here is that you can use bzr export --format=tgz to create it, and the advantage that you're using someone else's site to host your code.

You can use this release-info file template if you are consistent in your milestone naming. It assumes you used the name "main" for the series from which releases for chicken-install are made. You could alternatively add several uri entries, one per series, but it's still required that your egg's release versions are unique for your egg. Series are probably more suitable for distinguishing between major Chicken release versions.

(repo bzr "lp:MY-PROJECT/main")

(uri targz "http://launchpad.net/MY-PROJECT/main/{egg-release}/+download/{egg-name}.tar.gz")
(release "0.1")
(release "0.2")

Helper scripts

To make life easier, there are some helper scripts, hooks etcetera that could be used. Please add your own links here.

Subversion

There's an svn-egg-author egg that helps you to update the release-info and meta files and tag a release of your egg, all with one command.

If you run your own subversion server with Spiffy, you can consider using pseudo-meta-egg-info instead. If you do, you do not need to use svn-egg-author. If you're using the chicken-eggs repository hosted on call-cc.org, you don't need it either because call-cc is already running pseudo-meta-egg-info for all its eggs.

Mercurial

There's an egg-author extension for Mercurial that allows you to say hg eggtag 0.1 and have it automatically update the release-info file with the new tag name.

Git

There's a git-egg-author egg that helps you keep your release-info file up-to-date and tag a release with one command.

Bazaar

There's also a bzr-egg-author extension for Bazaar that allows you to say bzr eggtag 0.1 and have it automatically update the release-info file with the new tag name.

Moving an egg

When you decide to migrate to a different code hosting site, and don't want to move all releases, you can define multiple uris:

;; Main ("default") repo
(repo hg "https://bitbucket.org/YOUR-BITBUCKET-USERNAME/{egg-name}")
(uri targz "https://bitbucket.org/YOUR-BITBUCKET-USERNAME/{egg-name}/get/{egg-release}.tar.gz")
(release "2.1")
(release "2.0")

;; Old repo, using "old-svn-repo" alias
(repo svn "http://anonymous:@code.call-cc.org/svn/chicken-eggs/release/4/{egg-name}")
(uri files-list "http://code.call-cc.org/files-list?egg={egg-name};release={egg-release}" old-svn-repo)
(release "1.3" old-svn-repo)
(release "1.2" old-svn-repo)
(release "1.1" old-svn-repo)
(release "1.0" old-svn-repo)
(release "0.2" old-svn-repo)
(release "0.1" old-svn-repo)