The CHICKEN Gazette

The Gazette is produced weekly by a volunteer from the CHICKEN community. The latest issue can be found at http://gazette.call-cc.org or you can follow it in your feed reader at http://gazette.call-cc.org/feed.atom. The Gazette Archive is at http://gazette.call-cc.org/archive.html. If you'd like to write an issue, check out the instructions and come and find us in #chicken on Freenode!

Releases

The gazette will be released weekly on Mondays. The usual distribution channel is the chicken-users mailing list and the gazette site.

Who can be editor for a gazette?

Short answer: You can! You don't have to be an expert on scheme for this, indeed summarizing what's going on in chicken land helped me to learn a lot of new stuff too. So don't be shy and sign up!

How long does it take to write an issue?

4-7 hours, depending on the week and on your omelette recipe.

I am lost! Where can I get help?

Write to chicken-users or drop by on IRC in #chicken. We are there to help you.

Authors for upcoming issues

Issue 18 2010-12-27

Authors of past issues

Issue 17 2011-01-10
   by Moritz Heidkamp, Christian Kellermann and Felix Winkelmann
Issue 16 2010-12-13
   by Christian Kellerman
Issue 15 2010-12-06
   by Alaric Snell-Pym and Moritz Heidkamp
Issue 14 2010-11-29 +0000
   by Felix Winkelmann
Issue 13 2010-11-22 +0000
   by Peter Bex
Issue 12 2010-11-15 +0000
   by Alaric Snell-Pym
Issue 11 2010-11-08 +0000
   by Moritz Heidkamp and Christian Kellermann
Issue 10 2010-11-01 +0000
   by Mario Domenech Goulart
Issue 9 2010-10-25 +0000
   by Jim Ursetto
Issue 8 2010-10-18 +0000
   by Moritz Heidkamp
Issue 7 2010-10-11 +0000
   by Christian Kellermann
Issue 6 2010-10-03 +0000
   by Felix Winkelmann
Issue 5 2010-09-27 +0000
   by Alex Shinn
Issue 4 2010-09-17 +0000
   by Peter Bex
Issue 3 2010-09-10 +0000
   by Alaric Snell-Pym
Issue 2 2010-09-04 +0000
   by Christian Kellermann and Moritz Heidkamp
Issue 1 2010-08-27 +0000
   by Christian Kellermann

The Crazy Gazette Automation Plan

To reduce the tedious part of writing a gazette, user:alaric-snellpym has suggested we automate the boring bits.

The original plan was:

1) The users page on the wiki to, in parens or brackets or something,
after each person's name, list their various identities used in IRC /
svn usernames / etc. so they can be easily tallied together

2) A script that, when run in a local svn checkout, or maybe by talking
direct to SVN, lists all the eggs that have seen commits in a specified
time period (with the option of "right up to now" as the end of the time
period); and then lists the commit messages and revision numbers (for
deeper investigation, if required) for each. For extra points, make sure
that tagging a version of an egg is clearly indicated somehow (eg,
"TAGGED 1.5"). This should all be a relatively simple matter of parsing
the svn logs.

3) A script that, when run in a local git checkout, or maybe by talking
direct to the core git repo, lists all the commits in a specified time
period, grouped by branch.

4) All scripts should map usernames found in svn/git to displayed names
via a function that defaults to identity. Somebody else please write a
function to replace this, that looks in the wiki page and parses it to
map svn/git identities to Wiki markup for the user's user page with
their full name as the anchor text, that can be inserted into the above
scripts to make wonderful magic happen.

For bonus points, the output of scripts (2) and (3) could be actual
markup for putting straight into the gazette, say as a bulleted list,
just requiring editing to remove useless commits and to add editorial
insight.

Super special bonus point:

5) Write a script that, given a date range, parses the mailing list
archive into wiki markup for a list of links to the posts in the
archive, along with links to the user's pages as per (4), grouped by thread.

I think that the above are relatively bite-sized chunks that people who
want to see the Gazette continue should be able to manage between them;
if the above are done then, if needed, I'd be willing to pioneer alone
with running them each week (or every other week at worst) and writing
some content around them!

When asked for an example of the output of (2), I suggested:

 * [[egg:ugarit|ugarit]]
   * [r12345] Added more bugs [[users:alaric-snell-pym|Alaric Snell-Pym]]
   * [r56789] Tagged 0.9 [[users:alaric-snell-pym|Alaric Snell-Pym]]

And for (3):

 * experimental
   * [deadbeef] Implemented call/pc [[users:felix|Felix Winkelmann]]
 * master
   * [cafebabe] Tagged v4.9 [[users:felix|Felix Winkelmann]]

Andy Bennett produced a shell script for (3):

for b in `git branch -a --no-color | sed -e 's/^*//' -e 's/^\s*\(\S*\).*/\1/'` ; do echo " * On `echo $b | sed -e 's#^remotes/##'`:"; git log --pretty=format:'   * (%h): %s (%an, %ar)' --since="1 week ago" $b; echo ""; done;


I have just tested with NetBSD's sed and the 2nd part seems to cause
everything to turn blank...
It seems to be having trouble with the \S character class and the match
reference. Changing this argument to

-e 's/-> .*//'

produces a correct result.

And a Scheme script for (5):

(use http-client html-parser sxpath srfi-69 srfi-19)

(define base-url "http://lists.nongnu.org/archive/html/chicken-users/")

; Fetch the mail archive's index
(define tree (with-input-from-request base-url #f html->sxml))
(define month-list ((sxpath `(// li)) tree))

; The last week might span both this month and last month
;(define interesting-months `(,(car month-list) ,(cadr month-list)))
(define interesting-months `(,(car month-list)
			      ,(cadr month-list)
			      ))

; Fetch the thread listings for the interesting-months
(define (get-thread-path snippet)
  (cadadr ((sxpath `(// a @ href)) snippet)))

; Filter the HTML so that we only have the thread listings. Then convert it to SXML
(define thread-trees (map (lambda (thread-path)
			    (with-input-from-string (;apply
						     string-intersperse
						     (with-input-from-request (string-append base-url thread-path) #f
									      (lambda () (let ((flag #f))
											   (map
											     (lambda (line)

											       (cond ((string-match "<hr [^>]*>" line)
												      (begin
													(set! flag (not flag))
													line))
												     (flag line)
												     (else ""))
											       ;(string-substitute "<input [^>]*>" "" line)
											       ;(string-substitute "(<input [^>]*>)|(<p>)" "" line)
											       )
											     (read-lines))
											   )
										)))
			      html->sxml))
			  (map get-thread-path interesting-months)))


; Canonicalise the subject for each message and generate a list of (subject, author, date, link)s
(define messages ((sxpath `(// li)) thread-trees))

(define message-list
  (map (lambda (message)
	 (if (> (length message) 4)
	   (let ((link (car ((sxpath `(// a @ href *text*)) (second message))))
		 (subject (string-substitute "[ \t]+" " " (string-substitute (string-append "(Re: )?" (regexp-escape "[CHICKEN-users] ") "(Re: )?") "" (car ((sxpath `(// *text*)) (second message)))) 'all ))
		 (author (second (fourth message)))
		 (date (second (sixth message))))
	     `(,subject ,author ,date ,link)
	     )
	   '()
	   ))
       messages))

; Filter out everything except messages from the last week
(define (sent-this-week? date-string)
  (let ((date (scan-date date-string "~Y~m~d"))
	(last-week (date-subtract-duration (current-date) (make-duration #:days 8))))
    (if (>= (date-compare date last-week) 0)
      #t
      #f)))

(define filtered-message-list (map (lambda (message)
				     (if (= (length message) 4)
				       (if (sent-this-week? (third message))
					 message
					 '())
				       '()))
				   message-list))

; Regroup the filtered-message-list by subject then author
(define message-hash
  (fold
    (lambda (msg summary)
      (if (not (eq? msg '()))
	(begin
	  (hash-table-update! (second summary) (first msg)
			      (lambda (identity)
				(hash-table-update! identity (car (cdr msg))
						    (lambda (identity) (append identity `(,(cddr msg))))
						    (lambda () '())
						    ) identity)
			      (lambda () (make-hash-table))
			      )
	  (hash-table-update! (first summary) (first msg)
			      (lambda (identity) (+ identity 1))
			      (lambda () 0))
	  ))
      summary )
    `(,(make-hash-table) ,(make-hash-table))
    filtered-message-list))


; Render a wikified list of threads and links to messages
(define url-regex (regexp "(....)/(..)/.."))
(map
  (lambda (subj) (
		  (lambda (subj value)
		    (format #t "* ~A    (~A messages)" subj (hash-table-ref (first message-hash) subj))
		    (newline)
		    (hash-table-walk value (lambda (key value)
					     (format #t "** ~A" key )
					     (newline)
					     (format #t "*** ")
					     (map (lambda (value)
						    (format #t "[[~A~A/~A|~A]] " base-url (string-substitute url-regex "\\1-\\2" (first value)) (second value) (first value))) value)
					     (newline)
					     ))
		    (newline))
		  subj (hash-table-ref (second message-hash) subj)))
  (sort (hash-table-keys (first message-hash)) (lambda (a b) (> (hash-table-ref (first message-hash) a) (hash-table-ref (first message-hash) b)))))