User Manual

• Introduction
• Requirements
• Installation
• Usage
  • Compiling Scheme Programs
  • Compiling Scheme Libraries
  • Command Line Options
  • Generated Files
  • Interpreter
• Language Details
• Macros
  • Syntax Rules
  • Explicit Renaming
  • Debugging
• Multithreaded Programming
  • Thread Safety
• Foreign Function Interface
  • Writing a Scheme Function in C
  • Foreign Library
  • Including a C Header File
  • C Compiler Options
  • Linking to a C Library
  • Calling Scheme Functions from C
• Licensing
• References and Further Reading

Introduction

Cyclone is a brand-new Scheme-to-C compiler with the goal of supporting real-world application development using the R⁷RS Scheme Language standard. We provide modern features and a stable system capable of generating fast native binaries.

A variant of the Cheney on the MTA technique is used to implement full tail recursion, continuations, generational garbage collection, and native threads. This is the same technique proposed by Henry Baker (Cheney on the MTA) and implemented first by CHICKEN Scheme. The difference is that our compiler allows multiple native threads, each with their own stack. A tracing garbage collector is used to manage the second-generation heap and perform major collections without “stopping the world”.

Cyclone is developed by Justin Ethier.

Bug reports and patches are welcome! Please report any issues using the Issues Page.

Requirements

Cyclone has been tested under Linux on the x86, x86-64, and ARM platforms.

The following packages are required:

• make
• gcc

• Concurrency Kit

  NOTE: The best way to install libck is via a package manager such as apt-get. But if a package is not available for this library it can also be built from source. Just replace 0.6.0 below with the latest version available from their website:

    wget http://concurrencykit.org/releases/ck-0.6.0.tar.gz
    tar xfz ck-0.6.0.tar.gz ; cd ck-0.6.0 ; ./configure PREFIX=/usr && make all && sudo make install
    sudo ldconfig
  

On a Debian variant such as Ubuntu the necessary packages may be installed via the command:

sudo apt-get install libck-dev make gcc

The following command can be used to install dependencies on Fedora, though libck will also need to be built from source:

sudo yum install gcc make

Installation

Cyclone cannot be built directly on a system that does not have Cyclone binaries installed because the compiler is self-hosting. The easiest way to install Cyclone binaries is to build from source using cyclone-bootstrap:

$ git clone https://github.com/justinethier/cyclone-bootstrap.git
$ cd cyclone-bootstrap
$ ./install.sh

Once Cyclone is installed, it can be rebuilt directly from the cyclone repository:

$ make
$ make test
$ sudo make install

Usage

Compiling Scheme Programs

A Scheme program may be compiled using the cyclone command:

$ cyclone  examples/fac.scm
$ examples/fac
3628800

Compiling Scheme Libraries

Scheme code can be organized into libraries that are compiled separately from programs. Cyclone intends a library to represent a single C module (or file) when compiled.

Each library must be placed into a .sld file that corresponds to the library name. For example, the library

(scheme cyclone util) 

would be defined in its .sld file as:

(define-library (scheme cyclone util)
  ... )

and should be located in the file

scheme/cyclone/util.sld

Cyclone will automatically generate libraries when compiling a program. For example:

$ cd cyclone/examples/game-of-life
$ cyclone life.scm
$ ./life

Command Line Options

cyclone has the following command line options:

Generated Files

The following files are generated during the Cyclone compilation process:

When installing a library, all of the files indicated above must be installed as well as the corresponding .sld file.

Interpreter

Scheme code can be evaluated interactively using the icyc command:

    $ icyc
    cyclone> (write 'hello-world)
    hello-world

Language Details

Cyclone implements the Scheme language as documented by the R⁷RS Scheme Specification.

A R⁷RS Compliance Chart lists differences between the specification and Cyclone’s implementation.

API Documentation is available for the libraries provided by Cyclone.

Macros

Syntax Rules

High-level hygienic macros may be created using syntax-rules. This system is based on a template language specified by R⁷RS. The specification goes into more detail on how to work with these macros:

(define-syntax when
  (syntax-rules  ()
    ((when test result1 result2 ...)
     (if test
         (begin result1 result2 ...)))))

Explicit Renaming

Alternatively a low-level explicit renaming (ER) system is provided that allows defining macros using Scheme code, in a similar manner as defmacro.

This macro system provides the convenience functions (rename identifier) to hygienically rename an identifier and (compare identifier1 identifier2) to compare two identifiers:

(define-syntax when
  (er-macro-transformer
    (lambda (exp rename compare)
      (if (null? (cdr exp)) (error/loc "empty when" exp))
      (if (null? (cddr exp)) (error/loc "no when body" exp))
      `(if ,(cadr exp)
           ((lambda () ,@(cddr exp)))))))

Debugging

• From the interpreter one can use expand to perform macro expansion on the given expression:

  cyclone> (expand '(when #t (+ 1 2 3)))
  (if #t ((lambda () (+ 1 2 3))) )
  

• Alternatively when developing an ER macro, since its just a Scheme function, the macro can be defined as a lambda and passed a quoted expression to debug:

  (pretty-print
    ((lambda (exp rename compare)
       (if (null? (cdr exp)) (error/loc "empty when" exp))
       (if (null? (cddr exp)) (error/loc "no when body" exp))
       `(if ,(cadr exp)
            ((lambda () ,@(cddr exp)))))
    '(when #t  (write 1) (write 2)) #f #f))
  

• Finally, a Scheme file may be compiled with the -t option to write all of the intermediate transformations - including macro expansions - out to the corresponding .c file.

Multithreaded Programming

Overview

The srfi 18 library may be imported to provide support for multithreaded programs. See the SRFI 18 specification for more background information.

Many helper functions are provided by (cyclone concurrent) to make it easier to write multithreaded programs.

Thread Safety

Cyclone uses a generational garbage collector that automatically move objects from the first generation (on the stack) to the second generation (on the heap). This move is performed by the application thread that originally created the object. Without the proper safety measures in place this could cause problems as the address that another thread is using for an object may suddenly change.

To prevent race conditions Cyclone automatically relocates objects to the heap before they can be accessed by more than one thread. It is still necessary for application code to use the appropriate concurrency constructs - such as locks, atomics, etc - to ensure that an object is safely accessed by only one thread at a time.

Foreign Function Interface

Writing a Scheme Function in C

The define-c special form can be used to define a function containing user-defined C code. This code will be carried through from the Scheme file all the way to the compiled C file. For example:

 (define-c Cyc-add-exception-handler
   "(void *data, int argc, closure _, object k, object h)"
   " gc_thread_data *thd = (gc_thread_data *)data;
     make_pair(c, h, thd->exception_handler_stack);
     thd->exception_handler_stack = &c;
     return_closcall1(data, k, &c); ")

The arguments to define-c are:

• Function name. These functions can be used in Scheme code just like any other function.
• A string with C function arguments, enclosed in parentheses. Some of these are required:
  • data - Internal thread data. Do not modify this unless you know what you are doing!
  • argc - Number of function arguments.
  • A closure, typically unused in this context.
  • k - Current continuation, typically the code will call into k with a result.
• A string containing the C function body. Remember that runtime functions are not allowed to return. In the example above, the return_closcall1 macro is used to “return” a newly-allocated list to the current continuation.

Functions that may block must call the set_thread_blocked macro to let the system know the thread may block. After the blocking section is finished, the return_thread_runnable macro must be called to recover from the blocking operation and call into the current continuation. For example:

object Cyc_mutex_lock(void *data, object cont, object obj) {
  mutex m = (mutex) obj;
  Cyc_check_mutex(data, obj);
  set_thread_blocked(data, cont);
  if (pthread_mutex_lock(&(m->lock)) != 0) {
    fprintf(stderr, "Error locking mutex\n");
    exit(1);
  }
  return_thread_runnable(data, boolean_t);
}

The Cyclone runtime can be used as a reference for how to write your own C functions. A good starting point would be runtime.c and types.h.

Foreign Library

The (cyclone foreign) provides a convenient interface for integrating with C code, and provides higher-level constructs than define-c. See the API documentation for more information.

Including a C Header File

A C header may be included using the include-c-header special form. This special form may be used either as part of a library definition:

(define-library (example life)
  (include-c-header "../write-png.h")
  (export life)
  ... )

Or as part of a program (add any includes immediately after the import expression, if one is present):

(import (scheme base)
        (example life)
        (example grid))
(include-c-header "stdlib.h")
(include-c-header "<stdio.h>")

By default this will generate an #include preprocessor directive with the name of the header file in double quotes. However, if include-c-header is passed a text string with angle brackets (EG: "<stdio.h>"), the generated C code will use angle brackets instead.

C Compiler Options

A Cyclone library may use the `c-compiler-options expression to pass options directly to the C compiler. For example:

(define-library (my-lib)
  (c-compiler-options "-Imy-dir/include")
  ...

This expression may also be used at the top level of a program, EG:

(c-compiler-options "-Imy-dir/include")

Linking to a C Library

A Cyclone library may use the c-linker-options expression to instruct the compiler to include linker options when building an executable. For example:

(define-library (cyclone curl)
  (include-c-header "<curl/curl.h>")
  (export curl-version)
  (c-linker-options "-lcurl")

This expression may also be used at the top level of a program, EG:

(c-linker-options "-lcurl")

Calling Scheme Functions from C

C functions Cyc_scm_call and Cyc_scm_call_no_gc are provided to allow calling Scheme functions from C.

The best way to get started with these functions is with the code examples in the Cyclone source tree under examples/call-scm-from-c.

Licensing

Cyclone is available under the MIT license.

References and Further Reading

• CONS Should Not CONS Its Arguments, Part II: Cheney on the M.T.A., by Henry Baker
• CHICKEN Scheme
• Chibi Scheme
• Compiling Scheme to C with closure conversion, by Matt Might
• Implementing an on-the-fly garbage collector for Java, by Domani et al
• Lisp in Small Pieces, by Christian Queinnec
• Portable, Unobtrusive Garbage Collection for Multiprocessor Systems, by Damien Doligez and Georges Gonthier
• R⁵RS Scheme Specification
• R⁷RS Scheme Specification
• Structure and Interpretation of Computer Programs, by Harold Abelson and Gerald Jay Sussman
• The 90 minute Scheme to C compiler, by Marc Feeley