The Macrological Fascicle

In order to evaluate a Scheme program, the macro uses within the program must be expanded into core forms which the implementation can directly further process itself. The set of core forms is implementation-dependent: an implementation may consider any syntax form defined by this report to be either a core form or a macro; the difference is not observable to users. Uses of both macros and core forms are represented as syntax objects.

Macro uses and core forms are both distinguished in forms to be processed by syntax keywords. These keywords occupy the same namespace as variables. That is, within the same scope, an identifier can be bound as a variable or keyword, or neither, but not both, and local bindings of either kind may shadow other bindings of either kind. In order to use a macro or a core form, the corresponding keyword must be imported into a program or library, or introduced within it by means of special macro definition and binding forms.

Many uses of macros defined by the user have the form:

(keyword datum ...)

where keyword is an identifier which lexically refers to the binding established for the macro or core form. Macro uses can also take the form of improper lists, bare identifiers, or set! forms, where the second subform of the set! is the keyword:

(keyword datum ... . datum)

keyword

(set! keyword datum)

In the latter case, the set! keyword must lexically refer to the same binding as the set! form defined in this report, and the binding of keyword must explicitly allow this kind of macro use (see section 2.4).

Macros whose uses can take the form of bare identifiers are referred to as identifier macros.

Transformers

Syntax keywords are bound by user code to transformers.

Most transformers are ordinary Scheme procedures, called transformer procedures, which receive exactly one argument, a syntax object (see section 3) representing the form of a macro use, and return exactly one value, a syntax object representing the result of expanding the input macro use. The result of expanding the input form using the transformer procedure replaces the macro use in the place where it occurred.

Variable transformers (section 2.4) are another kind of transformer.

It is undefined behaviour to re-enter the dynamic extent of a call to a transformer by the expander after it has returned once.

Syntax definition and binding forms

(let ()
  (define even?
    (lambda (x)
      (or (= x 0) (odd? (- x 1)))))
  (define-syntax odd?
    (syntax-rules ()
      ((odd? x) (not (even? x)))))
  (even? 10))

(let ()
  (define-syntax bind-to-zero
    (syntax-rules ()
      ((bind-to-zero id) (define id 0))))
  (bind-to-zero x)
  x)

(let ((x 21))
  (splicing-let-syntax
      ((def (syntax-rules ()
              ((def stuff ...) (define stuff ...)))))
    (def foo 42))
  foo)

(let ((x 21))
  (let-syntax
      ((def (syntax-rules ()
              ((def stuff ...) (define stuff ...)))))
    (def foo 42))
  foo)

(define-syntax let-syntax
  (syntax-rules ()
    ((_ bindings body_0 body_1 ...)
     (splicing-let-syntax bindings
       (let () body_0 body_1 ...)))))

(letrec-syntax
    ((xor
      (syntax-rules ()
        ((_) #f)
        ((_ e)
         (if e #t #f))
        ((_ e_1 e_2 ...)
         (let ((temp e_1))
           (if temp
               (not (or e_2 ...))
               (xor e_2 ...)))))))
  (values (xor #t #f #f)
          (xor #t #t #f)))

(define-syntax letrec-syntax
  (syntax-rules ()
    ((_ bindings body_0 body_1 ...)
     (splicing-letrec-syntax bindings
       (let () body_0 body_1 ...)))))

Syntax parameters

Syntax parameters are a minor variation on ordinary syntax keyword bindings. They provide a mechanism for rebinding a macro definition within the dynamic extent of a macro expansion.

Among other uses, this provides a convenient solution to one of the most common types of unhygienic macro: those that reintroduce the same unhygienic binding each time the macro is used. With syntax parameters, instead of introducing the binding unhygienically each time, one instead creates a single binding for the keyword, which is adjusted when the keyword is supposed to have a different meaning. As no new bindings are introduced, hygiene is preserved. Using a syntax parameter also provides the advantage that the identifier for the binding can be renamed when it is imported, if the macro user so wishes.

(define-syntax-parameter return
  (erroneous-syntax "return used outside of lambda^"))

(define-syntax lambda^
  (syntax-rules ()
    ((lambda^ formals body_0 body_1 ...)
     (lambda formals
       (call-with-current-continuation
        (lambda (escape)
          (syntax-parameterize
              ((return (identifier-syntax escape)))
            body_0 body_1 ...)))))))

Variable transformers

Variable transformers are another kind of transformer besides transformer procedures. A variable transformer is a simple container for a procedure, created by calling make-variable-transformer on that procedure. Binding a syntax keyword to a variable transformer declares to the expander that the procedure contained within it also expects to process macro uses of the form (set! keyword datum). An attempt to expand a macro use of this form whose transformer is not a variable transformer is a syntax violation.

(define-syntax used-as
  (make-variable-transformer
   (lambda (stx)
     (cond ((identifier? stx)
            (quote-syntax (quote reference)))
           ((free-identifier=? (car (unwrap-syntax stx)) #'set!)
            `(,(quote-syntax cons)
              ,(quote-syntax (quote assignment))
              (,(quote-syntax quote)
               ,(cdr (unwrap-syntax
                      (cdr (unwrap-syntax stx)))))))
           (else
            `(,(quote-syntax cons) ,(quote-syntax (quote combination))
                                   (,(quote-syntax quote)
                                   ,(cdr (unwrap-syntax stx)))))))))

Identifier properties

During expansion, a set of properties can be associated with each identifier in a Scheme program. This allows arbitrary information to be associated with identifiers, which can be used by macro transformers to inform their treatment of particular identifiers. For example, the sample implementation of the syntax-case pattern matcher included with this report uses identifier properties to implement pattern variables.

Each property defined on an identifier associates a key (which must also be an identifier) with a value (which may be any object). When an identifier binding is created by definition or by a local binding construct, it is associated with a new, empty set of identifier properties. If the identifier bound shadows one from a containing lexical context, the identifier properties on the shadowed identifier effectively become hidden within the lexical extent of the new binding, in the same way its binding is hidden.

When an identifier property is defined on an identifier, the property belongs only to the lexical scope in which that property is defined. The property itself may shadow properties created on the same identifier and with the same key in containing lexical contexts.

When an identifier is imported from a library, it brings with it a copy of the set of identifier properties that were defined on it in that library. Additional identifier properties may be defined on it, and properties from the original library may be redefined within the context in which the identifier was imported, without those definitions or redefinitions being visible in the original library. If the identifier was imported into a library which subsequently re-exports it, the re-exported version has the identifier properties as they were (re-)defined in the library which re-exports it. If the same binding is then imported into another context from both the original and the re-exporting library, or from multiple re-exporting libraries which each defined their own properties on the identifier, the identifier in that context has a set of properties which is the union of the properties from all the libraries it is imported from. If two properties with the same key are imported on the same identifier, and the values of the properties are not the same in the sense of eqv?, it is an import error.

(import (scheme base)
        (rename (only (scheme base) cons) (cons make-pair)))

(define-syntax renamed?
  (erroneous-syntax "only an identifier property key"))
(define-property make-pair renamed? #t)

(define-syntax both-renamed?
  (lambda (stx)
    (and (identifier-property #'cons #'renamed?)
         (identifier-property #'make-pair #'renamed?))))

(values (free-identifier=? #'cons #'make-pair)
        (both-renamed?))

Expansion process

In order to expand a body (whether library, program, or other body), the expander processes the initial forms within from left to right. How the expander processes each form encountered depends upon the kind of form.

[Editorial note: The following has been formulated based on the equivalent expansion process defined by the R6RS, but assuming that R7RS will relax the restriction on the order of definitions and expressions in all bodies. R7RS already relaxed the restriction in library bodies compared to R6RS. If the restriction is not relaxed within regular bodies, only a small adjustment to the text, reverting to R6RS semantics for those bodies, is required. ]

[Editorial note: This process does not define semantics compatible with those prescribed for program bodies by the small language. Those semantics will be specified in a future fascicle. ]

macro use

The expander invokes the associated transformer to transform the macro use, then recursively performs whichever of these actions are appropriate for the resulting form.

define-syntax or define-syntax-parameter form

The expander expands and evaluates the right-hand-side expression and binds the keyword to the resulting transformer.

define form

The expander records the fact that the defined identifier is a variable but defers expansion of the right-hand-side expression until after all of the forms in the body have been processed.

define-property form

The expander expands and evaluates the value expression and creates or replaces a property for the key on the given identifier, associating it with the resulting value.

begin form

The expander splices the subforms into the list of body forms it is processing.

splicing-let-syntax or splicing-letrec-syntax form

The expander splices the inner body forms into the list of (outer) body forms it is processing, arranging for the keywords bound by the splicing-let-syntax and splicing-letrec-syntax to be visible only in the inner body forms.

expression, i.e., nondefinition

The expander defers the expansion of the expression until after all the forms in the body have been processed.

Once the rightmost form in the body has been processed, the expander makes a second pass over the forms deferred as the right-hand sides of variable definitions or as nondefinitions.

Note that this algorithm does not directly reprocess any form. It requires a single left-to-right pass over the definitions followed by a single pass (in any order) over the body expressions and deferred right-hand sides.

The behaviour is undefined if any definition in the sequence of forms to define any identifier whose binding is used to determine the meaning of the undeferred portions of the definition, or of any definition that precedes it in the sequence of forms. Similarly, the behaviour is undefined if the evaluation of any form in the sequence of forms uses or assigns the value of a defined variable whose definition is to the right of that form. For example, the behaviour of each of the following examples is undefined:

(define define 3)

(begin (define begin list))

(display (+ x 16))
(define x 32)

[Editorial note: Last example should be within a (let () ...) if the relaxation is accepted for all bodies. ]

The behaviour of the following example is not undefined, because the body of the internal increase procedure will not be evaluated by a call to it until after the value variable it closes over has been defined:

(define (make-counter)
  (define (increase)
    (set! value (+ value 1))
    value)
  (define value 0)
  increase)

Phases of evaluation and macro expansion

The algorithm for processing forms in bodies outlined above requires the expressions creating macro transformers to be evaluated before evaluation of the Scheme program as a whole can proceed. The environment in which such evaluation takes place is defined by dividing the evaluation of Scheme programs into phases. Each phase is identified by a non-negative integer, and the number of the phase in which evaluation is currently taking place at any time is denoted $\varphi$. If a macro definition appears in phase $n$ code, then its right-hand-side expression is evaluated in phase $n+1$. The expansion and evaluation of Scheme forms after all syntax keywords have been defined takes place at phase $0$; thus, at the top level of a body, the expansion and evaluation of the right-hand sides of all define-syntax forms and the transformer expressions of splicing-let-syntax and splicing-letrec-syntax bindings takes place at phase $1$.

The environment at each phase is defined as follows. The environment at the earliest phase of evaluation contains all bindings which the program or library has imported from other libraries. All of these bindings, whether they are variables or syntax keywords, are available at all phases of evaluation. All syntactic bindings created in the course of expansion are likewise available at all phases of evaluation within the scopes in which they are visible. Variable bindings are available only in the phase in which they are created. It is undefined behaviour to either attempt to access the binding of, or to rebind an identifier which is a variable defined in a different phase.