Chapter 7 Camlp4 library modules
7.1 Module MLast: abstract syntax tree
This library module is accessible, but not documented. The creation of
abstract syntax tree nodes must be done using quotations, defined in
the file q_MLast.cmo
. See appendix A.
However, some functions are provided to access the ``location'' field
in nodes. They are:
-
loc_of_expr
for expressions.
-
loc_of_patt
for patterns.
-
loc_of_ctyp
for types.
-
loc_of_module_type
for module types.
-
loc_of_module_expr
for module expressions.
-
loc_of_sig_item
for signature items.
-
loc_of_str_item
for structure items.
-
loc_of_phrase
for phrases.
7.2 Module Grammar: extensible grammars
-
This module implements the Camlp4 extensible grammars system.
Grammars entries can be extended using the EXTEND statement,
added by loading the Camlp4 pa_extend.cmo file.
type g;;
-
The type for grammars, holding entries.
val create : Token.lexer -> g;;
-
Create a new grammar, without keywords, using the lexer given
as parameter.
val tokens : g -> string -> (string * int) list;;
-
Given a grammar and a token pattern constructor, returns the list of
the corresponding values currently used in all entries of this grammar.
The integer is the number of times this pattern value is used.
Examples:
If the associated lexer uses ("", xxx) to represent a keyword
(what is represented by then simple string xxx in an EXTEND
statement rule), the call Grammar.token g "" returns the keywords
list.
The call Grammar.token g "IDENT" returns the list of all usages
of the pattern "IDENT" in the EXTEND statements.
module Entry :
sig
type 'a e;;
val create : g -> string -> 'a e;;
val parse : 'a e -> char Stream.t -> 'a;;
val parse_token : 'a e -> Token.t Stream.t -> 'a;;
val name : 'a e -> string;;
val of_parser : g -> string -> (Token.t Stream.t -> 'a) -> 'a e;;
val print : 'a e -> unit;;
val find : 'a e -> string -> Obj.t e;;
external obj : 'a e -> Gramext.g_entry = "%identity";;
end
;;
-
Module to handle entries.
* Entry.e is the type for entries returning values of type 'a.
* Entry.create g n creates a new entry named n in the grammar g.
* Entry.parse e returns the stream parser of the entry e.
* Entry.parse_token e returns the token parser of the entry e.
* Entry.name e returns the name of the entry e.
* Entry.of_parser g n p makes an entry from a token stream parser.
* Entry.print e displays the entry e using Format.
* Entry.find e s finds the entry named s in e's rules.
* Entry.obj e converts an entry into a Gramext.g_entry allowing
to see what it holds (Gramext is visible, but not documented).
val of_entry : 'a Entry.e -> g;;
-
Return the grammar associated with an entry.
val error_verbose : bool ref;;
-
Flag for displaying more information in case of parsing error;
default = False
val warning_verbose : bool ref;;
-
Flag for displaying warnings while extension; default = True
val strict_parsing : bool ref;;
-
Flag to apply strict parsing, without trying to recover errors;
default = False
Clearing grammars and entries
module Unsafe :
sig
val reinit_gram : g -> Token.lexer -> unit;;
val clear_entry : 'a Entry.e -> unit;;
end
;;
-
Module for clearing grammars and entries. To be manipulated with
care, because: 1) reinitializing a grammar destroys all tokens
and there may have problems with the associated lexer if it has
a notion of keywords; 2) clearing an entry does not destroy the
tokens used only by itself.
* Unsafe.reinit_gram g lex removes the tokens of the grammar
and sets lex as a new lexer for g. Warning: the lexer
itself is not reinitialized.
* Unsafe.clear_entry e removes all rules of the entry e.
Functions called by statements EXTEND and DELETE RULE
-
These functions are not to be used directly. The statements EXTEND
and DELETE_RULE use these functions generating type constraints which
ensure the typing consistency.
val extend :
(Gramext.g_entry * Gramext.position option *
(string option * Gramext.g_assoc option *
(Gramext.g_symbol list * Gramext.g_action) list)
list)
list ->
unit;;
val delete_rule : 'a Entry.e -> Gramext.g_symbol list -> unit;;
Functorial interface
-
Alternative for grammars use. Grammars are not yet Ocaml values:
there is no type for them. Modules generated preserve the
rule "an entry cannot call an entry of another grammar" by
normal Ocaml typing.
module type LexerType = sig val lexer : Token.lexer;; end;;
-
The input signature for the functor Grammar.Make.
lexer is the lexer used.
module type S =
sig
type parsable;;
val parsable : char Stream.t -> parsable;;
val tokens : string -> (string * int) list;;
module Entry :
sig
type 'a e;;
val create : string -> 'a e;;
val parse : 'a e -> parsable -> 'a;;
val parse_token : 'a e -> Token.t Stream.t -> 'a;;
val name : 'a e -> string;;
val of_parser : string -> (Token.t Stream.t -> 'a) -> 'a e;;
val print : 'a e -> unit;;
external obj : 'a e -> Gramext.g_entry = "%identity";;
end
;;
module Unsafe :
sig
val reinit_gram : Token.lexer -> unit;;
val clear_entry : 'a Entry.e -> unit;;
end
;;
val extend :
'a Entry.e -> Gramext.position option ->
(string option * Gramext.g_assoc option *
(Gramext.g_symbol list * Gramext.g_action) list)
list ->
unit;;
val delete_rule : 'a Entry.e -> Gramext.g_symbol list -> unit;;
end
;;
-
Signature type of the functor Grammar.Make. The types and
functions are almost the same than in generic interface, but:
* Grammars are not values. Functions holding a grammar as parameter
do not have this parameter yet.
* The type parsable is used in function parse instead of
the char stream, avoiding the possible loss of tokens.
module Make : functor (L : LexerType) -> S;;
7.3 Module Extfun: extensible functions
-
This module implements pattern matching extensible functions.
To extend, use syntax pa_extfun.cmo:
extfun e with [ pattern_matching ]
type ('a, 'b) t;;
-
The type of the extensible functions of type 'a -> 'b
val empty : ('a, 'b) t;;
-
Empty extensible function
val apply : ('a, 'b) t -> 'a -> 'b;;
-
Apply an extensible function
exception Failure;;
-
Match failure while applying an extensible function
val print : ('a, 'b) t -> unit;;
-
Print patterns in the order they are recorded
7.4 Module Pcaml: language grammar, entries and printers.
-
Hold variables to be set by language syntax extensions. Some of them
are provided for quotations management.
-
Parsers
val gram : Grammar.g;;
val interf : ((MLast.sig_item * MLast.loc) list * bool) Grammar.Entry.e;;
val implem : ((MLast.str_item * MLast.loc) list * bool) Grammar.Entry.e;;
val top_phrase : MLast.str_item option Grammar.Entry.e;;
val use_file : (MLast.str_item list * bool) Grammar.Entry.e;;
val module_type : MLast.module_type Grammar.Entry.e;;
val module_expr : MLast.module_expr Grammar.Entry.e;;
val sig_item : MLast.sig_item Grammar.Entry.e;;
val str_item : MLast.str_item Grammar.Entry.e;;
val expr : MLast.expr Grammar.Entry.e;;
val patt : MLast.patt Grammar.Entry.e;;
val ctyp : MLast.ctyp Grammar.Entry.e;;
val let_binding : (MLast.patt * MLast.expr) Grammar.Entry.e;;
val class_sig_item : MLast.class_sig_item Grammar.Entry.e;;
val class_str_item : MLast.class_str_item Grammar.Entry.e;;
val class_expr : MLast.class_expr Grammar.Entry.e;;
val class_type : MLast.class_type Grammar.Entry.e;;
-
Some grammar and entries of the language, set by pa_o.cmo and
pa_r.cmo.
val input_file : string ref;;
-
The file currently being parsed.
val output_file : string option ref;;
-
The output file, stdout if None (default)
val report_error : exn -> unit;;
-
Prints an error message, using the module Format.
val quotation_dump_file : string option ref;;
-
quotation_dump_file optionally tells the compiler to dump the
result of an expander if this result is syntactically incorrect.
If None (default), this result is not dumped. If Some fname, the
result is dumped in the file fname.
val version : string;;
-
The current version of Camlp4.
val add_option : string -> Arg.spec -> string -> unit;;
-
Add an option to the command line options.
val no_constructors_arity : bool ref;;
-
True: dont generate constructor arity.
val no_assert : bool ref;;
-
True: dont generate assertion checks.
val sync : (char Stream.t -> unit) ref;;
val handle_expr_quotation : MLast.loc -> string * string -> MLast.expr;;
val handle_expr_locate : MLast.loc -> int * string -> MLast.expr;;
val handle_patt_quotation : MLast.loc -> string * string -> MLast.patt;;
val handle_patt_locate : MLast.loc -> int * string -> MLast.patt;;
val expr_reloc : (MLast.loc -> MLast.loc) -> int -> MLast.expr -> MLast.expr;;
val patt_reloc : (MLast.loc -> MLast.loc) -> int -> MLast.patt -> MLast.patt;;
-
Allow user to catch exceptions in quotations
type err_ctx =
Finding | Expanding | ParsingResult of (int * int) * string | Locating
;;
exception Qerror of string * err_ctx * exn;;
-
Printers
open Spretty;;
val print_interf : ((MLast.sig_item * MLast.loc) list -> unit) ref;;
val print_implem : ((MLast.str_item * MLast.loc) list -> unit) ref;;
-
Some printers, set by pr_dump.cmo, pr_o.cmo and pr_r.cmo.
type 'a printer_t =
{ mutable pr_fun : string -> 'a -> string -> kont -> pretty;
mutable pr_levels : 'a pr_level list }
and 'a pr_level =
{ pr_label : string;
pr_box : 'a -> pretty Stream.t -> pretty;
mutable pr_rules : 'a pr_rule }
and 'a pr_rule =
('a, ('a curr -> 'a next -> string -> kont -> pretty Stream.t)) Extfun.t
and 'a curr = 'a -> string -> kont -> pretty Stream.t
and 'a next = 'a -> string -> kont -> pretty
and kont = pretty Stream.t
;;
val pr_str_item : MLast.str_item printer_t;;
val pr_sig_item : MLast.sig_item printer_t;;
val pr_expr : MLast.expr printer_t;;
val pr_patt : MLast.patt printer_t;;
val pr_expr_fun_args :
(MLast.expr, (MLast.patt list * MLast.expr)) Extfun.t ref;;
val find_pr_level : string -> 'a pr_level list -> 'a pr_level;;
val top_printer : 'a printer_t -> 'a -> unit;;
-
for system use
val warning : (int * int -> string -> unit) ref;;
val expr_eoi : MLast.expr Grammar.Entry.e;;
val patt_eoi : MLast.patt Grammar.Entry.e;;
val arg_spec_list : unit -> (string * Arg.spec * string) list;;
7.5 Module Plexer: a lexical analyzer
val make : unit -> Token.lexer;;
-
Some lexer provided. See the module Token. The tokens returned
follow the Objective Caml and the Revised syntax lexing rules.
The meaning of the tokens are:
* ("", s) is the keyword s.
* ("LIDENT", s) is the ident s starting with a lowercase letter.
* ("UIDENT", s) is the ident s starting with an uppercase letter.
* ("INT", s) is an integer constant whose string source is s.
* ("FLOAT", s) is a float constant whose string source is s.
* ("STRING", s) is the string constant s.
* ("CHAR", s) is the character constant s.
* ("QUOTATION", "t:s") is a quotation t holding the string s.
* ("ANTIQUOT", "t:s") is an antiquotation t holding the string s.
* ("LOCATE", "i:s") is a location directive at pos i holding s.
* ("EOI", "") is the end of input.
The associated token patterns in the EXTEND statement hold the
same names than the first string (constructor name) of the tokens
expressions above.
The lexer do not use global (mutable) variables: instantiations
of Plexer.make () do not perturb each other.
7.6 Module Quotation: quotation operations
type expander =
ExStr of (bool -> string -> string)
| ExAst of ((string -> MLast.expr) * (string -> MLast.patt))
;;
-
The type for quotation expanders kind:
* ExStr exp for an expander exp returning a string which
can be parsed to create a syntax tree. Its boolean parameter
tells whether the quotation is in position of an expression
(True) or in position of a pattern (False). Quotations expanders
created with this way may work for some particular language syntax,
and not for another one (e.g. may work when used with Revised
syntax and not when used with Ocaml syntax, and conversely).
* ExAst (expr_exp, patt_exp) for expanders returning directly
syntax trees, therefore not necessiting to be parsed afterwards.
The function expr_exp is called when the quotation is in
position of an expression, and patt_exp when the quotation is
in position of a pattern. Quotation expanders created with this
way are independant from the language syntax.
val add : string -> expander -> unit;;
-
add name exp adds the quotation name associated with the
expander exp.
val find : string -> expander;;
-
find name returns the expander of the given quotation name.
val default : string ref;;
-
default holds the default quotation name.
val translate : (string -> string) ref;;
-
function translating quotation names; default = identity
7.7 Module Stdpp: standard definitions
exception Exc_located of (int * int) * exn;;
-
Exc_located loc e is an encapsulation of the exception e with
the input location loc. To be used in quotation expanders
and in grammars to specify some input location for an error.
Do not raise this exception directly: rather use the following
function raise_with_loc.
val raise_with_loc : int * int -> exn -> 'a;;
-
raise_with_loc loc e, if e is already the exception Exc_located,
re-raise it, else raise the exception Exc_located loc e.
val loc_name : string ref;;
-
Name of the location variable used in grammars and in the predefined
quotations for OCaml syntax trees. Default: loc
7.8 Module Token: lexers for Camlp4 grammars
-
This module defines the Camlp4 lexer type to be used in extensible
grammars (see module Grammar). It also provides some useful functions
to create lexers (this module should be renamed Glexer one day).
Token type
type t = string * string;;
type pattern = string * string;;
-
Token and token patterns. Token are build by the lexer. Token
patterns come from the EXTEND statement.
The first string is the constructor name (must start with
an uppercase character). When it is empty, the second string
is supposed to be a keyword.
The second string is the constructor parameter. Empty if it
has no parameter.
The way tokens pattern are interpreted to parse tokens is
done by the lexer, function tparse below.
exception Error of string;;
-
An lexing error exception to be used by lexers.
Lexer type
type location = int * int;;
type location_function = int -> location;;
-
The type for a function associating a number of a token in a stream
(starting from 0) to its source location.
type lexer_func = char Stream.t -> t Stream.t * location_function;;
-
The type for a lexer function. The character stream is the input
stream to be lexed. The result is a pair of a token stream and
a location function for this tokens stream.
type lexer =
{ func : lexer_func;
using : pattern -> unit;
removing : pattern -> unit;
tparse : pattern -> (t Stream.t -> string) option;
text : pattern -> string }
;;
-
The type for a lexer used by Camlp4 grammars.
The field func is the main lexer function. See lexer_func
type above. This function may be created from a char stream parser
or for an ocamllex function using the functions below.
The field using is a function telling the lexer that the grammar
uses this token (pattern). The lexer can check that its constructor
is correct, and interpret some kind of tokens as keywords (to record
them in its tables). Called by EXTEND statements.
The field removing is a function telling the lexer that the grammar
does not uses the given token (pattern) any more. If the lexer has a
notion of "keywords", it can release it from its tables. Called by
DELETE_RULE statements.
The field tparse is a function returning either Some f, f being
the parsing function associated with the pattern, or None if it
is a standard parsing of the pattern.
The field text returns the name of some token pattern,
used in error messages.
val lexer_text : pattern -> string;;
-
A simple text function for lexers
Lexer from char stream parsers or ocamllex function
-
The functions below create lexer functions either from a char stream
parser or for an ocamllex function. With the returned function f,
the simplest Token.lexer can be written:
{ Token.func = f;
Token.using = (fun _ -> ());
Token.removing = (fun _ -> ());
Token.tparse = (fun _ -> None);
Token.text = Token.lexer_text }
Note that a better using function should check the used tokens
and raise Token.Error for incorrect ones. The other functions
removing, tparse and text may have other implementations
as well.
val lexer_func_of_parser : (char Stream.t -> t * location) -> lexer_func;;
-
A lexer function from a lexer written as a char stream parser
returning the next token and its location.
val lexer_func_of_ocamllex : (Lexing.lexbuf -> t) -> lexer_func;;
-
A lexer function from a lexer created by ocamllex
val make_stream_and_location :
(unit -> t * location) -> t Stream.t * location_function;;
Useful functions
val eval_char : string -> char;;
val eval_string : string -> string;;
-
Convert a char or a string token, where the backslashes have not
been interpreted into a real char or string; raise Failure if
bad backslash sequence found; Token.eval_char (Char.escaped c)
returns c and Token.eval_string (String.escaped s) returns s