Copyright © 2005 James Bielman <jamesjb at jamesjb.com>
Copyright © 2005, 2006 Luís Oliveira
<loliveira at common-lisp.net>
Copyright © 2006 Stephen Compall <s11 at member.fsf.org>
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
The software is provided “as is”, without warranty of any kind, express or implied, including but not limited to the warranties of merchantability, fitness for a particular purpose and noninfringement. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, whether in an action of contract, tort or otherwise, arising from, out of or in connection with the software or the use or other dealings in the software.
--- Dictionary ---
Foreign Types
Pointers
Strings
Variables
Functions
Libraries
Callbacks
Finalizers
CFFI is the Common Foreign Function Interface for ANSI Common Lisp systems. By foreign function we mean a function written in another programming language and having different data and calling conventions than Common Lisp, namely, C. CFFI allows you to call foreign functions and access foreign variables, all without leaving the Lisp image.
We consider this manual ever a work in progress. If you have difficulty with anything CFFI-specific presented in the manual, please contact the developers with details.
See What makes Lisp different, for an argument in favor of FFI in general.
CFFI's primary role in any image is to mediate between Lisp developers and the widely varying FFIs present in the various Lisp implementations it supports. With CFFI, you can define foreign function interfaces while still maintaining portability between implementations. It is not the first Common Lisp package with this objective; however, it is meant to be a more malleable framework than similar packages.
CFFI supports various free and commercial Lisp implementations: Allegro CL, Corman CL, clisp, CMUCL, ECL, LispWorks, OpenMCL, SBCL and the Scieneer CL.
There are also plans to support Digitool MCL, and GCL.
Tested platforms: linux/x86, linux/ppc, win32/x86, darwin/ppc.
Version 7.0 is supported. The 8.0 beta is also known to work. Earlier versions are untested and unsupported but patches to support them are welcome.
:long-long type.
Tested platforms: win32/x86.
Versions prior to 2.51 are untested and unsupported. Also, you will
need to avoid Corman's buggy COMPILE-FILE and fasl
loader. Please follow these instructions by Edi Weitz to setup ASDF for Corman CL in a way that
works around these issues.
foreign-funcall.
Tested platforms: linux/x86, linux/ppc, win32/x86, darwin/ppc.
Version is 2.34 or newer is required on win32/x86. For other platforms version 2.35 or newer is required.
Tested platforms: linux/x86, darwin/ppc.
Versions prior to 19B are untested. For darwin/ppc, the 2006-02 (19C) snapshot or later is recommended.
Tested platforms: needs testing...
As of November 2005, the CVS version of ECL is required. It is reported to pass all tests.
:long-long type.
:dffi is not present in *features*),
cffi:load-foreign-library does not work and you must use ECL's
own ffi:load-foreign-library with a constant string argument.
Tested platforms: linux/x86, win32/x86, darwin/ppc.
Versions prior to 4.4 are untested.
:long-long type.
Tested platforms: darwin/ppc, linux/ppc.
OpenMCL 1.0 or newer is recommended.
Tested platforms: linux/x86, linux/ppc, darwin/ppc.
Version 0.9.6 or newer is recommended.
Tested platforms: linux/x86, linux/amd64.
Version 1.2.10 or newer is recommended. Passes all tests. The x86 and AMD64 ports feature long-double support.
Users of many popular languages bearing semantic similarity to Lisp, such as Perl and Python, are accustomed to having access to popular C libraries, such as GTK, by way of “bindings”. In Lisp, we do something similar, but take a fundamentally different approach. This tutorial first explains this difference, then explains how you can use CFFI, a powerful system for calling out to C and C++ and access C data from many Common Lisp implementations.
The concept can be generalized to other languages; at the time of writing, only CFFI's C support is fairly complete, but C++ support is being worked on. Therefore, we will interchangeably refer to foreign functions and foreign data, and “C functions” and “C data”. At no time will the word “foreign” carry its usual, non-programming meaning.
This tutorial expects you to have a working understanding of both Common Lisp and C, including the Common Lisp macro system.
The following sums up how bindings to foreign libraries are usually implemented in other languages, then in Common Lisp:
PyObject
in Python, and whatever C object is called for, often using C
functions provided by the implementation. It also translates between
the calling conventions of the language and C.
We believe the advantages of the Common Lisp approach far outweigh any disadvantages. Incremental development with a listener can be as productive for C binding development as it is with other Lisp development. Keeping it “in the [Lisp] family”, as it were, makes it much easier for you and other Lisp programmers to load and use the bindings. Common Lisp implementations such as CMUCL, freed from having to provide a C interface to their own objects, are thus freed to be implemented in another language (as CMUCL is) while still allowing programmers to call foreign functions.
Perhaps the greatest advantage is that using an FFI doesn't obligate you to become a professional binding developer. Writers of bindings for other languages usually end up maintaining or failing to maintain complete bindings to the foreign library. Using an FFI, however, means if you only need one or two functions, you can write bindings for only those functions, and be assured that you can just as easily add to the bindings if need be.
The removal of the C compiler, or C interpretation of any kind,
creates the main disadvantage: some of C's “abstractions” are not
available, violating information encapsulation. For example,
structs that must be passed on the stack, or used as return
values, without corresponding functional abstractions to create and
manage the structs, must be declared explicitly in Lisp. This
is fine for structs whose contents are “public”, but is not so
pleasant when a struct is supposed to be “opaque” by convention,
even though it is not so defined.1
Without an abstraction to create the struct, Lisp needs to be able to lay out the struct in memory, so must know its internal details.
In these cases, you can create a minimal C library to provide the
missing abstractions, without destroying all the advantages of the
Common Lisp approach discussed above. In the case of structs,
you can write simple, pure C functions that tell you how many bytes a
struct requires or allocate new structs, read and write fields of the
struct, or whatever operations are supposed to be
public.2
Another disadvantage appears when you would rather use the foreign language than Lisp. However, someone who prefers C to Lisp is not a likely candidate for developing a Lisp interface to a C library.
The widely available libcurl is a library for downloading files
over protocols like HTTP. We will use libcurl with
CFFI to download a web page.
Please note that there are many other ways to download files from the
web, not least the cl-curl project to provide bindings to
libcurl via a similar FFI.3
libcurl-tutorial(3)
is a tutorial for libcurl programming in C. We will follow
that to develop a binding to download a file. We will also use
curl.h, easy.h, and the man pages for the
libcurl function, all available in the `curl-dev' package
or equivalent for your system, or in the cURL source code
package. If you have the development package, the headers should be
installed in /usr/include/curl/, and the man pages
may be accessed through your favorite man facility.
First of all, we will create a package to work in. You can save these
forms in a file, or just send them to the listener as they are. If
creating bindings for an ASDF package of yours, you will
want to add :cffi to the :depends-on list in your
.asd file. Otherwise, just use the asdf:oos function to
load CFFI.
(asdf:oos 'asdf:load-op :cffi) ;;; Nothing special about the "CFFI-USER" package. We're just ;;; using it as a substitute for your own CL package. (defpackage :cffi-user (:use :common-lisp :cffi)) (in-package :cffi-user) (define-foreign-library libcurl (:unix (:or "libcurl.so.3" "libcurl.so")) (t (:default "libcurl"))) (use-foreign-library libcurl)
Using define-foreign-library and use-foreign-library, we
have loaded libcurl into Lisp, much as the linker does when you
start a C program, or common-lisp:load does with a Lisp source
file or FASL file. We special-cased for unix machines
to always load a particular version, the one this tutorial was tested
with; for those who don't care, the define-foreign-library
clause (t (:default "libcurl")) should be satisfactory, and
will adapt to various operating systems.
libcurlAfter the introductory matter, the tutorial goes on to present the first function you should use.
CURLcode curl_global_init(long flags);
Let's pick this apart into appropriate Lisp code:
;;; A CURLcode is the universal error code. curl/curl.h says ;;; no return code will ever be removed, and new ones will be ;;; added to the end. (defctype curl-code :int) ;;; Initialize libcurl with FLAGS. (defcfun "curl_global_init" curl-code (flags :long))
Implementor's note: CFFI currently assumes the UNIX viewpoint that there is one C symbol namespace, containing all symbols in all loaded objects. This is not so on Windows and Darwin. The interface may be changed to deal with this.
Note the parallels with the original C declaration. We've defined
curl-code as a wrapping type for :int; right now, it
only marks it as special, but later we will do something more
interesting with it. The point is that we don't have to do it yet.
Looking at curl.h, CURL_GLOBAL_NOTHING, a possible value
for flags above, is defined as `0'. So we can now call
the function:
cffi-user> (curl-global-init 0)
=> 0
Looking at curl.h again, 0 means CURLE_OK, so it
looks like the call succeeded. Note that CFFI converted the
function name to a Lisp-friendly name. You can specify your own name
if you want; use ("curl_global_init" your-name-here) as
the name argument to defcfun.
The tutorial goes on to have us allocate a handle. For good measure, we should also include the deallocator. Let's look at these functions:
CURL *curl_easy_init( ); void curl_easy_cleanup(CURL *handle);
Advanced users may want to define special pointer types; we will explore this possibility later. For now, just treat every pointer as the same:
(defcfun "curl_easy_init" :pointer) (defcfun "curl_easy_cleanup" :void (easy-handle :pointer))
Now we can continue with the tutorial:
cffi-user> (defparameter *easy-handle* (curl-easy-init)) => *EASY-HANDLE* cffi-user> *easy-handle* => #<FOREIGN-ADDRESS #x09844EE0>
Note the print representation of a pointer. It changes depending on what Lisp you are using, but that doesn't make any difference to CFFI.
The libcurl tutorial says we'll want to set many options before
performing any download actions. This is done through
curl_easy_setopt:
CURLcode curl_easy_setopt(CURL *curl, CURLoption option, ...);
We've introduced a new twist: variable arguments. There is no obvious
translation to the defcfun form, particularly as there are four
possible argument types. Because of the way C works, we could define
four wrappers around curl_easy_setopt, one for each type; in
this case, however, we'll use the general-purpose macro
foreign-funcall to call this function.
To make things easier on ourselves, we'll create an enumeration of the
kinds of options we want to set. The enum CURLoption isn't the
most straightforward, but reading the CINIT C macro definition
should be enlightening.
(defmacro define-curl-options (name type-offsets &rest enum-args) "As with CFFI:DEFCENUM, except each of ENUM-ARGS is as follows: (NAME TYPE NUMBER) Where the arguments are as they are with the CINIT macro defined in curl.h, except NAME is a keyword. TYPE-OFFSETS is a plist of TYPEs to their integer offsets, as defined by the CURLOPTTYPE_LONG et al constants in curl.h." (flet ((enumerated-value (type offset) (+ (getf type-offsets type) offset))) `(progn (defcenum ,name ,@(loop for (name type number) in enum-args collect (list name (enumerated-value type number)))) ',name))) ;for REPL users' sanity (define-curl-options curl-option (long 0 objectpoint 10000 functionpoint 20000 off-t 30000) (:noprogress long 43) (:nosignal long 99) (:errorbuffer objectpoint 10) (:url objectpoint 2))
With some well-placed Emacs query-replace-regexps, you could
probably similarly define the entire CURLoption enumeration. I
have selected to transcribe a few that we will use in this tutorial.
If you're having trouble following the macrology, just macroexpand the
curl-option definition, or see the following macroexpansion,
conveniently downcased and reformatted:
(progn
(defcenum curl-option
(:noprogress 43)
(:nosignal 99)
(:errorbuffer 10010)
(:url 10002))
'curl-option)
That seems more than reasonable. You may notice that we only use the type to compute the real enumeration offset; we will also need the type information later.
First, however, let's make sure a simple call to the foreign function works:
cffi-user> (foreign-funcall "curl_easy_setopt"
:pointer *easy-handle*
curl-option :nosignal :long 1 curl-code)
=> 0
foreign-funcall, despite its surface simplicity, can be used to
call any C function. Its first argument is a string, naming the
function to be called. Next, for each argument, we pass the name of
the C type, which is the same as in defcfun, followed by a Lisp
object representing the data to be passed as the argument. The final
argument is the return type, for which we use the curl-code
type defined earlier.
defcfun just puts a convenient façade on
foreign-funcall.4 Our earlier call to
curl-global-init could have been written as follows:
cffi-user> (foreign-funcall "curl_global_init" :long 0
curl-code)
=> 0
Before we continue, we will take a look at what CFFI can and can't do, and why this is so.
In What makes Lisp different, we mentioned
that writing an FFI sometimes requires depending on
information not provided as part of the interface. The easy option
CURLOPT_WRITEDATA, which we will not provide as part of the
Lisp interface, illustrates this issue.
Strictly speaking, the curl-option enumeration is not
necessary; we could have used :int 99 instead of
curl-option :nosignal in our call to curl_easy_setopt
above. We defined it anyway, in part to hide the fact that we are
breaking the abstraction that the C enum provides. If the
cURL developers decide to change those numbers later, we
must change the Lisp enumeration, because enumeration values are not
provided in the compiled C library, libcurl.so.3.
CFFI works because the most useful things in C libraries —
non-static functions and non-static variables — are included
accessibly in libcurl.so.3. A C compiler that violated this
would be considered a worthless compiler.
The other thing define-curl-options does is give the “type”
of the third argument passed to curl_easy_setopt. Using this
information, we can tell that the :nosignal option should
accept a long integer argument. We can implicitly assume t
== 1 and nil == 0, as it is in C, which takes care
of the fact that CURLOPT_NOSIGNAL is really asking for a
boolean.
The “type” of CURLOPT_WRITEDATA is objectpoint.
However, it is really looking for a FILE*.
CURLOPT_ERRORBUFFER is looking for a char*, so there is
no obvious CFFI type but :pointer.
The first thing to note is that nowhere in the C interface includes
this information; it can only be found in the manual. We could
disjoin these clearly different types ourselves, by splitting
objectpoint into filepoint and charpoint, but we
are still breaking the abstraction, because we have to augment the
entire enumeration form with this additional
information.5
The second is that the CURLOPT_WRITEDATA argument is completely
incompatible with the desired Lisp data, a
stream.6 It is probably acceptable if we are controlling every file
we might want to use as this argument, in which case we can just call
the foreign function fopen. Regardless, though, we can't write
to arbitrary streams, which is exactly what we want to do for this
application.
Finally, note that the curl_easy_setopt interface itself is a
hack, intended to work around some of the drawbacks of C. The
definition of Curl_setopt, while long, is far less cluttered
than the equivalent disjoint-function set would be; in addition,
setting a new option in an old libcurl can generate a run-time
error rather than breaking the compile. Lisp can just as concisely
generate functions as compare values, and the “undefined function”
error is just as useful as any explicit error we could define here
might be.
We could use foreign-funcall directly every time we wanted to
call curl_easy_setopt. However, we can encapsulate some of the
necessary information with the following.
;;; We will use this typedef later in a more creative way. For ;;; now, just consider it a marker that this isn't just any ;;; pointer. (defctype easy-handle :pointer) (defmacro curl-easy-setopt (easy-handle enumerated-name value-type new-value) "Call `curl_easy_setopt' on EASY-HANDLE, using ENUMERATED-NAME as the OPTION. VALUE-TYPE is the CFFI foreign type of the third argument, and NEW-VALUE is the Lisp data to be translated to the third argument. VALUE-TYPE is not evaluated." `(foreign-funcall "curl_easy_setopt" easy-handle ,easy-handle curl-option ,enumerated-name ,value-type ,new-value curl-code))
Now we define a function for each kind of argument that encodes the
correct value-type in the above. This can be done reasonably
in the define-curl-options macroexpansion; after all, that is
where the different options are listed!
We could make cl:defun forms in the expansion that simply call
curl-easy-setopt; however, it is probably easier and clearer to
use defcfun. define-curl-options was becoming unwieldy,
so I defined some helpers in this new definition.
(defun curry-curl-option-setter (function-name option-keyword) "Wrap the function named by FUNCTION-NAME with a version that curries the second argument as OPTION-KEYWORD. This function is intended for use in DEFINE-CURL-OPTION-SETTER." (setf (symbol-function function-name) (let ((c-function (symbol-function function-name))) (lambda (easy-handle new-value) (funcall c-function easy-handle option-keyword new-value))))) (defmacro define-curl-option-setter (name option-type option-value foreign-type) "Define (with DEFCFUN) a function NAME that calls curl_easy_setopt. OPTION-TYPE and OPTION-VALUE are the CFFI foreign type and value to be passed as the second argument to easy_setopt, and FOREIGN-TYPE is the CFFI foreign type to be used for the resultant function's third argument. This macro is intended for use in DEFINE-CURL-OPTIONS." `(progn (defcfun ("curl_easy_setopt" ,name) curl-code (easy-handle easy-handle) (option ,option-type) (new-value ,foreign-type)) (curry-curl-option-setter ',name ',option-value))) (defmacro define-curl-options (type-name type-offsets &rest enum-args) "As with CFFI:DEFCENUM, except each of ENUM-ARGS is as follows: (NAME TYPE NUMBER) Where the arguments are as they are with the CINIT macro defined in curl.h, except NAME is a keyword. TYPE-OFFSETS is a plist of TYPEs to their integer offsets, as defined by the CURLOPTTYPE_LONG et al constants in curl.h. Also, define functions for each option named set-`TYPE-NAME'-`OPTION-NAME', where OPTION-NAME is the NAME from the above destructuring." (flet ((enumerated-value (type offset) (+ (getf type-offsets type) offset)) ;;map PROCEDURE, destructuring each of ENUM-ARGS (map-enum-args (procedure) (mapcar (lambda (arg) (apply procedure arg)) enum-args)) ;;build a name like SET-CURL-OPTION-NOSIGNAL (make-setter-name (option-name) (intern (concatenate 'string "SET-" (symbol-name type-name) "-" (symbol-name option-name))))) `(progn (defcenum ,type-name ,@(map-enum-args (lambda (name type number) (list name (enumerated-value type number))))) ,@(map-enum-args (lambda (name type number) (declare (ignore number)) `(define-curl-option-setter ,(make-setter-name name) ,type-name ,name ,(ecase type (long :long) (objectpoint :pointer) (functionpoint :pointer) (off-t :long))))) ',type-name)))
Macroexpanding our define-curl-options form once more, we
see something different:
(progn
(defcenum curl-option
(:noprogress 43)
(:nosignal 99)
(:errorbuffer 10010)
(:url 10002))
(define-curl-option-setter set-curl-option-noprogress
curl-option :noprogress :long)
(define-curl-option-setter set-curl-option-nosignal
curl-option :nosignal :long)
(define-curl-option-setter set-curl-option-errorbuffer
curl-option :errorbuffer :pointer)
(define-curl-option-setter set-curl-option-url
curl-option :url :pointer)
'curl-option)
Macroexpanding one of the new define-curl-option-setter
forms yields the following:
(progn
(defcfun ("curl_easy_setopt" set-curl-option-nosignal) curl-code
(easy-handle easy-handle)
(option curl-option)
(new-value :long))
(curry-curl-option-setter 'set-curl-option-nosignal ':nosignal))
Finally, let's try this out:
cffi-user> (set-curl-option-nosignal *easy-handle* 1)
=> 0
Looks like it works just as well. This interface is now reasonably
high-level to wash out some of the ugliness of the thinnest possible
curl_easy_setopt FFI, without obscuring the remaining
C bookkeeping details we will explore.
According to the documentation for curl_easy_setopt, the type
of the third argument when option is CURLOPT_ERRORBUFFER
is char*. Above, we've defined
set-curl-option-errorbuffer to accept a :pointer as the
new option value. However, there is a CFFI type :string,
which translates Lisp strings to C strings when passed as arguments to
foreign function calls. Why not, then, use :string as the
CFFI type of the third argument? There are two reasons, both
related to the necessity of breaking abstraction described in
Breaking the abstraction.
The first reason also applies to CURLOPT_URL, which we will use
to illustrate the point. Assuming we have changed the type of the
third argument underlying set-curl-option-url to
:string, look at these two equivalent forms.
(set-curl-option-url *easy-handle* "http://www.cliki.net/CFFI") == (with-foreign-string (url "http://www.cliki.net/CFFI") (foreign-funcall "curl_easy_setopt" easy-handle *easy-handle* curl-option :url :pointer url curl-code))
The latter, in fact, is mostly equivalent to what a foreign function
call's macroexpansion actually does. As you can see, the Lisp string
"http://www.cliki.net/CFFI" is copied into a char array and
null-terminated; the pointer to beginning of this array, now a C
string, is passed as a CFFI :pointer to the foreign
function.
Unfortunately, the C abstraction has failed us, and we must break it.
While :string works well for many char* arguments, it
does not for cases like this. As the curl_easy_setopt
documentation explains, “The string must remain present until curl no
longer needs it, as it doesn't copy the string.” The C string
created by with-foreign-string, however, only has dynamic
extent: it is “deallocated” when the body (above containing the
foreign-funcall form) exits.
If we are supposed to keep the C string around, but it goes away, what
happens when some libcurl function tries to access the
URL string? We have reentered the dreaded world of C
“undefined behavior”. In some Lisps, it will probably get a chunk
of the Lisp/C stack. You may segfault. You may get some random piece
of other data from the heap. Maybe, in a world where “dynamic
extent” is defined to be “infinite extent”, everything will turn
out fine. Regardless, results are likely to be almost universally
unpleasant.7
Returning to the current set-curl-option-url interface, here is
what we must do:
(let (easy-handle)
(unwind-protect
(with-foreign-string (url "http://www.cliki.net/CFFI")
(setf easy-handle (curl-easy-init))
(set-curl-option-url easy-handle url)
#|do more with the easy-handle, like actually get the URL|#)
(when easy-handle
(curl-easy-cleanup easy-handle))))
That is fine for the single string defined here, but for every string
option we want to pass, we have to surround the body of
with-foreign-string with another with-foreign-string
wrapper, or else do some extremely error-prone pointer manipulation
and size calculation in advance. We could alleviate some of the pain
with a recursively expanding macro, but this would not remove the need
to modify the block every time we want to add an option, anathema as
it is to a modular interface.
Before modifying the code to account for this case, consider the other
reason we can't simply use :string as the foreign type. In C,
a char * is a char *, not necessarily a string. The
option CURLOPT_ERRORBUFFER accepts a char *, but does
not expect anything about the data there. However, it does expect
that some libcurl function we call later can write a C string
of up to 255 characters there. We, the callers of the function, are
expected to read the C string at a later time, exactly the opposite of
what :string implies.
With the semantics for an input string in mind — namely, that the
string should be kept around until we curl_easy_cleanup the
easy handle — we are ready to extend the Lisp interface:
(defvar *easy-handle-cstrings* (make-hash-table) "Hashtable of easy handles to lists of C strings that may be safely freed after the handle is freed.") (defun make-easy-handle () "Answer a new CURL easy interface handle, to which the lifetime of C strings may be tied. See `add-curl-handle-cstring'." (let ((easy-handle (curl-easy-init))) (setf (gethash easy-handle *easy-handle-cstrings*) '()) easy-handle)) (defun free-easy-handle (handle) "Free CURL easy interface HANDLE and any C strings created to be its options." (curl-easy-cleanup handle) (mapc #'foreign-string-free (gethash handle *easy-handle-cstrings*)) (remhash handle *easy-handle-cstrings*)) (defun add-curl-handle-cstring (handle cstring) "Add CSTRING to be freed when HANDLE is, answering CSTRING." (car (push cstring (gethash handle *easy-handle-cstrings*))))
Here we have redefined the interface to create and free handles, to
associate a list of allocated C strings with each handle while it
exists. The strategy of using different function names to wrap around
simple foreign functions is more common than the solution implemented
earlier with curry-curl-option-setter, which was to modify the
function name's function slot.8
Incidentally, the next step is to redefine
curry-curl-option-setter to allocate C strings for the
appropriate length of time, given a Lisp string as the
new-value argument:
(defun curry-curl-option-setter (function-name option-keyword)
"Wrap the function named by FUNCTION-NAME with a version that
curries the second argument as OPTION-KEYWORD.
This function is intended for use in DEFINE-CURL-OPTION-SETTER."
(setf (symbol-function function-name)
(let ((c-function (symbol-function function-name)))
(lambda (easy-handle new-value)
(funcall c-function easy-handle option-keyword
(if (stringp new-value)
(add-curl-handle-cstring
easy-handle
(foreign-string-alloc new-value))
new-value))))))
A quick analysis of the code shows that you need only reevaluate the
curl-option enumeration definition to take advantage of these
new semantics. Now, for good measure, let's reallocate the handle
with the new functions we just defined, and set its URL:
cffi-user> (curl-easy-cleanup *easy-handle*) => NIL cffi-user> (setf *easy-handle* (make-easy-handle)) => #<FOREIGN-ADDRESS #x09844EE0> cffi-user> (set-curl-option-nosignal *easy-handle* 1) => 0 cffi-user> (set-curl-option-url *easy-handle* "http://www.cliki.net/CFFI") => 0
For fun, let's inspect the Lisp value of the C string that was created
to hold "http://www.cliki.net/CFFI". By virtue of the implementation of
add-curl-handle-cstring, it should be accessible through the
hash table defined:
cffi-user> (foreign-string-to-lisp
(car (gethash *easy-handle* *easy-handle-cstrings*)))
=> "http://www.cliki.net/CFFI"
Looks like that worked, and libcurl now knows what
URL we want to retrieve.
Finally, we turn back to the :errorbuffer option mentioned at
the beginning of this section. Whereas the abstraction added to
support string inputs works fine for cases like CURLOPT_URL, it
hides the detail of keeping the C string; for :errorbuffer,
however, we need that C string.
In a moment, we'll define something slightly cleaner, but for now,
remember that you can always hack around anything. We're modifying
handle creation, so make sure you free the old handle before
redefining free-easy-handle.
(defvar *easy-handle-errorbuffers* (make-hash-table) "Hashtable of easy handles to C strings serving as error writeback buffers.") ;;; An extra byte is very little to pay for peace of mind. (defparameter *curl-error-size* 257 "Minimum char[] size used by cURL to report errors.") (defun make-easy-handle () "Answer a new CURL easy interface handle, to which the lifetime of C strings may be tied. See `add-curl-handle-cstring'." (let ((easy-handle (curl-easy-init))) (setf (gethash easy-handle *easy-handle-cstrings*) '()) (setf (gethash easy-handle *easy-handle-errorbuffers*) (foreign-alloc :char :count *curl-error-size* :initial-element 0)) easy-handle)) (defun free-easy-handle (handle) "Free CURL easy interface HANDLE and any C strings created to be its options." (curl-easy-cleanup handle) (foreign-free (gethash handle *easy-handle-errorbuffers*)) (remhash handle *easy-handle-errorbuffers*) (mapc #'foreign-string-free (gethash handle *easy-handle-cstrings*)) (remhash handle *easy-handle-cstrings*)) (defun get-easy-handle-error (handle) "Answer a string containing HANDLE's current error message." (foreign-string-to-lisp (gethash handle *easy-handle-errorbuffers*)))
Be sure to once again set the options we've set thus far. You may wish to define yet another wrapper function to do this.
If you have been reading
curl_easy_setopt(3), you should have noticed that some options
accept a function pointer. In particular, we need one function
pointer to set as CURLOPT_WRITEFUNCTION, to be called by
libcurl rather than the reverse, in order to receive data as it
is downloaded.
A binding writer without the aid of FFI usually approaches this problem by writing a C function that accepts C data, converts to the language's internal objects, and calls the callback provided by the user, again in a reverse of usual practices.
The CFFI approach to callbacks precisely mirrors its differences
with the non-FFI approach on the “calling C from Lisp”
side, which we have dealt with exclusively up to now. That is, you
define a callback function in Lisp using defcallback, and
CFFI effectively creates a C function to be passed as a function
pointer.
Implementor's note: This is much trickier than calling C functions from Lisp, as it literally involves somehow generating a new C function that is as good as any created by the compiler. Therefore, not all Lisps support them. See Implementation Support, for information about CFFI support issues in this and other areas. You may want to consider changing to a Lisp that supports callbacks in order to continue with this tutorial.
Defining a callback is very similar to defining a callout; the main
difference is that we must provide some Lisp forms to be evaluated as
part of the callback. Here is the signature for the function the
:writefunction option takes:
size_t function(void *ptr, size_t size, size_t nmemb, void *stream);
Implementor's note: size_t is almost always an unsigned int. You can get this and many other types using feature tests for your system by using cffi-grovel.
The above signature trivially translates into a CFFI
defcallback form, as follows.
;;; Alias in case size_t changes. (defctype size :unsigned-int) ;;; To be set as the CURLOPT_WRITEFUNCTION of every easy handle. (defcallback easy-write size ((ptr :pointer) (size size) (nmemb size) (stream :pointer)) (let ((data-size (* size nmemb))) (handler-case ;; We use the dynamically-bound *easy-write-procedure* to ;; call a closure with useful lexical context. (progn (funcall (symbol-value '*easy-write-procedure*) (foreign-string-to-lisp ptr data-size nil)) data-size) ;indicates success ;; The WRITEFUNCTION should return something other than the ;; #bytes available to signal an error. (error () (if (zerop data-size) 1 0)))))
First, note the correlation of the first few forms, used to declare the C function's signature, with the signature in C syntax. We provide a Lisp name for the function, its return type, and a name and type for each argument.
In the body, we call the dynamically-bound
*easy-write-procedure* with a “finished” translation, of
pulling together the raw data and size into a Lisp string, rather than
deal with the data directly. As part of calling
curl_easy_perform later, we'll bind that variable to a closure
with more useful lexical bindings than the top-level
defcallback form.
Finally, we make a halfhearted effort to prevent non-local exits from
unwinding the C stack, covering the most likely case with an
error handler, which is usually triggered
unexpectedly.9 The reason is that most C code is written to
understand its own idiosyncratic error condition, implemented above in
the case of curl_easy_perform, and more “undefined behavior”
can result if we just wipe C stack frames without allowing them to
execute whatever cleanup actions as they like.
Using the CURLoption enumeration in curl.h once more, we
can describe the new option by modifying and reevaluating
define-curl-options.
(define-curl-options curl-option
(long 0 objectpoint 10000 functionpoint 20000 off-t 30000)
(:noprogress long 43)
(:nosignal long 99)
(:errorbuffer objectpoint 10)
(:url objectpoint 2)
(:writefunction functionpoint 11)) ;new item here
Finally, we can use the defined callback and the new
set-curl-option-writefunction to finish configuring the easy
handle, using the callback macro to retrieve a CFFI
:pointer, which works like a function pointer in C code.
cffi-user> (set-curl-option-writefunction
*easy-handle* (callback easy-write))
=> 0
With all options finally set and a medium-level interface developed, we can finish the definition and retrieve http://www.cliki.net/CFFI, as is done in the tutorial.
(defcfun "curl_easy_perform" curl-code
(handle easy-handle))
cffi-user> (with-output-to-string (contents)
(let ((*easy-write-procedure*
(lambda (string)
(write-string string contents))))
(declare (special *easy-write-procedure*))
(curl-easy-perform *easy-handle*)))
=> "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\"
....
Now fear, comprehensively</P>
"
Of course, that itself is slightly unwieldy, so you may want to define a function around it that simply retrieves a URL. I will leave synthesis of all the relevant REPL forms presented thus far into a single function as an exercise for the reader.
The remaining sections of this tutorial explore some advanced features of CFFI; the definition of new types will receive special attention. Some of these features are essential for particular foreign function calls; some are very helpful when trying to develop a Lispy interface to C.
We've occasionally used the defctype macro in previous sections
as a kind of documentation, much what you'd use typedef for in
C. We also tried one special kind of type definition, the
defcenum type. See defcstruct, for a definition macro that
may come in handy if you need to use C structs as data.
However, all of these are mostly sugar for the powerful underlying
foreign type interface called type translators. You can easily
define new translators for any simple named foreign type. Since we've
defined the new type curl-code to use as the return type for
various libcurl functions, we can use that to directly convert
cURL errors to Lisp errors.
The CURLcode enumeration seems to follow the typical error code
convention of `0' meaning all is well, and each non-zero integer
indicating a different kind of error. We can apply that trivially to
differentiate between normal exits and error exits.
(define-condition curl-code-error (error) (($code :initarg :curl-code :reader curl-error-code)) (:report (lambda (c stream) (format stream "libcurl function returned error ~A" (curl-error-code c)))) (:documentation "Signalled when a libcurl function answers a code other than CURLE_OK.")) (defmethod translate-from-foreign (value (name (eql 'curl-code))) "Raise a CURL-CODE-ERROR if VALUE, a curl-code, is non-zero." (if (zerop value) :curle-ok (error 'curl-code-error :curl-code value)))
The heart of this translator is new method
translate-from-foreign. By specializing the name
parameter on (eql 'type-name), we immediately modify the
behavior of every function that returns a curl-code to pass the
result through this new method.
To see the translator in action, try invoking a function that returns
a curl-code.
cffi-user> (set-curl-option-nosignal *easy-handle* 1)
=> :CURLE-OK
As the result was `0', the new method returned :curle-ok,
just as specified.10 I will leave disjoining the separate
CURLcodes into condition types and improving the :report
function as an exercise for you.
The creation of *easy-handle-cstrings* and
*easy-handle-errorbuffers* as properties of easy-handles
is a kluge. What we really want is a Lisp structure that stores these
properties along with the C pointer. Unfortunately,
easy-handle is currently just a fancy name for the foreign type
:pointer; the actual pointer object varies from Common Lisp
implementation to implementation, needing only to satisfy
pointerp and be returned from make-pointer and friends.
One solution that would allow us to define a new Lisp structure to
represent easy-handles would be to write a wrapper around every
function that currently takes an easy-handle; the wrapper would
extract the pointer and pass it to the foreign function. However, we
can use type translators to more elegantly integrate this
“translation” into the foreign function calling framework, using
translate-to-foreign.
(defclass easy-handle () ((pointer :initform (curl-easy-init) :documentation "Foreign pointer from curl_easy_init") (error-buffer :initform (foreign-alloc :char :count *curl-error-size* :initial-element 0) :documentation "C string describing last error") (c-strings :initform '() :documentation "C strings set as options")) (:documentation "I am a parameterization you may pass to curl-easy-perform to perform a cURL network protocol request.")) (defmethod initialize-instance :after ((self easy-handle) &key) (set-curl-option-errorbuffer self (slot-value self 'error-buffer))) (defun add-curl-handle-cstring (handle cstring) "Add CSTRING to be freed when HANDLE is, answering CSTRING." (car (push cstring (slot-value handle 'c-strings)))) (defun get-easy-handle-error (handle) "Answer a string containing HANDLE's current error message." (foreign-string-to-lisp (slot-value handle 'error-buffer))) (defun free-easy-handle (handle) "Free CURL easy interface HANDLE and any C strings created to be its options." (with-slots (pointer error-buffer c-strings) handle (curl-easy-cleanup pointer) (foreign-free error-buffer) (mapc #'foreign-string-free c-strings))) (defmethod translate-to-foreign (handle (name (eql 'easy-handle))) "Extract the pointer from an easy-HANDLE." (slot-value handle 'pointer))
While we changed some of the Lisp functions defined earlier to use CLOS slots rather than hash tables, the foreign functions work just as well as they did before.
The greatest strength, and the greatest limitation, of the type translator comes from its generalized interface. As stated previously, we could define all foreign function calls in terms of the primitive foreign types provided by CFFI. The type translator interface allows us to cleanly specify the relationship between Lisp and C data, independent of where it appears in a function call. This independence comes at a price; for example, it cannot be used to modify translation semantics based on other arguments to a function call. In these cases, you should rely on other features of Lisp, rather than the powerful, yet domain-specific, type translator interface.
CFFI provides a rich and powerful foundation for communicating with foreign libraries; as we have seen, it is up to you to make that experience a pleasantly Lispy one. This tutorial does not cover all the features of CFFI; please see the rest of the manual for details. In particular, if something seems obviously missing, it is likely that either code or a good reason for lack of code is already present.
Implementor's note: There are some other things in CFFI that might deserve tutorial sections, such as define-foreign-type, free-translated-object, or structs. Let us know which ones you care about.
CFFI's interface is designed for human programmers, being aimed at aesthetic as well as technical sophistication. However, there are a few programs aimed at translating C and C++ header files, or approximations thereof, into CFFI forms constituting a foreign interface to the symbols in those files.
These wrapper generators are known to support output of CFFI forms.
First, this manual does not describe use of these other programs; they have documentation of their own. If you have problems using a generated interface, please look at the output CFFI forms and verify that they are a correct CFFI interface to the library in question; if they are correct, contact CFFI developers with details, keeping in mind that they communicate in terms of those forms rather than any particular wrapper generator. Otherwise, contact the maintainers of the wrapper generator you are using, provided you can reasonably expect more accuracy from the generator.
When is more accuracy an unreasonable expectation? As described in the tutorial (see Breaking the abstraction), the information in C declarations is insufficient to completely describe every interface. In fact, it is quite common to run into an interface that cannot be handled automatically, and generators should be excused from generating a complete interface in these cases.
As further described in the tutorial, the thinnest Lisp interface to a C function is not always the most pleasant one. In many cases, you will want to manually write a Lispier interface to the C functions that interest you.
Wrapper generators should be treated as time-savers, not complete automation of the full foreign interface writing job. Reports of the amount of work done by generators vary from 30% to 90%. The incremental development style enabled by CFFI generally reduces this proportion below that for languages like Python.
Foreign types describe how data is translated back and forth between C and Lisp. CFFI provides various built-in types and allows the user to define new types.
Dictionary
These types correspond to the native C integer types according to the ABI of the Lisp implementation's host system.
For convenience, the above types are provided as shortcuts for
unsigned-char, unsigned-short, unsigned-int,
unsigned-long, long-long and unsigned-long-long,
respectively.
:long-long and :unsigned-long-long are not supported on
all implementations. When those types are not available, the
symbol cffi-features:no-long-long is pushed into
*features*.
Foreign integer types of specific sizes, corresponding to the C types
defined in stdint.h.
On all systems, the :float and :double types represent a
C float and double, respectively. On most but not all
systems, :float and :double represent a Lisp
single-float and double-float, respectively. It is not
so useful to consider the relationship between Lisp types and C types
as isomorphic, as simply to recognize the relationship, and relative
precision, among each respective category.
This type is only supported on SCL.
A foreign pointer to an object of any type, corresponding to
void *.
No type at all. Only valid as the return type of a function.
CFFI also provides a few useful types that aren't built-in C types.
The :string type performs automatic conversion between Lisp and
C strings. Note that, in the case of functions the converted C string
will have dynamic extent (i.e. it will be automatically freed after
the foreign function returns).
In addition to Lisp strings, this type will also convert
Lisp arrays of element type (unsigned-byte 8) and will pass
foreign pointers unmodified.
A method for free-translated-object is specialized for this type. So, for example, foreign strings allocated by this type and passed to a foreign function will be freed after the function returns.
CFFI> (foreign-funcall "getenv" :string "SHELL" :string) => "/bin/bash" CFFI> (with-foreign-string (str "abcdef") (foreign-funcall "strlen" :string str :int)) => 6 CFFI> (let ((str (make-array 4 :element-type '(unsigned-byte 8) :initial-element 65))) (foreign-funcall "strlen" :string str :int)) => 4
Like :string but returns a list with two values when convert
from C to Lisp: a Lisp string and the C string's foreign pointer.
CFFI> (foreign-funcall "getenv" :string "SHELL" :string+ptr) => ("/bin/bash" #.(SB-SYS:INT-SAP #XBFFFFC6F))
The :boolean type converts between a Lisp boolean and a C
boolean. It canonicalizes to base-type which is :int by
default.
(convert-to-foreign nil :boolean) => 0 (convert-to-foreign t :boolean) => 1 (convert-from-foreign 0 :boolean) => nil (convert-from-foreign 1 :boolean) => t
The :wrapper type stores two symbols passed to the to-c
and from-c arguments. When a value is being translated to or
from C, this type funcalls the respective symbol.
:wrapper types will be typedefs for base-type and will
inherit its translators, if any.
Here's an example of how the :boolean type could be defined in
terms of :wrapper.
(defun bool-c-to-lisp (value) (not (zerop value))) (defun bool-lisp-to-c (value) (if value 1 0)) (defctype my-bool (:wrapper :int :from-c bool-c-to-lisp :to-c bool-lisp-to-c)) (convert-to-foreign nil 'my-bool) => 0 (convert-from-foreign 1 'my-bool) => t
Typedefs are similar to typedefs in C, except they are more
like “type wrappers” than aliases, for reasons that will become
clear in the next section.
Defining a typedef is as simple as giving defctype a new name
and the name of the type to be wrapped. Here is how a simpler version
of the built-in :boolean type could be defined:
;;; Define MY-BOOLEAN as an alias for the built-in type :INT. (defctype my-boolean :int)
With this type definition, one can declare arguments to foreign
functions as having the type my-boolean, and they will be
passed as integers. No conversion is taking place—if nil is
passed as a my-boolean, a type error will be signalled.
Type translators are used to automatically convert Lisp values to or
from foreign values. For example, using type translators, one can
define a boolean type which converts a Lisp generalized boolean
(nil vs. non-nil) to a C boolean (zero vs.
non-zero).
We created the my-boolean type in the previous section. To
tell CFFI how to automatically convert Lisp values to
my-boolean values, specialize the generic function
translate-to-foreign using an eql-specializer with the
name of your type, in this case the symbol my-boolean:
;;; Define a method that converts Lisp booleans to C booleans. (defmethod translate-to-foreign (value (type (eql 'my-boolean))) (if value 1 0))
Now, when an object is passed as a my-boolean to a foreign
function, this method will be invoked to convert the Lisp value to an
integer. To perform the inverse operation, which is needed for
functions that return a my-boolean, specialize the
translate-from-foreign generic function in the same manner as
above:
;;; Define a method that converts C booleans to Lisp booleans. (defmethod translate-from-foreign (value (type (eql 'my-boolean))) (not (zerop value)))
When a translate-to-foreign method requires allocation of
foreign memory, you may specialize the free-translated-object
method to free the memory once the foreign object is no longer needed.
This is called automatically by CFFI when passing objects to
foreign functions.
A type translator does not necessarily need to convert the value. For
example, one could define a typedef for :pointer that ensures,
in the translate-to-foreign method, that the value is not a
null pointer, signalling an error if a null pointer is passed. This
will prevent some pointer errors when calling foreign functions that
cannot handle null pointers.
Please note: these methods are meant as extensible hooks
only, and you should not call them directly. Use
convert-to-foreign, convert-from-foreign and
free-converted-object instead. These will take care of
following the typedef chain, for example, calling all the applicable
translators. They will also work for CFFI's built-in types, such
as enums.
See Defining new types, for a more involved tutorial example of type translators.
Being based on generic functions, the type translation mechanism described above can add a bit of overhead. This is usually not significant, but we nevertheless provide a way of getting rid of the overhead for the cases where it matters.
A good way to understand this issue is to look at the code generated
by defcfun. Consider the following example using the
my-boolean type defined above:
CFFI> (macroexpand-1 '(defcfun foo my-boolean (x my-boolean))) (DEFUN FOO (X) (MULTIPLE-VALUE-BIND (#:G3148 #:PARAM3149) (TRANSLATE-TYPE-TO-FOREIGN X #<FOREIGN-TYPEDEF MY-BOOLEAN>) (UNWIND-PROTECT (PROGN (TRANSLATE-TYPE-FROM-FOREIGN (%FOREIGN-FUNCALL "foo" :INT #:G3148 :INT) #<FOREIGN-TYPEDEF MY-BOOLEAN>)) (FREE-TYPE-TRANSLATED-OBJECT #:G3148 #<FOREIGN-TYPEDEF MY-BOOLEAN> #:PARAM3149))))
In order to get rid of those generic function calls, CFFI has
another set of extensible generic functions that provide functionality
similar to CL's compiler macros:
expand-to-foreign-dyn, expand-to-foreign and
expand-from-foreign. Here's how one could define
my-boolean with them:
(defmethod expand-to-foreign (value (type (eql 'my-boolean))) `(if ,value 1 0)) (defmethod expand-from-foreign (value (type (eql 'my-boolean))) `(not (zerop ,value)))
And here's what the macroexpansion of foo now looks like:
CFFI> (macroexpand-1 '(defcfun foo my-boolean (x my-boolean))) (DEFUN FOO (X) (LET ((#:G3182 (IF X 1 0))) (NOT (ZEROP (%FOREIGN-FUNCALL "foo" :INT #:G3182 :INT)))))
Much better.
The expansion interface has no equivalent of
free-translated-object; you must instead define a method on
expand-to-foreign-dyn, the third generic function in this
interface. This is especially useful when you can allocate something
much more efficiently if you know the object has dynamic extent, as is
the case with function calls that don't save the relevant allocated
arguments. Consider the following example:
;;; This type inherits :string's translators. (defctype stack-allocated-string :string) (defmethod expand-to-foreign-dyn (value var body (type (eql 'stack-allocated-string))) `(with-foreign-string (,var ,value) ,@body))
To short-circuit expansion and use the translate-* functions
instead, simply call the next method. Return its result in cases
where your method cannot generate an appropriate replacement for it.
The expand-* methods have precedence over their
translate-* counterparts and are guaranteed to be used in
defcfun, foreign-funcall, defcvar and
defcallback. If you define a method on each of the
expand-* generic functions, you are guaranteed to have full
control over the expressions generated for type translation in these
macros.
They may or may not be used in other CFFI operators that need to
translate between Lisp and C data; you may only assume that
expand-* methods will probably only be called during Lisp
compilation.
expand-to-foreign-dyn has precedence over
expand-to-foreign and is only used in defcfun and
foreign-funcall, only making sense in those contexts. If you
do not define a method on expand-to-foreign-dyn, however,
please note that this expand method for the hypothetical type
my-string is not the same as defining no method at all:
(defmethod expand-to-foreign (value-form (type-name (eql 'my-string)))
(call-next-method))
Without this method, your runtime translate-to-foreign method
will be called, and its result will be passed to
free-translated-object. However, if you define this method,
translate-to-foreign will still be called, but its result will
not be passed to free-translated-object. If you need to free
values with this interface, you must define an
expand-to-foreign-dyn method.
Important note: this set of generic functions is called at
macroexpansion time. Methods are defined when loaded or evaluated,
not compiled. You are responsible for ensuring that your
expand-* methods are defined when the foreign-funcall or
other forms that use them are compiled. One way to do this is to put
the method definitions earlier in the file and inside an appropriate
eval-when form; another way is to always load a separate Lisp
or FASL file containing your expand-* definitions
before compiling files with forms that ought to use them. Otherwise,
they will not be found and the runtime translators will be used
instead.
For more involved C types than simple aliases to built-in types, such
as you can make with defctype, CFFI allows declaration of
structures and unions with defcstruct and defcunion.
For example, consider this fictional C structure declaration holding some personal information:
struct person {
int number;
char* reason;
};
The equivalent defcstruct form follows:
(defcstruct person
(number :int)
(reason :string))
CFFI knows how to align C structs, and how to figure in
padding between struct elements.
Please note that this interface is only for those that must know about
the values contained in a relevant struct. If the library you are
interfacing returns an opaque pointer that needs only be passed to
other C library functions, by all means just use :pointer or a
type-safe definition munged together with defctype and type
translation.
Implementor's note: Which “operations” are worth going over here? –stephen
See Allocating Foreign Memory.
convert-to-foreign.
This is an external interface to the type translation facility. In the implementation, all foreign functions are ultimately defined as type translation wrappers around primitive foreign function invocations.
This function is available mostly for inspection of the type translation process, and possibly optimization of special cases of your foreign function calls.
Its behavior is better described under translate-from-foreign's
documentation.
CFFI-USER> (convert-to-foreign "a boat" :string) => #<FOREIGN-ADDRESS #x097ACDC0> => (T) CFFI-USER> (convert-from-foreign * :string) => "a boat"
convert-to-foreign
translate-from-foreign
free-converted-object along with the foreign value for that to
work.
This is an external interface to the type translation facility. In the implementation, all foreign functions are ultimately defined as type translation wrappers around primitive foreign function invocations.
This function is available mostly for inspection of the type translation process, and possibly optimization of special cases of your foreign function calls.
Its behavior is better described under translate-to-foreign's
documentation.
CFFI-USER> (convert-to-foreign t :boolean) => 1 => (NIL) CFFI-USER> (convert-to-foreign "hello, world" :string) => #<FOREIGN-ADDRESS #x097C5F80> => (T) CFFI-USER> (code-char (mem-aref * :char 5)) => #\,
convert-from-foreign
free-converted-object
translate-to-foreign
masks ::= [docstring] { (symbol value) }*
name-and-options ::= name | (name &optional (base-type :int))
The defbitfield macro is used to define foreign types that map
lists of symbols to integer values.
If value is omitted, it will be computed as follows: find the greatest value previously used, including those so computed, with only a single 1-bit in its binary representation (that is, powers of two), and left-shift it by one. This rule guarantees that a computed value cannot clash with previous values, but may clash with future explicitly specified values.
Symbol lists will be automatically converted to values and vice versa when being passed as arguments to or returned from foreign functions, respectively. The same applies to any other situations where an object of a bitfield type is expected.
Types defined with defbitfield canonicalize to base-type
which is :int by default.
(defbitfield open-flags (:rdonly #x0000) :wronly ;#x0001 :rdwr ;... :nonblock :append (:creat #x0200)) ;; etc... CFFI> (foreign-bitfield-symbols 'open-flags #b1101) => (:RDONLY :WRONLY :NONBLOCK :APPEND) CFFI> (foreign-bitfield-value 'open-flags '(:rdwr :creat)) => 514 ; #x0202 (defcfun ("open" unix-open) :int (path :string) (flags open-flags) (mode :uint16)) ; unportable CFFI> (unix-open "/tmp/foo" '(:wronly :creat) #o644) => #<an fd> ;;; Consider also the following lispier wrapper around open() (defun lispier-open (path mode &rest flags) (unix-open path flags mode))
foreign-bitfield-value
foreign-bitfield-symbols
name-and-options ::= structure-name | (structure-name &key size)
doc-and-slots ::= [docstring] { (slot-name slot-type &key count offset) }*
1 as such an array and a single element
are semantically equivalent.
This defines a new CFFI aggregate type akin to C structs.
In other words, it specifies that foreign objects of the type
structure-name are groups of different pieces of data, or
“slots”, of the slot-types, distinguished from each other by
the slot-names. Each structure is located in memory at a
position, and the slots are allocated sequentially beginning at that
point in memory (with some padding allowances as defined by the C
ABI, unless otherwise requested by specifying an
offset from the beginning of the structure (offset 0).
In other words, it is isomorphic to the C struct, giving
several extra features.
There are two kinds of slots, for the two kinds of CFFI types:
:long or :pointer. Used for simple
CFFI types.
The use of CLOS terminology for the structure-related features is intentional; structure definitions are very much like classes with (far) fewer features.
(defcstruct point "Pointer structure." (x :int) (y :int)) CFFI> (with-foreign-object (ptr 'point) ;; Initialize the slots (setf (foreign-slot-value ptr 'point 'x) 42 (foreign-slot-value ptr 'point 'y) 42) ;; Return a list with the coordinates (with-foreign-slots ((x y) ptr point) (list x y))) => (42 42)
;; Using the :size and :offset options to define a partial structure. ;; (this is useful when you are interested in only a few slots ;; of a big foreign structure) (defcstruct (foo :size 32) "Some struct with 32 bytes." ; <16 bytes we don't care about> (x :int :offset 16) ; an int at offset 16 (y :int) ; another int at offset 16+sizeof(int) ; <a couple more bytes we don't care about> (z :char :offset 24) ; a char at offset 24 ; <7 more bytes ignored (since size is 32)> ) CFFI> (foreign-type-size 'foo) => 32
foreign-slot-pointer
foreign-slot-value
with-foreign-slots
doc-and-slots ::= [docstring] { (slot-name slot-type &key count) }*
A union is a structure in which all slots have an offset of zero. It
is isomorphic to the C union. Therefore, you should use the
usual foreign structure operations for accessing a union's slots.
(defcunion uint32-bytes
(int-value :unsigned-int)
(bytes :unsigned-char :count 4))
foreign-slot-pointer
foreign-slot-value
The defctype macro provides a mechanism similar to C's
typedef to define new types.
The new type inherits base-type's translators.
(defctype my-string :string :documentation "My own string type.") (defctype long-bools (:boolean :long) :documentation "Booleans that map to C longs.") (defctype my-float :float :translate-p nil)
enum-list ::= [docstring] { keyword | (keyword value) }* name-and-options ::= name | (name &optional (base-type :int))
The defcenum macro is used to define foreign types that map
keyword symbols to integer values, similar to the C enum type.
If value is omitted its value will either be 0, if it's the first entry, or it it will continue the progression from the last specified value.
Keywords will be automatically converted to values and vice-versa when
being passed as arguments to or returned from foreign functions,
respectively. The same applies to any other situations where an object
of an enum type is expected.
Types defined with defcenum canonicalize to base-type
which is :int by default.
(defcenum boolean :no :yes) CFFI> (foreign-enum-value 'boolean :no) => 0
(defcenum numbers (:one 1) :two (:four 4)) CFFI> (foreign-enum-keyword 'numbers 2) => :TWO
foreign-enum-value
foreign-enum-keyword
The macro define-foreign-type defines a new parameterized type
called type-name. Given the arguments specified in
lambda-list, executing body should return a type
specifier which will determine the behaviour of type-name. The
behaviour of parameterized types can be further customized by
specializing translate-to-foreign,
translate-from-foreign, and free-translated-object.
Unlike defctype, which is used to define simple C-like
typedefs, define-foreign-type provides a mechanism for type
aliases to take arguments. The following examples illustrate this
capability.
Taken from CFFI's :boolean type definition:
(define-foreign-type :boolean (&optional (base-type :int)) "Boolean type. Maps to an :int by default. Only accepts integer types." (ecase base-type ((:char :unsigned-char :int :unsigned-int :long :unsigned-long) base-type))) CFFI> (canonicalize-foreign-type :boolean) => :INT CFFI> (canonicalize-foreign-type '(:boolean :long)) => :LONG CFFI> (canonicalize-foreign-type '(:boolean :float)) ;; error--> signalled by ECASE.
This next example is hypothetical as there is no :array type
yet.
(define-foreign-type int-array (&rest dimensions)
`(:array :int ,@dimensions))
nil.
The function foreign-bitfield-symbols returns a possibly shared
list of symbols that correspond to value in type.
(defbitfield flags (flag-a 1) (flag-b 2) (flag-c 4)) CFFI> (foreign-bitfield-symbols 'boolean #b101) => (FLAG-A FLAG-C)
defbitfield
foreign-bitfield-value
bitfield type.
The function foreign-bitfield-value returns the value that
corresponds to the symbols in the symbols list.
(defbitfield flags (flag-a 1) (flag-b 2) (flag-c 4)) CFFI> (foreign-bitfield-value 'flags '(flag-a flag-c)) => 5 ; #b101
defbitfield
foreign-bitfield-symbols
enum type.
foreign-enum-keyword returns
nil.
The function foreign-enum-keyword returns the keyword symbol
that corresponds to value in type.
An error is signaled if type doesn't contain such value and errorp is true.
(defcenum boolean :no :yes) CFFI> (foreign-enum-keyword 'boolean 1) => :YES
enum type.
foreign-enum-value returns
nil.
The function foreign-enum-value returns the value that
corresponds to keyword in type.
An error is signaled if type doesn't contain such keyword, and errorp is true.
(defcenum boolean :no :yes) CFFI> (foreign-enum-value 'boolean :yes) => 1
The function foreign-slot-names returns a potentially shared
list of slot names for the given structure type. This list
has no particular order.
(defcstruct timeval (tv-secs :long) (tv-usecs :long)) CFFI> (foreign-slot-names 'timeval) => (TV-SECS TV-USECS)
defcstruct
foreign-slot-offset
foreign-slot-value
foreign-slot-pointer
The function foreign-slot-offset returns the offset in
bytes of a slot in a foreign struct type.
(defcstruct timeval (tv-secs :long) (tv-usecs :long)) CFFI> (foreign-slot-offset 'timeval 'tv-secs) => 0 CFFI> (foreign-slot-offset 'timeval 'tv-usecs) => 4
defcstruct
foreign-slot-names
foreign-slot-pointer
foreign-slot-value
Returns a pointer to the location of the slot slot-name in a foreign object of type type at ptr. The returned pointer points inside the structure. Both the pointer and the memory it points to have the same extent as ptr.
For aggregate slots, this is the same value returned by
foreign-slot-value.
(defcstruct point "Pointer structure." (x :int) (y :int)) CFFI> (with-foreign-object (ptr 'point) (foreign-slot-pointer ptr 'point 'x)) => #<FOREIGN-ADDRESS #xBFFF6E60> ;; Note: the exact pointer representation varies from lisp to lisp.
defcstruct
foreign-slot-value
foreign-slot-names
foreign-slot-offset
For simple slots, foreign-slot-value returns the value of the
object, such as a Lisp integer or pointer. In C, this would be
expressed as ptr->slot.
For aggregate slots, a pointer inside the structure to the beginning
of the slot's data is returned. In C, this would be expressed as
&ptr->slot. This pointer and the memory it points to have the
same extent as ptr.
There are compiler macros for foreign-slot-value and its
setf expansion that open code the memory access when
type and slot-names are constant at compile-time.
(defcstruct point "Pointer structure." (x :int) (y :int)) CFFI> (with-foreign-object (ptr 'point) ;; Initialize the slots (setf (foreign-slot-value ptr 'point 'x) 42 (foreign-slot-value ptr 'point 'y) 42) ;; Return a list with the coordinates (with-foreign-slots ((x y) ptr point) (list x y))) => (42 42)
defcstruct
foreign-slot-names
foreign-slot-offset
foreign-slot-pointer
with-foreign-slots
The function foreign-type-alignment returns the
alignment of type in bytes.
CFFI> (foreign-type-alignment :char) => 1 CFFI> (foreign-type-alignment :short) => 2 CFFI> (foreign-type-alignment :int) => 4
(defcstruct foo (a :char)) CFFI> (foreign-type-alignment 'foo) => 1
The function foreign-type-size return the size of
type in bytes. This includes any padding within and following
the in-memory representation as needed to create an array of
type objects.
(defcstruct foo (a :double) (c :char)) CFFI> (foreign-type-size :double) => 8 CFFI> (foreign-type-size :char) => 1 CFFI> (foreign-type-size 'foo) => 16
convert-to-foreign;
used to implement the third argument to free-translated-object.
The return value is unspecified.
This is an external interface to the type translation facility. In the implementation, all foreign functions are ultimately defined as type translation wrappers around primitive foreign function invocations.
This function is available mostly for inspection of the type translation process, and possibly optimization of special cases of your foreign function calls.
Its behavior is better described under free-translated-object's
documentation.
CFFI-USER> (convert-to-foreign "a boat" :string) => #<FOREIGN-ADDRESS #x097ACDC0> => (T) CFFI-USER> (free-converted-object * :string '(t)) => NIL
convert-from-foreign
convert-to-foreign
free-translated-object
translate-to-foreign.
defctype.
translate-to-foreign.
This generic function may be specialized by user code to perform automatic deallocation of foreign objects as they are passed to C functions.
Any methods defined on this generic function must EQL-specialize the
type-name parameter on a symbol defined as a foreign type by
the defctype macro.
Foreign Type Translators
translate-to-foreign
defctype.
foreign-value to Lisp code.
This generic function is invoked by CFFI to convert a foreign value to a Lisp value, such as when returning from a foreign function, passing arguments to a callback function, or accessing a foreign variable.
To extend the CFFI type system by performing custom translations, this
method may be specialized by eql-specializing type-name on a
symbol naming a foreign type defined with defctype. This
method should return the appropriate Lisp value to use in place of the
foreign value.
The results are undefined if the type-name parameter is
specialized in any way except an eql specializer on a foreign type
defined with defctype. Specifically, translations may not be
defined for built-in types.
Foreign Type Translators
translate-to-foreign
free-translated-object
defctype.
lisp-value to foreign code.
free-translated-object.
This generic function is invoked by CFFI to convert a Lisp value to a foreign value, such as when passing arguments to a foreign function, returning a value from a callback, or setting a foreign variable. A “foreign value” is one appropriate for passing to the next-lowest translator, including the low-level translators that are ultimately invoked invisibly with CFFI.
To extend the CFFI type system by performing custom translations, this
method may be specialized by eql-specializing type-name on a
symbol naming a foreign type defined with defctype. This
method should return the appropriate foreign value to use in place of
the Lisp value.
In cases where CFFI can determine the lifetime of the foreign object
returned by this method, it will invoke free-translated-object
on the foreign object at the appropriate time. If
translate-to-foreign returns a second value, it will be passed
as the param argument to free-translated-object. This
can be used to establish communication between the allocation and
deallocation methods.
The results are undefined if the type-name parameter is
specialized in any way except an eql specializer on a foreign type
defined with defctype. Specifically, translations may not be
defined for built-in types.
Foreign Type Translators
translate-from-foreign
free-translated-object
The with-foreign-slots macro creates local symbol macros for
each var in vars to reference foreign slots in ptr of
type. It is similar to Common Lisp's with-slots macro.
(defcstruct tm (sec :int) (min :int) (hour :int) (mday :int) (mon :int) (year :int) (wday :int) (yday :int) (isdst :boolean) (zone :string) (gmtoff :long)) CFFI> (with-foreign-object (time :int) (setf (mem-ref time :int) (foreign-funcall "time" :pointer (null-pointer) :int)) (foreign-funcall "gmtime" :pointer time tm)) => #<A Mac Pointer #x102A30> CFFI> (with-foreign-slots ((sec min hour mday mon year) * tm) (format nil "~A:~A:~A, ~A/~A/~A" hour min sec (+ 1900 year) mon mday)) => "7:22:47, 2005/8/2"
defcstruct
defcunion
foreign-slot-value
All C data in CFFI is referenced through pointers. This includes defined C variables that hold immediate values, and integers.
To see why this is, consider the case of the C integer. It is not only an arbitrary representation for an integer, congruent to Lisp's fixnums; the C integer has a specific bit pattern in memory defined by the C ABI. Lisp has no such constraint on its fixnums; therefore, it only makes sense to think of fixnums as C integers if you assume that CFFI converts them when necessary, such as when storing one for use in a C function call, or as the value of a C variable. This requires defining an area of memory11, represented through an effective address, and storing it there.
Due to this compartmentalization, it only makes sense to manipulate
raw C data in Lisp through pointers to it. For example, while there
may be a Lisp representation of a struct that is converted to C
at store time, you may only manipulate its raw data through a pointer.
The C compiler does this also, albeit informally.
Dictionary
Manipulating pointers proper can be accomplished through most of the
other operations defined in the Pointers dictionary, such as
make-pointer, pointer-address, and pointer-eq.
When using them, keep in mind that they merely manipulate the Lisp
representation of pointers, not the values they point to.
CFFI provides support for stack and heap C memory allocation.
Stack allocation, done with with-foreign-object, is sometimes
called “dynamic” allocation in Lisp, because memory allocated as
such has dynamic extent, much as with let bindings of special
variables.
This should not be confused with what C calls “dynamic” allocation,
or that done with malloc and friends. This sort of heap
allocation is done with foreign-alloc, creating objects that
exist until freed with foreign-free.
When manipulating raw C data, consider that all pointers are pointing
to an array. When you only want one C value, such as a single
struct, this array only has one such value. It is worthwhile
to remember that everything is an array, though, because this is also
the semantic that C imposes natively.
C values are accessed as the setf-able places defined by
mem-aref and mem-ref. Given a pointer and a CFFI
type (see Foreign Types), either of these will dereference the
pointer, translate the C data there back to Lisp, and return the
result of said translation, performing the reverse operation when
setf-ing. To decide which one to use, consider whether you
would use the array index operator [n] or the pointer
dereference * in C; use mem-aref for array indexing and
mem-ref for pointer dereferencing.
The foreign-free function frees a ptr previously
allocated by foreign-alloc. The consequences of freeing a given
pointer twice are undefined.
CFFI> (foreign-alloc :int) => #<A Mac Pointer #x1022E0> CFFI> (foreign-free *) => NIL
foreign-alloc
with-foreign-pointer
The foreign-alloc function allocates enough memory to hold
count objects of type type and returns a
pointer. This memory must be explicitly freed using
foreign-free once it is no longer needed.
If initial-element is supplied, it is used to initialize the count objects the newly allocated memory holds.
If an initial-contents sequence is supplied, it must have a length less than or equal to count and each of its elements will be used to initialize the contents of the newly allocated memory.
If count is omitted and initial-contents is specified, it
will default to (length initial-contents).
initial-element and initial-contents are mutually exclusive.
When null-terminated-p is true,
(1+ (max count (length initial-contents))) elements
are allocated and the last one is set to NULL. Note that in
this case type must be a pointer type (ie. a type that
canonicalizes to :pointer), otherwise an error is signaled.
CFFI> (foreign-alloc :char) => #<A Mac Pointer #x102D80> ; A pointer to 1 byte of memory. CFFI> (foreign-alloc :char :count 20) => #<A Mac Pointer #x1024A0> ; A pointer to 20 bytes of memory. CFFI> (foreign-alloc :int :initial-element 12) => #<A Mac Pointer #x1028B0> CFFI> (mem-ref * :int) => 12 CFFI> (foreign-alloc :int :initial-contents '(1 2 3)) => #<A Mac Pointer #x102950> CFFI> (loop for i from 0 below 3 collect (mem-aref * :int i)) => (1 2 3) CFFI> (foreign-alloc :int :initial-contents #(1 2 3)) => #<A Mac Pointer #x102960> CFFI> (loop for i from 0 below 3 collect (mem-aref * :int i)) => (1 2 3) ;;; Allocate a char** pointer that points to newly allocated memory ;;; by the :string type translator for the string "foo". CFFI> (foreign-alloc :string :initial-element "foo") => #<A Mac Pointer #x102C40>
;;; Allocate a null-terminated array of strings. ;;; (Note: FOREIGN-STRING-TO-LISP returns NIL when passed a null pointer) CFFI> (foreign-alloc :string :initial-contents '("foo" "bar" "baz") :null-terminated-p t) => #<A Mac Pointer #x102D20> CFFI> (loop for i from 0 below 4 collect (mem-aref * :string i)) => ("foo" "bar" "baz" NIL) CFFI> (progn (dotimes (i 3) (foreign-free (mem-aref ** :pointer i))) (foreign-free **)) => nil
foreign-free
with-foreign-object
with-foreign-pointer
nil.
The function foreign-symbol-pointer will return a foreign
pointer corresponding to the foreign symbol denoted by the string
foreign-name. If a foreign symbol named foreign-name
doesn't exist, nil is returned.
ABI name manglings will be performed on foreign-name by
foreign-symbol-pointer if necessary. (eg: adding a leading
underscore on darwin/ppc)
Important note: do not keep these pointers across saved Lisp cores as the foreign-library may move across sessions.
CFFI> (foreign-symbol-pointer "errno") => #<A Mac Pointer #xA0008130> CFFI> (foreign-symbol-pointer "strerror") => #<A Mac Pointer #x9002D0F8> CFFI> (foreign-funcall * :int (mem-ref ** :int) :string) => "No such file or directory" CFFI> (foreign-symbol-pointer "inexistent symbol") => NIL
The function inc-pointer will return a new-pointer pointing
offset bytes past pointer.
CFFI> (foreign-string-alloc "Common Lisp") => #<A Mac Pointer #x102EA0> CFFI> (inc-pointer * 7) => #<A Mac Pointer #x102EA7> CFFI> (foreign-string-to-lisp *) => "Lisp"
incf-pointer
make-pointer
pointerp
null-pointer
null-pointer-p
setf place.
The incf-pointer macro takes the foreign pointer from
place and creates a new-pointer incremented by
offset bytes and which is stored in place.
CFFI> (defparameter *two-words* (foreign-string-alloc "Common Lisp")) => *TWO-WORDS* CFFI> (defparameter *one-word* *two-words*) => *ONE-WORD* CFFI> (incf-pointer *one-word* 7) => #.(SB-SYS:INT-SAP #X00600457) CFFI> (foreign-string-to-lisp *one-word*) => "Lisp" CFFI> (foreign-string-to-lisp *two-words*) => "Common Lisp"
inc-pointer
make-pointer
pointerp
null-pointer
null-pointer-p
The function make-pointer will return a foreign pointer
pointing to address.
CFFI> (make-pointer 42) => #<FOREIGN-ADDRESS #x0000002A> CFFI> (pointerp *) => T CFFI> (pointer-address **) => 42 CFFI> (inc-pointer *** -42) => #<FOREIGN-ADDRESS #x00000000> CFFI> (null-pointer-p *) => T
inc-pointer
null-pointer
null-pointer-p
pointerp
pointer-address
pointer-eq
mem-ref
(setf (mem-aref ptr type &optional (index 0)) new-value)
The mem-aref function is similar to mem-ref but will
automatically calculate the offset from an index.
(mem-aref ptr type n) ;; is identical to: (mem-ref ptr type (* n (foreign-type-size type)))
CFFI> (with-foreign-string (str "Hello, foreign world!") (mem-aref str :char 6)) => 32 CFFI> (code-char *) => #\Space CFFI> (with-foreign-object (array :int 10) (loop for i below 10 do (setf (mem-aref array :int i) (random 100))) (loop for i below 10 collect (mem-aref array :int i))) => (22 7 22 52 69 1 46 93 90 65)
CFFI> (with-foreign-string (ptr "Saluton") (setf (mem-ref ptr :char 3) (char-code #\a)) (loop for i from 0 below 8 collect (code-char (mem-ref ptr :char i)))) => (#\S #\a #\l #\a #\t #\o #\n #\Null) CFFI> (setq ptr-to-int (foreign-alloc :int)) => #<A Mac Pointer #x1047D0> CFFI> (mem-ref ptr-to-int :int) => 1054619 CFFI> (setf (mem-ref ptr-to-int :int) 1984) => 1984 CFFI> (mem-ref ptr-to-int :int) => 1984
NULL pointer.
The function null-pointer returns a null pointer.
CFFI> (null-pointer) => #<A Null Mac Pointer> CFFI> (pointerp *) => T
T or NIL.
The function null-pointer-p returns true if ptr is a null
pointer and false otherwise.
CFFI> (null-pointer-p (null-pointer))
=> T
(defun contains-str-p (big little) (not (null-pointer-p (foreign-funcall "strstr" :string big :string little :pointer)))) CFFI> (contains-str-p "Popcorns" "corn") => T CFFI> (contains-str-p "Popcorns" "salt") => NIL
T or NIL.
The function pointerp returns true if ptr is a foreign
pointer and false otherwise.
In Allegro CL, foreign pointers are integers thus in this
implementation pointerp will return true for any ordinary integer.
CFFI> (foreign-alloc 32) => #<A Mac Pointer #x102D20> CFFI> (pointerp *) => T CFFI> (pointerp "this is not a pointer") => NIL
The function pointer-address will return the address of
a foreign pointer ptr.
CFFI> (pointer-address (null-pointer)) => 0 CFFI> (pointer-address (make-pointer 123)) => 123
make-pointer
inc-pointer
null-pointer
null-pointer-p
pointerp
pointer-eq
mem-ref
T or NIL.
The function pointer-eq returns true if ptr1 and
ptr2 point to the same memory address and false otherwise.
The representation of foreign pointers varies across the various Lisp
implementations as does the behaviour of the built-in Common Lisp
equality predicates. Comparing two pointers that point to the same
address with EQ Lisps will return true on some Lisps, others require
more general predicates like EQL or EQUALP and finally
some will return false using any of these predicates. Therefore, for
portability, you should use POINTER-EQ.
This is an example using SBCL, see the implementation-specific notes above.
CFFI> (eql (null-pointer) (null-pointer)) => NIL CFFI> (pointer-eq (null-pointer) (null-pointer)) => T
bindings ::= {(var type &optional count)}*
The macros with-foreign-object and with-foreign-objects
bind var to a pointer to count newly allocated objects
of type type during body. The buffer has dynamic extent
and may be stack allocated if supported by the host Lisp.
CFFI> (with-foreign-object (array :int 10) (dotimes (i 10) (setf (mem-aref array :int i) (random 100))) (loop for i below 10 collect (mem-aref array :int i))) => (22 7 22 52 69 1 46 93 90 65)
The with-foreign-pointer macro, binds var to size
bytes of foreign memory during body. The pointer in var
is invalid beyond the dynamic extend of body and may be
stack-allocated if supported by the implementation.
If size-var is supplied, it will be bound to size during body.
CFFI> (with-foreign-pointer (string 4 size) (setf (mem-ref string :char (1- size)) 0) (lisp-string-to-foreign "Popcorns" string size) (loop for i from 0 below size collect (code-char (mem-ref string :char i)))) => (#\P #\o #\p #\Null)
As with many languages, Lisp and C have special support for logical arrays of characters, going so far as to give them a special name, “strings”. In that spirit, CFFI provides special support for translating between Lisp and C strings.
The :string type and the symbols related below also serve as an
example of what you can do portably with CFFI; were it not
included, you could write an equally functional strings.lisp
without referring to any implementation-specific symbols.
Dictionary
(unsigned-byte 8).
The foreign-string-alloc function allocates a foreign string
containing a Lisp string or (unsigned-byte 8) array.
This string must be freed with foreign-string-free.
CFFI> (setq str (foreign-string-alloc "Hello, foreign world!")) => #<FOREIGN-ADDRESS #x00400560> CFFI> (foreign-funcall "strlen" :pointer str :int) => 21
foreign-string-free
with-foreign-string
foreign-string-alloc.
The foreign-string-free function frees a foreign string
allocated by foreign-string-alloc.
array-total-size-limit, by default.
The foreign-string-to-lisp function copies at most size
characters from ptr into a Lisp string.
When null-terminated-p is true (the default), characters are
copied until size is reached or a NULL character is
found.
If ptr is a null pointer, returns nil.
Note that the :string type will automatically convert between
Lisp strings and foreign strings.
CFFI> (foreign-funcall "getenv" :string "HOME" :pointer) => #<FOREIGN-ADDRESS #xBFFFFFD5> CFFI> (foreign-string-to-lisp *) => "/Users/luis"
lisp-string-to-foreign
foreign-string-alloc
(unsigned-byte 8) array.
The lisp-string-to-foreign function copies at most size-1
characters from a Lisp string or (unsigned-byte 8) arrayto
ptr. The foreign string will be null-terminated.
CFFI> (with-foreign-pointer-as-string (str 255) (lisp-string-to-foreign "Hello, foreign world!" str 6)) => "Hello" CFFI> (with-foreign-pointer-as-string (str 255) (lisp-string-to-foreign (make-array 6 :element-type '(unsigned-byte 8) :initial-contents '(65 66 67 68 69 60)) str 4)) => "ABC"
foreign-string-alloc
foreign-string-to-lisp
with-foreign-pointer-as-string
(unsigned-byte 8).
The with-foreign-string macro will bind var to a newly
allocated foreign string containing lisp-string-or-ub8-array.
CFFI> (with-foreign-string (foo "12345") (foreign-funcall "strlen" :pointer foo :int)) => 5 CFFI> (let ((array (coerce #(84 117 114 97 110 103 97) '(array (unsigned-byte 8))))) (with-foreign-string (foreign-string array) (foreign-string-to-lisp foreign-string))) => "Turanga"
foreign-string-alloc
with-foreign-pointer-as-string
The with-foreign-pointer-as-string macro is similar to
with-foreign-pointer except that var, as a Lisp string, is
used as the returned value of an implicit progn around body.
CFFI> (with-foreign-pointer-as-string (str 6 str-size)
(lisp-string-to-foreign "Hello, foreign world!" str str-size))
=> "Hello"
foreign-string-alloc
with-foreign-string
Dictionary
name ::= lisp-name | foreign-name | (foreign-name lisp-name)
The defcvar macro
When one of lisp-name or foreign-name is omitted, the other is automatically derived using the following rules:
CFFI> (defcvar "errno" :int) => *ERRNO* CFFI> (foreign-funcall "strerror" :int *errno* :string) => "Inappropriate ioctl for device" CFFI> (setf *errno* 1) => 1 CFFI> (foreign-funcall "strerror" :int *errno* :string) => "Operation not permitted"
Trying to modify a read-only foreign variable:
CFFI> (defcvar ("errno" +error-number+) :int :read-only t) => +ERROR-NUMBER+ CFFI> (setf +error-number+ 12) ;; error--> Trying to modify read-only foreign var: +ERROR-NUMBER+.
Note that accessing errno this way won't work with every
C standard library.
defcvar.
The function get-var-pointer will return a pointer to the
foreign global variable symbol previously defined with
defcvar.
CFFI> (defcvar "errno" :int :read-only t) => *ERRNO* CFFI> *errno* => 25 CFFI> (get-var-pointer '*errno*) => #<A Mac Pointer #xA0008130> CFFI> (mem-ref * :int) => 25
Dictionary
The defcfun macro provides a declarative interface for defining
Lisp functions that call foreign functions.
When one of lisp-name or foreign-name is omitted, the other is automatically derived using the following rules:
If you place the symbol &rest in the end of the argument list
after the fixed arguments, defcfun will treat the foreign
function as a variadic function. The variadic arguments
should be passed in a way similar to what foreign-funcall would
expect. Unlike foreign-funcall though, defcfun will take
care of doing argument promotion. Note that in this case
defcfun will generate a Lisp macro instead of a
function and will only work for Lisps that support
foreign-funcall.
(defcfun "strlen" :int (n :string)) CFFI> (strlen "123") => 3
(defcfun ("abs" c-abs) :int (n :int)) CFFI> (c-abs -42) => 42
Variadic function example:
(defcfun "sprintf" :int (str :pointer) (control :string) &rest) CFFI> (with-foreign-pointer-as-string (s 100) (sprintf s "%c %d %.2f %s" :char 90 :short 42 :float pi :string "super-locrian")) => "A 42 3.14 super-locrian"
arguments ::= { arg-type arg }* [return-type]
:void by default.
The foreign-funcall macro is the main primitive for calling
foreign functions.
Note: The return value of foreign-funcall on functions with a :void return type is still undefined.
foreign-funcall. On
implementations that don't support foreign-funcall
cffi-features:no-foreign-funcall will be present in
*features*. Note: in these Lisps you can still use the
defcfun interface.
CFFI> (foreign-funcall "strlen" :string "foo" :int)
=> 3
Given the C code:
void print_number(int n)
{
printf("N: %d\n", n);
}
CFFI> (foreign-funcall "print_number" :int 123456) -| N: 123456 => NIL
Or, equivalently:
CFFI> (foreign-funcall "print_number" :int 123456 :void)
-| N: 123456
=> NIL
CFFI> (foreign-funcall "printf" :string (format nil "%s: %d.~%")
:string "So long and thanks for all the fish"
:int 42 :int)
-| So long and thanks for all the fish: 42.
=> 41
Dictionary
Almost all foreign code you might want to access exists in some kind of shared library. The meaning of shared library varies among platforms, but for our purposes, we will consider it to include .so files on unix, frameworks on Darwin (and derivatives like Mac OS X), and .dll files on Windows.
Bringing one of these libraries into the Lisp image is normally a two-step process.
define-foreign-library top-level form.
use-foreign-library form or by calling the function
load-foreign-library.
See Loading foreign libraries, for a working example of the above two steps.
Looking at the libcurl library definition presented earlier,
you may ask why we did not simply do this:
(define-foreign-library libcurl
(t (:default "libcurl")))
Indeed, this would work just as well on the computer on which I tested the tutorial. There are a couple of good reasons to provide the .so's current version number, however. Namely, the versionless .so is not packaged on most unix systems along with the actual, fully-versioned library; instead, it is included in the “development” package along with C headers and static .a libraries.
The reason CFFI does not try to account for this lies in the meaning of the version numbers. A full treatment of shared library versions is beyond this manual's scope; see Library interface versions, for helpful information for the unfamiliar. For our purposes, consider that a mismatch between the library version with which you tested and the installed library version may cause undefined behavior.12
Implementor's note: Maybe some notes should go here about OS X, which I know little about. –stephen
A list, in which each element is a string, a pathname, or a simple Lisp expression.
A list containing the following, in order: an expression corresponding
to Darwin path ~/Library/Frameworks/,
#P"/Library/Frameworks/", and
#P"/System/Library/Frameworks/".
The meaning of “simple Lisp expression” is explained in *foreign-library-directories*. In contrast to that variable, this is not a fallback search path; the default value described above is intended to be a reasonably complete search path on Darwin systems.
CFFI> (load-foreign-library '(:framework "OpenGL"))
=> #P"/System/Library/Frameworks/OpenGL.framework/OpenGL"
*foreign-library-directories*
define-foreign-library
load-clause ::= (feature { library }*)
Creates a new library designator called name. The
load-clauses describe how to load that designator when passed to
load-foreign-library or use-foreign-library.
When trying to load the library name, the relevant function searches the load-clauses in order for the first one where feature evaluates to true. That happens for any of the following situations:13
cffi-features package, is
present in common-lisp:*features*.
(first feature),
a keyword:
:and(rest feature) are
true.
:or(rest feature)
is true.
:not(second feature) is not true.
t, this load-clause is
picked unconditionally.
Upon finding the first true feature, the library loader then loads each library. The meaning of “library designator” is described in load-foreign-library.
See Loading foreign libraries.
A list, in which each element is a string, a pathname, or a simple Lisp expression.
The empty list.
You should not have to use this variable.
Most, if not all, Lisps supported by CFFI have a reasonable default
search algorithm for foreign libraries. For example, Lisps for
unix usually call
dlopen(3), which in turn looks in the system library
directories. Only if that fails does CFFI look for the named
library file in these directories, and load it from there if found.
Thus, this is intended to be a CFFI-only fallback to the library search configuration provided by your operating system. For example, if you distribute a foreign library with your Lisp package, you can add the library's containing directory to this list and portably expect CFFI to find it.
A simple Lisp expression is intended to provide functionality commonly used in search paths such as ASDF's14, and is defined recursively as follows:15
$ ls -| liblibli.so libli.lisp
In libli.lisp:
(pushnew #P"/home/sirian/lisp/libli/" *foreign-library-directories* :test #'equal) (load-foreign-library '(:default "liblibli"))
*darwin-framework-directories*
define-foreign-library
Load the library indicated by library. A library designator is defined as follows:
define-foreign-library.
*foreign-library-directories* with
cl:probe-file; if found, the absolute path is passed to the
implementation's loader.
(first library):
:framework*darwin-framework-directories*, and loaded
when found.
:or:defaultIf the load fails, signal a load-foreign-library-error.
Please note: For system libraries, you should not need to specify the directory containing the library. Each operating system has its own idea of a default search path, and you should rely on it when it is reasonable.
On ECL platforms where its dynamic FFI is not supported (ie. when
:dffi is not present in *features*),
cffi:load-foreign-library does not work and you must use ECL's
own ffi:load-foreign-library with a constant string argument.
See Loading foreign libraries.
*darwin-framework-directories*
define-foreign-library
*foreign-library-directories*
load-foreign-library-error
use-foreign-library
load-foreign-library-error, error,
serious-condition, condition, t
Signalled when a foreign library load completely fails. The exact
meaning of this varies depending on the real conditions at work, but
almost universally, the implementation's error message is useless.
However, CFFI does provide the useful restarts retry and
use-value; invoke the retry restart to try loading the
foreign library again, or the use-value restart to try loading
a different foreign library designator.
See load-foreign-library, for the meaning of “library
designator”. This is intended to be the top-level form used
idiomatically after a define-foreign-library form to go ahead
and load the library.
Finally, on implementations where the regular evaluation rule is
insufficient for foreign library loading, it loads it at the required
time.16
See Loading foreign libraries.
Dictionary
The callback macro is analogous to the standard CL special
operator function and will return a pointer to the callback
denoted by the symbol name.
CFFI> (defcallback sum :int ((a :int) (b :int)) (+ a b)) => SUM CFFI> (callback sum) => #<A Mac Pointer #x102350>
arguments ::= ({ (arg-name arg-type) }*)
The macro defcallback defines a Lisp function the can be called
from C (but not from Lisp). The arguments passed to this function will
be converted to the appropriate Lisp representation and its return
value will be converted to its C representation.
This Lisp function can be accessed by the callback macro or the
get-callback function.
Portability note: defcallback will not work correctly
on some Lisps if it's not a top-level form.
(defcfun "qsort" :void (base :pointer) (nmemb :int) (size :int) (fun-compar :pointer)) (defcallback < :int ((a :pointer) (b :pointer)) (let ((x (mem-ref a :int)) (y (mem-ref b :int))) (cond ((> x y) 1) ((< x y) -1) (t 0)))) CFFI> (with-foreign-object (array :int 10) ;; Initialize array. (loop for i from 0 and n in '(7 2 10 4 3 5 1 6 9 8) do (setf (mem-aref array :int i) n)) ;; Sort it. (qsort array 10 (foreign-type-size :int) (callback <)) ;; Return it as a list. (loop for i from 0 below 10 collect (mem-aref array :int i))) => (1 2 3 4 5 6 7 8 9 10)
This is the functional version of the callback macro. It
returns a pointer to the callback named by symbol suitable, for
example, to pass as arguments to foreign functions.
CFFI> (defcallback sum :int ((a :int) (b :int)) (+ a b)) => SUM CFFI> (get-callback 'sum) => #<A Mac Pointer #x102350>
Although object “finalization” is orthogonal to FFI, it often is a useful feature to have when implementing wrapper objects, for example. Therefore, CFFI provides a portable API for registering and canceling finalizers.
Dictionary
The function finalize pushes function to object's
list of finalizers, which will be invoked when object is
garbage-collected.
function should take no arguments.
For portability reasons, function should not attempt to look at object by closing over it because, in some lisps, object will already have been garbage-collected and is therefore not accessible when function is invoked.
(defclass wrapper-object () ...) (defmethod initialize-instance :after ((self wrapper-object) &key) (let ((foreign-pointer ...)) ... (finalize self (lambda () (foreign-free foreign-pointer)))))
The cancel-finalization function cancels all of object's
finalizers, if any.
For example: use this when you want to free a wrapper object's resources immediately in order to avoid a double free when the finalizer is invoked (see the example for finalize).
These are CFFI's limitations across all platforms; for information on the warts on particular Lisp implementations, see Implementation Support.
structs cannot be passed by value.
CFFI does some platform tests on loading. The details vary between Lisps; in fact, the purpose is to unify the list of available platform features for use elsewhere in the CFFI code. These features are also part of the public interface; see define-foreign-library.
The exact meanings of the features follow. Though you will usually
refer to these symbols as keywords, CFFI internally views them in
the package cffi-features.
darwinno-foreign-funcallforeign-funcall is not available. On such
platforms, the only way to call a foreign function is through
defcfun. See foreign-funcall, and defcfun.
no-long-longlong long type is not available as a foreign
type.
no-finalizersppc32unixwindowsx86When using type translators, the foreign value is the lower-level Lisp
value derived from the object passed to translate-to-foreign
(see translate-to-foreign). This value should be a Lisp number or
a pointer (satisfies pointerp), and it can be treated like any
general Lisp object; it only completes the transformation to a true
foreign value when passed through low-level code in the Lisp
implementation, such as the foreign function caller or indirect memory
addressing combined with a data move.
In other contexts, this refers to a value accessible by C, but which
may only be accessed through CFFI functions. The closest you can
get to such a foreign value is through a pointer Lisp object, which
itself counts as a foreign value in only the previous sense.
:boolean: Other Types:char: Built-In Types:double: Built-In Types:float: Built-In Types:int: Built-In Types:int16: Built-In Types:int32: Built-In Types:int64: Built-In Types:int8: Built-In Types:llong: Built-In Types:long: Built-In Types:long-double: Built-In Types:long-long: Built-In Types:pointer: Built-In Types:short: Built-In Types:string: Other Types:string+ptr: Other Types:uchar: Built-In Types:uint: Built-In Types:uint16: Built-In Types:uint32: Built-In Types:uint64: Built-In Types:uint8: Built-In Types:ullong: Built-In Types:ulong: Built-In Types:unsigned-char: Built-In Types:unsigned-int: Built-In Types:unsigned-long: Built-In Types:unsigned-long-long: Built-In Types:unsigned-short: Built-In Types:ushort: Built-In Types:void: Built-In Types:wrapper: Other Typescallback: callbackcancel-finalization: cancel-finalizationconvert-from-foreign: convert-from-foreignconvert-to-foreign: convert-to-foreigndefbitfield: defbitfielddefcallback: defcallbackdefcenum: defcenumdefcfun: defcfundefcstruct: defcstructdefctype: defctypedefcunion: defcuniondefcvar: defcvardefine-foreign-library: define-foreign-librarydefine-foreign-type: define-foreign-typefinalize: finalizeforeign-alloc: foreign-allocforeign-bitfield-symbols: foreign-bitfield-symbolsforeign-bitfield-value: foreign-bitfield-valueforeign-enum-keyword: foreign-enum-keywordforeign-enum-value: foreign-enum-valueforeign-free: foreign-freeforeign-funcall: foreign-funcallforeign-slot-names: foreign-slot-namesforeign-slot-offset: foreign-slot-offsetforeign-slot-pointer: foreign-slot-pointerforeign-slot-value: foreign-slot-valueforeign-string-alloc: foreign-string-allocforeign-string-free: foreign-string-freeforeign-string-to-lisp: foreign-string-to-lispforeign-symbol-pointer: foreign-symbol-pointerforeign-type-alignment: foreign-type-alignmentforeign-type-size: foreign-type-sizefree-converted-object: free-converted-objectfree-translated-object: free-translated-objectget-callback: get-callbackget-var-pointer: get-var-pointerinc-pointer: inc-pointerincf-pointer: incf-pointerlisp-string-to-foreign: lisp-string-to-foreignload-foreign-library: load-foreign-libraryload-foreign-library-error: load-foreign-library-errormake-pointer: make-pointermem-aref: mem-arefmem-ref: mem-refnull-pointer: null-pointernull-pointer-p: null-pointer-ppointer-address: pointer-addresspointer-eq: pointer-eqpointerp: pointerptranslate-from-foreign: translate-from-foreigntranslate-to-foreign: translate-to-foreignuse-foreign-library: use-foreign-librarywith-foreign-object: with-foreign-objectwith-foreign-objects: with-foreign-objectwith-foreign-pointer: with-foreign-pointerwith-foreign-pointer-as-string: with-foreign-pointer-as-stringwith-foreign-slots: with-foreign-slotswith-foreign-string: with-foreign-string[1] Admittedly, this is an advanced issue, and we encourage you to leave this text until you are more familiar with how CFFI works.
[2] This does not apply to structs whose contents are intended to be part of the public library interface. In those cases, a pure Lisp struct definition is always preferred. In fact, many prefer to stay in Lisp and break the encapsulation anyway, placing the burden of correct library interface definition on the library.
[3] Specifically,
UFFI, an older FFI that takes a somewhat different
approach compared to CFFI. I believe that these days (December
2005) CFFI is more portable and actively developed, though not as
mature yet. Consensus in the free unix Common Lisp community
seems to be that CFFI is preferred for new development, though
UFFI will likely go on for quite some time as many projects
already use it. CFFI includes the UFFI-COMPAT package for
complete compatibility with UFFI.
[4] This isn't entirely true; some Lisps
don't support foreign-funcall, so defcfun is implemented
without it. defcfun may also perform optimizations that
foreign-funcall cannot.
[5] Another possibility is to allow the caller to specify the desired C type of the third argument. This is essentially what happens in a call to the function written in C.
[6] See Other Kinds of Streams, for a GNU-only way to extend the FILE*
type. You could use this to convert Lisp streams to the needed C
data. This would be quite involved and far outside the scope of this
tutorial.
[7] “But I thought Lisp was supposed to protect me from all that buggy C crap!” Before asking a question like that, remember that you are a stranger in a foreign land, whose residents have a completely different set of values.
[8] There are advantages and
disadvantages to each approach; I chose to (setf
symbol-function) earlier because it entailed generating fewer magic
function names.
[9] Unfortunately, we can't protect against
all non-local exits, such as returns and throws,
because unwind-protect cannot be used to “short-circuit” a
non-local exit in Common Lisp, due to proposal minimal in
ANSI issue Exit-Extent. Furthermore, binding an
error handler prevents higher-up code from invoking restarts
that may be provided under the callback's dynamic context. Such is
the way of compromise.
[10] It might be better to return
(values) than :curle-ok in real code, but this is good
for illustration.
[11] The definition of memory includes the CPU registers.
[12] Windows programmers may chafe at adding a
unix-specific clause to define-foreign-library. Instead,
ask why the Windows solution to library incompatibility is “include
your own version of every library you use with every program”.
[13] This is described in
cffi-feature-p in libraries.lisp.
[14] See Using asdf to load systems, for information on
asdf:*central-registry*.
[15] See mini-eval in libraries.lisp for
the source of this definition. As is always the case with a Lisp
eval, it's easier to understand the Lisp definition than the
english.
[16] Namely, CMUCL. See
use-foreign-library in libraries.lisp for details.