SWIG and Complex C structures
I had to use SWIG to access a kernel module’s chardev interface through python and found SWIG examples are not enough, so adding my own.
Lets take the following example header file.
I will explain how to access all the members in complex_struct_t from python.
Also extend these structures so that python code would look little better.
[codegroup]
[c tab=”header”]
/* test.h */
#ifndef _TEST_STRUCT_H
#define _TEST_STRUCT_H
/* a simple structure – no problem with SWIG */
typedef struct simple_struct {
int int_var;
long long_var;
float float_var;
} simple_struct_t;
typedef struct tlv_base {
int type;
int length;
unsigned char value[];
} tlv_base_t;
typedef struct tlv_type1 {
tlv_base_t base;
int stat;
int info;
long something;
} tlv_type1_t;
/* relatively complex C structure. */
typedef struct complex_struct {
char string[10]; //SWIG considers this as null terminated string
unsigned char bytes[10]; //SWIG wont considers this as string
simple_struct_t embedded;
int pointer_array_count;
simple_struct_t *pointer_array; //SWIG supports only accessing first element.
tlv_base_t tlv; //How do cast this to derived_struct_1 ?
} complex_struct_t;
complex_struct_t * alloc_complex_struct(int array_count);
void free_complex_struct(complex_struct_t *cs);
void print_complex_struct(complex_struct_t *cs);
#endif
[/c]
[c tab=’source’]
/* test.c */
#include
#include
#include “test.h”
complex_struct_t *
alloc_complex_struct(int array_count)
{
complex_struct_t *result;
size_t size;
result = (complex_struct_t *)malloc(sizeof(complex_struct_t) + sizeof(tlv_type1_t));
if (result == NULL){
return NULL;
}
result->tlv.type = 1;
result->tlv.length = sizeof(tlv_type1_t);
size = sizeof(simple_struct_t) * array_count;
result->pointer_array = (simple_struct_t *)malloc(size);
if (result->pointer_array == NULL) {
free(result);
return NULL;
}
memset(result->pointer_array, 0, size);
result->pointer_array_count = array_count;
return result;
}
void
free_complex_struct(complex_struct_t *cs)
{
free(cs->pointer_array);
free(cs);
}
static inline void
print_simple_struct(simple_struct_t *ss)
{
printf(“int %d long %ld float %f\n”, ss->int_var, ss->long_var, ss->float_var);
}
void
print_complex_struct(complex_struct_t *cs)
{
int i;
printf(“String = %s\n”, cs->string);
printf(“Embedded : “);
print_simple_struct(&cs->embedded);
printf(“External : \n”);
for (i=0; i
printf(“%d) “, i + 1);
print_simple_struct(&cs->pointer_array[i]);
}
}
[/c]
[raw tab=’interface’]
// test_swig.i
%module test_struct
%{
#include “../test.h”
%}
%include “test.h”
[/raw]
[shell tab=’commands’]
# Commands to make the shared library
$ mkdir -p _build
$ gcc -I /usr/include/python2.7/ -fPIC -c -o _build/test.o test.c
$ swig -python -outdir _build -o _build/test_swig_wrap.c test_swig.i
$ gcc -I /usr/include/python2.7/ -fPIC -c -o _build/test_swig_wrap.o _build/test_swig_wrap.c
$ ld -shared _build/test.o _build/test_swig_wrap.o -o _build/_test_struct.so
$ rm _build/test_swig_wrap.c
[/shell]
[raw tab=’makefile’]
#Makefile
BLD_DIRECTORY = _build
SWIG_SRCS = test_swig.i
C_SRCS = test.c
CFLAGS = -I /usr/include/python2.7/ -fPIC -c -Wall
OBJS = $(patsubst %.c, $(BLD_DIRECTORY)/%.o, $(C_SRCS)) \
$(patsubst %.i, $(BLD_DIRECTORY)/%_wrap.o, $(SWIG_SRCS))
$(BLD_DIRECTORY)/%.o: %.c %.h
gcc $(CFLAGS) -o $@ $<
$(BLD_DIRECTORY)/%.o: $(BLD_DIRECTORY)/%.c
gcc $(CFLAGS) -o $@ $<
$(BLD_DIRECTORY)/%_wrap.c: %.i
swig -python -outdir $(BLD_DIRECTORY) -o $@ $<
cp $@ $@.bak
$(BLD_DIRECTORY):
mkdir -p $(BLD_DIRECTORY)
clean:
rm -rf $(BLD_DIRECTORY)
all: $(BLD_DIRECTORY) $(OBJS) $(C_SRCS) $(SWIG_SRCS)
ld -shared $(OBJS) -o $(BLD_DIRECTORY)/_test_struct.so
.PHONY: all clean
.DEFAULT_GOAL := all
[/raw]
[/codegroup]
With this simple interface file, SWIG would be able to create a _test_struct.so and test_struct.py which is perfect for most cases.
[raw tab=’interface’]
%module test_struct
%{
#include “../test.h”
%}
%include “test.h”
[/raw]
[python]
import test_struct as ts
cs = ts.alloc_complex_struct(1)
cs.string = “Hello”
cs.embedded.int_var = 9
cs.embedded.long_var = 10
cs.embedded.float_var = 11.23
ts.print_complex_struct(cs)
ts.free_complex_struct(cs)
[/python]
This shows SWIG’s ability to convert C string to python string and vice versa.
Similarly accessing primitive structure members is very easy.
Here is the output of the program when ran from _build directory.
[raw]
String = Hello
Embedded : int 9 long 10 float 11.230000
Pointer Array :
1) int 0 long 0 float 0.000000
[/raw]
It also shows how to call function call into C library. If you have noticed this program looks more like a C program rather than a python program – mainly because it manages the memory allocation/free. Python can informed that [c] alloc_complex_struct() [/c] returns a new object and it is the responsibility of the caller to free it by using the SWIG typemap [python] newobject [/python]. Now python garbage collector will free the object when there is no reference. But python does not know how to free the object(complex_struct_t) – this can be done by using [python]newfree[/python] typemap.
By adding the following to the test_swig.i, we can avoid calling free_complex_struct() in python program.
[c]
%typemap(newobject) alloc_complex_struct;
%typemap(newfree) complex_struct_t * {
free_complex_struct($1);
}
[/c]
Lets modify the program a little bit and access the pointer_array elements.
[python]
import test_struct as ts
cs = ts.alloc_complex_struct(5)
print ‘Pointer array count ‘, cs.pointer_array_count
print cs.pointer_array[0]
ts.free_complex_struct(cs)
[/python]
This will fail with the following error:
[raw]
Pointer array count 5
Traceback (most recent call last):
File “./test.py”, line 4, in
print cs.pointer_array[0]
TypeError: ‘simple_struct_t’ object does not support indexing
[/raw]
The reason is SWIG does not really know [c]simple_struct_t *pointer_array;[/c] actually points to an array of [c]simple_struct_t[/c]. In other words SWIG safely assumes it points to a single entry. If pointer_array was “array of simple_struct_t pointers” then carrays.i macro would have been helped. But pointer_array is actually “pointer to array of simple_struct_t” so carrays.i won’t help.
The easiest way is extending complex_struct_t and add a new member(kind of) function to it.
[c]
%extend complex_struct_t{
simple_struct_t *get_array_element(int i) {
return &$self->pointer_array[i];
}
}
[/c]
This way cs.get_array_element(4) will return 4th element in the array.
Similarly tlv elements can be accessed also – but this time I decided to override indexing operator([]).
[c]
%extend complex_struct_t{
unsigned char __getitem__(int i) {
return $self->tlv[i];
}
}
[/c]
However this is not useful since python cant cast from [c](struct tlv_base *)[/c] to [c]struct tlv_type1 *[/c]. To cast, a C function can be coded or SWIG’s cpointer.i can be used.
Here is the full test_swig.i
[c]
%module test_struct
%{
#include “../test.h”
%}
%include “test.h”
%extend complex_struct{
simple_struct_t *get_array_element(int i) {
return &$self->pointer_array[i];
}
}
%typemap(newobject) alloc_complex_struct;
%typemap(newfree) complex_struct_t * {
free_complex_struct($1);
}
%include
%pointer_cast(tlv_base_t *, tlv_type1_t *, cast_to_tlv_type1);
[/c]
And test code:
[python]
import test_struct as ts
cs = ts.alloc_complex_struct(5)
cs.string = ‘Hello’
print ‘Pointer array count ‘, cs.pointer_array_count
for i in range(cs.pointer_array_count):
simple_struct = cs.get_array_element(i)
simple_struct.int_var = i * 10
simple_struct.long_var = i * 20
simple_struct.float_var = i * 3.3
ts.print_complex_struct(cs)
tlv = ts.cast_to_tlv_type1(cs.tlv)
print tlv.stat, tlv.info, tlv.something
[/python]