python lists and c-extension, almost there but still need help.

Hi,
sorry for bothering you again, but this is getting a real problem for me. I
need to write some c extensions that will receive large lists (float only)
do some calcs and return other lists with the results.
I have learned a lot about Swig and typemaps and below you find an example
of what I have right now. If I pass an list argument (input) to my c
extension it is transformed into a contiguous Numeric array and a second var
holding its size. The pointer to the data field of this array and the size
var are passed to the c-function.

If the results have to be stored in another list, I have to pass the size of
this resultlist to my python function. The second typemap allocates a memory
block of this size and passes the pointer and the
size to the c-function.

When returning from the c-function the "out-lists" are transformed into
python Numeric array objects and are attached to the result tuple.

so i can write:

import Numeric
import ctest
result=ctest([1,2,3,4],8) #inputlist, size of resultlist

and will receive:
(30, [1,2,3,4,2,4,6,8]) #return value of c function and resultlist

all this is really wonderful. But if I do:

for i in range(1000):
    result=ctest(longlist,largeresult)

i can watch my heap grow and grow. I have played around with some
Py_DECREF here and there but this isn't getting me anywhere.
I can't get rid of these memory leaks and I have no idea what to do now.

Please,please, if someone has written something similar or knows what to do
... let me know.

Thanks
marcus

P.S. There is a remark of Konrad Hinsen that PyArray_FromDimsAndData() might
never be freed. Although I don't understand the reason I tried
PyArray_FromDims with copying the data one by one. -> Slow and didn't help.

#--------------------snip-------------------
%module ctest
%{
#include "ctest.h"
%}
//--------------------------------------------------------------------------
---
typedef double d_list_in;
typedef double d_list_out;
//--------------------------------------------------------------------------
---
%include "typemaps.i"
//--------------------------------------------------------------------------
---
// doublelist coming in
%typemap(in) (d_list_in *, int) {
  int size;
  if (!PySequence_Check($input)) {
    PyErr_SetString(PyExc_ValueError,"Expected a sequence as double (in)");
    return NULL;
  }
  size=PySequence_Length($input);
  PyArrayObject  *data;
  data = (PyArrayObject *) PyArray_ContiguousFromObject($input,
PyArray_DOUBLE, 1, 0);
  if (data == NULL){
    PyErr_SetString(PyExc_ValueError,"PyArray_ContiguousFromObject returns
NULL");
     return NULL;
  }
  $1= (double *) data->data;
  $2= size;
}
%typemap(freearg) double * {
   if ($1) free($1);
}
//--------------------------------------------------------------------------
---
//--------------------------------------------------------------------------
---
// result list that will be filled with doubles
%typemap(in) ( d_list_out *, int) {
   if (!PyInt_Check($input)) {
       PyErr_SetString(PyExc_ValueError, "Expecting an integer (size of
double resultlist)");
       return NULL;
   }
   $2 = PyInt_AsLong($input);
   if ($2 < 0) {
       PyErr_SetString(PyExc_ValueError, "Positive integer expected");
       return NULL;
   }
   $1 = (double *) malloc($2*sizeof(double));
}
//
%typemap(freearg) d_list_out * {
   if ($1) free($1);
}
//--------------------------------------------------------------------------
---
//--------------------------------------------------------------------------
---
//filled out list that will be appended to current result
%typemap(argout) (d_list_out *, int) {
    PyObject *retarray;
    PyObject *o2, *o3;
    int size[1];

    size[0] = $2
    size[1] = 0;

    retarray = (PyObject *)
PyArray_FromDimsAndData(1,size,PyArray_DOUBLE,(char *) $1);

    if (retarray == NULL){
      PyErr_SetString(PyExc_ValueError,"PyArray_FromDims returns NULL");
       return NULL;
    }

    if ((!$result) || ($result == Py_None)) {
        $result = retarray;
    } else {
        if (!PyTuple_Check($result)) {
            PyObject *o2 = $result;
            $result = PyTuple_New(1);
            PyTuple_SET_ITEM($result,0,o2);
        }
        o3 = PyTuple_New(1);
        PyTuple_SetItem(o3,0,retarray);
        o2 = $result;
        $result = PySequence_Concat(o2,o3);
        Py_DECREF(o2);
        Py_DECREF(o3);
    }
}
//--------------------------------------------------------------------------
---
//include header file or enter declarations
//--------------------------------------------------------------------------
---
double calclist (d_list_in *, int, d_list_out *, int);
//--------------------------------------------------------------------------
---

_______________________________________________
Swig maillist  -  Swig <at> cs.uchicago.edu
http://mailman.cs.uchicago.edu/mailman/listinfo/swig