Variadic functions and libffi

Hi,
  As part of the linaro toolchain effort for ARM, I've been asked to
look at Variadic function calls in libffi,
specifically for the ARM 'hard float' (aka PCS_VFP or FFI_VFP, armhf)
variant and wanted to see what
people thought before I dived in to it.

  The ARM hard float variant is a variant of the ABI in which floating
point values are passed in floating point
registers (as opposed to the standard ABI where FP values are still
passed in integer registers).  What makes
this ABI a little unusual is that for variadic functions it just falls
back to the standard all-integer-register ABI, and
thus, as an example, in the function:

   double foo(double bar, char *str, ...)

the return value and the 'bar' would be passed in integer registers
even though they aren't actually variadic
parameters.

It doesn't seem like there is any way out of this except for an API
change to libffi to state that a variadic function is being
called - so as I see it the questions then are:

   1) Does libffi really support variadic functions and/or is there a
desire to?  There are big warnings in various places saying
 it doesn't - but then the testcases call variadic functions, and at
least Python and SWIG both show examples of using it (with FP values).
On the other hand there is an ancient gcc bug
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=26744 to fix variadic on
PPC MacOS
marked as 'wishlist' because libffi doesn't support variadic.

   2) What's a good/minimal API change.

   3) Which FFI users actually use variadics and are hitting problems.

As for an API change, as far as ARM hard float is concerned we don't
actually care which parameters are
variadic, we just care that the function is declared variadic, so a
minimal change would be to define a FFI_DEFAULT_VARIADIC_ABI
that is the same as FFI_DEFAULT_ABI on everything except ARM hard
float, in which case it's set to FFI_SYSV, and when calling
ffi_prep_cif you would pass FFI_DEFAULT_VARIADIC_ABI rather than
FFI_DEFAULT_ABI for variadic calls.

That's simple for ARM but it seems unfair on any other architecture
that needs to fix things up.

I've had some discussion with Marcus Shawcroft (cc'd) and it looks
like the right solution is to declare a

'ffi_prep_cif_variadic'

that is just like ffi_prep_cif but also takes a 'nnamedargs' to say
the number of fixed args (and hence which arg is the 1st variadic).
For all architectures that don't care this can just call ffi_prep_cif
and lose the extra arg, this should mean it won't break anything else,
but any architecture that did need to know which were variadic args
would have that information.

I know that there are some other architectures that currently don't
work with variadic FP args; some might have to do something
special, but it's possible that some may be fixable without them
needing this ABI change - e.g. some might be able to pass
the FP values both in memory and FP registers and get away with this.

Closures are a separate question; I wouldn't propose supporting
variadic closures - but closures that take a fixed number of
parameters
but then call a variadic function sound doable using the same
ffi_prep_cif_variadic modification.

I've started putting together a list of FFI users with comments on
whether they use variadics - it's here
https://wiki.linaro.org/WorkingGroups/ToolChain/FFIusers
and I'd appreciate all input - I'm especially interested if anyone has
a real world app/test that uses variadics through ffi with fp.

All comments welcome, if people are happy with the proposed change,
I'm happy to do the change to libffi for ARM and make sure it doesn't
break the other arch that I have access to.

Dave