Re: Feature request: Check in system directory when cimporting module

On 31 January 2012 17:50, Stefan Behnel <stefan_ml <at> behnel.de> wrote:
> Thomas Wiecki, 31.01.2012 16:26:
>> On Tue, Jan 31, 2012 at 9:47 AM, Stefan Behnel wrote:
>>> Thomas Wiecki, 31.01.2012 14:52:
>>>> it seems that cimport is only looking in the local and Cython/Includes
>>>> directory for a pxd module when using cimport. However, for people to
>>>> start building cython packages I think there should be an option to
>>>> install pxd modules that then can get cimported system-wide. Is there
>>>> a current way to do this? I did not have luck so far using CythonGSL.
>>>> I guess the most straight forward way would be to look in the python
>>>> dist-packages directory for pxd modules as then one could simply build
>>>> a setup.py and put the pxd files as package_data.
>>>
>>> I may be mistaken, but I thought that the compiler searched the PYTHONPATH
>>> when encountering a cimport. Doesn't it do that?
>>
>> It at least doesn't seem that way. When I install into PYTHONPATH,
>> cimport of my pxd packages doesn't work, but it does when I put it
>> into Cython/Includes. It'd be great if that was a bug (either on my or
>> cython's side).
>
> Looks like you're right. It only searches in the "include path" (which is a
> bad name that persists due to historical reasons), and that path doesn't
> include the PYTHONPATH.
>
> I think it should. We should fix this for 0.16.
>
> http://trac.cython.org/cython_trac/ticket/761
>

+1. This means that the Cython include path continues to work as-is
(perhaps with some documentation on the matter :), but it would be
extended with entries from sys.path, right?

>>>> I guess another option would be to allow installing into
>>>> Cython/Includes via some distutils extension.
>>>
>>> Not a good idea. External .pxd files should be provided by and in the
>>> packages that ship them.
>>
>> Well, but wouldn't you want to support cython-only packages (like
>> CythonGSL)?
>
> Sure. Exactly the way we support Python packages. CythonGSL seems to have
> the appropriate package layout already. (Although it's not really a
> "wrapper" as the setup.py states, more of a set of declarations to use the
> library from Cython code).
>
>
>> E.g. libc is provided by cython, but if that wasn't the
>> case, you would not want every package that uses libc to provide those
>> wrappers and instead enable installation of a third-party extension
>> that provides the wrapper system-wide, no? (I'm just talking about
>> that general idea, not necessarily on how to implement it).
>
> Right. And that means someone writes a set of declarations and ships it as
> an external package full of .pxd files and (preferably) an __init__.py
> package file that contains the version of the package and maybe also a way
> to figure out additional .h include paths etc., so that external packages
> get a limited form of package introspection and build support. (You could
> do the same thing in CythonGSL, i.e. provide a get_include() function that
> calls "gsl-config" appropriately.)
>
> BTW, I noticed that CythonGSL actually presents an include statement as the
> right way of using it in the README file. Looks like a left-over to me. In
> any case, a cimport of .pxd files is the best way of doing this because the
> import mechanism gives the users the greatest freedom in how they get your
> declarations into their namespace.
>
> Stefan