8 Mar 2011 08:25
Re: correct way to compile cython project with icc?
Dag Sverre Seljebotn <d.s.seljebotn <at> astro.uio.no>
2011-03-08 07:25:39 GMT
2011-03-08 07:25:39 GMT
On 03/06/2011 05:20 AM, Zak Stone wrote: > Hello everyone, > > I would like to compile a Cython project with Intel's compiler icc to > benchmark the resulting binary against one produced by gcc. It appears > that calling "python setup.py build_ext" with CC="icc" is enough to > switch the Cython compilation step to icc, but it is difficult to > prevent distutils from passing the wrong flags to icc, and distutils > insists on performing the linking step with gcc. > > It is possible to follow NumPy's lead and patch distutils to support > icc -- that only requires a minor change to ccompiler.py and the > addition of a file like intelccompiler.py -- but I'd rather not > maintain a patch on my system-wide Python installation, and it appears > that virtualenv doesn't make it easy to patch Python itself in an > isolated local environment. It might be possible to use a modified > local copy of distutils only as NumPy seems to, but then I would also > have to modify Cython to interface with the modified copy of > distutils, and the resulting toolchain would be rather fragile. > > Of course, there could be deeper problems still, considering that the > system Python installation was compiled with gcc, as were NumPy and > SciPy; I don't know whether gcc and icc play well together. > > Is the "right solution" to use icc to compile a completely separate > installation of Python, NumPy, SciPy, and Cython and then to develop > my project in that environment? If so, could anyone recommend tools to > build and maintain such an environment? (pythonbrew appears to be a > step in the right direction.) Or is there a simpler route? For > example, the binary produced with CC="icc" in my current test case > appears to run properly despite the various gcc confounds, but I don't > know whether to trust it. If you want this kind of flexibility I'd switch to a real build tool, such as 'waf'. Here's one example; it does more than you need since it also builds and links with Fortran code: https://github.com/dagss/healpix4py/blob/9a730266a15ca8986165cfdc9077a8537ed595e9/wscript https://github.com/dagss/healpix4py/blob/9a730266a15ca8986165cfdc9077a8537ed595e9/healpix/wscript The downside is you can't that easily distribute your package in standard ways, but "bento" is here to help with that: http://cournape.wordpress.com/2011/03/06/adding-a-distutils-compatibility-layer-to-bento/ (I didn't try this yet, as I don't need to easy_install my packages.) So, it's outside of the beaten path, but in the end it may be better than wasting your time with distutils. Dag Sverre