PyFAI is a library to deal with diffraction images for data reduction. This chapter describes the project from the computer engineering point of view.
PyFAI is an open source project licensed under the GPL mainly written in Python (v2.6, 2.7) and heavily relying on the python scientific ecosystem: numpy, scipy and matplotlib. It provides high performances image treatment thanks to cython and OpenCL... but only a C-compiler is needed to build that.
PyFAI is a Python project but uses many programming languages:
The OpenCL code has been tested using:
The project is hosted by GitHub: https://github.com/pyFAI/pyFAI
Which provides the issue tracker in addition to Git hosting. Collaboration is done via Pull-Requests in github’s web interface:
Everybody is welcome to fork the project and adapt it to his own needs: CEA-Saclay, Synchrotrons Soleil, Desy and APS have already done so. Collaboration is encouraged and new developments can be submitted and merged into the main branch via pull-requests.
A mailing list: pyfai@esrf.fr is publicly available. To subscribe, send an email to sympa@esrf.fr with “subscribe pyfai” as subject. On this mailing list, you will have information about release of the software, new features available and meet experts to help you solve issues related to azimuthal integration. This mailing list is archived and can be consulted at: http://www.edna-site.org/lurker
In addition to the run dependencies, pyFAI needs a C compiler. There is an issue with MacOSX (v10.8 onwards) where the default compiler (Xcode5 or 6) switched from gcc 4.2 to clang which dropped the support for OpenMP (clang v3.5 supports OpenMP under linux but not directly under MacOSX). Multiple solution exist, pick any of those:
C files are generated from cython source and distributed. Cython is only needed for developing new binary modules. If you want to generate your own C files, make sure your local Cython version supports memory-views (available from Cython v0.17 and newer), unless your Cython files will not be compiled or used.
python setup.py build install
There are few specific options to setup.py:
Setting the environment variable http_proxy can be necessary (depending on your network):
export http_proxy=http://proxy.site.org:3128
Especially at ESRF, the configuration of the network proxy can be obtained by phoning on the hotline: 24-24.
PyFAI comes with 30 test-suites (172 tests in total) representing a coverage of 67%. This ensures both non regression over time and ease the distribution under different platforms: pyFAI runs under Linux, MacOSX and Windows (in each case in 32 and 64 bits). Test may not pass on computer featuring less than 2GB of memory or 32 bit computers.
Name | Stmts | Miss | Cover |
---|---|---|---|
pyFAI/__init__ | 14 | 5 | 64% |
pyFAI/_version | 31 | 1 | 97% |
pyFAI/azimuthalIntegrator | 1209 | 311 | 74% |
pyFAI/blob_detection | 520 | 323 | 38% |
pyFAI/calibrant | 360 | 112 | 69% |
pyFAI/detectors | 1147 | 287 | 75% |
pyFAI/directories | 30 | 8 | 73% |
pyFAI/geometry | 815 | 208 | 74% |
pyFAI/geometryRefinement | 477 | 304 | 36% |
pyFAI/gui_utils | 66 | 41 | 38% |
pyFAI/io | 457 | 214 | 53% |
pyFAI/massif | 189 | 60 | 68% |
pyFAI/multi_geometry | 66 | 4 | 94% |
pyFAI/ocl_azim | 269 | 78 | 71% |
pyFAI/ocl_azim_csr | 225 | 46 | 80% |
pyFAI/ocl_azim_lut | 219 | 45 | 79% |
pyFAI/opencl | 191 | 52 | 73% |
pyFAI/peak_picker | 732 | 537 | 27% |
pyFAI/spline | 397 | 249 | 37% |
pyFAI/third_party/__init__ | 0 | 0 | 100% |
pyFAI/third_party/six | 393 | 184 | 53% |
pyFAI/units | 42 | 6 | 86% |
pyFAI/utils | 744 | 315 | 58% |
test_all | 81 | 1 | 99% |
test_azimuthal_integrator | 241 | 67 | 72% |
test_bilinear | 82 | 8 | 90% |
test_bispev | 66 | 16 | 76% |
test_blob_detection | 54 | 5 | 91% |
test_bug_regression | 41 | 5 | 88% |
test_calibrant | 126 | 35 | 72% |
test_convolution | 54 | 6 | 89% |
test_csr | 88 | 23 | 74% |
test_detector | 171 | 13 | 92% |
test_distortion | 56 | 8 | 86% |
test_dummy | 27 | 6 | 78% |
test_export | 87 | 9 | 90% |
test_flat | 112 | 9 | 92% |
test_geometry | 91 | 6 | 93% |
test_geometry_refinement | 64 | 7 | 89% |
test_histogram | 234 | 19 | 92% |
test_integrate | 139 | 12 | 91% |
test_io | 108 | 30 | 72% |
test_marchingsquares | 42 | 9 | 79% |
test_mask | 218 | 55 | 75% |
test_multi_geometry | 79 | 20 | 75% |
test_openCL | 225 | 27 | 88% |
test_peak_picking | 89 | 11 | 88% |
test_polarization | 57 | 6 | 89% |
test_saxs | 105 | 31 | 70% |
test_sparse | 44 | 5 | 89% |
test_split_pixel | 74 | 6 | 92% |
test_utils | 97 | 6 | 94% |
test_watershed | 40 | 6 | 85% |
TOTAL | 11585 | 3857 | 67% |
Note that the test coverage tool does not count lines of Cython, nor those of OpenCL Updated 27/05/2014
Continuous integration is made by a home-made scripts which checks out the latest release and builds and runs the test every night. Nightly builds are available for debian6-64 bits in: http://www.edna-site.org/pub/debian/binary/
$ git log --pretty='%aN##%s' | grep -v 'Merge pull' | grep -Po '^[^#]+' | sort | uniq -c | sort -rn