Item 1: Topic Discussion: Absorption Correction

SUMMARY: The effect of absorption by macromolecular crystals can only be implicitly taken into account by a purely empirical approach, the classic inter-image scaling. A significant improvement of this approach involves the use of spherical harmonics and is now programmed in some data reduction programs (eg. Scalepack and Scala), which usually leads to significantly improved intensity estimations, especially in the case of anomalous scattering. However, this approach needs to have enough redundancy of measurements and should be used with care.

Dr. Lothar Esser (NCI): In continuing the discussion about absorption correction ( of which we had an excellent introduction by Dr. Dauter ), the following cases where it
might help come to mind:

1) Heavy atom derivatives. Data processing of crystals that contain heavy atoms with Z >> 16 may be improved by applying an absorption correction. I know of one case where the resulting phases of a mercury derivative were substantially improved after an absorption correction was applied. Similar arguments apply if the native crystal already contains atoms heavier than sulfur ( i.e. Ca, Fe, Cu, Zn, Se, Mo etc.).

2) MAD data collection. Here the wavelength of the beam is deliberately fine-tuned to maximize absorption. Despite the enormous success of SeMet MAD experiments in that superb density maps were obtained, there is certainly a chance that a proper absorption correction even further improves the quality of the data.

However it seems that absorption correction overall is not critical for obtaining useful data ( with very few exceptions ). So the question should not be if it is time to perform absorption correction but who should do it and when. Which crystal is likely to benefit the most from it? Dr. Dauter already pointed out that conventional macromolecular data collection of highly redundant data, compensates for absorption phenomena to some extend and that this again is a sink for all sorts of
physical / machine stability related phenomena that lead to the fluctuation of the intensity of symmetry related reflections. What is the effect of not performing absorption correction of an absorbing non-spherical crystal ? Well, usually the positions of all atoms but mostly heavier atoms are effected by it. This leads to a distortion of bond length and angles. Macromolecular refinement with its heavy use of geometric restraints saves us here from disaster. Another effect of uncorrected data becomes visible in anisotropic refinement of heavy atoms: Instead of nice round (isotropic) probability ellipsoids, one might obtain oblong (US-football-shaped) ones.
Again no problem for restrained refinement as it is common practice in macromolecular crystallography.

So in the end, those people who benefit the most are those who are blessed with stupendously well diffracting crystals which then allow those fortunate people to refine atoms without restraints. But this does not mean that you should not give it a try even if you have only 2.5A data of a highly redundant (!) SeMet MAD data set and see if it improves your maps. However, as Dr. Dauter stressed, absorption correction in its current form needs to be applied with good judgment. An improved Rmerge does not necessarily mean that you improved your data quality [Is it time for an Rfree(merge) ? ].

One practical aspect: If you use Scalepack to do a semi-empirical absorption correction and you have multiple sets of data, make sure that they all have the same reference zone. Otherwise, the absorption correction is meaningless. If you have data collected at Kappa=0 and merge them with data at Kappa=45, don't try absorption correction, either. BTW Scalepack will not give you a warning, even if you do it wrong.

Dr. Zbigniew Dauter (NCI): The most proper way to correct for the effect of the crystal
absorption of X-rays requires the exact measurements of the crystal dimensions and indexing of its faces as well as the knowledge of the mass absorption coefficients of its contents. This leads to the analytical absorption correction, which is sometimes used in small molecule crystallography. The other, empirical method of North and Phillips is based on the so called phi-scan using the four-circle diffractometer, where the crystal is rotated around the diffraction vector of one of the reflections and the resulting intensity function applied to rescale other reflections.

Unfortunately, these methods are not applicable to crystals of macromolecules, which have shapes difficult to measure and are usually surrounded by some other absorbing material, either a capillary or a frozen solvent in the loop. The psi-scan is
impossible on the usual single-axis goniostat and a two-dimensional detector.

In fact, the classic inter-image scaling, based on a comparison of intensities of the symmetry equivalent reflections, to some extent mimics the effect of the psi-scan method, and implicitly takes into account part of the effect of absorption, as well as
some other effects (for example the varying volume of the crystal irradiated by the X-ray beam). This is a purely empirical approach involving only two parameters (the scale and B-factor) per diffraction image.

A significant improvement of this approach, which is now programmed in some data reduction programs (eg. Scalepack and Scala) is based on the representation of the relative scales of all measured reflections as a three-dimensional function expressed in terms of the linear combination of spherical harmonics of various orders. With this approach, the scales vary not only as a function of the oscillation angle, but can have more complicated shape in reciprocal space. This method usually leads to significantly improved intensity estimations, especially in case of anomalous scattering.

However, this approach should be used with care. The spherical harmonics can be simply treated as Lagrange coefficients. Therefore, with increased number of such coefficients, the resulting Rmerge will always be smaller, even if the procedure has no physical meaning. It means that it is pointless to use spherical harmonics up to the order of 10 if only 10 degrees of data have been collected. In principle to use this method effectively, the total rotation range should be large, preferably spanning the whole reciprocal sphere, i.e. it needs to have enough redundancy of measurements, which also depends on the crystal symmetry.

The use of spherical harmonics is a purely empirical approach, taking into account not only absorption, but also various other phenomena influencing intensities of diffracted reflections. These additional effects include the sub-optimal calibration of
the detector, bad (not synchronized) behavior of the X-ray shutter, the uneven motion of the goniostat motors and many others. Unfortunately, it is not easy to deduce this syndrome from the individual values of harmonic coefficients, and only Dr. Zbyszek Otwinowski knows how to do it properly...