XDLDATAMAN (CCP4: Supported Program)

NAME

xdldataman - X-windows tool; manipulation, analysis and reformatting of reflection files.

SYNOPSIS

xdldataman [-font1 | font2 | font3 | font4 | font5]
[menu-driven command selection; interactive parameter input]

DESCRIPTION

xdldataman can be used to read, write, analyse and manipulate ASCII reflection files from most biomacromolecular refinement packages packages. No binary reflection file types are supported (i.e.: MTZ files can *not* be read).

The program runs in interactive mode only. It uses the XDL_VIEW toolkit of J.W. Campbell to provide an easy-to-use interface. Commands are selected by clicking on the desired menu option; some menu options give a pop-up sub-menu with further options (indicated by "-->" in the menu name).

File formats are selected with pop-up menus; all other parameters are set in pop-up dialogue boxes (cut-and-paste is supported). In most cases, default values are given in [square brackets]. To accept these defaults, hit the RETURN key. If multiple numbers are to be input (e.g., cell constants), and if only the first one needs to be changed, for instance, typing the new value for this first number followed by the RETURN key will preserve the values for the other five numbers.

There is a command line option (-font?) which will determine the size of the menu font. These fonts refer to the xdl fonts which are defined from 1 to 5. This can be useful if the window size is too large for the screen. The default font size is 2. The font definitions can be changed in the .Xdefaults file or xrdb, however all xdl programs will be then be effected.

Output from the program is written to a separate area of the main window. Output can be scrolled and cut and pasted into other documents.

For lengthy operations a progress bar shows how much of the operation has been completed.

HISTORY

xdldataman is a CCP4 special version of DATAMAN (part of the Uppsala RAVE averaging package). This version is entirely interactive and has a user-friendly interface. However, this version can only handle one dataset at a time, and some of the functionality of the parent program is absent.

DATAMAN was originally written as a simple format-exchange program, to convert MTZDUMP files to X-PLOR reflection files. It has grown quite a bit since then to include other formats and to carry out several everyday manipulations on datasets. It also includes several programs that were previously stand-alone jiffies (such as the Gemini twin analysis command).

Some of the operations are implementations of Gerard Bricogne's algorithms as described in Volume B of the International Tables.

MENU OPTIONS

Commands are issued by moving the pointer over the desired menu option and clicking with the left mouse button.

READ A DATASET

Provide the filename and then select the appropriate file type from the pop-up menu. See FORMATS.

LIST DATASET INFORMATION

This prints some information regarding the dataset that is currently in memory.

STATISTICS OF THE DATASET

This lists the mean, standard deviation, minimum and maximum values of all properties of the dataset that are known.

EFFECTIVE RESOLUTION

This calculates an estimate of the effective resolution of the dataset, defined as "the resolution at which the actual number of reflections in the current dataset would constitute a 100% complete dataset" (B. Hazes, personal communication). This number is listed for all reflections, and for all reflections with F > n * sigma(F), with n=1,...,5. The lattice type and the number of asymmetric units need to be provided (they are used to estimate the volume of reciprocal space covered by the data).

RSYM HKL/KHL REFLECTIONS

This calculates Rsym (on Fs and on Is, assuming that I=F*F) for all reflection pairs HKL and KHL. If the spacegroup is, for instance, P3x or I4x and this Rsym is low, there may be a spacegroup error (e.g., the spacegroup is P41212 instead of P41).

CELL CONSTANTS

Enter the cell constants (needed to calculate the resolution of the reflections).

SYMMETRY OPERATORS

Provide the name of a symmetry-operator file in O format.

CALCULATE/DEDUCE/CONVERT

This command has several sub-options:
RESOLUTION
Calculate the resolution of all reflections.
CENTRICS/ACENTRICS
Assign centric/acentric flag to each reflection.
ORBITAL MULTIPLICITY
Calculate the orbital multiplicity of each reflection.
F -> I CONVERSION
Convert Fs to Is by using: I ~ F*F.
I -> F CONVERSION
Convert Is to Fs by using: F ~ sqrt(I) (I>=0).

TYPE SOME REFLECTIONS

Use this command to list some reflections. Provide the number of the first and last reflection and the step size (e.g., step size 10 will list every 10-th reflection). Providing a negative number for the last reflection is taken to mean the actual last reflection. Providing a value of zero for the step size will print only the first and the last reflection. Providing a negative step size means "show N reflections equally spread between the first and the last", with N being -1 * the step size.

SHOW SOME REFLECTIONS

This command has several sub-options:
FOBS >
Provide a number; all reflections with F greater than this number will be listed.
FOBS <
Provide a number; all reflections with F smaller than this number will be listed.
SIGMA >
Provide a number; all reflections with sigma(F) greater than this number will be listed.
SIGMA <
Provide a number; all reflections with sigma(F) smaller than this number will be listed.
F/SIGMA >
Provide a number; all reflections with F/sigma(F) greater than this number will be listed.
F/SIGMA <
Provide a number; all reflections with F/sigma(F) smaller than this number will be listed.
RESOLUTION >
Provide a number; all reflections with a resolution greater than this number will be listed (note: greater means "lower resolution" !).
RESOLUTION <
Provide a number; all reflections with a resolution smaller than this number will be listed (note: smaller means "higher resolution" !).

SPECIAL REFLECTIONS

List all reflections of a certain type. Provide a template of the type of reflections to be shown (containing the characters H, K, L and/or 0). For example: HHH, 0K0, KK0, etc.

ABSENCES LIST

List reflections that are systematically absent according to the current spacegroup symmetry operators. This can sometimes be used to make educated guesses concerning the nature of certain screw axes (e.g., in P4x, if only 00L, with L=4N, are strong reflections x is probably 1 or 3).

TWIN STATISTICS

Print some intensity statistics that may or may not be able to provide information with respect to possible twinning.

GEMINI TWIN ANALYSIS

This implements intensity analysis options as described by Stanley (1972) and Rees (1980), that may be of help in investigating possible merohedral twinning. The output consists of statistics and an estimate of the most likely twin fraction ("0.0" means no merohedral twinning). In addition, two PostScript files are produced showing 1N(z,alpha) as a function of z for centro- and non-centrosymmetric reflections. See the original papers for more information.

TEMPERATURE FACTOR APPLY

Apply a temperature factor to the Fs.

CHANGE INDEX

Re-index data. This may be necessary when the data-processing program yields a cell with beta < 90 in a monoclinic spacegroup, or when two datasets cannot be merged due to indexing along equivalent, but different axes (e.g., in P3x, P4x, etc.). Provide expressions for the new H, K and L.

PROD/PLUS

This command has several sub-options:
FOBS
Provide two numbers X and Y; all Fs will be replaced by X*F+Y.
SIGMA(FOBS)
Provide two numbers X and Y; all sigmas will be replaced by X*sigma+Y.
BOTH
Provide two numbers X and Y; all Fs and sigmas will be replaced by X*F+Y and X*sigma+Y, respectively.

LAUE GROUP APPLY

Move the reflections into the asymmetric unit appropriate for the Laue group of the dataset. This is sometimes necessary when the data-processing program outputs a non-standard asymmetric unit (for instance, R-AXIS processing software in P4x gives an asymmetric unit which is incompatible with the CCP4 standard). The Laue group is selected from a pop-up menu.

Implemented Laue groups and their asymmetric units are: 


  1bar,   hkl:h>=0  0kl:k>=0   00l:l>=0
  1bar,   hkl:k>=0  h0l:l>=0   h00:h>=0
  1bar,   hkl:l>=0  hk0:h>=0   0k0:k>=0
  2/m,    hkl:k>=0, l>=0     hk0:h>=0, k>=0
  2/m,    hkl:h>=0, l>=0     0kl:k>=0, l>=0 (2-nd sett)
  mmm,    hkl:h>=0, k>=0, l>=0
  4/m,    hkl:h>=0, k>0, l>=0 with  k>=0 for h=0
  4/mmm,  hkl:h>=0, h>=k>=0, l>=0
  3bar,   hkl:h>=0, k<0, l>=0 including 00l
  3bar,   hkl:h>=0, k>0  including  00l:l>0
  3barm,  hkl:h>=0, k>=0 with k<=h; if h=k l>=0
  6/m,    hkl:h>=0, k>0, l>=0  with  k>=0 for h=0
  6/mmm,  hkl:h>=0, h>=k>=0, l>=0
  m3,     hkl:h>=0, k>=0, l>=0 with l>=h, k>=h for l=h, k>h if l>h
  m3m,    hkl:k>=l>=h>=0

SORT REFLECTIONS

Sort the reflections by their indices H, K and L. The sort order is determined by the user.

KILL SOME REFLECTIONS

This command has the same sub-options as SHOW SOME REFLECTIONS:
FOBS >
Provide a number; all reflections with F greater than this number will be deleted.
FOBS <
Provide a number; all reflections with F smaller than this number will be deleted.
SIGMA >
Provide a number; all reflections with sigma(F) greater than this number will be deleted.
SIGMA <
Provide a number; all reflections with sigma(F) smaller than this number will be deleted.
F/SIGMA >
Provide a number; all reflections with F/sigma(F) greater than this number will be deleted.
F/SIGMA <
Provide a number; all reflections with F/sigma(F) smaller than this number will be deleted.
RESOLUTION >
Provide a number; all reflections with a resolution greater than this number will be deleted (note: greater means "lower resolution" !).
RESOLUTION <
Provide a number; all reflections with a resolution smaller than this number will be deleted (note: smaller means "higher resolution" !).

ERASE OPTIONS

This command has the several sub-options:
ROGUES
Delete "rogue" reflections simply by providing their HKL indices. This can be used to remove individual reflections which are suspect for some reason or other.
ODD H/K/L
Delete all reflections for which either H, K or L is odd.
EVEN H/K/L
Delete all reflections for which either H, K or L is even.

RFREE OPTIONS

This command has the several sub-options:
INITIALISE
Set the seed for the random-number generator. Providing a negative seed will use the current machine clock value as the seed; a positive number will be used itself as the seed.
LIST CURRENT STATUS
This lists the current partitioning of the dataset in WORK and TEST reflections. If there are very few or very many TEST reflections, a warning message will be printed. In general, 10% of the data with a minimum of ~500 and a maximum of ~2000 TEST reflections is considered to be reasonable. Note that the error in Rfree is roughly equal to 1/SQRT(number of TEST reflections), so that for 500 TEST reflections the error is ~4.5% and for 2000 TEST reflections ~2.2%.
RESET ALL RFREE FLAGS
This sets all Rfree flags to zero, i.e. all reflections are flagged as WORK reflections. See RFREE FLAGS.
GENERATE RANDOM RFREE FLAGS
Provide either the *percentage* or (roughly) the *number* of TEST reflections. Randomly chosen reflections will be assigned as TEST reflections (the same way X-PLOR does this). This is actually the worst possible way to select TEST reflections, since (through the G-function) every reflection will be related to its neighbours (in reciprocal space) and, in the case of NCS, to its "NCS mates" and their neighbours.
SHELLS OF RFREE REFLECTIONS
Provide the *percentage* or (roughly) the *number* of TEST reflections and the number of resolution bins. The data will be sorted by resolution and divided into bins. From every bins the appropriate fraction of reflections from its centre will be flagged as being TEST reflections. This is to counter couplings in the case of NCS.
SPHERES OF RFREE REFLECTIONS
Provide the *percentage* or (roughly) the *number* of TEST reflections and the radius of reciprocal-space spheres. Reflections are picked at random, and they and their neighbours inside a small sphere (in reciprocal space) are all assigned as TEST reflections. This is to counter couplings due to bulk solvent in the absence of NCS.
GSHELDRICKS METHOD
This simply assigns every N-th reflection to be a TEST reflection, where the value of N is provided by the user (in SHELX, N=10).
COMPLETE CROSS-VALIDATION SETS
The number N of datasets to be generated is provided. Every reflection will be assigned to be a TEST reflection in exactly one of the N datasets. The N datasets are written in X-PLOR format (but can, of course, be converted into other formats with this program).

WRITE DATASET TO FILE

Provide the filename and select the desired file type from the pop-up menu.

DELETE CURRENT DATASET

Remove the current dataset from memory.

HELP

This prints some brief information. Subsequently, click on *any* menu command to get a brief explanation of what that command does.

QUIT

Stop working with the program.

FORMATS

Supported input formats :
* (free format)
MTZDUMP (user or free format)
XPLOR (no format required)
SHELXS (fixed format)
TNT (free format)
PROTEIN (user or free format)
MKLCF (user or free format)
HKLFS (user or free format)
RFREE (user or free format)
ELEANOR (user or free format)

Supported output formats :

* (fixed format)
RXPLOR (no format required)
SHELXS (fixed format)
TNT (fixed format)
PROTEIN (user or fixed format)
CIF (user or fixed format)
MKLCF (user or fixed format)
HKLFS (user or fixed format)
RFREE (user or fixed format)
ELEANOR (user or fixed format)
XPLOR (no format required)

Notes on formats :

*/HKLFS - reads/writes HKL F Sigma
RXPLOR - X-PLOR format with Rfree flags
SHELXS - fixed format, no Rfree flags
TNT - no FOMs, phases or Rfree flags
PROTEIN - no Sigmas
MKLCF - integer F and Sigma
RFREE - HKL F Sigma and integer Rfree flags
ELEANOR - ditto, but real (1.0-Rfree) flags
MTZDUMP - reads unedited MTZDUMP log file
CIF - output only; Rfree flags UNofficial
Use the Calculate option to convert I<->F if needed.

RFREE FLAGS

xdldataman uses the X-PLOR convention, i.e. the Rfree flag is an integer number (0 or 1), and a value of "1" means that the reflection belongs to the TEST set which is *not* used in refinement. CCP4 has a different convention: reflections are divided into a number of equal-sized sets, one of which (usually flagged "0") represents the TEST set, see program FREERFLAG. The CCP4 convention is supported (and converted) through the "ELEANOR" format.

REFERENCES

  1. XDLDATAMAN:
    G.J. Kleywegt & T.A. Jones (1996), Acta Cryst. D52, 826-828.
  2. XDL_VIEW:
    J.W. Campbell (1995). "XDL_VIEW, an X-windows-based toolkit for crystallographic and other applications", J. Appl. Cryst. 28, 236-242.
  3. RAVE:
    G.J. Kleywegt & T.A. Jones (1994). "Halloween ... Masks and Bones", in "From First Map to Final Model" (S. Bailey, R. Hubbard & D. Waller, Eds.), SERC Daresbury Laboratory, pp. 59-66.
  4. O:
    T.A. Jones, J.Y. Zou, S.W. Cowan, & M. Kjeldgaard (1991). "Improved methods for building protein models in electron density maps and the location of errors in these models", Acta Cryst. A47, 110-119.
  5. GEMINI:
    E. Stanley (1972). "The identification of twins from intensity statistics", J. Appl. Cryst. 5, 191-194.
  6. GEMINI:
    D.C. Rees (1980). "The influence of twinning by merohedry on intensity statistics", Acta Cryst. A36, 578-581.
  7. RFREE:
    A.T. Brunger (1992). "Free R value: a novel statistical quantity for assessing the accuracy of crystal structures", Nature 355, 472-475.
  8. CCP4:
    Collaborative Computational Project Number 4 (1994). "The CCP4 suite: programs for protein crystallography", Acta Cryst. D50, 760-763.

KNOWN BUGS

None (at the time of writing).
If you improve the program, please notify GJK of your changes so that they can be implemented in future versions and the entire community may benefit from them (E-mail a brief description and the relevant pieces of code to "gerard@xray.bmc.uu.se").

AUTHORS

Originators: G.J. Kleywegt & T.A. Jones, Uppsala

SEE ALSO

xdlmapman, mtzdump