 |
CCP4i: Graphical User Interface |
| Reflection Data Utilities Module |
|
- Convert to MTZ & Standardise (Import)
- Import - Task Window Layout
- Import - The Import Task
- Import - The Import File Format
- Import - NA4toMTZ
- Import - CIF2MTZ
- Import - The MTZ Column Types
- The Reflection Data Utilities Module contains tasks to manipulate reflection data files:
- Calculate Fs & Phases
- Convert to MTZ & Standardise
- Convert from MTZ
- Edit MTZ File (Sftools)
- Merge MTZ Files (Cad)
- Edit MTZ Project&Dataset
- Reindex Reflections
- Phase Analysis (Phistats)
- SF File Analysis
- SIGMAA
- Specialist Help is available on:
- FreerUnique - how to create a complete unique list of reflections with your data
The layout of each task window, i.e. the number of folders present, and whether these folders are open or closed by default, depends on the choices made in the Protocol folder of the task (see Introduction). Although certain folders are closed by default, there are specific reasons why you should or may want to look at them. These reasons are described in the Task Window Layout sections below.
Calculate Fs & Phases
This is an interface to the SFALL program which generates Fcalc and PHIcalc from either input coordinates or a map. You must give the name of an input coordinate or map file and also an MTZ file into which the calculated Fs and PHIs are added. You will need to identify the FP and SIGFP in the MTZ file. If a FreeR column is also identified, the FreeR set of reflections will be excluded from the calculation. By default the calculated Fs and PHIs will cover the same resolution range as the one already in the MTZ file.
SFALL - Task Window Layout
The Task Window for the SFALL Task is relatively simple. The Protocol folder has the choice between generating structure factors from coordinates or a map (after which the Files folder is adapted accordingly), and the option to scale the input Fp to the Fcalc.
After the Protocol and Files folders, the 'Cell Parameters' folder has the option of adapting resolution range, space group and/or cell parameters. The then following 'Program Parameters' folder allows the adaptation of Maximum atom radius used to build atom/residue mask and the amount to add to atomic Bfactors.
See program documentation: SFALL.
Convert to MTZ & Standardise
Standard CCP4 programs deal with reflection files in so-called MTZ format. Files in different formats, namely ascii MTZ ('na4'), X-PLOR/CNS, SHELX, MULTAN, TNT, mmCIF and user defined ascii, may be converted to MTZ format. Additionally, MTZ files may be extended to create a full unique set of reflections with various options to deal with freeRflags. The default setting is to create this set. At the start of a structure determination, it is advisable to always create this full unique set of reflections, which contains (h, k, l, F, SIGF, (I, SIGI), ..., FreeR_flag) for all observed reflections to the resolution limit available, plus entries for any unobserved reflection, all with freeRflags assigned. This never does any harm, and it is much harder to do satisfactorily at a later stage. For more information about a full unique set with freeRs, see also FreerUnique.
Import - Data Harvesting
Import - Data Harvesting - TRUNCATE
TRUNCATE, part of the Import Task when SHELX or MULTAN data is imported, is needed to convert intensities to structure factors. This is a genuine call for data harvesting, which is why the first folder beneath the Protocol and Files folders is concerned with 'Data Harvesting'. The user has the choice whether to allow data harvesting or not, but it is, of course, recommended. Input sensible (and/or appropriate, if the have already been assigned) project and dataset names.
See Data Harvesting in CCP4i for implications for the Interface.
Import - Data Harvesting - F2MTZ
F2MTZ is one of the data preparation programs that can handle data harvesting keywords PNAME and DNAME. See Data Harvesting in CCP4i for implications for the Interface.
To make things easier in the long run, it would be wise to think this through at this stage, and choose sensible project and dataset names. Input these in the 'MTZ Project & Dataset Names' folder directly beneath the Files folder.
Import - Task Window Layout
Features to look out for in the Convert to MTZ & Standardise Task are:
| Protocol option | Folder title | Importance | Comment |
|---|---|---|---|
| all | Creating full/unique dataset | freeR | see FreerUnique |
| systematic absences | chance to include them in MTZ file | ||
| extend resolution | use artificially high resolution if more data is anticipated | ||
| all (not MTZ, ascii MTZ, mmCIF) | Detailed specification of import file format | Fortran format, data type and labels | Essential for the successful running of the Task with user defined option. See below |
| SHELX and MULTAN | Convert Is to Fs | TRUNCATE protocol to be followed | Conversion to structure factors either by truncation according to French&Wilson, or by simply taking the square root of the intensities. See TRUNCATE documentation. |
| Data Harvesting and Convert Is to Fs | role of TRUNCATE in Data Harvesting | see above | |
| all (not MTZ, ascii MTZ) | Extra information to be saved in MTZ file | symmetry, cell | Add in symmetry and cell information if needed |
Import - The Import Task
The full "import" (i.e. Convert to MTZ & Standardise) task contains the following steps:
- F2MTZ - to convert any ASCII style reflection file to MTZ format. If no full unique set of reflections with freeRflags is required, this will be the only step (but this is not the default). For "ascii MTZ" and mmCIF, the specialised programs NA4TOMTZ and CIF2MTZ are used instead.
- unique - to create a full unique list of reflections (indices) with dummy
F and SIGF, to the required resolution. This file is then used as a template
list in the following steps.
Then, depending on whether there were freeRflags already in your data or not, i.e. whether you chose to- keep existing FreeR data
- import FreeR data from another MTZ file
- generate FreeR data
- If there were freeRflags already in your data set:
- CAD - to combine your data (from step 1, i.e. option a, or from an existing MTZ file,
i.e. option b) with
the unique template list (from step 2), sort it to standard order, and place it
in the CCP4 asymmetric unit. Reflections not featuring in your data set will
acquire a "missing number flag" ("NaN" by default), which
can be replaced with real data as and when it becomes available.
N.B. CAD can not deal with more than 29 columns. - freerflag - in COMPLETE mode, to add a random freeRflag to all reflections which do not already have one.
- mtzutils - to remove superfluous columns (created by step 2) from the file.
- CAD - to combine your data (from step 1, i.e. option a, or from an existing MTZ file,
i.e. option b) with
the unique template list (from step 2), sort it to standard order, and place it
in the CCP4 asymmetric unit. Reflections not featuring in your data set will
acquire a "missing number flag" ("NaN" by default), which
can be replaced with real data as and when it becomes available.
- If freeRflags are to be assigned from scratch for all reflections (i.e. option c):
- freerflag - to add a random freeRflag to all reflections in the file created in step 2, according to the chosen fraction.
- CAD - as in step 3, using all columns from your data, but using only the freeRflag column from the unique file from step 2, so that the superfluous columns are automatically removed at this stage.
- freerflag - as in step 3, to ensure freeRflags are assigned to every reflection.
Import - The Import File Format
Pay special attention to the folder Detailed specification of import file format:
- The program F2MTZ requires all values to be described
as real numbers, i.e. Fortran format type F. The Interface
has the default Fortran format for all types of reflection
files, but this may have to be changed if a non-standard reflection file is
used or if more or fewer columns are to be included. This is an example of
a Fortran format as it should be entered:
(5X,3F5.0,6X,2F10.0,7X,F10.0,/,10X,2F10.0,8X,F10.0,6X,F10.0)
This complicated statement describes the following reflection file format:
INDE 6 0 0 FOBS= 1259.884 0.000 SIGMA= 38.561 FPART= 0.000 0.000 WEIGHT= 1.000 TEST= 0- For more information on Fortran formats, see a Fortran Manual.
- TNT input is split into two files (working and test sets separately). The Interface will automatically merge those files and insert freeRflags but can only do so if you add F8.0 to the format statement.
- If there are header lines
in the reflection file not containing actual data, these lines should be
skipped. The program will not run if the SKIP value is incorrect.
- If the input is from TNT, the merging of the files can only happen successfully if the headers of the two input files contain the same number of lines, i.e. the SKIP value is the same for both files.
- The output data type and label must be given correctly for every column
of data in the reflection file.
- For X-PLOR input, the Interface is assuming freeRflags are already assigned, and will initiate the necessary swap of the values (see FreerUnique) by denoting the column type X for the freeRflag column.
- If the input is from SHELX, the Intensities have to be converted to Fs. The column labels for the program Truncate, which is to perform the conversion, need to match those of the input file exactly.
Import - NA4toMTZ
The CCP4 Suite of programs is released with a set of example files. MTZ files are distributed in 'na4' format, which is a type of ASCII. In order to convert these to the MTZ format needed to run the programs, the program NA4toMTZ is provided. In the Interface, this is included within the 'F2MTZ' part of the 'Convert to MTZ & Standardise' task.
When browsing for an 'na4' file, do not forget to change the file extension to '.na4'.
Import - CIF2MTZ
Structure factor files obtained from the PDB will be in mmCIF format. This option can be used to convert these files to MTZ format. With a well-formed mmCIF file, you should not need to change the defaults or add any information.
Import - The MTZ Column Types
- H
- index h,k,l
- F
- structure amplitude, F
- D
- anomalous difference
- Q
- standard deviation of anything: J,F,D or other
- P
- phase angle in degrees
- W
- weight (of some sort)
- A
- phase probability coefficients (Hendrickson/Lattmann)
- B
- BATCH number
These data types are not usually present, but are legal:
- S
- 1/d**2 = 4 (sin theta/lambda)**2
- J
- intensity
- Y
- M/ISYM, packed partial/reject flag and symmetry number
- I
- any other integer
- R
- any other real
- U
- partial FC (see CAD)
- V
- partial PHIC (see CAD)
In addition, the program F2MTZ accepts the column type X for freeRflags originating from X-PLOR/CNS, and will reset the column type to I in the resulting MTZ file (see FreerUnique).
N.B. The column types of structure amplitudes and phases may not be any other than F and P, respectively. It is also essential to have correct column types for phases and anomalous differences, to distinguish:
- phases which will require changing if the reflection is moved to a symmetry equivalent;
- anomalous differences which require changing sign if the reflection is changed to a Friedel partner.
See program documentation: F2MTZ, NA4toMTZ, CIF2MTZ, UNIQUE, FREERFLAG, CAD, MTZUTILS, MTZLIB
Convert from MTZ
MTZ files may be converted to several other formats, i.e. X-PLOR/CNS, MULTAN/RANTAN, SHELX, TNT, MAIN, SCALEPACK, USER and mmCIF. To output an mmCIF file for deposition, use the task interface "Structure Factors for Deposition" in the Validation & Deposition module instead.
Export - Task Window Layout
Features to look out for in the Convert from MTZ Task are:
| Protocol option | Folder title | Importance | Comment |
|---|---|---|---|
| CIF | CIF Format Details | one data block per dataset | See Notes on CIF in the MTZ2VARIOUS Program Documentation |
| CIF, SHELX and XPLOR/CNS | MTZ File Labels | anomalous data | Friedel pairs are output as separate reflections if the anomalous difference is assigned. See also OUTPUT anomalous data in MTZ2VARIOUS Program Documentation |
| SCALEPACK | MTZ File Labels | I(+) and I(-) | pseudo-SCALEPACK output useable as input to the SOLVE package |
| USER | User Defined Output Format | See OUTPUT options and Notes on INPUT and OUTPUT in the MTZ2VARIOUS Program Documentation | |
| all | Infrequently Used Options | output modifications | get output in desired form |
See program documentation: MTZ2VARIOUS, MTZDUMP,MTZDMP, MTZTONA4,NA4TOMTZ, MTZUTILS
For more information on mmCIF, see The mmCIF Home Page at the IUCr.
Edit MTZ File (SFTOOLS)
Analyse, manipulate and convert structure factor files.
SFTOOLS - Task Window Layout
Features to look out for in the Edit MTZ File Task are:
| Protocol option | Folder title | Importance | Comment |
|---|---|---|---|
| set or change header information | default option | ||
| Change column labels and types | Change column label to | chance to make column label names that mean something to you | |
| reduce to asymmetric unit | choose CCP4, BIOMOL, TNT or user-defined | ||
| expand to lower symmetry | specify spacegroup to expand into, and choose asymmetric unit as before. Can also be done with CAD | ||
| sort reflections | sorting criteria | sorting on index can also be done with CAD, see below | |
| reindex reflections | transformation matrix | complicated transformations with translations are better dealt with by redoing the data reduction | |
| merge reflections | use only when you know why you have multiple reflections with identical indices | ||
| delete reflections | |||
| delete columns | |||
| generate Hendrickson-Lattman coeffs | |||
See program documentation: SFTOOLS, MTZUTILS, REINDEX, CAD
Merge MTZ Files (CAD)
This task interfaces to the CAD program which can be used to:
- Delete columns from an MTZ file
- Merge data from two or more MTZ files
- Reset the resolution range in the MTZ file(s)
- Change the sort order or HKL limits or space group
To input more than one MTZ file, click on the Add input MTZ file button. By default all the data in the input MTZ file is put into the output file but you can change the Input option from 'all columns' to 'selected columns' and then select the columns using the Add column button. If you want to have the majority of the columns in the file, then click on the List All Columns button and then delete the columns you do not require using the Delete selected item option under the Edit list menu button. You will then need to select the column by clicking on one of the fields for that column with the right mouse button. See also Extending Frames and Toggle Frames.
CAD can not deal with more than 29 columns.
Do not include columns H, K and L in input. These are transferred to output automatically, and only upset the program.
Two special data types are used to signal that you are preparing data for translation functions of various types. They are:
- U
- partial FC
- V
- partial PHIC
There must be only one FCpart PHICpart per input file, and they must be the last items specified for LABIN. CAD generates equivalent reflections using only the ROTATIONAL part of the primitive symmetry operator (i.e. if the spacegroup is P212121, these reflections are analysed as though the spacegroup was P222). This is allowed for in the TFFC and RSEARCH programs.
CAD - Task Window Layout
Features to look out for in the CAD Task are:
| Folder title | Importance | Comment |
|---|---|---|
| Define MTZ Output | override space group, cell dimensions, sort order, hkl limits etc. | can also be done with SFTOOLS, see above |
See program documentation: CAD, SFTOOLS
Edit MTZ Project&Dataset
The columns of an MTZ file are divided between a number of datasets, which are each identified by a Project name (PNAME) and a Dataset name (DNAME). See the description of Data Harvesting for background information. The program CAD can be used to change the attributes of a dataset, and to change the assignment of columns to datasets, and this functionality is accessible through the Edit MTZ Project & Dataset Names interface.
The top half of the interface lists the current datasets and their attributes. Any attribute can be edited, and datasets may be added or deleted. The lower half provides a pull-down menu for each column allowing the user to select a dataset from the current list. Note that the input to CAD is taken from the list of columns, so that surplus datasets are ignored, and datasets deleted from the dataset list but retained in the column list are kept.
In principle, dataset information should be generated and propagated automatically, so this interface should only be needed to fix problems.
See Data Harvesting in CCP4i for implications for the Interface.
See program documentation: CAD
Reindex Reflections - Reindex
This is the option to use for reindexing 'standard' MTZ files, i.e. non-multi-record MTZ.
See program documentation: REINDEX
Phase Analysis (Phistats)
This task will take an MTZ file with two phase sets and analyse the differences between them.
See program documentation: PHISTATS
SF File Analysis
This is an interface to the data analysis tools of SFTOOLS which will perform several forms of analysis of Structure Factor data. The user should select an MTZ file and the data columns to analyse. The results are displayed automatically by The Loggraph Utility. The results are also saved in a .log file which can subsequently be viewed with the View Log Graphs option in the View Files from Job menu on the right hand side of the Main Window of CCP4i.
See program documentation: SFTOOLS
SIGMAA
See program documentation: SIGMAA
|
|