MTZUTILS (CCP4: Supported Program)
NAMEmtzutils - Reflection data files utility program
SYNOPSISmtzutils hklin foo_in_i.mtz [ hklin2 foo_in_j.mtz ] hklout foo.mtz
DESCRIPTIONThe MTZ utility program MTZUTILS is provided for the purpose of creating a new re-arranged or edited MTZ reflection data file from one or two existing files. Consider also CAD, which has similar functions and should be used if you are not sure your input files are in the same region of reciprocal space.
KEYWORDED INPUTThe program reads one or two input reflection data files and creates a single output reflection data file. The basic functions are selected using the file control option specification keywords. The available keywords and options are:
RUN | GO | ENDTerminate the keyworded input and start processing. [Optional, but advisable.]
SYMMETRYSpace group name or number. Replace Symmetry information in the output MTZ file.
SORT_ORDER h,k,lInput a combination of the strings H, K, and L. This is stored but not currently used.
CELL <a> <b> <c> [ <alpha> <beta> <gamma> ]Replace cell information in the output MTZ file (alpha, beta, gamma default to 90.00).
HISTORY <string>Add to the history stack.
The string is added to existing history, in order.
up until MAXHIS reached
TITLE <title>Edit MTZ Titles. Examples:
TITLE 1 NOCHANGE TITLE 2 NOCHANGE TITLE NOCHANGE # ==> from File_Number_1 TITLE REPLACE string # ==> from File_Number_1 TITLE 1 REPLACE string # ==> from File_Number_1 TITLE 2 REPLACE string # ==> from File_Number_2 TITLE ADD string # ==> from File_Number_1 TITLE 1 ADD string # ==> from File_Number_1 TITLE 2 ADD string # ==> from File_Number_2
COLUMN_LABELS [ <file> ] <program label>=<file label>...Edit column label names. Examples:
COLUMN_LABELS Tom=Huey Dick=Dewey Harry=Luey # ==> from File_Number_1 COLUMN_LABELS 1 Tom=Huey Dick=Dewey Harry=Luey # ==> from File_Number_1 COLUMN_LABELS 2 Tom=Huey Dick=Dewey Harry=Luey # ==> from File_Number_2For the MERGE option NO column editing is allowed.
For Column Editing FIRST for assignments with a set of files with column labels:
File_1 H K L A B File_2 H K L A C using keywords as: COLU 2 A=D INCLUDE 1 A B INCLUDE 2 D C
HEADER BRIEF | HIST | ALLControls printing of MTZ information. STRING is one of following:
RESOLUTION <limit1> <limit2>Followed by minimum and maxmimum resolution for the output file.
resmin resmax (any order) OR smin smax (any order) resolution limits for output file.
AXIS <zone>Output reflection file restricted to given zone(s)
Use one or more of: H00, 0K0, 00L, HH0, -HH0, HHH, HK0, 0KL, H0L, HHL
RZONE <rzone>To select a zone then RZONE must be followed by 5 integers e.g.
+h +k +l = 3n ==> RZONE 1 1 1 3 0 and l = 2n + 1 ==> RZONE 0 0 1 2 1
SCALE ...Scale column labels by multiplying them with an input scale factor.
Input may be in one of the following styles:
ONEFILEThis is compulsory if only one file is being used with the EXCLUDE/INCLUDE options and you have assigned HKLIN2. If HKLIN2 isn't assigned, ONEFILE will be assumed.
FILE CONTROL OPTION SPECIFICATIONThese options enable the user to select a particular task. Available options are INCLUDE, EXCLUDE, UNIQUE, CONCAT and MERGE. Options INCLUDE and EXCLUDE are specific to a single input file, while options UNIQUE, CONCAT and MERGE apply to both input files. Option ONEFILE is required if there is just a single input file. As only two input files are allowed, the file specification options INCLUDE and EXCLUDE cannot be used with options UNIQUE, MERGE or CONCAT.
If there is only one input file then the ONEFILE option is needed after the keywords for the first input file and succeeding keywords (except the cell dimensions, if required) are omitted.
When operating on two files, the data items that are not defined in a record, will be set to the missing number flag. For instance when using the MERGE option, the unique columns in file 2 will not be defined when writing out a record from file 1. The converse is also true. Note that multi-record files should not contain missing reflections, thus the output file could not be used as a normal multi-record file.
(a) File control option INCLUDEThis option applies to one of the input files. The file control option keyword containing the code INCLUDE containing a list of column labels of data items to be copied to the output file. Column labels for h, k and l should not be given among these labels. If the input file has not got any of the labels requested for inclusion, the job will be aborted. Title editing and column label editing instructions are allowed.
(b) File control option EXCLUDEThis option applies to one of the input files. The file control option keyword containing the code EXCLUDE containing a list of column labels of data items to be excluded when creating the output file. Column labels corresponding to h, k and l should not appear among these label strings as they are taken care of automatically. The program is aborted if any requested label string is not found among edited column labels of the file. Title editing and column label editing keywords are allowed.
EXCLUDE Tom Dick Harry # ==> from File_Number_1 EXCLUDE 1 Tom Dick Harry # ==> from File_Number_1 EXCLUDE 2 Tom Dick Harry # ==> from File_Number_2 INCLUDE Tom Dick Harry # ==> from File_Number_1 INCLUDE 1 Tom Dick Harry # ==> from File_Number_1 INCLUDE 2 Tom Dick Harry # ==> from File_Number_2 INCLUDE # ==> ALL columns from File_Number_1 INCLUDE ALL # ==> ALL columns from File_Number_1 INCLUDE 1 ALL # ==> ALL columns from File_Number_1 INCLUDE 2 ALL # ==> ALL columns from File_Number_2
(c) File control option ONEFILEIf the option INCLUDE or EXCLUDE is used when only one input file is required, the file control option ONEFILE should be used to indicate to the program that only one input file exists.
(d) File control option UNIQUEThis file control option specifies that each column of the two input files with a unique label is to be copied to the output file and that whenever a particular reflection appears in both the input files, the data should be merged into a single record of the output file. Note that unique columns are recognised from the edited labels of the input files. If a column label is found in both the input files then the data value from the first file is copied to the output record unless it is the distinguished missing value, in which case the value from the second file (HKLIN2) will be copied to the output.
Both the input files should have identical labels for h, k and l otherwise the output file will contain three extra columns containing the values of h, k and l derived from the second input file. The output file from this option is of the single record/reflection type (a normal MTZ file).
WARNING: if the input data sets are not properly sorted on h, k and l (the first three columns), the output file may become of mixed type with both types of record present.
(e) File control option CONCATThis file control option specifies that the data records of the two input files are to be copied to the output file. The option is used to create a multiple record type output file from the two input files by merging them. The output file contains edited titles from both the input files and edited labels from the first input file. The labels of the first input file should be edited to become identical with the existing labels in the second input file failing which the program will abort.
In this option the reflections are sorted. For sorting purposes the missing number flag, for both files, is changed to a large negative value. This is similar to SORTMTZ. The missing number flag is then reset to that of the first file or the default (NaN).
(f) File control option MERGEThis option creates a multi-record type merged MTZ file from two input MTZ files. The columns in the two input files need not be identical. The column labels in the output file will be the common labels from the two files, the unique labels from file 1 and the unique labels from file 2.
Title Editing OptionThe title of the output file is derived from the titles of the input files after appropriate editing based on the title editing options. At present, the available options are REPLACE, NOCHANGE and ADD. For each input file there may be one keyword of this type. The option REPLACE indicates that the existing title of the input file should be replaced by a new title before copying to the output file. The new title information is given on the same line. The option NOCHANGE (default) indicates that the title information of the input file should be copied as it is the output file. The option ADD indicates that the output file should have all the title information supplied on the rest of the line, along with that of the input file.
Column Label Editing OptionsThese keywords allow the re-naming of column labels of an input file before copying to the output file. For the options UNIQUE and CONCAT, these apply to the first input file. The keywords contain entries of the type
label1=label2where label1 is an existing column label and label2 is the replacing label for the particular column. There should be at least one blank between two such assignment statements. The statements may be spread over COLUMN keywords if required and are terminated by. If label1 is not found for a particular assignment then label2 is tried and if still a failure the program will continue after giving an error message.
INPUT AND OUTPUT FILESThe input files are:
The output file is a reflection data file which is normally in standard MTZ format though it may be of mixed record type if the option UNIQUE is used with unsorted input files or will be of multiple record type if the option CONCAT is used. The missing number flag for the output file is either defined as the one set in the first file or the default NaN. The input files need not have the same missing number flag.
NOTESWhere a value is not taken from an input file, a default value of 0 will be supplied for the output file.
PRINTER OUTPUTThe line printer output gives details of the input reflection data files as they are read, details of the commands input to the program and details of the output reflection data file which has been created.
PROGRAM FUNCTIONThe MTZ utility program is provided for the purpose of creating a new re-arranged or edited MTZ file from one or two existing files. The programs cannot perform any calculation on the data items appearing in the data records. The program is meant to create a new file with
The program cannot be used to exclude any data records or to create a file with multiple header labels. The program leaves the input files unaltered and deletes dummy labels before copying to the output file.
BUGS<! -- Dave Love dixit (following Jan Zelinka): -- > The functions of this program should be expressed in terms of the relational calculus operations (join, project, select etc.). (MTZ files basically contain two RDB tables, the header information keyed on keywords (like CELL) and the reflections keyed on the combined H, K and L columns.) If it was rewritten with this in mind it might be clearer and less buggy.
#!/bin/sh mtzutils hklin1 fvb_f.mtz hklin2 2hfl_vhsearch.mtz hklout unique.mtz << eof HISTORY testing unique CELL 86.16 111.93 71.71 90.0 90.0 90.0 HEADER ALL UNIQUE RUN eof