Powder v1.00
A short help file

  
 
 
 
      Tokyo University, Hongo campus,  march '98 

This picture has nothing to do with this program, except maybe that here I keep my bicycle when working late in night

 



 
 
About this program. Some history.
 
This program consists in a collection of tools for X-ray powder diffraction. This is a variant of a previously one, known as Convert v3 which was capable of various data file conversions. These functions are still available in the program Powder. The goal is to write a comprehensive package for the X-ray diffraction on powders, I have to say that this version is far from that goal. This small help file should give some info about this program, probably a heuristic method will be better for the user. However, at this moment the conversion function is still the main function of this program. A graphic interface for peak hunting and peak profile analysis will be added later sometime..

Since I start working on X-ray powder diffractometry I noticed that almost each program required a different form of the data file (quite normal, different authors, different programs..) There was quite a pain to convert the data file from an instrument to an appropriate format for use with another software. So, not willing to do the conversion by hand, I wrote some small routines in some variants of Basic and/or Fortran (very good Format function) to do the job. There were no fancy interfaces or help. I had to write then nicer routines for some of my colleagues. And I had then the (good ???) idea to write a more general program for such types of conversion. This was Convert 1.0. That version was written when doing my PhD at Université Paris Sud (1994; this wasn't what I supposed to do...!); it was written in a french version of Visual Basic 3.0. Another version and a bug hunting followed when I was working at University of Bucharest. The version 3 was written in '97 at University of Tokyo (Visual Basic v5, a japanese version; so some error messages may look funny on your computer).
 




 

Update notes.
 

v.01 a
compiled on march 8, 98
 

v.01 b.
hkl generation added.
correcting a bug in Decimate function
compiling date: march 28, 1998

v.01 c
energy dispersive refinement is now possible
compiling date: april 3, 1998
 

 back to top




 

About version and author
 
For the exact version look in the About form (a window is called here Form), the help file may not be up to date.
This is the version 1.00 of Powder, a 32 bits version and can run only in Windows 95 or NT 3.5 or, I hope, above. I didn't find any bugs in this version but I think that it may contain some (an even number...). If you find any, hopefully sooner, please let me know and I will try to correct them.
I can not guarantee a support to this program since I have a job; that will depend entirely on my free time. However, before asking me something read the help file and follow the advises in the notice.
 

 
The author of this program is:
 
Dr. N. Dragoe
Department of Applied Chemistry,
Faculty of Engineering, University of Tokyo
7-3-1 Hongo, Bunkyo-ku, Tokyo 113, Japan
 
tdragoe@hongo.ecc.u-tokyo.ac.jp
 

The permanent address is given in the About form, which can be seen from the Help menu.

 

and e-mail:

ndragoe@math.math.unibuc.ro
ndragoe@dopey.univermed-cdgm.ro
 
this is my permanent address, for a while (1997-1999 ?) you can reach me at the address in Tokyo/Japan.
 

 
It will be nice to have some comments from you...
 
back to top




 

Features

It is useful mainly for people doing X-ray diffraction experiments although the program can be used with different data types or instruments (I am using it for conversion of UV/VIS spectra ascii data where the X values are in a decreasing order; so a negative step can revert this...).
This program can read and write some of the data files I know : Dbws, Gsas, Philips, Siemens, Scintag, ascii files, MXP18 data, Wppf....Some other files can also be read with this program.
A quite different function now is a unit cell refinement calculations and HKL generation which can also be performed with this program.




 

Forms

There are three different windows in the program.

About-which contains the version number and with which the program itself starts. This form can be viewed at any time from the Help/About menu. To start the program you must click on the icon or in the text area in the form.

Main form- which is practically the old Convert 3 program.

PowderCell-is a new form added for utilities related to unit cell refinement or powder pattern generation



 
 

Main form - menus

The main form of the program has one text region where some info is printed from time to time and few commands available in a menu. The form look like this:

There are few menus in this form. Click on the appropriate keyword for more information.
 
File
    Open
    Save as
    Send data
    Quit
 
Header
    I don't have a header...
    Ignore first ...
 
Tools
    Unit cell
    Read HSTDMP from GSAS lst file
    Read REFLIST from GSAS/rfl file
    Compute d/angle
    Remove strange characters
    Swap XY
    Open file in Wordpad
    Show report
    Clear report
 
Help
    About
    Contents - HTML file
    Author WWW page
 
 
back to top




 

File
This menu contains the following sub menus:

    Open
    Save as
    Quit
 


 

File-Open

This command open a raw data file and read the data in memory. You should choose the file format you have otherwise strange effects may occur.
The program accepts the following commands (as sub-menus) while in File-Open menu:

DBWS file
GSAS-CW ESD
GSAS-CW STD
Philips PC-UDF
Scintag data
Siemens, ascii
Wppf/Profit 1
Wppf/Profit 2
Line
    The first value as Y
    The first 2 values as X, Y
    The first 3 values as X, Y, Z
Y-free ascii
XY free ascii
XYZ free ascii
MXP18, binary




 

Open DBWS file

DBWS format consists in: starting 2theta, step, final 2theta and a title (named in fortran as 3F8, A56), this being the first line in the file, called in Powder as header. The rest of the file are in 8F7,1X format. Only the first 56 spaces are read since some old data files have a comment or card number in the columns 72 through 80 (this area was reserved for comments in the old days). The values read from the header will be later used to generate the X values, if necessary. Although it reads a title here, the program will prompt you to set or modify the title when trying to save the data. You can use this command only for data files having 8 values on a line, otherwise you may obtain strange results.
Refer to the Header commands if you want to find out how to ignore the header read from this file.
 




 

GSAS - CW ESD  and GSAS - CW STD

GSAS is the well known program (in my opinion the best free package, and better than many commercial ones, for XRD) written by A.C. Larson and Bob von Dreele from LANL.
You can read a GSAS-ESD constant wavelength. In Powder there are some limitations regarding Gsas files. Some of the records (except starting two theta, step and the number of points) are ignored in this program. For instance "Instrument parameter" which is optional in the Gsas file is completely ignored here, as well as BINTYP info (read the GSAS manual for more information). The data is read as 5 values and corresponding ESD's on a line until the EOF marker is found. The normal GSAS data may have few BANK records in a file, here only one is read.
 
Same general comments are available for STD file too,  ten values in each line are accepted here. For use of the GSAS files I strongly recommend to read the instructions in the GSAS User Manual. Note that new versions of GSAS contain a file converter.



  
 

Philips PC-UDF

With this function you can read a Philips UDF file. The UDF files which I know consist in a header (about 20 lines) and then the diffraction data as eight characters in a row, separated by commas. From the UDF header some fields are ignored. If you don't want to use a header, the same file can be read by Y, free ascii command.




 

Scintag data

This file type is similar to that of Philips, i.e. a big header which is mostly ignored by Powder and 2theta, intensity and esd. Although it is not convenient, the same file can be read ignoring the header, as XYZ free ascii.




 

Siemens ascii

The format of this file is quite similar to the Philips or Scintag ones, a header followed by data written as 8 values on each line. As before, from header only some fields are requested.




 

WPPF/Profit

WPPF and Profit are two different programs written by H. Toraya. These programs can read two distinct data files either as 8 or 10 values in a line. These data types are named hereafter as format 1 and respectively 2. The program detect the data type you have and read it either as 8 or 10 values in a row, as appropriate. From the header of this file, some info as the wavelength or counting time are ignored.
 




 

 Line

     the first value, as Y
     the first 2 values, as XY
     the first 3 values, as XYZ

On this menu, three commands are available giving the opportunity for reading only the first value, the first 2 values, or the first 3 values from each line. This is useful only if  there are more data in a row than x, y. These commands do not allow a header, you will be prompted for additional information if necessary. If the data file contains some titles, use the available commands to ignore these lines, see the Headers menu.




 

Y; free ascii

You can read Y values no matter the format is, as long as the data are separated by comma or tab or spaces and no empty lines are in between values.
 
X, Y; free ascii

Read X and the corresponding Y in any format as long as the values are in pairs (i.e. in each line the final value must be for an Y value.)
 
X, Y, Z; ascii free

Read X, Y, Z from any ascii file. Z is usually considered the ESD value for each Y. However, in this program no ESD estimations/correction are made, if necessary Z will be filled with 1.0.




 

MXP-18 binary

This is a binary data file from Mac Science MXP18 diffractomers.
Read some details from a header as well as X and Y. Instructions relative to ascii file do not apply here (for instance, ignore lines, etc)

back to top




 

File-Save As
 
Write the data in the appropriate format. If necessary, the program will ask details about the data. Some notes applies for each format. The program accepts the following commands (as sub-menus):

DBWS
GSAS - CW ESD
GSAS - CW STD
Philips UDF
Scintag
Siemens
WPPF/Profit 1
WPPF/Profit 2
Y-ascii
XY-ascii
XYZ-ascii
DPLOT file




 

DBWS file

This command save a file in a Dbws file format.


 
 

GSAS - CW ESD
GSAS - CW STD

The output file for GSAS is limited at some constants. For more advanced data type use the built-in convert program from the GSAS package (or edit manually). No "Instrument parameter" is written in the output file, the BANK number is always 1. Check the data file for 80 characters in length, GSAS requires this (Powder fill with spaces up to 80 chars but if you edit this files with other editors check if you still have 80 characters on each line).
 




 

Philips UDF ,  Siemens ascii and Scintag

In Philips, Siemens and Scintag files, most of the header is fixed, you can manually change the values to your preferences.



 
 

WPPF/Profit 1 or  2

Both format types in WPPF/Profit neglect the wavelength value and the counting time. The wavelength will be always of copper Kalpha 1. You can change later this value, see the Wppf or Profit user manual for this.
 




 

Y; ascii
X, Y; ascii
X, Y, Z; ascii

Not much to explain here, the output is in ascii with one record per line.
 




 

Dplot file

Save the data in a Dplot file. Dplot is a very good freeware program for scientific graphs. I quote from the Dplot help file: "DPLOT is a general purpose X-Y plotting program that allows you to view, manipulate, and produce hard copies of data in a variety of formats".
The package can be found at http://sliris.wes.army.mil.


back to top




 

Header
 
I don't have a header

Selecting this option (it is ON if there is a check mark on the menu) allows the reading of data file which do not have a header definition.
 

Ignore first ...

With this option, first lines from data file can be ignored. The user should choose how many lines to be ignored when reading a data file. For instance, if you have an ascii data file and in the first 3 lines there are some comments, you can choose Ignore first then input the value 3. In this case the program will ignore the first 3 lines in your data file.
Or, if you want to read a DBWS file and modify the 2 theta values, select I don't have a header option and then Ignore 1st line. The program will skip the 1st line and then start to read data directly, not searching for the X values. It will ask at a later occasion, if necessary.
 
back to top 




 

Tools
 
The menu Tools contains the following commands:
 

    Unit cell
    Read HSTDMP from GSAS lst file
    Read REFLIST from GSAS/rfl file
    Compute d/angle
    Remove strange characters
    Swap XY
    Open file in Wordpad
    Show report
    Clear report
 




 

Unit cell

This command will open a routine related to Unit cell refinement and/or hkl generation. See the Unit cell commands.




 

Read HSTDMP from GSAS .lst file and Read REFLIST from GSAS .rfl file

These commands are related to Gsas package. The first one  request a GSAS results file, *.lst. It will search in this file the occurrence of "Program HSTDMP" string. When it can find this, a file named _hstdmp.dat is created in the application directory (Warning - it will erase an old _hstdmp.dat file). This file contains the data output from HSTDMP program. If two or more statements are found the data set will be attached one after another without a separator. This command may depends on the GSAS version you have. You can use the _hstdmp.dat file for another application (your preferred graph editor..)
 
Read Reflist from GSAS/rfl file request a .rfl file written by GSAS. This file can be saved by the Tools/Reflist menu command with the option R/ascii file. With this function you can read h,k,l Fo and esd(Fo) from Gsas and save a Shelx .hkl file (for structure determination or use of these data with another software). The program Powder will ask for a batch number which is an optional field in Shelx (for those making batch refinements). The output file will be merged. The format in the output file is as required by Shelx 97, I don't know if works with older versions but I guess it does so..
 




 

Remove strange characters

This option allows to skip all characters from the extended code and from command code (smaller than 30). It is useful when the data come from some Unix software which have a different ascii character set, as my MXP18. It is a time consuming option (this is relative, 5 seconds with 5000 data on my laptop). To be honest, often I use this command without an obvious reason (some files, depending on the programs saving them, give errors when read by Powder. I guess that this is related to the EOL chars).



 
Compute d/angle

Toggle a small utility for computing d or diffraction angle for a given radiation. When working with this routine, the report -a kind of activity log- is not visible anymore; however it is still there (if you want to read the report toggle OFF this function and ON the Show report command)
 




 

Swap X, Y

This function interchange the X and Y data in memory. Generally it is not useful for X-ray diffraction data but for other type of data file.
 




 

Open file in Wordpad

This command open a shell to Wordpad, where you can edit or print a datafile or the report pad, etc...
 




 

Show report

In the main form, a text area shows what is going on, what was the filename, how many lines it found in it, and so on. All output from the Unit Cell routine is directed here.  May also be useful for some debugging purposes.
You can toggle the visibility of this report pad (it is still in memory). From time to time (when this pad has 32kb) the report will be appended to a file called _rpbackup.dat and will be erased from the screen.
This file is in the working directory (Warning- NOT the application directory); take care and clean this file from time to time as it can get very large...




 

Clear report

A self explaining command, be careful : no UNDO is available.
You can also clear or copy/paste data to this report at any time. Use the RightClick button of the mouse to see a PopUp Menu with common commands under Win95 (as Select, Copy,...).
 

back to top



 
 
Help
 
About

Shows the About window of the program. Nothing interesting there except the version number of the program (or the author ?..). When you have trouble and want to ask some questions (after reading this help file and after you let me know you use the program !) , report this version number.
 
Contents-html file

Open this Help file in the default browser. You can also edit this file and change whatever you like (I will be glad to put your comments in the future help files!)
 
Author WWW page

Open the default browser and go to my WWW page (there might be an update or another version of this program). The page location is read from the ascii file _wwwauthor.page from the application directory. You can edit this file and change the URL of the author page if this will change. If this file isn't there, the program will try at the address internally stored. This command may not be available all the time, this is a temporary job and so a temporary home page (I guess it will be there until the end of 1999...)
 
back to top




 

PowderCell

This form will help you in refining a unit cell and to generate hkl and d's. It is also useful when reading some data (it can read a specified column from the file, etc..). The form looks like this:

There are few menus which will are discussed later in this file.
The unit cell, both for cell refinement and hkl computing can be selected from the listbox named Cell, just click on it. In its right side there is a similar one where you can select the Wavelength or, if you use something rare -as a synchrotron- put the actual value in the associated text field.
The grid in the middle of the image is referred here as Data grid, where you -or the program- will put some data. you can change the data type in the column 4 just by clicking on the top of the grid (you can toggle between d/A, theta, 2 theta or E/keV for energy dispersive data). You can put a lot of data here (I hope not, because the program will become really slow) although I am not sure about the upper limit of the MSFlex grid, I guess you can put more than 15000 rows...). For refinement you have to insert data there, no empty lines allowed, the order is meaningless.
The grid in the right part of the form is the standard grid where you should insert the expected and observed values for your standard material. Important note: use the same data type as in the data grid, therefore you can put here, d, theta or 2theta depending on what you have in the data grid..
The data inserted in the standard grid will be used for a calibration, if and only if, the associated check box is ON (named Use standard).
The correction for 2theta is made with an interpolation polynomial for the difference in theta/2theta or d relative to theta.
The program computes such a difference with a pseudo-Inverse routine i.e. an overdetermined system (frankly speaking, a least squares!).
You should remember some mathematical basics, the polynomial interpolation may give strange results. However the corrections  are printed on the report pad in the main form; check them out. In this version, the polynomial degree can be comprised between 0 and 8; although I recommend a value between 1 and 4. Two observations here:
-do not use a high degree polynomial (I suggest you not to go over 4 or 5) unless you have a lot of data somehow homogeneously distributed over the 2theta domain (about 20 to 30 data for a good conditioning) and not too far from the initial and final values.
-0 polynomial is not a zero error if the inserted data is d.

There are checkboxes associated with text fields, such as a, b, c...etc. In the refinement, the meaning is to refine or not the corresponding parameter; there is no meaning for them in the hkl generation routine.
For instance, in the following example:

the cell selected is monoclinic (note the constraints, you can not modify alpha and gamma for this cell). The parameters a,b,c and zero error will be refined while the wavelength and beta will not. No standard correction is to be made.
The lower left part shows three text fields with width, steps and refine. The meaning of these parameters will be explained later; however for a normal refinement (i.e. almost always) only Steps/Cycles is take into account and its meaning is in fact Cycles.
In this example, when the method of refinement is Least squares (see later), 10 refinement cycles will be made.

Another important note: by weight In this program the value of w in the relation is considered:

w(Yo-Yc), so don't put the esd's on the column 5 but 1/esd.
 
 

The available menus are:
 

 
back to top



 

Cell-File

The commands in this menu are:

    Ignore:
    Import data (ascii)
    Import calibration data
    Export data grid to ascii file
    Quit




 

Ignore

Similar to the Ignore function from the Main form. The program will ask how many lines from a data file should be ignored. Usually, the data files contains some headers and comments, by this command one can import data by ignoring few lines and thus avoiding editing and saving it under other name, etc..
Initially this menu will have the caption "Ignore:none" and after activated this menu will show how many lines will be ignored.




 

Import data

This function will import data from an ascii file and put these data in grid. Note: each record should contain four data or five data (the last is for a weight). The selection between the two option is made with submenus. These data can be either each record on one line either sequentially in an ascii file. If the four values are read, the weight column will be filled with 1. Special functions for editing the weight are available in the Edit menu.
If the data file are in a different format it still can be read with the command Input data in column, available in Edit menu.




 

Import calibration data

A two column ascii file will be read into standard grid. Warning:If the data in the standard grid are not the same type as those in the data grid, you'll get a mess.




 

Export data grid

The data from grid can be saved into an ascii file. This function is useful either for storing the datafile for further reference or to export a generated hkl, d pattern. See the menu command Compute.




 

Quit

Close the Cell refinement form and loose all the data you have in grid.




 

Edit command menus
 
The following commands are available here:




 

Insert Row, Delete Row, Delete Column

These functions refer to the data grid only. You can insert or delete rows in the grid, if you want to modify your data (delete column will delete data but not the column...). Delete row is in fact equivalent to a Cut command and is very useful because  you can not have empty lines in the data grid (the program will "read" how many data you actually have by checking an empty line.)

 
 




 

Add to Column
    a constant value
    a column
    1/column
    to weight:1
Multiply Column
    with a constant value
    with column
    with 1/column

These commands are similar, they will add a value (or multiply with one) to a column in the data grid. The program will ask which column you want to modify (H is considered the first column). You can add or multiply weight with the value which is in another column (but the same row). By this mechanism you can change the weighting scheme for the refinement.
Take care: if you put 0 in the weight column that data row will be neglected in the refinement.




 

Input data in Column

This is a very powerful function which allows you to read only one column from a data file (or read some data and skip others in a regular manner). The program will ask in which column the data are to be inserted (again, the column 1 is H). The second question is which values will be skipped (read one values and then skip n), you'll be asked about this n; n=0 means to read all the data.




 

Clear data grid
Clear std. grid

You can clear one or both of the grid at any time.




 

Set data rows

When inserting data from files, to avoid errors the program automatically will increase the number of rows (often to much). By this command you can set the number of rows to a reasonable value. Sometimes this can save some time. If the grid has (because of unfortunate input files..) 1000 rows but you have only 100 data points it is better to set the data rows to 100 to save some memory (anyway, in my opinion -from time to time- 5 seconds more or less it is not a big deal !).
 

 back to top




 

Compute

This is the main function in this form. There are two possibilities, shown in the sub-menus:

Unit cell parameters
    least squares
    overdetermined system, Newton
    multidimensional search
HKL, d and theta
    send to report pad
    send to data grid
 



 

Unit cell parameters
    least squares
    energy dispersive data
    overdetermined system, Newton
    multidimensional search
 

The unit cell parameters are calculated by least squares (all three methods have some least squares procedures inside) and I recommend you to use the first option (however, is the only one who gives the standards deviations for the cell parameters).
Both least squares and overdetermined system, Newton use the same algorithm of Newton, the difference is in the solution of the linear system (first one is a Crout type and the second one is a Gauss-Jordan). The second method is less convergent than the first one because of a dumping factor I imposed, but seems to be more stable. You should check if you arrived at a convergence, the values for the sq. deviations at each cycle are printed.
A fancy called  multidimensional search  is just a search for the minimum of the squared difference between Q and its expression reciprocal terms. This is done by a modified Interval halving method. I will explain this method because is very easy to put in code (although its value is limited) and because it is not a recommended method for refinement, especially for lower symmetries.

Starting from an initial guess (no matter how far from the minimum) one can find the minimum of a function f=(y1-y2)2 where y1 is an experimental value and y2 is a model depending of some parameters x1, ..,xwhich have to be estimated. If n is not too high one can make a search in an interval and compute the function f at each step, the minimum f will be an estimate of the solution x. The error will depend on the step size and on the fact that the actual minim is comprised in the search range.
In this function, if the minimum is not comprised in the search range, the width will be adjusted toward the minimum.
This procedure is implemented here (test it if you like but think how long it will take!). The width shows how far from the initial point it will search the minimum, the steps/cycles is how many steps will make on each variable on each interval and refine shows how many times to do this with a halving of the width for each cycle.
For each refine the step will be smaller with a 1/q factor where q is the number of refine in the textbox. The fields Width and refine are used only in the search algorithm and have no meaning in the least squares.
It is very slow because, for n parameters problem one need stepsn functions evaluations and this multiply by refine value.
It is a very time consuming tehnique but probably in ten years from now....
Actually,  the reason of existence for this function here lies in other routines I will add later and in some educational purposes (manual indexations...). One can make an indexation according to Ito's procedure and get any strange triclinic cell. By a subsequent Delaunay transformation we can get a better cell.

Note: the results are printed in the Report pad and not in the Cell Form.

At this moment I don't know much about the accuracy of this program compared to other known programs (the minimizations use Double precision for variables but Visual Basic is not made for scientists!). I think that the eventual error is smaller than 0.001A which for me is good enough.
I am waiting some input from you here...




 

HKL, d and theta
    send to report pad
    send to data grid

Both commands perform the same calculation tasks but the output is directed either to the report pad or to data grid.
The HKL and d are computed for the cell you have and with the parameters found in the Cell form.The hkl limits and 2theta are to be input by the user as well as Bravais symbols.




 

Decimate

This menu contains three functions which are inspired by the Armel LeBail's Overlap. Reading Fullprof FOU, Gsas RFL and/or Shelx HKL files, eliminating the reflections which are too close (the two theta limit will be requested) and save what's left in a Shelx .hkl file for further direct methods or whatever.
The cell values are taken from the Cell form, the symmetry is not requested.
 

back to top



 
 

Notice
 
There are some limitations in the use of this program and probably there are some bugs too. It may be slow but it can read as many data as you have, a special attention was paid to the "friendly" characteristic. Some comments may help you out of trouble.
And now, the usual stuff:
 
License Agreement

This software is provided "as is" and without warranties of any kind; use it on your own risk.
If you don't like it don't blame me but don't use it. I don't guarantee for any accuracy and/or meaning of the output.
You are hereby licensed to use this software if you are from a non-profit making institution only.  You may make as many copies of the software as you wish.  You may give exact copies (including this help file) of the original software version and redistribute the complete and unmodified version via electronic means. You are specifically prohibited from charging a fee, or requesting donations, for any such copies.
 
I you want to use this software, you are requested to send an e-mail to the author, stating your name, institution and the version you have. This is for bug reports and updates. Future correspondence regarding this program will be made only to the people on my "user list" where you can stay as long as you want (and to encourage you, I will not answer to any questions if you are not on that list).
 



 
back to top