Hacker Chronicles 2/HACKER2.BIN/445.ANALYSIS.DOC

Text File | 1993-03-31 | 19KB | 547 lines

          __________________________________________________

                             ANALYSIS (v4.0)
          __________________________________________________

              Copyright 1992, 1993 by Gene V. Wallenstein
                          All rights reserved.

                        Last revision: 03/31/93

                                   -


                  Software for Science and Engineering

                    Gene V. Wallenstein
                    5990 Pine Cone Court #406d1
                    Lake Worth, Florida 33463 USA

                    CompuServe 75110,77
                    Telephone  [phone]
                    Internet - [e-mail]




































T A B L E   O F  C O N T E N T S

1. Introduction

2. Registration
   2.1. What you get when you register
        2.1.1 Upgrades - What's next?
        2.1.2 User's Manual and Tutorial
        2.1.3 Direct Support
   2.2. Registration Form
   2.3. Legal Stuff

3. ANALYSIS Tools
   3.1. Fast Fourier Transform (FFT)
   3.2. Wavelet Analysis
   3.3. Autocorrelation
   3.4. Crosscorrelation
   3.5. Coherence Analysis
   3.6. Moments and the Distribution
   3.7. Wigner-Ville Analysis
   3.8. Digital Filtering
   3.9. Graphics (2D and 3D)



























1.  Introduction and System Requirements

ANALYSIS is a time-series analysis package which includes a
wide diversity of transformations and other techniques to
locate and quantify patterns in a data set. Even though you
may not be analyzing time-series data, ANALYSIS can still
give the same kinds of information as might be extracted
from time dependent events, however, the interpretation
changes. The only requirement is that you simply have the
data in ASCII single column format. Some types of analyses
also require that you specify two data sets. You will easily
know which modules require this by the directions on the
first menu screen for each type of analysis. If you are
required to import two data sets, you must concatenate them
into a single file which is of course twice as long as each
original file.

There are two other general requirements to use this
package.

1)  The first is simply that you should have installed the
ANSI.SYS in your CONFIG.SYS file. This was probably already
done for you depending on your version of DOS, but just the
same take a peak at the CONFIG.SYS file. This is a normal
text file, so if you have to edit it be sure to save the
file in text mode and avoid any special control characters
that a word processor might put in it. Check to see if you
have in your CONFIG.SYS file, a line which reads:

DEVICE=ANSI.SYS

if you don't, edit the file and add this line to it. You can
still run ANALYSIS without ANSI.SYS, but the cursor control
used by this package is dependent on the ANSI standard and
will probably behave a bit strange without it.

2)  The second requirement is that your screen be at least
640 by 480. As with the ANSI.SYS requirement, ANALYSIS will
still run on smaller screens, but some of the graphic
displays may clip on screens much smaller than this. Give it
a try.


2.  Registration

ANALYSIS is distributed through a unique marketing concept
called "shareware". The idea behind shareware is that you
get the software and try it for a short grace period and if
you intend to use it beyond that you register your copy with
the author. The entire concept is based on an honor system
among programmers and users, which has led to an outstanding
base of high quality software without the high price usually
associated with it. Also, shareware users are in the unique
position of being able to contact the authors of the
software they are using and get direct support in the form
of manuals, phone help, and possible further customization
of the software. Please read the list of added features you
will receive when you become a registered user of ANALYSIS.


2.1.  What do I get when I register ANALYSIS?

Most authors of shareware offer further features to
registered users. Below is a list of the additional items
provided to registered users of ANALYSIS.

2.1.1.  Upgrades - What's next?

Registered users of ANALYSIS pay a one time registration
price and receive all upgrades free. The only cost is a
minimal fee to cover the postage and handling of the disk.
As a registered user you will receive notifications through
the mail for updates of ANALYSIS.

Several users have asked what is in store for new versions
of ANALYSIS. Since in general we update ANALYSIS every 8-10
months, I will simply provide a broad sketch of planned
upgrades.

Most users have commented that, above all, they would like
to see more "number crunching" routines with as much
flexibility as possible. In light of this, the following
additions will be made to the next version(s) of ANALYSIS:

1) WAVELET - Additions to the wavelet module have already
begun. These include a broader family of "mother wavelets",
which can be viewed prior to use. We are also looking into
algorithms for allowing the user to define their own "mother
wavelet".

2) NON-LINEAR ANALYSIS - Several routines have already been
written to estimate quantities such as the correlation
dimension and Lyapunov exponents of an attractor. These are
quantities used to characterize the geometry and dynamics of
a system's attractor. They can thus be used to determine if
a system is chaotic or not.

3) MODELING - Both linear and nonlinear modeling in the form
of ordinary differential equations (ODEs). The idea here is
to specify a general form of the ODEs and solve for the
parameters by using optimality constraints on the fit and
stability of the equations.

4) STATISTICAL TESTS - Some users have requested the
inclusion of various statistical tests such as ANOVA and T-
tests. We did not plan to incorporate much in the way of
statistics beyond the first four moments and distribution,
but considering the feedback obtained from our users, a
general section for additional statistics will be added in
the future.

We encourage all our users to provide feedback about
ANALYSIS along with any ideas they may have for future
releases. This is the primary means by which we guide our
updates.

2.1.2.  User's Manual and Tutorial

A laser-printed user's manual and tutorial which is actually
and expanded version of section 3 of this file will be given
to all registered users. This manual includes various
examples of data analysis and explains which techniques are
best in certain circumstances and the type of information
which can be obtained from such methods. Several graphic
examples are also used to assist the reader in the
interpretation of the results obtained from using ANALYSIS,
along with an extended list of references for future
reading.

2.1.3.  Direct Support

All registered users of ANALYSIS will receive full support
either over the telephone, internet, or via the regular
mail, by using one of the contact points on the cover of
this document. This information is also listed on the first
information screen of the program as well. Typical support
consists of addressing any problems which are encountered
while running ANALYSIS on certain machines. NOTE: The
leading cause of reported problems is simply not using the
proper file format. Please refer to the screen accompanying
the particular module you are using for any additional
requirements on file format beside those already stated.















2.2.  ANALYSIS Registration Form

There is a separate form in the file REGISTER.DOC, which is
the registration form. To get a hardcopy of the file you can
"copy" it to your printer by using C:>COPY REGISTER.DOC
PRN. This command prints out the document assuming the
REGISTER.DOC file is in the C:\ directory. You do not have
to use this form, but try to include the following
information when registering ANALYSIS:

Name________________________________________________________

Address_____________________________________________________

City/State/Zip______________________________________________

Disk Size  5.25_________  3.5__________

Where did you find ANALYSIS?________________________________


The following fees apply to registration:

1) Single user - $25.00

2) Institution (under 10 machines) - $150.00

3) Institution (over 10 machines) - $300.00

Send check or money order to:

Gene V. Wallenstein
5990 Pine Cone Court #406d1
Lake Worth, Florida 33463 USA




















2.3.  Legal Stuff

1)  License - The user is granted the right (and encouraged)
to copy this unregistered version and distribute it with no
charge assuming it is unaltered . The user is also granted a
grace period of 30 days in which to decide to register the
software or not. During this grace period the software may
be used free of charge.

2)  Copyright - ANALYSIS is protected under United States
copyright law and international treaty provisions. Copyright
1993 by Gene V. Wallenstein; All rights reserved.

3)  Reverse Engineering - Any form of reverse engineering
such as reverse compiling or disassembly is strictly
prohibited.

4)  Disclaimer of Warranty - This software and manual are
distributed "as is" and without warranties as to performance
of merchantability or any other warranties whether expressed
or implied. Because of the various hardware and software
environments into which this program may be placed, no
warranty of fitness for a particular purpose is offered.
Good data processing procedure dictates that any program be
thoroughly tested with non-critical data before relying on
it. The user must assume the entire risk of using the
program. Any liability of the seller will be limited
exclusively to product replacement or refund of purchase
price.


3.  ANALYSIS Tools

The following is a brief description of the tools used by
ANALYSIS. It is by no means exhaustive in its description
but is merely intended to get the reader "up and running". A
more detailed description of the techniques as well as a
tutorial is provided to registered users (section 2.1.).

3.1.  Fast Fourier Transform (FFT)

The Fast Fourier Transform (FFT) is a technique used to
locate regularities in a data set. It is a way of
representing a very complicated data pattern in terms of its
linear projection onto sinusoids of various frequency. Thus
the data can be seen as the sum of many simple periodic
patterns each with a specific frequency. The POWER SPECTRUM
is a power by frequency graph (uses a Parzen window) which
shows quantitatively how much of the original pattern is
composed by each of the frequencies.

A second quantity which can be derived from the FFT is the
phase angle. Since the original signal can be represented as
a sum of periodic patterns, it may be important to know the
temporal relationship among each of the simpler patterns.
That is, if they are time-shifted with respect to one
another or not. The PHASE SPECTRUM tells the user the phase
angle at each frequency. Generally, the user is only
interested in those frequencies containing relatively large
amounts of the total power in the spectrum. Therefore,
one may compare for instance the phase angle for numerous
data sets at a specific frequency and its higher harmonics.

In general, the FFT is a good method to decompose a data set
which has multiple frequencies embedded in it.

3.2.  Wavelet Analysis

Wavelet analysis is fairly new to mainstream signal
processing. It is a projective technique similar to the FFT,
however, rather than simply decomposing the signal into
sinusoids of varying frequency, the data is represented as
projections onto the affine group (translations and
dilations). This means the data set can be represented as
time translations of the mother wavelet (a basis function)
and/or time dilations (i.e. shrinking or expanding the time
scale of observation). ANALYSIS uses what is called the
Mexican Hat as the mother wavelet (the function looks like a
slice through a sombrero). The TRANSLATION SPECTRUM plots
the relative magnitude of the original signal when projected
onto the mother wavelet under various translations. The
DILATION SPECTRUM is the same thing, only now the projection
is onto dilations of the mother wavelet.

Typically, wavelet analysis is an excellent method for
locating short-lived, high frequency patterns. This is
because it avoids the time vs. frequency resolution problem
inherent in Fourier analysis.

3.3.  Autocorrelation

The autocorrelation function of a signal is a time dependent
plot of the correlation of the signal with itself at
different time lags. If the signal is periodic, the
autocorrelation of the signal will also be. The initial
value in the autocorrelation should be 1, since this is the
first time point in the signal correlated with itself (zero
time lag). Following this, a time lag of one point is
introduced each time a correlation is computed until the
final lag of (n-1) time points. The range of the
autocorrelation signal is thus 1 to -1.

Another use for the autocorrelation is to see where the
signal loses all correlation with itself (i.e. Markov
processes). This can be seen as the first zero crossing of
the autocorrelation. Consequently, even a periodic signal
will have a zero crossing in the autocorrelation function.
By examing the autocorrelation function of a signal, one can
tell when in time the signal is correlated with itself and
to what degree.

3.4.  Crosscorrelation

The crosscorrelation is similar to the autocorrelation
function except it describes the correlation between two
signals shifted in time. To use the crosscorrelation module
in ANALYSIS, you must concatenate the two files into one in
single column format. Thus the two original files must be of
exactly the same number of points. This module is very
useful for extracting information about the temporal
dependency of one signal on another.

3.5.  Coherence Analysis

Coherence is a frequency dependent measure which quantifies
the magnitude of shared power between two signals. To use
this module, the data must be in the same format as that
used by the crosscorrelation analysis. Similar to the FFT,
the POWER SPECTRUM is a magnitude by frequency graph, which
in the case of coherence is normalized between 0 and 1. Thus
if two signals have exactly the same amount of a given
frequency in their composition, the coherence at that
frequency will be 1. If they have little in common at a
given frequency, the coherence magnitude is 0.

A second quantity of the coherence measure is the relative
phase angle between the two signals at a given frequency.
The measure of relative phase between two signals yields
information about the synchrony of the signals at a specific
frequency. The two signals may be in perfect synchrony (a
relative phase of 0 degrees) or be exactly out of phase or
perfect syncopation (180 degrees). This analysis provides a
means of quantifying the relative degree of synchrony
between any two signals at each of the frequencies of
interest.


3.6.  Moments and the Distribution

This module returns the first four moments of the data set
(the mean, variance, skew, and kurtosis) and generates a
graph showing its distribution. This information is of use
when attempting to get a general feel for the data set and
its characteristics. The information can also of course be
used as a means to statistically quantify the data set in
comparison with other data.

3.7.  Wigner-Ville Analysis

The Wigner-Ville method of analysis is a technique which has
similarities to both the FFT and Wavelet methods. It is a
projection of a data signal onto a family of sinusoids as
is the case of the FFT, however, it also depends on time
translations of the signal. The Wigner-Ville method is
particularly good at uncovering periodic patterns in data
sets where the signal is not stationary (i.e. the data may
have a strong trend). By using the Wigner-Ville analysis,
the data signal can be decomposed into its frequency
components which are of varying temporal duration.

When using the TRANSLATION SPECTRUM of this module, one must
specify which frequency of interest to calculate the time
dilation across, or if the interest is in the FREQUENCY
SPECTRUM, the user must specify the time dilation to
calculate across each frequency.

If you are dealing with a data set which has slow trends
which shift the moments of the signal dramatically in time,
the Wigner-Ville can be used as a check on the FFT
calculation.

3.8.  Digital Filtering

The digital filtering module in ANALYSIS is a general
purpose tool for filtering data sets at prescribed band
widths. For example, if the user wishes to create a new data
signal consisting of only the 8.0 to 12.0 Hz frequency
components, one simply needs to specify the high-pass (8.0),
the low-pass (12.0), and the sampling rate of the original
data set. The resulting output can be viewed and sent to a
file for further analysis. Digital filtering is often the
best way to "clean up" a signal so that its true structure
can be viewed more easily. This, for example, is often the
case when the signal contains a large amount of higher
frequency noise. The user wishing to get rid of this could
simply use this module and set the high-pass to 0.0 Hz and
the low-pass setting to whatever is sufficient to eliminate
the "noise". However, caution must obviously be taken when
deciding what does and does not constitute noise in a
signal.

3.9.  Graphics (2D and 3D)

The graphics routines in ANALYSIS will plot data in two and
three dimensions. The 2D routine simply takes the single
column of data and plots it with time as the X axis. The 3D
routine takes the single column of data and breaks it down
in such a way that the first three numbers represent the
first point in 3 dimensional space, the next three the
second point, and so on. For the 3D routine, the user is
asked whether a connected plot or circles are desired to
represent the data. Also, the user specifies the desired
rotation in the polar and azimuthal directions. A polar
rotation of 0 degrees would be looking directly down at the
data from the north pole. A 90 degree polar rotation would
place the line of view directly at the equator. A rotation
in the azimuthal direction would be like a walk around the
equator. Try an initial polar rotation of 65 degrees and an
azimuthal rotation of about 35 degrees.