Bickford, Randall L. (Orangevale, CA, US)
Herzog, James P. (Downers Grove, IL, US)

Plaque It!

11/492329

08/19/2008

07/24/2006

Images are available in PDF form when logged in. To view PDFs, Login  or  Create Account (Free!)

Click for automatic bibliography generation

Intellectual Assets LLC (Lake Tahoe, NV, US)

702/178

702/181

G01R17/00; G06F15/00

702/178-185, 702/1-6

5223207	Expert system for online surveillance of nuclear reactor coolant pumps	June, 1993	Gross et al.
5410492	Processing data base information having nonwhite noise	April, 1995	Gross et al.
5459675	System for monitoring an industrial process and determining sensor status	October, 1995	Gross et al.
5586066	Surveillance of industrial processes with correlated parameters	December, 1996	White et al.
5629872	System for monitoring an industrial process and determining sensor status	May, 1997	Gross et al.	702/116
5680409	Method and apparatus for detecting and identifying faulty sensors in a process	October, 1997	Qin et al.
5745382	Neural network based system for equipment surveillance	April, 1998	Vilim et al.	706/16
5761090	Expert system for testing industrial processes and determining sensor status	June, 1998	Gross et al.
5764509	Industrial process surveillance system	June, 1998	Gross et al.
5774379	System for monitoring an industrial or biological process	June, 1998	Gross et al.
5987399	Ultrasensitive surveillance of sensors and processes	November, 1999	Wegerich et al.
6107919	Dual sensitivity mode system for monitoring processes and sensors	August, 2000	Wilks et al.
6119111	Neuro-parity pattern recognition system and method	September, 2000	Gross et al.
6131076	Self tuning system for industrial surveillance	October, 2000	Wegerich et al.
6181975	Industrial process surveillance system	January, 2001	Gross et al.
6202038	Ultrasensitive surveillance of sensors and processes	March, 2001	Wegerich et al.
6240372	System for surveillance of spectral signals	May, 2001	Gross et al.
6245517	Ratio-based decisions and the quantitative analysis of cDNA micro-array images	June, 2001	Chen et al.
6892163	Surveillance system and method having an adaptive sequential probability fault detection test	May, 2005	Herzog et al.	702/181
7082379	Surveillance system and method having an adaptive sequential probability fault detection test	July, 2006	Bickford et al.	702/178
20010049590	Complex signal decomposition and modeling	December, 2001	Wegerich

K. Humenik, et al, Sequential Probability Ratio Tests for Reactor Signal Validation and Sensor Surveillance Applications, Nuclear Science and Engineering, vol. 105, pp. 383-390, 1990.
K.C. Gross, et al, Sequential Probability Ratio Test for Nuclear Plant Component Surveillance, Nuclear Technology, vol. 93, pp. 131, Feb. 1991.
A. Racz, Comments on the Sequential Probability Ratio Testing Methods, Annals of Nuclear Energy, vol. 23, No. 11, pp. 919-934, 1996.
K. Kulacsy, Further Comments on the Sequential Probability Ratio Testing Methods, prepared for Annals of Nuclear Energy by the KFKI Atomic Energy Research Institute, Budapest, Hungary, Report No. KFKI-1996-10/G, 1996.
R. M. Singer, et al, Model-Based Nuclear Power Plant Monitoring and Fault Detection: Theoretical Foundations, Proceedings. 9th International Conference on Intelligent Systems Applications to Power Systems, Seoul, Korea, 1997.
K.C. Gross, et al, Application of a Model-based Fault Detection System to Nuclear Plant Signals, Proceedings 9th International Conference on Intelligent Systems Applications to Power Systems, Seoul, Korea, 1997.
R. M. Singer, et al, Power Plant Surveillance and Fault Detection: Applications to a Commercial PWR, International Atomic Energy Agency, IAEA-TECDOC-1054, pp. 185-200, Sep. 1997.
K. Kulacsy, Tests of the Bayesian Evaluation of SPRT Outcomes on PAKS NPP Data, KFKI Atomic Energy Research Institute, Budapest, Hungary, Report No. KFKI-1997-07/G, 1997.
J.P. Herzog, et al, Dynamics Sensor Validation for Reusable Lauch Vehicle Propulsion, AIAA98-3604, 34th Joint Propulsion Conference, Cleveland, Ohio, 1998.
Hylko, J.M., New Al Technique Detects Instruments, Power, Nov. 1998, Printed in USA by Power.
Herzog, J.P., et al, MSET Modeling of Crystal River-3 Venturi Flow Meters, 6th International Conference on Nuclear Engineering, 1998, Printed in USA by ASME.
Bickford, R.L., et al, Real-Time Space Shuttle Main Engine Sensor Validation, National Aeronautics and Space Administration, Aug. 1995, Printed in USA by ExperTech & Intelligent Software Associates, Inc.
Bickford, R.L., et al, Real-Time Flight Data Validation For Rocket Engines, AIAA, 1996, Printed in USA by ExperTech & NYMA, Inc.
Bickford, R.L., et al, Real-Time Sensor Validation for Autonomous Flight Control, AIAA, Jul. 1997, Printed in USA by Expert Microsystems, Inc. & Intelligent Software Associates, Inc. & Beoing Defense and Space Group.
Bickford, R.L., et al, Real-Time Sensor Validation For Propulsion Systems, American Institute of Aeronautics and Astronautics, 1998, Printed in USA by Expert Microsystems, Inc & Dynacs Engineering Co.
Bickford, R.L., et al, Real-Time Sensor Data Validation For Space Shuttle Main Engine Telemetry Monitoring, AIAA, Jun. 1999, Printed in USA by Expert Microsystems, Inc.& Intelligent Software Associates, Inc. & Dynacs Engineering Company & NASA Glenn Research Center.
Singer, R.M., et al, A Pattern-recognition-based, Fault-tolerant Monitoring and Diagnostic Technique, 7th Symp. on Nuclear Reactor Surveillance, Jun. 1995, Printed in USA by Argonne National Laboratory.
Bickford, R.L., et al, Online Signal Validation for Assured Data Integrity, 47th International Instrumentation Symposium, May 2001, Printed in USA by Expert Microsystems, Inc., and NASA Glenn Research Center.
Wegerich, S., et al, Challenges Facing Equipment Condition Monitoring Systems, MARCOM 2001, May 2001, Printed in USA by SmartSignal Corporation.
Zavaljevski, N., et al, Sensor Fault Detection in Nuclear Power Plants Using Multivariate State Estimation Technique and Support Vector Machines, 3rd Intl. Conf. of Yugoslav Nuclear Society, Oct. 2000, Printed in USA by Argonne National Laboratory.
Zavaljevski, N., et al, Sensor Fault Detection in Nuclear Power Plants Using Multivariate State Estimation Technique and Support Vector Machines, 3rd Intl. Conf. of Yugoslav Nuclear Society, Oct. 2000, Printed in USA by Argonne National Laboratory.
Bickford, R.L., et al, Sensor Validation Tools and SSME Network, Final Report, Apr. 2000, Printed in USA by Expert Microsystems, Inc.
Miron, A., et al, The Effects of Parameter Variation on MSET Models of the Crystal River-3 Feedwater Flow Systems, ANS Annual Meeting, Jun. 1998, Printed in USA by Argonne National Laboratory.
Wrest, D.J., et al., Instrument Surveillance and Calibration Verification through Plant Wide Monitoring Using Autoassociative Neural Networks, Specialists Meeting on Monitoring and Diagnosis Systems to Improve Nuclear Power Plant Reliability and Safety, May 1996, printed by the International Atomic Energy Agency.

Barlow Jr., John E.

Taylor, Victor J.

Deboo, Dennis A.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of continuation patent application Ser. No. 11/027,998, filed Dec. 31, 2004 now U.S. Pat. No. 7,082,379 which claims the benefit and is a continuation of patent application Ser. No. 10/095,835, filed Mar. 8, 2002, now U.S. Pat. No. 6,892,163, the entire disclosure of both incorporated herein by reference.

We claim:

1. A method for performing surveillance an asset, the steps including: acquiring residual data correlative to expected asset operation; fitting a mathematical model to the acquired residual data and storing the mathematical model in a memory means; collecting a current set of observed signal data values from the asset; using the mathematical model in a sequential hypotheses test performed as a function of at least one of the observed signal data values for detecting a fault condition; and determining if a detected fault condition is indicative of an actual presence of a fault for providing asset surveillance.

2. The method of claim 1 wherein the step of acquiring residual data includes a step of acquiring residual data correlative to at least one mode of asset operation.

3. The method of claim 2 wherein the step of acquiring residual data correlative to at least one mode of asset operation includes a step of acquiring past observations of signal data values from the asset and obtaining deviations between the past observations of signal data values and expected signal values obtained by using a model of expected asset operation.

4. The method of claim 1 wherein the step of fitting the mathematical model to the acquired residual data and storing the mathematical model in the memory means includes a step of fitting at least one mathematical function correlative to a frequency distribution to the acquired residual data.

5. The method of claim 4 wherein the step of fitting at least one mathematical function correlative to a frequency distribution to the acquired residual data includes a step of fitting at least one function defining a frequency curve to the acquired residual data.

6. The method of claim 4 wherein the step of fitting at least one mathematical function correlative to a frequency distribution to the acquired residual data includes a step of fitting at least one function defining a frequency histogram to the acquired residual data.

7. The method of claim 4 wherein the step of fitting at least one mathematical function correlative to a frequency distribution to the acquired residual data includes a step of fitting at least one function defining a frequency polygon to the acquired residual data.

8. The method of claim 1 wherein the step of using the mathematical model in a sequential hypotheses test performed as a function of at least one of the observed signal data values for detecting a fault condition includes the step of performing at least one of a group of sequential hypothesis tests comprised of a positive mean test, a negative mean test, a nominal variance test, and a inverse variance test.

9. The method of claim 1 further including a step of performing a control action when an actual presence of a fault is determined.

10. The method of claim 1 further including a step of sounding an alarm when an actual presence of a fault is determined.

11. A method for performing surveillance of an asset, the steps including: acquiring residual data correlative to expected asset operation; fitting a mathematical model to the acquired residual data and storing the mathematical model in a memory means; collecting a current set of observed signal data values from the asset; using the mathematical model in a sequential hypotheses test performed as a function of at least one of the observed signal data values for determining indices correlative to asset status; evaluating the indices for detecting a fault condition; and determining if a detected fault condition is indicative of an actual presence of a fault for providing asset surveillance.

12. The method of claim 11 wherein the step of acquiring residual data includes a step of acquiring residual data correlative to at least one mode of asset operation.

13. The method of claim 11 wherein the step of fitting the mathematical model to the acquired residual data and storing the mathematical model in the memory means includes a step of fitting at least one mathematical function correlative to a frequency distribution to the acquired residual data.

14. The method of claim 13 wherein the step of fitting at least one mathematical function correlative to a frequency distribution to the acquired residual data includes a step of fitting at least one function defining a frequency curve to the acquired residual data.

15. The method of claim 13 wherein the step of fitting at least one mathematical function correlative to a frequency distribution to the acquired residual data includes a step of fitting at least one function defining a frequency histogram to the acquired residual data.

16. The method of claim 13 wherein the step of fitting at least one mathematical function correlative to a frequency distribution to the acquired residual data includes a step of fitting at least one function defining a frequency polygon to the acquired residual data.

17. The method of claim 11 wherein the step of using the mathematical model in a sequential hypotheses test performed as a function of at least one of the observed signal data values for detecting a fault condition includes the step of performing at least one of a group of sequential hypothesis tests comprised of a positive mean test, a negative mean test, a nominal variance test, and a inverse variance test.

18. The method of claim 11 further including a step of performing a control action when an actual presence of a fault is determined.

19. The method of claim 11 further including a step of sounding an alarm when an actual presence of a fault is determined.

20. A method for performing surveillance of an asset, the steps including: acquiring residual data correlative to expected asset operation; fitting a mathematical model to the acquired residual data and storing the mathematical model in a memory means; collecting a current set of observed signal data values from the asset; using the mathematical model in a sequential hypotheses test performed as a function of at least one of the observed signal data values for determining a time series of indices correlative to asset status; evaluating the time series of indices for detecting a fault condition; and determining if a detected fault condition is indicative of an actual presence of a fault for providing asset surveillance.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH AND DEVELOPMENT

The invention described herein was made in the performance of work under NASA Small Business Technology Transfer Research (STTR) Contract NAS8-98027, and is subject to the provisions of Public Law 96-517 (35 USC 202) and the Code of Federal Regulations 48 CFR 52.227-11 as modified by 48 CFR 1852.227-11, in which the contractor has elected to retain title.

The United States Government has rights in this invention pursuant to Contract No. W-31-109-ENG-38 between the United States Government and the University of Chicago representing Argonne National Laboratory.

FIELD OF THE INVENTION

The instant invention relates generally to a system and method for process fault detection using a statistically based decision test and, in particular, to a system and method for performing high sensitivity surveillance of an asset such as a process and/or apparatus wherein the surveillance is performed using an adaptive sequential probability (ASP) fault detection test comprised of a probability density function model empirically derived from a numerical analysis of asset operating data.

BACKGROUND OF THE INVENTION

Conventional process surveillance schemes are sensitive only to gross changes in the mean value of a process signal or to large steps or spikes that exceed some threshold limit value. These conventional methods suffer from either a large number of false alarms (if thresholds are set too close to normal operating levels) or from a large number of missed (or delayed) alarms (if the thresholds are set too expansively). Moreover, most conventional methods cannot perceive the onset of a process disturbance or sensor signal error that gives rise to a signal below the threshold level or an alarm condition. Most conventional methods also do not account for the relationship between measurements made by one sensor relative to another redundant sensor or between measurements made by one sensor relative to predicted values for the sensor.

Recently, improved methods for process surveillance have developed from the application of certain aspects of artificial intelligence technology. Specifically, parameter estimation methods have been developed using either statistical, mathematical or neural network techniques to learn a model of the normal patterns present in a system of process signals. After learning these patterns, the learned model is used as a parameter estimator to create one or more predicted (virtual) signals given a new observation of the actual process signals. Further, high sensitivity surveillance methods have been developed for detecting process and signal faults by analysis of a mathematical comparison between an actual process signal and its virtual signal counterpart. In particular, such a mathematical comparison is most often performed on a residual error signal computed as, for example, the difference between an actual process signal and its virtual signal counterpart.

Parameter estimation based surveillance schemes have been shown to provide improved surveillance relative to conventional schemes for a wide variety of assets including industrial, utility, business, medical, transportation, financial, and biological systems. However, parameter estimation based surveillance schemes have in general shown limited success when applied to complex processes. Applicants recognize and believe that this is because the parameter estimation model for a complex process will, in general, produce residual error signals having a non-Gaussian probability density function. Moreover, a review of the known prior-art discloses that virtually all such surveillance systems developed to date utilize or assume a Gaussian model of the residual error signal probability density function for fault detection. Hence, a significant shortcoming of the known prior-art is that, inter alia, parameter estimation based surveillance schemes will produce numerous false alarms due to the modeling error introduced by the assumption of a Gaussian residual error signal probability density function. The implication for parameter estimation based surveillance schemes is that the fault detection sensitivity must be significantly reduced to prevent false alarms thereby limiting the utility of the method for process surveillance. An alternative for statistically derived fault detection models is to mathematically pre-process the residual error signals to remove non-Gaussian elements prior to using the residual error signals in the fault detection model; however this approach requires an excess of additional processing and also limits the sensitivity of the surveillance method. Therefore, the implication of assuming a Gaussian residual error signal probability density function for a parameter estimation based surveillance scheme is simply that the system becomes less accurate thereby degrading the sensitivity and utility of the surveillance method.

Many attempts to apply statistical fault detection techniques to surveillance of assets such as industrial, utility, business, medical, transportation, financial, and biological processes have met with poor results in part because the fault detection models used or assumed a Gaussian residual error signal probability density function.

In one specific example, a multivariate state estimation technique based surveillance system for the Space Shuttle Main Engine's telemetry data was found to produce numerous false alarms when a Gaussian residual error fault detection model was used for surveillance. In this case, the surveillance system's fault detection threshold parameters were desensitized to reduce the false alarm rate; however, the missed alarm rate then became too high for practical use in the telemetry data monitoring application.

Moreover, current fault detection techniques for surveillance of assets such as industrial, utility, business, medical, transportation, financial, and biological processes either fail to recognize the surveillance performance limitations that occur when a Gaussian residual error model is used or, recognizing such limitations, attempt to artificially conform the observed residual error data to fit the Gaussian model. This may be attributed, in part, to the relative immaturity of the field of artificial intelligence and computer-assisted surveillance with regard to real-world process control applications. Additionally, a general failure to recognize the specific limitations that a Gaussian residual error model imposes on fault decision accuracy for computer-assisted surveillance is punctuated by an apparent lack of known prior art teachings that address potential methods to overcome this limitation. In general, the known prior-art teaches computer-assisted surveillance solutions that either ignore the limitations of the Gaussian model for reasons of mathematical convenience or attempt to conform the actual residual error data to the artificial Gaussian model, for example, by using frequency domain filtering and signal whitening techniques.

For the foregoing reasons, there is a need for a surveillance system and method that overcomes the significant shortcomings of the known prior-art as delineated hereinabove.

BRIEF SUMMARY OF THE INVENTION

The instant invention is distinguished over the known prior art in a multiplicity of ways. For one thing, an embodiment of the instant invention provides a surveillance system and method having a fault detection model of unconstrained probability density function form and having a procedure suitable for overcoming a performance limiting trade-off between probability density function modeling complexity and decision accuracy that has been unrecognized by the known prior-art. Specifically, an embodiment of the instant invention can employ any one of a plurality of residual error probability density function model forms, including but not limited to a Gaussian form, thereby allowing a surveillance system to utilize the model form best suited for optimizing surveillance system performance.

Moreover, an embodiment of the instant invention provides a surveillance system and method that uses a computer-assisted learning procedure to automatically derive the most suitable form of the residual error probability density function model by observation and analysis of a time sequence of process signal data and by a combination of a plurality of techniques. This ability enables surveillance to be performed by the instant invention with lower false alarm rates and lower missed alarm rates than can be achieved by the known prior-art systems and methods.

Thus, an embodiment of the instant invention provides a surveillance system and method that performs its intended function much more effectively by enabling higher decision accuracy. Further, an embodiment of the instant invention is suitable for use with a plurality of parameter estimation methods thereby providing a capability to improve the decision performance of a wide variety of surveillance systems.

In one embodiment, the instant invention provides a surveillance system and method that creates and uses, for the purpose of process surveillance, a multivariate state estimation technique parameter estimation method in combination with a statistical hypothesis test fault detection method having a probability density function model empirically derived from a numerical analysis of asset operating data.

Particularly, and in one embodiment, the instant invention provides a surveillance system and method for providing surveillance of an asset such as a process and/or apparatus by providing training and surveillance procedures.

In accordance with one embodiment, the instant invention provides a training procedure comprised of a calibrate parameter estimator procedure and a calibrate fault detector procedure.

The calibrate parameter estimator procedure creates a parameter estimation model(s) and trains a parameter estimation model by utilizing training data correlative to expected asset operation and, for example, utilizing a multivariate state estimation technique (MSET) procedure. The calibrate parameter estimator procedure further stores this model as a new parameter estimation model in a process model. The process model may contain one or more parameter estimation models depending upon design requirements. Additionally, each parameter estimation model contained in the process model may be created to implement any one of a plurality of parameter estimation techniques.

The calibrate fault detector procedure makes use of the parameter estimation models to provide estimated values for at least one signal parameter contained in the training data. Generally, the calibrate fault detector procedure will create a separate and distinct fault detection model for each sensor or data signal associated with the asset being monitored for the presence of fault conditions during the surveillance procedure.

The calibrate fault detector procedure includes, for example, a method of fitting a standard Gaussian probability density function (PDF) to a training data residual distribution (computed as a function of estimated process parameters and the training data) and then adding successive higher order terms of a remainder function to the standard Gaussian PDF for the purpose of defining a general PDF that better fits the computed training data residual distribution. Other techniques for fitting a general PDF to the training data are similarly feasible and useful in accordance with the instant invention and include, for example, a technique for fitting a polynomial function to the training data.

The training procedure is completed when all training data has been used to calibrate the process model. At this point, the process model preferably includes parameter estimation models and fault detection models for each sensor or data signal associated with the asset being monitored for the presence of fault conditions during the surveillance procedure. The process model is thereafter used for performing surveillance of the asset.

The surveillance procedure is performed using an adaptive sequential probability (ASP) fault detection test comprised of the general probability density function model empirically derived from a numerical analysis of the asset training data.

The surveillance procedure acquires and digitizes current asset data and then estimates process parameters as a function of the acquired digitized current asset data and the parameter estimation model(s) obtained from the calibrate parameter estimator procedure. Then, fault detection is determined by first computing data residuals as a function of the estimated process parameters and the acquired digitized current asset data and then performing the ASP test(s) as a function of the fault detection models and thus, as a function of the fitted general PDF obtained in the calibrate fault detector procedure. Each ASP test returns one of three possible states: a not null state which rejects the probability that a null hypothesis is true and excepts an alternative hypothesis correlative to unexpected operation of the asset; a null state which accepts the probability that a null hypothesis is true and excepts the null hypothesis correlative to expected operation of the asset; and an in-between state which excepts neither the null hypothesis nor the alternative hypothesis as being true and requires more data to reach a conclusion.

The results of the fault detection procedure are then analyzed according to the instant invention such that an alarm and/or a control action are taken when the analysis determines that the results indicate unexpected operation of the asset.

Hence, the instant invention is distinguished over the known prior-art by providing a surveillance system and method for performing surveillance of an asset by acquiring residual data correlative to expected asset operation; fitting a mathematical model to the acquired residual data; storing the mathematical model in a memory means; collecting a current set of observed signal data values from the asset; using the mathematical mode in a sequential hypotheses test to determine if the current set of observed signal data values indicate unexpected operation of the asset indicative a fault condition; and outputting a signal correlative to each detected fault condition for providing asset surveillance.

Accordingly, the instant invention provides a new, novel and useful surveillance system and method having an adaptive sequential probability fault detection test.

In one embodiment of the instant invention the surveillance system and method includes an unconstrained form of a residual error probability density function model used in said surveillance system's fault detection method.

In one embodiment of the instant invention the surveillance system and method can perform high sensitivity surveillance for a wide variety of assets including industrial, utility, business, medical, transportation, financial, and biological processes and apparatuses wherein such process and/or apparatus asset preferably has at least one pair of redundant actual and/or virtual signals.

In one embodiment of the instant invention the surveillance system and method includes a statistical hypothesis test surveillance decision procedure that uses a fault detection model comprised of a probability density function model of a residual error signal that is of an unconstrained form.

In one embodiment of the instant invention the surveillance system and method creates an improved fault detection model for a process surveillance scheme using recorded operating data for an asset to train a fault detection model.

In one embodiment of the instant invention the surveillance system and method provides an improved system and method for surveillance of on-line, real-time signals, or off-line accumulated signal data.

In one embodiment of the instant invention the surveillance system and method provides an improved system and method for surveillance of signal sources and detecting a fault or error state of the signal sources enabling responsive action thereto.

In one embodiment of the instant invention the surveillance system and method provides an improved system and method for surveillance of signal sources and detecting a fault or error state of the asset processes and apparatuses enabling responsive action thereto.

In one embodiment of the instant invention the surveillance system and method provides an improved decision as to the accuracy or validity for at least one process signal parameter given an observation of at least one actual signal from the asset.

In one embodiment of the instant invention the surveillance system and method provides an improved system and method for ultra-sensitive detection of a fault or error state of signal sources and/or asset processes and apparatuses wherein the parameter estimation technique used for the generation of at least one virtual signal parameter is a multivariate state estimation technique (MSET) having any one of a plurality of pattern recognition matrix operators, training procedures, and operating procedures.

In one embodiment of the instant invention the surveillance system and method provides an improved system and method for ultra-sensitive detection of a fault or error state of signal sources and/or asset processes and apparatuses wherein the parameter estimation technique used for the generation of at least one virtual signal parameter is a neural network having any one of a plurality of structures, training procedures, and operating procedures.

In one embodiment of the instant invention the surveillance system and method provides an improved system and method for ultra-sensitive detection of a fault or error state of signal sources and/or asset processes and apparatuses wherein the parameter estimation technique used for the generation of at least one virtual signal parameter is a mathematical process model having any one of a plurality of structures, training procedures, and operating procedures.

In one embodiment of the instant invention the surveillance system and method provides an improved system and method for ultra-sensitive detection of a fault or error state of signal sources and/or asset processes and apparatuses wherein the parameter estimation technique used for the generation of at least one virtual signal parameter is an autoregressive moving average (ARMA) model having any one of a plurality of structures, training procedures, and operating procedures.

In one embodiment of the instant invention the surveillance system and method provides an improved system and method for ultra-sensitive detection of a fault or error state of signal sources and/or asset processes and apparatuses wherein the parameter estimation technique used for the generation of at least one virtual signal parameter is a Kalman filter model having any one of a plurality of structures, training procedures, and operating procedures.

In one embodiment of the instant invention the surveillance system and method provides a novel system and method for using at least one of a plurality of methods to classify the state of a residual error signal produced by the mathematical difference between two signals, said two signals being either actual and/or predicted signals, into one of at least two categories.

In one embodiment of the instant invention the surveillance system and method provides a novel system and method to classify the state of a residual error signal wherein said classification is made to distinguish between a normal signal and a abnormal signal.

In one embodiment of the instant invention the surveillance system and method provides a novel system and method to classify the state of a residual error signal wherein said classification is performed using a statistical hypothesis test having any one of a plurality of probability density function models, training procedures, and operating procedures.

In one embodiment of the instant invention the surveillance system and method provides a novel system and method to classify the state of a residual error signal wherein said classification is performed using a probability density function model having any one of a plurality of structures, training procedures, and operating procedures.

Accordingly, having thus summarized the invention, it should be apparent that numerous modifications and adaptations may be resorted to without departing from the scope and fair meaning of the present invention as set forth hereinbelow by the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a surveillance system of an embodiment in accordance with the instant invention.

FIG. 2 is a schematic functional flow diagram of an embodiment in accordance with the instant invention.

FIG. 3 is a schematic functional flow diagram of a method and system for training a process model consisting of at least one parameter estimation model and at least one fault detection model using recorded observations of the actual process signals in accordance with the instant invention.

FIG. 4 is a schematic functional flow diagram of a method and system for the fault detection model training procedure in accordance with the instant invention.

FIG. 5 is a schematic functional flow diagram of a preferred method and system for performing surveillance of an asset using at least one parameter estimation model and at least one fault detection model in accordance with the instant invention.

FIG. 6 is a schematic functional flow diagram of a method and system for the fault detection surveillance procedure in accordance with the instant invention.

FIG. 7 is a schematic functional flow diagram of a method and system for performing parameter estimation and fault detection using a redundant sensor.

FIG. 8 is a schematic functional flow diagram of a method and system for performing parameter estimation and fault detection using a generalized parameter estimation model, such as a multivariate state estimation model, a neural network model, an analytical model, or a Kalman filter model.

FIG. 9 is a flow diagram of a training and surveillance procedure in accordance with the instant invention.

FIG. 10 illustrates characteristics of the null and alternate hypotheses for the prior-art sequential probability ratio test (SPRT) mean tests.

FIG. 11 illustrates characteristics of the null and alternate hypotheses for the prior-art SPRT variance tests.

FIG. 12 illustrates acquired operating data, estimated parameter data, and residual error data for a typical Space Shuttle Main Engine accelerometer.

FIG. 13 illustrates a probability density function of the residual error data for a typical Space Shuttle Main Engine accelerometer with comparison to a Gaussian probability density function.

FIG. 14 illustrates an un-optimized one-term expansion probability density function model of the residual error data for a typical Space Shuttle Main Engine accelerometer with comparison to the actual residual error data and a Gaussian probability density function.

FIG. 15 illustrates an un-optimized two-term expansion probability density function model of the residual error data for a typical Space Shuttle Main Engine accelerometer with comparison to the actual residual error data and a Gaussian probability density function.

FIG. 16 illustrates an un-optimized three-term expansion probability density function model of the residual error data for a typical Space Shuttle Main Engine accelerometer with comparison to the actual residual error data and a Gaussian probability density function.

FIG. 17 lists the root mean square errors for five different un-optimized probability density function models computed for each of six Space Shuttle Main Engine accelerometers used for feasibility testing of a preferred embodiment in accordance with the instant invention.

FIG. 18 lists the root mean square errors for five different optimized probability density function models computed for each of six Space Shuttle Main Engine accelerometers used for feasibility testing of a preferred embodiment in accordance with the instant invention.

FIG. 19 illustrates the optimized two-term expansion probability density function model of the residual error data for a typical Space Shuttle Main Engine accelerometer with comparison to the actual residual error data and a Gaussian probability density function.

FIG. 20 illustrates the empirical false alarm rates for feasibility testing of a preferred embodiment in accordance with the instant invention with comparison to the prior art SPRT method.

DETAILED DESCRIPTION OF THE INVENTION

Considering the drawings, wherein like reference numerals denote like parts throughout the various drawing figures, reference numeral 10 is directed to the system according to the instant invention.

In its essence, and referring to FIGS. 1 and 2, the system 10 is generally comprised of a method and apparatus for performing high sensitivity surveillance of a wide variety of assets including industrial, utility, business, medical, transportation, financial, and biological processes and apparatuses wherein such process and/or apparatus asset preferably, has at least one distinct measured or observed signal or sequence comprised of characteristic data values which are processed by the system 10 described herein for providing ultra-sensitive detection of the onset of sensor or data signal degradation, component performance degradation, and process operating anomalies. The system 10 includes a training procedure 30 carried out on a computer 22 such that a process model 50 of an asset 12 (e.g., a process and/or apparatus) is stored in an associated memory means 24 after being learned from historical operating data using at least one of a plurality of computer-assisted procedures in accordance with the instant invention. The historical operating data includes a set or range of observations from expected or typical operation of the asset 12 that are acquired and digitized by a data acquisition means 20 and stored in a memory means 24 as training data 34 by using any combination of electronic data acquisition hardware and signal processing software 20 known to those having ordinary skill in the art, and informed by the present disclosure. As will be delineated in detail infra, one hallmark of the instant invention is the process model 50 for the asset 12 that is derived during the training procedure 30 .

The system 10 further includes a surveillance procedure 60 wherein the stored process model 50 is used for high sensitivity computer-assisted surveillance of the asset 12 for the purpose of determining whether a process fault or failure necessitates an alarm or control action. The process model 50 is comprised of a parameter estimation model 52 or collection of parameter estimation models 52 as necessary to provide an estimated value for each sensor or data signal 14 of asset 12 to be monitored for the presence of fault conditions during the surveillance procedure 60 . The process model 50 is further comprised of a fault detection model 54 or collection of fault detection models 54 such that at least one fault detection model 54 is provided for each sensor or data signal 14 of asset 12 to be monitored for the presence of fault conditions during the surveillance procedure 60 . The fault detection model 54 is another hallmark of the instant invention and will be delineated in further detail hereinbelow.

Process Model Training Procedure:

More specifically, and referring to FIGS. 1 and 3, the training procedure 30 of the system 10 includes a method and apparatus for training or preparing the process model 50 using historical operating data from the asset 12 that has been acquired by the data acquisition means 20 using any combination of conventional electronic data acquisition hardware and signal processing software as is well known in the art. The historical operating data is acquired in digital format and stored in memory means 24 using a data storage procedure 32 to create a training data set 34 .

The training data set 34 includes at least N discrete observations of the asset 12 wherein each single observation, herein denoted Xobs, is comprised of a vector of data values for each signal parameter to be included in the process model 50 . For the purposes of the training procedure 30 , the number of observations, N, acquired is at least great enough to adequately bound the operating state space of the asset 12 . Thus, the training data set 34 provides a representative sample of the signals produced by the asset 12 during all normal modes of operation.

Upon acquiring the training data set 34 , the unique training procedure 30 can be implemented in accordance with instant invention.

The unique training procedure 30 is comprised of a calibrate parameter estimator procedure 36 and a calibrate fault detector procedure 38 . The calibrate parameter estimator procedure 36 creates the parameter estimation model 52 and trains the parameter estimation model 52 using the training data 34 . The calibrate parameter estimator procedure 36 further stores this model as a new parameter estimation model 52 in the process model 50 .

The process model 50 may contain one or more parameter estimation models 52 depending upon the requirements of the approach taken by a designer. Continuing to refer to FIG. 3, the training procedure 30 may be, in general, performed using any parameter estimation method suitable for defining a parameter estimation model 52 useful for estimating the values of one or more process signals. Methods suitable for the calibrate parameter estimator procedure 36 include, but are not limited to, a plurality of redundant sensor techniques, a plurality of multivariate state estimation techniques, a plurality of neural network techniques, a plurality of mathematical model techniques, a plurality of autoregressive moving average techniques, and a plurality of Kalman filter techniques. Each parameter estimation model 52 contained in the process model 50 may be created to implement any one of a plurality of parameter estimation techniques. Further, the parameter estimation technique implemented for an individual parameter estimation model 52 is not constrained to be the same as the parameter estimation technique implemented for any other parameter estimation model 52 contained in the process model 50 .

One example of the calibrate parameter estimator procedure 36 would be the computation of a bias term between two redundant sensors wherein the parameter estimation model 52 used for estimating the value of one sensor during the surveillance procedure 60 consisted of summing the observed value of a second redundant sensor with a bias term computed during the training procedure 30 as the mean difference between the two sensor values over the training data set 34 . More sophisticated examples of the training procedure 30 using multivariate state estimation techniques will be described herein below.

Still referring to FIG. 3, the calibrate fault detector procedure 38 makes use of the parameter estimation models 52 to provide estimated values for at least one signal parameter contained in the training data. Generally, the calibrate fault detector procedure 38 will create a separate and distinct fault detection model 54 for each sensor or data signal of asset 12 to be monitored for the presence of fault conditions during the surveillance procedure 60 . As delineated infra, one hallmark of the instant invention is the fault detection model 54 element of the process model 50 for the asset 12 that is derived during the training procedure 30 . In particular, the instant invention encompasses a statistical hypothesis test type of fault detection model 54 having novel and unique characteristics and calibration procedures described herein including but not limited to having a probability density function model empirically derived from a numerical analysis of asset operating data.

Continuing to refer to FIG. 3, the training procedure 30 is completed when all training data 34 has been used to calibrate the process model 50 . At this point, the process model 50 includes parameter estimation models 52 and fault detection models 54 for each sensor or data signal of asset 12 to be monitored for the presence of fault conditions during the surveillance procedure 60 . The process model array 50 is thereafter useful for performing surveillance of the asset 12 .

Referring to FIG. 4, the training procedure 30 is illustrated in additional detail. A designer 16 initializes the calibrate parameter estimators procedure 36 by specifying a set of parameter estimator methods and settings 40 . The parameter estimator methods and settings 40 are then used to operate on the training data 34 via a nominal signal behavior modeling procedure 42 , for example using an MSET training procedure as described herein in detail, in order to create the parameter estimation models 52 , which are stored in the process model 50 .

Still referring to FIG. 4, the training procedure 30 next proceeds to the calibrate fault detectors procedure 38 wherein the parameter estimation models 52 are an input to the procedure. The designer 16 initializes the calibrate fault detectors procedure 38 by specifying a set of fault detector methods and settings 46 . Next, the estimate process parameters procedure 64 operates the parameter estimation models 52 over the training data 34 to generate an estimated value for each monitored signal value contained in the training data 34 . It is important that the estimate process parameters procedure 64 used in the calibrate fault detectors procedure 38 be the same estimate process parameters procedure 64 that will later be used in the surveillance procedure 60 (reference FIGS. 1 and 2 for surveillance procedure 60 ). Next, A compute training data residuals procedure 44 calculates the training data residuals for each monitored signal, which are calculated as the difference between the training data value and the corresponding estimated data value for each monitored signal. The training data residuals are next used by a compute nominal residual probability density function (PDF) procedure 48 to create a fault detection model 54 for each monitored signal. In accordance with the instant invention, the fault detection models 54 is typically comprised of mathematical descriptions of the probability density function that best characterizes or best fits the training data residual for the monitored signal. The training data is presumed to accurately characterize the expected normal operating states of the asset 12 . Therefore, the training data residuals are characteristic of the expected normal deviations between the observed signal values and the values estimated using the parameter estimation models 52 . The fault detection models 54 are stored in the process model 50 thereby completing the training procedure.

One hallmark of the instant invention is the method and system for computing the fault detection models 54 by the means of the compute nominal residual probability density function (PDF) procedure 48 . As will be described mathematically herein below, the compute nominal residual probability density function (PDF) procedure 48 fits a general open-ended probability function to the training data residuals and employs this fitted function when implementing a herein named Adaptive Sequential Probability (ASP) method and system for computing the fault detection model 54 and thereafter employing said fault detection model 54 for the purpose of performing a fault detection procedure 66 of the surveillance procedure 60 .

Surveillance Procedure:

Referring to FIG. 5, the surveillance procedure 60 is comprised of acquiring successive vectors of current operating data and determining for each such observation vector whether the current operating data is indicative of a fault or failure of the asset 12 . The surveillance procedure 60 further includes implementing an alarm or control action 70 for the purpose of notifying an operator and/or taking a corrective action in response to a detected fault or failure of the asset 12 . The surveillance procedure 60 is in general an open-ended data acquisition and analysis loop that continues until such time as the operator chooses to terminate the surveillance 74 .

More specifically, and referring to FIG. 5, the surveillance procedure 60 begins 58 with an acquire current operating data procedure 62 that employs the data acquisition and digitization means 20 (FIG. 1) to acquire a current set of signal data from the monitored asset 12 . The current set of signal data is provided to the estimate process parameters procedure 64 that uses the parameter estimation models 52 to estimate values for one or more of the current signal data values. The observed and estimated data are next provided to a perform fault detection procedure 66 that uses the fault detection models 54 to determine whether a fault is found 68 in the current operating data. If a fault is found 68 is true, an alarm and/or control action 70 is taken. Upon completing the fault found procedure 68 , the surveillance procedure 60 then repeats for the next available set of signal data for as long as a surveillance complete decision procedure 72 determines that additional surveillance data are available or terminates at surveillance complete step 74 when no more surveillance data are available or when terminated by an operator.

Referring now to FIG. 6, the perform fault detection procedure 66 of surveillance procedure 60 is illustrated in additional detail. For each current set of signal data values acquired the estimate process parameters procedure 64 uses the parameter estimation models 52 to estimate values for one or more of the current signal data values. The compute data residuals procedure 76 performs a mathematical transformation on the acquired and estimated values to produce a current set of residual data values. Said mathematical transformation is most typically a simple mathematical difference, however, any appropriate transformation may be used including transformations that remove correlated and uncorrelated noise from the residual data values. The residuals produced and transformed in the compute data residuals procedure 76 are next processed by a perform ASP tests procedure 78 that uses the fault detection models 54 to produce a ASP fault indication that is a hallmark of the method and system of the instant invention. Next, the fault found decision procedure 68 is performed on the basis of the ASP fault indication results produced by the perform ASP tests procedure 78 . The fault found decision procedure 68 may have any one of a plurality of structures and procedures, including but not limited to methods and systems to perform false alarm filtering by means of observing a time series of ASP fault indication results for the purposes of determining the actual presence of a fault. In one preferred embodiment of the instant invention, a conditional probability fault found decision procedure 68 is used to perform said false alarm filtering.

Continuing to refer to FIG. 6, the estimate process parameters procedure 64 uses the parameter estimation models 52 to estimate values for one or more of the current signal data values wherein the parameter estimation method used may have any one of a plurality of structures and procedures, including but not limited to, a plurality of redundant sensor techniques, a plurality of multivariate state estimation techniques, a plurality of neural network techniques, a plurality of mathematical model techniques, a plurality of autoregressive moving average techniques, and a plurality of Kalman filter techniques.

Referring to FIG. 7, one possible redundant sensor technique for the estimate process parameters procedure 64 is illustrated. The acquire current operating data procedure 62 is used to acquire current signal data values from signals 14 monitored from asset 12 via sensors 18 . The estimated value for a first redundant sensor signal is computed using a mathematical transformation on the acquired value of a second redundant sensor signal. Said mathematical transformation is the estimate process parameters procedure 64 that in this case may be a simple equivalence or may include biasing, de-noising or other signal processing. The compute data residuals procedure 76 is then performed followed by the perform ASP tests procedure 78 as described hereinabove and further delineated hereinbelow.

Referring to FIG. 8, one possible multivariable parameter estimation technique for the estimate process parameters procedure 64 is illustrated. The acquire current operating data procedure 62 is used to acquire current signal data values from signals 14 monitored from asset 12 via sensors 18 . The estimated value for one or more sensor signals is computed using a mathematical transformation on the acquired values of one or more sensor signals. Said mathematical transformation is the estimate process parameters procedure 64 that in this case may implement any feasible parameter estimation technique or procedure, including but not limited to a plurality of multivariate state estimation techniques, a plurality of neural network techniques, a plurality of mathematical model techniques, and a plurality of Kalman filter techniques. The compute data residuals procedure 76 is then performed followed by the perform ASP tests procedure 78 as described hereinabove and further delineated hereinbelow.

Referring again to FIG. 6, the usefulness of the instant invention is, inter alia, the improvement achieved in the accuracy of the fault decision procedure 68 that results from the improvement achieved in the accuracy of perform fault detection procedure 66 made possible by the novel perform ASP tests procedure 78 that is a hallmark of the instant invention. Improving the accuracy of the fault decision procedure 68 accomplishes a reduction in the number of false alarms sent to a process operator or control system that can in turn result in an erroneous alarm or control action by the alarm or control action procedure 70 . Further, improving the accuracy of the fault decision procedure 68 accomplishes a reduction in the number of missed alarms thereby accomplishing more timely alarm or control action by the alarm or control action procedure 70 . The instant invention thereby enables improved operating safety, improved efficiency and performance, and reduced maintenance costs for a wide variety of industrial, utility, business, medical, transportation, financial, and biological processes and apparatuses wherein such process and/or apparatus asset 12 has at least one characteristic data signal suitable for surveillance.

In use and operation, and in one preferred form, FIGS. 1 and 9 outline a general surveillance procedure of the system 10 when employing the novel fault detection model 54 contained in the process model 50 and the accompanying novel fault detection procedure 66 having the perform ASP tests procedure 78 . In a typical surveillance procedure, the asset 12 is the source of at least one process signal 14 that is acquired and digitized using conventional data acquisition means 20 for providing the data acquisition procedure for the purpose of computer-assisted surveillance. The digitized signal data is generally evaluated using a computer 22 having computer software modules implementing the estimate process parameters procedure 64 , and the perform fault detection procedure 66 . The estimate process parameters procedure 64 is used to produce an estimated signal value for at least one process signal 14 emanating from the asset 12 . The estimate process parameters procedure 64 in general makes use of the process model 50 stored in a memory means 24 associated with the computer 22 to produce the estimated signal values. The estimated signal values are then generally evaluated using the perform fault detection procedure 66 to identify faults or operating anomalies in the asset 12 . The results of the fault detection evaluation are thereafter communicated by a conventional communications means 26 (as is known to those having ordinary skill in the art, and informed by the present disclosure) to an operator console 80 and/or automated process control system 82 for possible alarm and/or control action.

The computer 22 along with the associated memory means 24 can also be employed to perform the training and surveillance procedures 30 , 60 as delineated hereinabove and to produce and store all the data associated with these procedures, for example, the historical training data, designer defined settings, and the process model.

Multivariate State Estimation Technique (MSET) for Estimate Process Parameters Procedure:

In one embodiment of the instant invention, the estimate process parameters procedure 64 uses a multivariate state estimation technique (MSET) procedure having an MSET parameter estimation model 52 structure. The US Department of Energy's Argonne National Laboratory originally developed the implementation of MSET described herein for surveillance of sensors and components in nuclear power plant applications. However, other implementations of a multivariate state estimation technique are possible and useful in conjunction with the instant invention. MSET is in general a statistically derived parameter estimation algorithm that uses advanced pattern recognition techniques to measure the similarity or overlap between signals within a defined domain of asset operation (set of asset operating states). MSET “learns” patterns among the signals by numerical analysis of historical asset operating data. These learned patterns or relationships among the signals are then used to identify the learned state that most closely corresponds with a new signal data observation. By quantifying the relationship between the current and learned states, MSET estimates the current expected response of the asset signals. MSET then uses threshold comparisons or a form of statistical hypothesis testing, such as a Sequential Probability Ratio Test (SPRT) as disclosed in U.S. Pat. No. 5,459,675 which is hereby incorporated by reference in its entirety or the Adaptive Sequential Probability (ASP) procedure that is a hallmark of the instant invention to compare the current estimated value of a signal with its observed value. The MSET procedure provides an accurate and widely applicable method to estimate the operating signal values for an asset. However, other implementations of the parameter estimation procedure are possible and useful in conjunction with the instant invention.

An MSET model is created for the asset 12 using the MSET training algorithms to learn the inherent data relationships within a set of historical asset operating data. The trained MSET model is then used with the MSET parameter estimation and fault detection algorithms to perform the process surveillance function when presented with a new observation of signal data values. The following sections will first provide a mathematical overview of the MSET algorithms and procedures useful for training a parameter estimation model and for using this trained model for process surveillance. The description is followed by a detailed description of a preferred embodiment of the instant invention using a novel Adaptive Sequential Probability (ASP) procedure for fault detection during process surveillance.

MSET Training and Parameter Estimation Procedures:

The MSET methods are generally described in the following two US Government documents produced and maintained by the US Department of Energy's Argonne National Laboratory, Argonne, Ill., disclosure of which is incorporated in its entirety herein by reference.

J. P. Herzog, S. W. Wegerich, R. M. Singer, and K. C. Gross, “Theoretical Basis of the Multivariate State Estimation Technique (MSET),” Argonne National Laboratory, ANL-NT-49, December 1997.

J. P. Herzog, S. W. Wegerich, K. C. Gross, and R. M. Singer, “MSET: Code Structure and Interface Development Guide,” ANL-NT-48, August 1997.

The MSET algorithm uses pattern recognition with historical operating data from an asset to generate a parameter estimation model. If data is collected from a process over a range of operating states, this data can be arranged in matrix form, where each column vector (a total of m) in the matrix represents the measurements made at a particular state. Thus, this matrix will have the number of columns equal to the number of states at which observations were made and the number of rows equal to the number of measurements (a total of n signal data values) that were available at each observation. We begin by defining the set of measurements taken at a given time t _j as an observation vector X(t _j ),
{right arrow over (X)} ( t _j )=[ x ₁ ( t _j ), x ₂ ( t _j ), . . . , x _n ( t _j )] ^T (E1)

where x _i (t _j ) is the measurement from signal i at time t _j . We then define the data collection matrix as the process memory matrix D:

$\begin{array}{cc}\overleftrightarrow{D}=\left[\begin{array}{cccc}{d}_{1,1}& d_{1,2}& \dots & d_{1,m}\\ d_{2,1}& d_{2,2}& \dots & d_{2,m}\\ ⋮& ⋮& ⋮& ⋮\\ d_{n,1}& d_{n,2}& \dots & d_{n,m}\end{array}\right]\equiv \left[\stackrel{->}{X}\left(t_1\right),\stackrel{->}{X}\left(t_2\right),\dots ,\stackrel{->}{X}\left(t_m\right)\right]& \left(\mathrm{E2}\right)\end{array}$

Each of the column vectors (X(t _j )) in the process memory matrix represents an operating state of the asset. Any number of observation vectors can be assigned to the process memory matrix. Training an MSET model includes collecting enough unique observation vectors from historical operation of the asset during normal conditions such that the process memory matrix encompasses the full dynamic operating range of the asset. Computation of the D matrix is the first of three steps in the method for training an MSET model based on historical operating data.

One of at least two algorithms is used by MSET to select the vectors in the D matrix. The MinMax algorithm extracts vectors that bound the vector space defined by the training data and returns the smallest process memory matrix that will produce an effective system model (see also U.S. Pat. No. 5,764,509 which is hereby incorporated by reference in its entirety). A vector ordering algorithm selects and includes representative vectors from the inner regions of the vector space producing a more accurate system model.

Once the process memory matrix has been constructed, MSET is used to model the dynamic behavior of the system. For each current observation of the system (Xobs), MSET compares the observation vector to the stored operating states to calculate an estimate of the process parameter values. The parameter estimate of the current process state (Xest) is an n-element vector that is given by the product of the process memory matrix and a weight vector, W:
{right arrow over (X)} _est =·{right arrow over (W)} (E3)

The weight vector represents a measure of similarity between the estimate of the current state and the process memory matrix. To obtain the weight vector, we minimize the error vector, R, where:
{right arrow over (R)}={right arrow over (X)} _obs −{right arrow over (X)} _est (E4)

The error is minimized for a given state when:
{right arrow over (W)} =( ^T {circle around (x)} ) ⁻¹ ·( ^T {circle around (x)}{right arrow over (X)} _obs ) (E5)

This equation represents a “least squares” minimization when the pattern recognition operator {circle around (x)} is the matrix dot product. Several advanced pattern recognition operators have been defined that provide excellent parameter estimation performance. Pattern recognition operators used by MSET include, but are not limited to, the System State Analyzer (SSA) method (see also U.S. Pat. No. 4,937,763 which is hereby incorporated by reference in its entirety), the Bounded Angle Ratio Test (BART) method (see also U.S. Pat. No. 5,987,399 which is hereby incorporated by reference in its entirety), the Vector Pattern Recognizer (VPR) method, the Vector Similarity Evaluation Technique (VSET) method, and the Probabilistic State Estimation Method (PSEM).

Once the weight vector is found, the resulting current state estimate of the system (i.e., the parameter estimate vector) is given by:
{right arrow over (X)} _est ={umlaut over (D)}·({umlaut over (D)} ^T {circle around (x)}{umlaut over (D)} ) ⁻¹ ·({umlaut over (D)} ^T {circle around (x)}{right arrow over (X)} _obs ) (E6)

The first application of the pattern recognition operator in equation E6 (D ^T {circle around (x)}D) involves a comparison between the row vectors in the D ^T matrix and each of the column vectors in the D matrix. If we define G=D ^T {circle around (x)}D, then G, the similarity matrix, is an m by m matrix. The element in the i-th row and j-th column of the matrix (g _i,j ) represents a measure of the similarity between the i-th and j-th column vectors (i.e., memorized states) in the process memory matrix. The second application of the pattern recognition operator in equation E6 (D ^T {circle around (x)}X _obs ) involves a comparison between the row vectors in the D ^T matrix and each of the elements in the observation vector Xobs. If we define A=D ^T {circle around (x)}X _obs , then A, the similarity vector, is an m by 1 vector. Each element in the similarity vector is a measure of the similarity between the observation vector and the i-th column vector (i.e., memorized state) in the process memory matrix.

Note that the similarity matrix is a function of the process memory matrix only. Thus, the similarity matrix and its inverse Ginv=(D ^T {circle around (x)}D) ⁻¹ can be calculated as soon as the process memory matrix has been derived thereby making the application of MSET to an on-line surveillance system more computationally efficient. Computation of the Ginv matrix initializes the parameter estimation model and completes the second of three steps in the procedure for training an MSET model based on historical operating data.

The third and final step in the MSET training procedure includes analyzing the historical training data using equation E4 to produce a residual error vector, R, for each observation vector in the training data. The collection of residual error vectors comprises the training data residuals necessary for training the fault detection model 54 using any one of a plurality of techniques, including but not limited to the SPRT technique, and the novel ASP technique that is a hallmark of the instant invention.

The Sequential Probability Ratio Test (SPRT) technique is a statistical hypothesis test fault detection algorithm historically used for MSET process surveillance. The SPRT technique is described in U.S. Pat. No. 5,459,675, which is incorporated herein by reference in its entirety. The SPRT analyzes a sequence of discrete residual error values from a signal to determine whether the sequence is consistent with normal signal behavior or with some other abnormal behavior. When the SPRT reaches a decision about the current signal behavior, e.g., that the signal is behaving normally or abnormally, the decision is reported and the test continues analyzing the signal data. For any SPRT, signal behavior is defined to be normal when the signal data adheres to a Gaussian probability density function (PDF) with mean 0 and variance σ ² . Normal signal behavior is referred to as the null hypothesis, H ₀ . MSET employs four specific SPRT hypothesis tests. Each test determines whether current signal behavior is consistent with the null hypothesis or one of four alternative hypotheses. The four tests are known as the positive mean test, the negative mean test, the nominal variance test, and the inverse variance test. For the positive mean test, the corresponding alternative hypothesis, H ₁ , is that the signal data adhere to a Gaussian PDF with mean +M and variance σ ² . For the negative mean test, the corresponding alternative hypothesis, H ₂ , is that the signal data adheres to a Gaussian PDF with mean −M and variance σ ² . For the nominal variance test, the corresponding alternative hypothesis, H ₃ , is that the signal data adheres to a Gaussian PDF with mean 0 and variance Vσ ² . For the inverse variance test, the corresponding alternative hypothesis, H ₄ , is that the signal data adheres to a Gaussian PDF with mean 0 and variance σ ² /V. The user-assigned constants M and V control the sensitivity of the tests.

Limitations of the SPRT Fault Detector Training and Surveillance Method and System:

One significant shortcoming of the SPRT technique is found in the assumptions underlying its mathematical formulation. Specifically, the SPRT technique presumes that the residual error signals adhere to a Gaussian probability density function. For residual error signals that are non-Gaussian, the fault detector false alarm rates and/or missed alarm rates specified by the designer are not accomplished by the SPRT procedure thereby degrading the fault decision accuracy of the asset control and/or surveillance system. The novel ASP technique of the instant invention specifically removes the assumption that the residual error signals adhere to a Gaussian probability density function. The ASP technique implements any one of a plurality of methods to numerically fit a probability density function to the observed residual error signal distribution that is characteristic of normal asset operation. The derived probability density function is then used to perform a dynamic statistical hypothesis test thereby achieving the fault detector false alarm and missed alarm rates specified by the designer and improving the fault decision accuracy of the asset control and/or surveillance system.

Fault Detection Using Statistical Hypothesis Test Procedures:

The general theory underlying the statistical hypothesis test will now be delineated below. Next, the SPRT implementation of a dynamic statistical hypothesis test will be described. Finally, the novel ASP implementation of a dynamic statistical hypothesis test for non-Gaussian residual error signals will be delineated in detail along with a delineation of its reduction to practice.

Bayes' Rule for a Single Observation:

Statistical decision problems in which there are just two possible outcomes constitute an important class called binary hypothesis testing problems. The possible states of a system are called hypotheses and each individual state of the system is termed a simple hypothesis. A simple hypothesis is a complete specification of a probability distribution for the system (i.e., the distribution of possible observations or samples from the system). The “hypothesis” being tested is that the particular distribution is the correct one.

The basic operation in a binary hypothesis test is to evaluate the veracity of a hypothesis, H, given a piece of evidence or observation, e, from the system being studied. Because of the unpredictability or element of chance inherent in the system, the test deals with the probabilities that events occur or that hypotheses are true. The probability that a hypothesis is true given a piece of evidence is written as P(H|e). The notation identifies a conditional probability—namely the probability that the hypothesis is true under the condition that the event has occurred with absolute certainty. Bayes' well known inversion rule, as described in Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, by Judea Pearl, Morgan Kaufmann Publishers, Inc., San Mateo, Calif., 1988, Second Edition at pages 29 through 39, provides a means for evaluating the conditional probability of a hypothesis, P(H|e), which is also known as the posterior probability,

$\begin{array}{cc}P\left(H❘e\right)=\frac{P\left(e❘H\right)P\left(H\right)}{P\left(e\right)}& \left(\mathrm{E7}\right)\end{array}$

P(e|H) is the probability that the observation would occur if the hypothesis is true. P(H) is the probability that the hypothesis is true before any observations of the system have been made, also known as the prior probability. The denominator, P(e), is the unconditional probability that the observation occurs.

Basic axioms of set theory can be used to prove the following identity for two events R and S: R=(R∩S)∪(R∩ S), where S is the converse of event S. In probability theory, the analog of this identity is
P ( R )= P ( R,S )+ P ( R, S ), (E8)

where the notation P(R,S) is used represent the probability of the joint event R∩S. The multiplication law states that the probability of two events occurring jointly can be expressed as a function of the conditional probability of one event based on the other,
P ( R,S )= P ( R|S ) P ( R ). (E9)

If R and S are independent events, then P(R|S)=P(R) and the multiplication law simplifies to P(R,S)=P(R) P(S).

Bayes' rule can be simplified by eliminating the denominator in equation E7. Combining equations E8 and E9, and substituting e for R and H for S, the denominator can be written as
P ( e )= P ( e|H ) P ( H )+ P ( e| H ) P ( H ) (E10)

Therefore, Bayes' rule becomes

$\begin{array}{cc}P\left(H❘e\right)=\frac{P\left(e|H\right)P\left(H\right)}{P\left(e|H\right)P\left(H\right)+P\left(e❘H\right)P\left(H\right)}.& \left(\mathrm{E11}\right)\end{array}$

The power of Bayesian techniques comes primarily from the fact that in causal reasoning the relationship P(e|H) is local, namely, given that H is true, the probability of e can be estimated naturally and is not dependent on many other propositions. For instance, given that the measurements from an experiment adhere to a particular PDF, the probability that any single measurement will occur is easily computed.

The complementary form of Bayes' rule provides the posterior probability for the converse of the hypothesis. It is evaluated by substituting H for each instance of H in equation E11 and noting that ( H)=H:]

$\begin{array}{cc}P\left(⫬H❘e\right)=\frac{P\left(e❘⫬H\right)P\left(⫬H\right)}{P\left(e❘H\right)P\left(H\right)+P\left(e❘⫬H\right)P\left(⫬H\right)}.& \left(\mathrm{E12}\right)\end{array}$

An alternate form of Bayes' rule is produced by dividing it (i.e., equation E11) with its complementary form (i.e., equation E12) to obtain

$\begin{array}{cc}\frac{P\left(H❘e\right)}{P\left(⫬H❘e\right)}=\frac{P\left(e❘H\right)P\left(H\right)}{P\left(e❘⫬H\right)P\left(⫬H\right)}.& \left(\mathrm{E13}\right)\end{array}$

This form of Bayes' rule is further manipulated by first defining the prior odds on hypothesis H as

$\begin{array}{cc}O\left(H\right)=\frac{P\left(H\right)}{P\left(⫬H\right)},& \left(\mathrm{E14}\right)\end{array}$

the likelihood ratio as

$\begin{array}{cc}L\left(e❘H\right)=\frac{P\left(e❘H\right)}{P\left(e❘⫬H\right)},& \left(\mathrm{E15}\right)\end{array}$

and the posterior odds on H as

$\begin{array}{cc}O\left(H❘e\right)=\frac{P\left(H❘e\right)}{P\left(⫬H❘e\right)}.& \left(\mathrm{E16}\right)\end{array}$

Bayes' rule then specifies the posterior odds as the product of the likelihood ratio and the prior odds,
O ( H|e )= L ( e|H ) O ( H ) (E17)

Bayes' Rule for a Time Series:

The formulation above specifies Bayes' rule for a single observation and a binary hypothesis. For the application of binary hypothesis tests to real world signals, the formulation must be able to handle a sequence of discrete observations. This is accomplished by beginning with a single observation and successively updating Bayes' rule for each successive observation. Let the sequence {Y _n } be an ordered set of n elements, {Y _n }=y ₁ , y ₂ , . . . , y _n , in which the elements are observations of the signal made at n discrete moments in time such that t ₁ <t ₂ < . . . <t _n . Bayes' rule for the first observation (y ₁ ) in the time series is

$\begin{array}{cc}P\left(H❘y_1\right)=\frac{P\left(y_1❘H\right)P\left(H\right)}{P\left(y_1\right)}.& \left(\mathrm{E18}\right)\end{array}$

Adding the second observation from the time series, Bayes' rule for the joint event y ₁ ∩y ₂ is

$\begin{array}{cc}P\left(H❘y_1,y_2\right)=\frac{P\left(y_1,y_2❘H\right)P\left(H\right)}{P\left(y_1,y_2\right)}.& \left(\mathrm{E19}\right)\end{array}$

With the aid of the multiplication law (equation E9), the joint event probabilities are converted to conditional probabilities so that the right hand side of this equation can be rewritten as

$\begin{array}{cc}P\left(H❘y_1,y_2\right)=\frac{P\left(y_2❘y_1,H\right)P\left(y_1❘H\right)P\left(H\right)}{P\left(y_2❘y_1\right)P\left(y_1\right)}.& \left(\mathrm{E20}\right)\end{array}$

Note that the probability of the joint event y ₁ ∩y ₂ is written as P(y ₁ , y ₂ )=P(y ₂ |y ₁ )P(y ₁ ) instead of the equivalent P(y ₁ , y ₂ )=P(y ₁ |y ₂ )P(y ₂ ) because of the temporal dependency of the data. The second form of the multiplication law reduces to P(y ₁ , y ₂ )=P(y ₁ )P(y ₂ ) because earlier events (e.g., y ₁ ) in a time series cannot be dependent on later events (e.g., y ₂ ).

The multiplication law is used for each successive observation in the time series to derive the form of Bayes' rule for the joint event (y ₁ ∩y ₂ ∩ . . . ∩y _n ):

$\begin{array}{cc}P\left(H❘y_1,y_2,\dots ,y_n\right)=\frac{P\left(y_n❘y_{n-1},\dots ,y_1,H\right)\dots P\left(y_2❘y_1,H\right)P\left(y_1❘H\right)P\left(H\right)}{P\left(y_n❘y_{n-1},\dots ,y_1\right)\dots P\left(y_2❘y_1\right)P\left(y_1\right)}.& \left(\mathrm{E21}\right)\end{array}$

Since the probabilities for each of the later observations in the time series are conditioned on earlier observations, Bayes' rule is difficult to solve in the general case. But if the observations in the series are independent of each other (i.e., random), then the probabilities will be dependent on the hypothesis only. In this case the conditional probability for the ith observation, P(y _i |y _i−1 , . . . , y ₁ , H), is simply P(y _i |H). Thus, Bayes' rule for independent observations in a time series is

$\begin{array}{cc}P\left(H❘y_1,y_2,\dots ,y_n\right)=\frac{P\left(y_n❘H\right)\dots P\left(y_1❘H\right)P\left(H\right)}{P\left(y_n\right)\dots P\left(y_1\right)}=P\left(H\right)\prod _{i=1}^n\frac{P\left(y_i❘H\right)}{P\left(y_i\right)}& \left(\mathrm{E22}\right)\end{array}$

If an explicit time dependency can be established for the observations in the time series (i.e., a function is found that relates earlier events to later events), then the general form of Bayes' rule (equation E21) can be used to develop failure detection models for serially-correlated signals. However, the residual signals formed by the difference between the observed and estimated signal values are in general random signals; thus, Bayes' rule for random time series is used as the basis for the fault detection models.

Dividing by the complementary form of Bayes' rule for random time series and utilizing the definition of the posterior odds, prior odds, and likelihood ratio from above, an alternate form of Bayes' rule for a time series is developed:

$\begin{array}{cc}O\left(H❘y_1,y_2\dots ,y_n\right)=O\left(H\right)\prod _{i=1}^{n}L\left(y_i❘H\right).& \left(\mathrm{E23}\right)\end{array}$

If we take the logarithm of this equation, the incremental nature of Bayesian formulation becomes more apparent. Equation E24 shows the log of the likelihood ratio as a weight, carried by each observation in the sequence, which additively sways the belief in the hypothesis one way or the other.

$\begin{array}{cc}\ln \left[O\left(H❘y_1,y_2,\dots ,y_n\right)\right]=\ln \left[O\left(H\right)\right]+\sum _{i=1}^n\ln \left[L\left(y_i❘H\right)\right]& \left(\mathrm{E24}\right)\end{array}$

Sequential Hypothesis Tests:

Wald first presented and studied the following sequential test of a simple hypothesis against a simple alternative, as described in Sequential Analysis, by A. Wald, John Wiley and Sons, Inc., New York, 1947. Let H ₀ be a specific probability density function called the null hypothesis. Then the probability that the time series {Y _n } contains samples drawn from H ₀ is P(y ₁ , y ₂ , . . . , y _n |H ₀ ). Let H _j be a different probability density function called the alternative hypothesis. Then the probability that the time series {Y _n } contains samples drawn from H _j is P(y ₁ , y ₂ , . . . , y _n |H _j ). Two threshold limits A and B are chosen, with A<B, and after each observation in the series the following statistic (Λ _j,n ) is calculated:

$\begin{array}{cc}{\Lambda }_{j,n}=\frac{P\left(y_1,y_2,\dots ,y_n❘H_j\right)}{P\left(y_1,y_2,\dots ,y_n❘H_0\right)}.& \left(\mathrm{E25}\right)\end{array}$

The test procedure is then as follows. If the statistic is greater than or equal to the upper threshold limit (i.e., Λ _j,n ≧B), then a decision is made to accept hypothesis H _j as true. If the statistic is less than or equal to the lower threshold limit (i.e., Λ _j,n ≦A), then a decision is made to accept hypothesis H ₀ as true. If the statistic falls between the two limits (i.e., A<Λ _j,n <B), then neither hypothesis can yet be accepted to be true and sampling continues. If the observations in the series are independent random variables, then the test statistic reduces to a product of conditional probabilities:

$\begin{array}{cc}{\Lambda }_{j,n}=\prod _{i=1}^n\frac{P\left(y_i❘H_j\right)}{P\left(y_i❘H_0\right)}.& \left(\mathrm{E26}\right)\end{array}$

In the sequential hypothesis tests, the logarithm of the test statistic is often easier to work with:

$\begin{array}{cc}\ln {\Lambda }_{j,n}=\sum _{i=1}^n\ln \left[\frac{P\left(y_i❘H_j\right)}{P\left(y_i❘H_0\right)}\right].& \left(\mathrm{E27}\right)\end{array}$

The sequential hypothesis test consists of calculating the logarithm of the test statistic for each observation in the series and comparing the result to the logarithms of the lower and upper threshold limits.

The statistic in the sequential hypothesis test is a product of a sequence of likelihood ratios. Each term in the product is the ratio of a probability conditioned on one hypothesis to a probability conditioned on a second hypothesis. The difference between the likelihood ratios in the sequential hypothesis tests and those in Bayes' rule (see equations E15 and E23), is that in Bayes' rule the probabilities are conditioned on a hypothesis and its converse, whereas in the sequential hypothesis tests the probabilities are conditioned on two hypotheses from a set of related hypotheses. In principle, the two hypotheses in the sequential hypothesis tests could be the converse of each other (i.e., the set contains two elements), but in practice the hypotheses are selected from an infinite set of exhaustive and mutually exclusive hypotheses. For instance, suppose the null hypothesis is a Gaussian PDF with a mean of 1 and a variance 10. Then a sequential hypothesis test can be defined in which the alternate hypothesis is any Gaussian PDF in which the mean is not 1 and/or the variance is not 10.

It is informative to compare the sequential hypothesis tests to Bayes' rule for a time series. Given a null hypothesis H ₀ and an alternative hypothesis H _j , the likelihood ratio for an observation e conditioned on the two hypotheses is defined as

$\begin{array}{c}{L}_j\left(e❘H_0\right)=\frac{P\left(e❘H_j\right)}{P(e}\end{array}$