Hints from Life to AI,  edited by Ugur HALICI, METU, 1994 ã

 

 

connectionist expert system

 

Ferda N. Alpaslan

Mehmet R. Tolun

Department of Computer Engineering

Middle East Technical University

06531 Ankara - Turkey

{ferda,mrt}@ metu.edu.tr

The construction of a classical expert system needs to define IF-THEN rules explicitly. In contrast, to build a connectionist expert system, training examples must be used by employing the generalization capability of neural  networks in which the network is coded in the rules of expert system. The neural network models depend on the processing elements which are connected through weighted connections. The knowledge in these systems are represented by these weights and the topology of the connections are explicit representations of the rules. This paper first makes a comparison of expert systems and neural networks and second describes how these can be integrated to obtain connectionist expert systems.

 

1. Introduction

 

The term expert system was originally used to denote systems that utilize a significant amount of expert knowledge about a particular domain to solve problems in that domain. In general, an expert system may be defined as a computer-based system that searches for a solution within a body of knowledge formulated by an expert in a particular area of expertise. The power of an expert system is characterized by the contents of its knowledge base and its capabilities to explain or advise as efficient as possible like human expert. Most classical expert systems has the following components:

 

The foundation of an expert system is its knowledge base which is often the most difficult, time consuming, and expensive component to build and debug. Knowledge for expert system must be derived from sources like experts in the given field, journal articles, texts, reports, manuals, databases, and so on. Elicitation of  the right knowledge can take a lot of time and cost too much. This process of knowledge acquisition is recognized as one of the major bottlenecks in building knowledge-based expert systems. Recognizing this has led to finding new ways to improve and speed up the knowledge acquisition process. One of these methods utilizing neural network models proved to be appropriate and yielded itself in constructing flexible knowledge bases that can also cut down the knowledge acquisition time [7,25].

 

Expert systems have been applied to many areas of research, which prove, in fact, their ability to handle problems effectively. Designing and implementing an expert system is a difficult job, and usually takes experimentation and experience to achieve high performance. One of the most important characteristics of an expert system is that, it should be easy to modify, since expert systems evolve generally. This evolutionary or incremental development technique has to be noticed as the dominant methodology in the expert system area. Moreover, the expert knowledge should be arranged and formalized by defining the problem, discovering the basic steps involved.

 

Generally, neural networks consist of processing units -which resemble the neurons in human brain- connected by directed arcs, each of which has a value called weight. Each weight represents the effect of one of these processing elements on the other one which is directly connected to it. These weights may have positive or negative values, which represent reinforcement or inhibition, respectively. These weights are subject to change during learning process in order to adapt the networks to its inputs from the actual process allowing representation of complex systems which is not easy to represent in knowledge-based systems. The application of these neural networks has a greater degree of fault-tolerance because of their distributed representation, and the ability to adapt and continue learning to improve performance [18,24,26,28]. another advantage is that they can accommodate variations in their input and still produce the correct output [4,9,20].

 

Neural networks accomplish the needed generalizations by virtue of their structure rather than through elaborate programming [2,9,13]. Neural networks are trained by examples rather than programming. Neurons are viewed as processing elements combining input signals using differential equations [14,16,1,2,27]. These processing elements are treated as a simple transfer function. The network of these neurons can be viewed as associative memory, in which they associate input patterns with the desired output patterns. The number of associations in the network does not determine the required time of this association between input and output. However, the association time depends on the stability of the network, in which the actual response is obtained after training the network many times. This training process varies between supervised and unsupervised learning methods [1,3]

 

2. A Comparison of Expert Systems and Neural Networks

 

Expert systems (in general all symbolic systems) and neural networks are two "rival" approaches to Artificial Intelligence, both having different application areas within this scope. expert system design may take months of gathering information and testing. On the other hand, a neural network may be designed and trained in a few days after most examples related with the domain are gathered.

 

Assessing the available data is the most critical step in deciding on a neural network or expert system solution. One can waste a lot of time training, redesigning, and retraining a neural network that cannot learn to solve the problem because of a poor training example set [17]. In a similar fashion, an expert system can fail if there are too many conflicting and ambiguous rules. A good rule of thumb is if the rules are not well defined and there is a lot of training data then use neural network approach. If there are no examples available then a classical expert system approach may be taken. Apart from this, there are of course other criteria for deciding which approach to follow:

 

 

3. Connectionist Expert Systems - A Hybrid Approach

 

In recent years, neural network approaches have been used to simulate human behavior in areas such as vision, speech, and recognition. In fact, recognition and classification tasks can be done neural nets better than human beings. Neural nets can find a solution by making associations between known inputs and outputs by observing a large number of data.

 

Expert systems have been used in conjunction with neural network technology [5,7,10,11,12] to eliminate the time-consuming part of constructing and debugging the knowledge base. Neural networks can be used to recognize patterns, such as financial or sensory data, which are then acted on by the rules of an expert system. A neural network can be used as a model that is run by an expert system to decide how to control something. An expert system can train a neural network and a neural network can generate rules for an expert system.

 

The use of an artificial neural network greatly simplifies the knowledge engineering task because it allows the system itself to construct the knowledge base from a set of training examples. Because the correct output of the expert system is known according to given inputs, it is more appropriate to deploy a supervised learning algorithm to train the system. The resulting trained system is used to make expert decisions after it has been provided with a set of test instances. Expert systems that use artificial neural networks to develop their knowledge bases and to make inferences are called connectionist expert systems [7].

 

Compared to rule-based expert systems, connectionist expert  systems seem to be a better model of human-like reasoning. Although these models can be applied to any type of decision problems, they are especially suitable when the creation of if-then rules is not possible or the information is contradictory. A connectionist expert system usually contains three major parts: knowledge base, inference engine, and user interface. The knowledge base is a problem-dependent part that contains expert knowledge.  The connectionist expert system data base is composed of neurons and connections among them. The inference engine is a problem-independent driver program which is responsible for reasoning. The user interface is a link between the inference engine and the external user. The architecture of a connectionist expert system is illustrated in Figure 1.

 

Figure 1. A Connectionist Expert System

 

Experts are often not as much help as one would like, because it is hard for experts to specify exactly what it is they are doing to make them expert [21]. Furthermore, discovering the rules of the expert systems domain by observations is a difficult task and requires powerful learning procedures. Multilayered neural networks are of one source of such learning procedures [6,8,15,22]. These networks have the ability to learn arbitrary associations from a set of known variables to a set of target variables or actions. The generalization ability of these networks makes them much powerful than classical expert systems in which they generalize from a set of examples to the broader class of situations they may be confronted with. These networks can exhibit the rule chaining behavior knowledge based expert systems without containing any explicit representations of the rules. Instead, the rules are represented by cells which are interacting through the weighted set of connections of real values. The knowledge in these models are represented by the values of these weights and the topology of the connections. The representation of different or complex engineering systems can be facilitated by these models by modifying the weighted connections during the learning process. However, the representations of these systems would also be represented by traditional expert systems.

 

Much work has been done in the area of connectionist expert systems. Gallant [7] introduces a two-program package for constructing connectionist expert systems from training examples:

"The first program is a network knowledge base generator that  uses several connectionist learning techniques, and the second is a stand-alone expert system inference engine that interprets such knowledge bases. the system has been tested on a large number of artificial problems and on several medical and economic problems  where data were readily available" [7, p.152].

 

The connectionist medical expert system described in [12] uses a learning algorithm developed by the authors that is based on a potential function approach to supervised learning. Bradshaw et al. introduce a three-layer connectionist expert system that learns to forecast flares [5]. They conclude that their connectionist expert system can perform the same task as well as a conventionally built rule-based system. The architectures for executing probabilistic rule-bases in a parallel manner using information-theoretic models is discussed in [10]. The main concern in this research is automatic generation of connectionist expert systems from data using a rule-based representation. Hudli et al. [11] propose using artificial neural networks to construct an expert system model consisting of  vertically organized sets of simple neural networks, each realizing a simple task. Two reasoning procedures, backward and forward chaining, of expert systems have been implemented using neural network formalism.

 

The following example describes how training and test patterns are generated from expert system rules.

         

The knowledge base for a medical expert system that contains the rules to diagnose some illnesses and to recommend proper treatment can be represented as a three layered-neural network. Each node represents a parameter of the original expert system. The expert system contains the rules defined to describe the illnesses and proper treatments. An example of one of the rules is listed in Figure 2.

 

 

 

 

 Figure 2. An Expert System Rule

 

The training and test patterns have been generated from the rules defined above. Training patterns are composed of 0's (unknown), 1's (true), and -1's (false). When generating a  pattern from a rule, the nodes corresponding to the expert system parameters whose values are used in the rule positively are represented as 1 whereas those whose values are used negatively are represented as -1. All the remaining nodes are 0. As an example, to generate a training pattern from the rule

 

The system, then, can be trained and tested by using a supervised learning algorithm, like backpropagation.

 

4. Conclusion

 

The idea of using neural networks to develop knowledge bases for expert systems was suggested several years ago as a way of eliminating the tedious chore of encoding knowledge from the experts. Expert systems developed from neural networks are known as connectionist expert systems. The advantage of using a connectionist expert system approach over other conventional techniques to build expert systems are as follows:

 

References

 

1.    Abu-Salameh W. and M.R. Tolun. Knowledge Representation in Neural Networks. Technical Report 89-9. METU, Ankara, Turkey, 1989.

2.    Anderson, J.A. and G.Murphy. Psychological Concepts in Parallel Systems. Elsevier Science Publishers. 1989.

3.    Atiya A. An Unsupervised Learning Technique for Artificial Neural Networks. Neural Network, 1988.

4.    Buello, J., A.W.Mackie, and J.A. Anderson. Sytactic Category Diambiguation with Neural Networks. Computer Speech language, 203-217, 1989.

5.    Bradshaw, G., R. Fazzard, and L. Ceci.  A Connectionist Expert System That Actually Works.  Touretzky (ed). Neural Information Processing Systems 1. organ Kaufman Publishers, San Mateo, CA, 1989, 248-255.

6.    Gallant S.I. Automatic generation of Experts from Examples. Proc. 2nd Intl.Conf. on AI Applications. FL, Dec 11-13, 1985.

7.    Gallant, S.I.  Connectionist Expert Systems. Comm. ACM. 31, Feb.1988, 2, 152-169.

8.    Gardner, E.J., N. Strund, and D.J. Walke. Training with Noise and the Storage of Correlated Patterns in a Neural Network Model. J Physics, 22, 1989.

9.    Giles, C.I, and T. Maxwell. Learning Invarience, and Generalization in Higher-Order Neural Network. Applied Optics. 26,23, 1987.

10.  Goodman, R.M., J.W. Miller, and P.Smyth.  An Information Theoretic Approach to Rule-Based Connectionist Expert Systems. Touretzky (ed). Neural Information Processing Systems 1. Morgan Kaufman Publishers, San Mateo, CA, 1989, 256-263.

11.  Hudli, A.V., M.J. Palakal, and M.J. Zoran.  A Neural Network Based Expert System Model. Proc. IEEE Intl. Conf. on Tools for AI. San Jose, CA, Nov 1991, 145-149.

12.  Hudson, D.L. and M.E. Cohen. A Connectionist  Medical Expert  System. Proc. Sixth IASTED. M.H. Hamza (ed). ACTA Press, Dec 14-16, 1989,  104-107.

13.  Kohonen, T.Self-Organization and Associative Memory. Springer-Verlag, Berlin, 1984.

14.  Kohonen, T. Adaptive, Associative, and Self-Organizing Funcrions in Neural Computing. applied Optics. 26,23,1987.

15.  Kohonen, T. An Introduction to Neurocomputing. Neural Network. 1, 13-16, 1988.

16.  Kodko, B., Constructing an Associative Memory. BYTE, Sept, 1987.

17.  Lawrence, J. Introduction to Neural Networks and Expert Systems. California Scientific Software, Nevada City, CA, 1992.

18.  Lippmann, R.P., An Introduction to Computing with Neural Nets. IEEE ASSP Mag. 4, 4-22, 1987.

19.  Matheus, L.J. and E. Hohnsee. Learning in Artificial Neural Systems. Technical Report R-87-1394. Univ. of Illinois at Urbana Champaign, 1987.

20.  Mel, B.W. A Computational Model for Neural Extraction. Technical report UIUCDCS R-87-1357. Univ. of Illinois at Urbana Champaign,      1987.

21.  Michael, C. Mozer. RAMBOT: A Connectionist Expert System that Learns    by Examples.ICS Report. UCSD , 1986.

22.  Minsky, M. and S. Papert. Perceptrons : An Introduction to Computational Geometry. MIT Press. Cambridge, MA, 1969.

23.  Rumelhart, D.E., G.E. Hinton, and R.J. Williams.  Learning Representations by Back-propagating Errors. Nature, 1986, 323, 533- 536.

24.  Rumelhart, D.E., and D. Zipser. Future Discovery by Competitive Learning. Cognitive Science, 9, 75-112, 1985.

25.  Tolun, M.R. and F.N. Civelek. An Overview of Expert systems and Neural Networks. Proc. 3rd ISCIS, Nova Science Publishers, Oct. 1987.

26.  William, R.J. Inference of Spatial Relations by Self-Organization Networks. ICS Report, UCSD, 1985.

27.  William, R.J. Reinforcement Learning in Connectionist Networks: A Mathematical Analysis. ICS Report, UCSD, 1986.

28. Yager, R.R. On the Aggregation of Processing Units in Neural Networks. Technical report M II-708, Iona College, New Rochelle, 1988.

 

 

contents                                     home