Tree based methods for rule extraction from artificial neural networks

Dancey, Darren (2008) Tree based methods for rule extraction from artificial neural networks. Doctoral thesis (PhD), Manchester Metropolitan University.

Preview

Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (8MB) | Preview

Abstract

Artificial Neural Networks are powerful and flexible tools for pattern recognition, inspired by biological neurons, such as those making up the human brain. Artificial neural networks have successfully found widespread use, but further adoption is hindered in many areas because, like their biological counterparts, artificial neural networks do not reveal the knowledge they have learnt in a readily understandable form. This thesis presents new algorithms for extracting decision trees from artificial neural networks. Decision trees, unlike neural networks, are a graphical representation of a decision process that are intuitively easy to understand. This thesis extends previous algorithms in this area by making use of new developments in decision trees. The algorithms developed do not require specialised neural network architectures or training algorithms and can be applied to existing neural networks and other classifier types that are black boxes. In addition to algorithms for the extraction from classification domains, this thesis also presents algorithms to extract model trees from artificial neural networks trained on regression problems. Artificial neural networks make excellent models for function appromimation, but extraction from such neural networks has been a neglected area of research. To show the real-world applicability of these algorithms an empirical evaluation was completed on 16 real-world datasets from the standard machine learning benchmark repositories. This evaluation confirms that the algorithms are capable of extracting decision trees that achieve higher predictive classification accuracy than decision trees directly induced on the datasets, and also maintains high level of fidelity with the neural network from which they are extracted.