Interpretability of Linguistic Fuzzy Rule-Based Systems: An Overview of Interpretability Measures

This Website contains additional material to the SCI2S research paper on "Interpretability of Linguistic Fuzzy Rule-Based Systems". It is maintained by M. J. Gacto and R. Alcalá.

M.J. Gacto, R. Alcalá, F. Herrera, Interpretability of Linguistic Fuzzy Rule-Based Systems: An Overview of Interpretability Measures, Information Sciences, 181:20 (2011) 4340–4360, doi: 10.1016/j.ins.2011.02.021 .

The web is organized according to the following summary:

M.J. Gacto, R. Alcalá, F. Herrera, Interpretability of Linguistic Fuzzy Rule-Based Systems: An Overview of Interpretability Measures, Information Sciences, 181:20 (2011) 4340–4360, doi: 10.1016/j.ins.2011.02.021 .

Paper abstract.

Linguistic fuzzy modelling, developed by linguistic fuzzy rule-based systems, allows us to deal with the modelling of systems by building a linguistic model which could become interpretable by human beings. Linguistic fuzzy modelling comes with two contradictory requirements: interpretability and accuracy. In recent years the interest of researchers in obtaining more interpretable linguistic fuzzy models has grown.

Whereas the measures of accuracy are straightforward and well-known, interpretability measures are difficult to define since interpretability depends on several factors; mainly the model structure, the number of rules, the number of features, the number of linguistic terms, the shape of the fuzzy sets, etc. Moreover, due to the subjectivity of the concept the choice of appropriate interpretability measures is still an open problem.

In this paper, we present an overview of the proposed interpretability measures and techniques for obtaining more interpretable linguistic fuzzy rule-based systems. To this end, we will propose a taxonomy based on a double axis: Complexity versus Semantic Interpretability" considering the two main kinds of measures; and Rule Base versus Fuzzy Partitions" considering the different components of the knowledge base to which both kinds of measures can be applied. The main aim is to provide a well established framework in order to facilitate a better understanding of the topic and well founded future works.

Taxonomy: A double axis classification of interpretability measures for Linguistic FRBSs

In this section, we present a specific classification that can help us better understand how the interpretability aspect has been taken into account in the particular framework of linguistic FRBSs. Different works (José M. Alonso and Luis Magdalena and Gil González Rodríguez, "Looking for a good fuzzy system interpretability index: An experimental approach", International Journal of Approximate Reasoning, vol 51, pp. 115-134, 2009 doi: 10.1016/j.ijar.2009.09.004, and C. Mencar and A.M. Fanelli, "Interpretability constraints for fuzzy information granulation", Information Sciences, vol. 178, no. 24, pp. 4585-4618, 2008 doi: 10.1016/j.ins.2008.08.015, and S. M. Zhou and J. Q. Gan, "Low-level interpretability and high-level interpretability: A unified view of data-driven interpretable fuzzy system modelling", Fuzzy Sets and Systems, vol. 159, no. 23, pp. 3091-3131, 2008 doi: 10.1016/j.fss.2008.05.016) have proposed interesting taxonomies as a way to study interpretability aspects within the more general area of fuzzy systems.

In this particular case of linguistic FRBSs, the two main kinds of approaches to take into account the interpretability of linguistic FRBSs are:

• Complexity-based Interpretability: These approaches are devoted to decreasing the complexity of the obtained model (usually measured as number of rules, variables, labels per rule, etc.).
• Semantics-based Interpretability: These approaches are devoted to preserving the semantics associated with the Membership Functions (MFs). We can find approaches trying to ensure semantic integrity by imposing constraints on the MFs or approaches considering measures such as distinguishability, coverage, etc.

Since both kinds of measures, complexity-based interpretability and semantic-based interpretability, should be considered in both Knowledge Base (KB) components, linguistic fuzzy partition and Rule Base (RB), we will follow a taxonomy based on a double axis:

1. Complexity versus Semantic interpretability.
2. Rule Base versus Fuzzy Partition.

In this way, the said taxonomy comes from combining both axes. This leads to the appearance of the following quadrants devoted to analyzing the interpretability of linguistic FRBSs (see Table 1):

Q1: Complexity at the RB level.

Q2: Complexity at the fuzzy partition level.

Q3: Semantics at the RB level.

Q4: Semantics at the fuzzy partition level.

Table 1. A taxonomy to analyze the interpretability of linguistic FRBSs

 Rule Base level Fuzzy Partition level Complexity-based Interpretability Q1 Number of rules Number of conditions Q2 Number of Membership Functions Number of Features Semantic-based Interpretability Q3 Consistency of rules Rules fired at the same time Transparency of rule structure Cointension Q4 Completeness or Coverage Normalization Distinguishability Complementarity Relative measures

Most of the works in the literature consider measures included inside quadrants Q1 and Q2 which are considered as the classic interpretability measures. These measures are widely well-known as the number of rules, number of conditions, number of features and number of MFs. This is not the case of semantic interpretability measures: there are still no widely accepted measures, because many of them appear as new measures in the literature.

In the following, we present a short description of these less known measures included in the quadrants Q3 and Q4. Some of the more commonly used measures included in the quadrant Q3 are the following:

1. Consistency of the RB, is the absence of contradictory rules in RB, i.e., rules with similar premise parts should have similar consequent parts.
2. Number of rules fired at the same time, which consists of minimizing the number of rules firing that are activated for a given input.
3. Cointension, is a proximity of the input/output relations of the object of modeling and the model. A model is cointensive if its proximity is high.

The most common constraints or absolute measures in the quadrant Q4 are:

1. Completeness or Coverage: The universe of discourse of a variable should be covered by the MFs, and every data point should belong to at least one of the fuzzy sets and have a linguistic representation, i.e., it is required that membership values should not be zero for all the linguistic variable domains.
2. Normalization: MFs are normal if there is at least one data point in the universe of discourse with a membership value equal to one, in respect to the maximum membership degree.
3. Distinguishability: An MF should represent a linguistic term with a clear semantic meaning and should be easily distinguishable from the remaining MFs of the corresponding variable.
4. Complementarity: For each element of the universe of discourse, the sum of all its membership values should be near to one. This guarantees a uniform distribution of the meanings among the elements.

There is an additional possibility for Q4 which is considering relative measures, i.e., using a measure that takes into account the user MF definitions if they are available or that tries to maintain a previous accepted definition obtained from automatic techniques (probably based on absolute measures). The first proposal in this research line can be found in M.J. Gacto, R. Alcalá and F. Herrera, "Integration of an Index to Preserve the Semantic Interpretability in the Multiobjective Evolutionary Rule Selection and Tuning of Linguistic Fuzzy Systems", IEEE Transactions on Fuzzy Systems, vol. 18, no. 3, pp. 515-531, 2010 doi: 10.1109/TFUZZ.2010.2041008.

Categorizing the current state-of-the-art to assess the interpretability of linguistic FRBSs. Bib text with the bibliography reference

In order to show the existent works chronologically, Table 2 shows a summary of the works that consider the interpretability for Linguistic FRBSs grouped by quadrants and by publication date: by years and within each year by alphabetical order. This table represents a categorization of the current state-of-the art to assess the interpretability of linguistic FRBSs.

Table 2. Summary of the current state-of-the-art to assess the interpretability of linguistic FRBSs by years

NR = Number of Rules, NC = Number of Conditions, Nmf = Number of membership functions, Nfeat. = Number of Features,
Cons. = Consistency, Rfired. = Number of Rules fired at the same time, Coin. = Cointension, Ctr. = Constraints, Meas. = Measures;
Clas. = Classification, Reg. = Regression, Ctl. = Control, Am = Absolute Measures, Rm = Relative Measures.

 Using Q1 Q2 Q3 Q4 Authors Ref. Year Type MOEAs NR NC Nmf Nfeat. Cons. Rfired. Coin. Ctr. Meas. Ishibuchi et al. 35, 36, 39; bib 1995, 1997, 1995 Clas. x,✓,x ✓ Pedrycz et al. 62; bib 1996 Reg. Am Oliveira et al. 59,60; bib 1999 Ctl. ✓ Am Jin et al. 44, 45 ; bib 2003, 1999 Ctl. ✓ ✓ Cheong et al. 15, 16 ; bib 2000, 2003 Ctl. ✓ ✓ Espinosa et al. 23; bib 2000 Reg. ✓ ✓ Jin 43; bib 2000 Ctl. ✓ ✓ ✓ ✓ Cordón et al. O. Cordón and F. Herrera and L. Magdalena and P. Villar, A Genetic Learning Process for the Scaling Factors, Granularity and Contexts of the Fuzzy Rule-Based System Data Base, Information Science (2001) 136 85-107">18, F. Herrera and P. Villar, Generating the Knowledge Base of a Fuzzy Rule-Based System by the Genetic Learning of Data Base, IEEE Transactions on Fuzzy Systems (2001) 9:4 667-674">19; bib 2001 Reg. ✓ ✓ Ishibuchi et al. 37; bib 2001 Clas. ✓ ✓ ✓ Suzuki et al. 25, 70, 71; bib 2001, 2003 Reg. ✓ Am Cordón et al. O. Cordón and M.J. del Jesus and F. Herrera and L. Magdalena and P. Villar, A Multiobjective Genetic Learning Process for Joint Feature Selection and Granularity and Context Learning in Fuzzy Rule-Based Classification Systems, Interpretability issues in fuzzy modeling (2003) 79-99">20; bib 2003 Clas. ✓ ✓ ✓ Guillaume et al. 30; bib 2003 Clas. ✓ ✓ Ishibuchi et al. 41; bib 2003 Clas. ✓ Ishibuchi et al. 40; bib 2003 Reg. ✓ ✓ Nauck et al. 58; bib 2003 Clas. ✓ ✓ Am Pedrycz et al. 61; bib 2003 Reg. ✓ Pena-Reyes et al. 63; bib 2003 Clas. ✓ ✓ ✓ Tikk et al. 73; bib 2003 Clas. ✓ Vanhoucke et al. 74; bib 2003 Clas. ✓ Guillaume et al. 31; bib 2004 Clas. ✓ ✓ ✓ Am Ishibuchi et al. 42; bib 2004 Clas. ✓ ✓ ✓ Casillas et al. J. Casillas and O. Cordón and M. J. del Jesus and F. Herrera, Genetic tuning of fuzzy rule deep structures preserving interpretability and its interaction with fuzzy rule set reduction, IEEE Trans. Fuzzy Syst. (2005) 13:1 13-29">14; bib 2005 Clas. ✓ Narukawa et al. 57; bib 2005 Clas. ✓ ✓ ✓ Mikut et al. 55; bib 2005 Clas. ✓ ✓ Alcalá et al. R. Alcalá and J. Alcalá-Fdez and F. Herrera and J. Otero, Genetic Learning of Accurate and Compact Fuzzy Rule Based Systems Based on the 2-Tuples Linguistic Representation, International Journal of Approximate Reasoning (2007) 44:1 45-64">2; bib 2007 Reg. ✓ ✓ Alcalá et al. R. Alcalá and J. Alcalá-Fdez and F. Herrera, A proposal for the genetic lateral tuning of linguistic fuzzy systems and its interaction with rule selection, IEEE Trans. Fuzzy Syst. (2007) 15:4 616-635">1; bib 2007 Reg. \ Ctl. ✓ Alcalá et al. R. Alcalá and M. J. Gacto and F. Herrera and J. Alcalá-Fdez, A multi-objective genetic algorithm for tuning and rule selection to obtain accurate and compact linguistic fuzzy rule-based systems, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems (2007) 15:5 539-557">4; bib 2007 Reg. ✓ ✓ Cococcioni et al. 17; bib 2007 Reg. ✓ ✓ Fazendeiro et al. 24; bib 2007 Reg. ✓ Am Ishibuchi et al. 38; bib 2007 Ctl. ✓ ✓ ✓ Liu et al. 47; bib 2007 Clas. ✓ ✓ Mencar et al. 52 ; bib 2007 Clas. ✓ Alonso et al. 9; bib 2008 Clas. ✓ ✓ ✓ Pulkkinen et al. 65; bib 2008 Clas. ✓ ✓ ✓ ✓ ✓ Am Pulkkinen et al. 66; bib 2008 Clas. ✓ ✓ ✓ ✓ Alcalá et al. R. Alcalá and P. Ducange and F. Herrera and B. Lazzerini and F. Marcelloni, A Multi-Objective Evolutionary Approach to Concurrently Learn Rule and Data Bases of Linguistic Fuzzy Rule-Based Systems, IEEE Transactions on Fuzzy Systems (2009) 17:5 1106-1122">3; bib 2009 Reg. ✓ ✓ Alonso et al. 10; bib 2009 Clas. ✓ ✓ ✓ Botta et al. 11; bib 2009 Reg. ✓ Am Gacto et al. R. Alcalá and F. Herrera, Adaptation and application of multi-objective evolutionary algorithms for rule reduction and parameter tuning of fuzzy rule-based systems, Soft Computing (2009) 13:5 419-436">27; bib 2009 Reg. ✓ ✓ Pulkkinen et al. 64, 67; bib 2009, 2010 Clas. \ Reg. ✓ ✓ ✓ Alonso et al. O. Cordón, Embedding {HILK} in a three-objective evolutionary algorithm with the aim of modeling highly interpretable fuzzy rule-based classifiers, 4th International Workshop on Genetic and Evolving Fuzzy Systems (GEFS2010) (2010) 15-20">6; bib 2010 Clas. ✓ ✓ ✓ Alonso et al. 8; bib in press Clas. ✓ ✓ ✓ ✓ Gacto et al. R. Alcalá and F. Herrera, Integration of an Index to Preserve the Semantic Interpretability in the Multi-Objective Evolutionary Rule Selection and Tuning of Linguistic Fuzzy Systems, IEEE Transactions on Fuzzy Systems (2010) 18:3 515-531">28; bib 2010 Reg. ✓ ✓ Rm Márquez et al. 51; bib 2010 Reg. ✓ ✓ ✓ Mencar et al. 53; bib in press Clas. ✓ ✓

Taking into account the situation depicted in Table, most of the works consider measures included inside the quadrants Q1 and Q2 which are considered as the classic interpretability measures. The number of rules in the quadrant Q1 is one of the more used measures in the literature, for which reason it is possible to consider it as a good measure of complexity at the rule base level. However the total number of conditions seems to be a more complete way since it can consider both the length of the rules and the number of rules, in an simple measure. In the quadrant Q2 most of the works simply impose restrictions on the maximum number of MFs allowed, even though, depending on the problem (particularly in high dimensional problems) decreasing the number of features should be preferred. The large quantity of works published in 2003 was motivated by the following two books (J. Casillas and O. Cordón and F. Herrera and L. Magdalena, "Interpretability issues in fuzzy modeling", vol. 128, 2003 and J. Casillas and O. Cordón and F. Herrera and L. Magdalena, "Accuracy improvements in linguistic fuzzy modeling" , vol. 129, 2003) on the interpretability-accuracy trade-off in the field of FRBSs. However, apart from this, the interest of researchers has increased particularly from 2007, giving rise to the appearance of many works from this year to the present. Inside quadrant Q3 there are a few measures but some works propose promising measures such as the consistency of the rules and the more recent number of rules fired at the same time and cointension. In quadrant Q4 there are a lot of works imposing constraints. However, recently new absolute or relative semantic interpretability measures have arisen, which are more suitable to be taken into account for optimization processes. For Q3 and Q4 quadrants, there are still no widely accepted measures, which will arise with their use as happened with the number of rules for Q1. Additionally, the use of Multi-Objective Evolutionary Algorithms (MOEAs) has emerged as a good way to handle interpretability since they allow both complexity and semantic interpretability measures to be optimized together by also taking into account the accuracy of the model.

Related Bibliography and Special Issues focused on Interpretability Issues of FRBSs

This section tries to present a quick snapshot of the Interpretability of FRBs status stressing the following points:

Related Bibliography

For beginners, we present next the related bibliography associated to the books and review papers published in the specialized literature:

1) Books.

2) Review papers.

• C. Mencar and A.M. Fanelli, "Interpretability constraints for fuzzy information granulation" , Information Sciences, vol. 178, no. 24, pp. 4585-4618, 2008 doi: 10.1016/j.ins.2008.08.015
• S. M. Zhou and J. Q. Gan, "Low-level interpretability and high-level interpretability: A unified view of data-driven interpretable fuzzy system modelling" , Fuzzy Sets and Systems, vol. 159, no. 23, pp. 3091-3131, 2008 doi: 10.1016/j.fss.2008.05.016

Journal Special Issues

Next, we provide a list of the journal special issues devoted to Interpretability of FRBSs, including important contributions. We include a global table of contents for these special issues.

• Jose M. Alonso and Luis Magdalena. Special Issue on Interpretable Fuzzy Systems. Information Science Volume 181, Issue 20 (2011), pages 4331-4822. Table of contents

The collection of papers that we can find on this special issue is briefly described in the following:

• The special issue of 2011, co-edited by Jose M. Alonso and Luis Magdalena, gives an overview of some of the hottest topics in the field of interpretable fuzzy systems. (M.J. Gacto, R. Alcalá and F. Herrera, Interpretability of linguistic fuzzy rule-based systems: An overview of interpretability measures, Information Science 181:20 (2011) 4340-4360, doi: 10.1016/j.ins.2011.02.021), present a thorough chronological review of papers dealing with interpretability of linguistic fuzzy rule-based systems. The main goal consists of establishing a reference framework for interpretable fuzzy systems with the aim of helping readers to have better and deeper understanding of the topic. (C. Mencar, C. Castiello, R. Cannone and A.M. Fanelli, Design of fuzzy rule-based classifiers with semantic cointension, Information Science 181:20 (2011) 4361-4377, doi: 10.1016/j.ins.2011.02.014), describe a novel methodology for designing interpretable fuzzy rule-based classifiers. The main contribution of this work arises from the fact that the proposed design process is guided by a new interpretability measure grounded on what is named as semantic cointension approach. (Andri Riid and Ennu Rüstern, Identification of transparent, compact, accurate and reliable linguistic fuzzy models, Information Science 181:20 (2011) 4378-4393, doi: 10.1016/j.ins.2011.01.041), tackle the search of a good interpretability-accuracy trade-off in the context of 0-th order Takagi–Sugeno fuzzy systems where priority is usually given to accuracy disregarding interpretability. (Adolfo R. de Soto, A hierarchical model of a linguistic variable, Information Science 181:20 (2011) 4394-4408, doi: 10.1016/j.ins.2011.04.006), introduces a theoretical hierarchical model of linguistic variables paying attention to both syntactic and semantic definitions. (Serge Guillaume and Brigitte Charnomordic ,Learning interpretable fuzzy inference systems with FisPro, Information Science 181:20 (2011) 4409-4427, doi: 10.1016/j.ins.2011.03.025), present an open source software for fuzzy inference system design and optimization called FisPro.

Slides for Interpretability of FRBSs Presentations

R. Alcalá (August 2011) . A double Axis Classification of Interpretability Measures for Linguistic Fuzzy Rule-Based Systems

The slides of this talk are associated to the paper: M.J. Gacto, R. Alcalá, F. Herrera, Interpretability of Linguistic Fuzzy Rule-Based Systems: An Overview of Interpretability Measures, Information Sciences, 181:20 (2011) 4340–4360 doi: 10.1016/j.ins.2011.02.021. These slides were presented in the 9th International Workshop on Fuzzy Logic and Applications (WILF 2011)