M.J. Gacto, R. Alcalá, F. Herrera, Interpretability of Linguistic Fuzzy Rule-Based Systems: An Overview of Interpretability Measures, Information Sciences, 181:20 (2011) 4340–4360, doi: 10.1016/j.ins.2011.02.021 .

Paper abstract.

Linguistic fuzzy modelling, developed by linguistic fuzzy rule-based systems, allows us to deal with the modelling of systems by building a linguistic model which could become interpretable by human beings. Linguistic fuzzy modelling comes with two contradictory requirements: interpretability and accuracy. In recent years the interest of researchers in obtaining more interpretable linguistic fuzzy models has grown.

Whereas the measures of accuracy are straightforward and well-known, interpretability measures are difficult to define since interpretability depends on several factors; mainly the model structure, the number of rules, the number of features, the number of linguistic terms, the shape of the fuzzy sets, etc. Moreover, due to the subjectivity of the concept the choice of appropriate interpretability measures is still an open problem.

In this paper, we present an overview of the proposed interpretability measures and techniques for obtaining more interpretable linguistic fuzzy rule-based systems. To this end, we will propose a taxonomy based on a double axis: ``Complexity versus Semantic Interpretability" considering the two main kinds of measures; and ``Rule Base versus Fuzzy Partitions" considering the different components of the knowledge base to which both kinds of measures can be applied. The main aim is to provide a well established framework in order to facilitate a better understanding of the topic and well founded future works.

Taxonomy: A double axis classification of interpretability measures for Linguistic FRBSs

In this section, we present a specific classification that can help us better understand how the interpretability aspect has been taken into account in the particular framework of linguistic FRBSs. Different works (José M. Alonso and Luis Magdalena and Gil González Rodríguez, "Looking for a good fuzzy system interpretability index: An experimental approach", International Journal of Approximate Reasoning, vol 51, pp. 115-134, 2009 doi, and C. Mencar and A.M. Fanelli, "Interpretability constraints for fuzzy information granulation", Information Sciences, vol. 178, no. 24, pp. 4585-4618, 2008 doi, and S. M. Zhou and J. Q. Gan, "Low-level interpretability and high-level interpretability: A unified view of data-driven interpretable fuzzy system modelling", Fuzzy Sets and Systems, vol. 159, no. 23, pp. 3091-3131, 2008 doi) have proposed interesting taxonomies as a way to study interpretability aspects within the more general area of fuzzy systems.

In this particular case of linguistic FRBSs, the two main kinds of approaches to take into account the interpretability of linguistic FRBSs are:

Complexity-based Interpretability: These approaches are devoted to decreasing the complexity of the obtained model (usually measured as number of rules, variables, labels per rule, etc.).

Semantics-based Interpretability: These approaches are devoted to preserving the semantics associated with the Membership Functions (MFs). We can find approaches trying to ensure semantic integrity by imposing constraints on the MFs or approaches considering measures such as distinguishability, coverage, etc.

Since both kinds of measures, complexity-based interpretability and semantic-based interpretability, should be considered in both Knowledge Base (KB) components, linguistic fuzzy partition and Rule Base (RB), we will follow a taxonomy based on a double axis:

1. 1. Complexity versus Semantic interpretability.

2. 2. Rule Base versus Fuzzy Partition.

In this way, the said taxonomy comes from combining both axes. This leads to the appearance of the following quadrants devoted to analyzing the interpretability of linguistic FRBSs (see Table 1):

1. Q1: Complexity at the RB level.

2. Q2: Complexity at the fuzzy partition level.

3. Q3: Semantics at the RB level.

4. Q4: Semantics at the fuzzy partition level.

Most of the works in the literature consider measures included inside quadrants Q1 and Q2 which are considered as the classic interpretability measures. These measures are widely well-known as the number of rules, number of conditions, number of features and number of MFs. This is not the case of semantic interpretability measures: there are still no widely accepted measures, because many of them appear as new measures in the literature.

In the following, we present a short description of these less known measures included in the quadrants Q3 and Q4. Some of the more commonly used measures included in the quadrant Q3 are the following:

• 1. Consistency of the RB, is the absence of contradictory rules in RB, i.e., rules with similar premise parts should have similar consequent parts.

• 2. Number of rules fired at the same time, which consists of minimizing the number of rules firing that are activated for a given input.

• 3. Cointension, is a proximity of the input/output relations of the object of modeling and the model. A model is cointensive if its proximity is high.

The most common constraints or absolute measures in the quadrant Q4 are:

• 1. Completeness or Coverage: The universe of discourse of a variable should be covered by the MFs, and every data point should belong to at least one of the fuzzy sets and have a linguistic representation, i.e., it is required that membership values should not be zero for all the linguistic variable domains.

• 2. Normalization: MFs are normal if there is at least one data point in the universe of discourse with a membership value equal to one, in respect to the maximum membership degree.

• 3. Distinguishability: An MF should represent a linguistic term with a clear semantic meaning and should be easily distinguishable from the remaining MFs of the corresponding variable.

• 4. Complementarity: For each element of the universe of discourse, the sum of all its membership values should be near to one. This guarantees a uniform distribution of the meanings among the elements.

There is an additional possibility for Q4 which is considering relative measures, i.e., using a measure that takes into account the user MF definitions if they are available or that tries to maintain a previous accepted definition obtained from automatic techniques (probably based on absolute measures). The first proposal in this research line can be found in M.J. Gacto, R. Alcalá and F. Herrera, "Integration of an Index to Preserve the Semantic Interpretability in the Multiobjective Evolutionary Rule Selection and Tuning of Linguistic Fuzzy Systems", IEEE Transactions on Fuzzy Systems, vol. 18, no. 3, pp. 515-531, 2010 doi.

Categorizing the current state-of-the-art to assess the interpretability of linguistic FRBSs. Bib text with the bibliography reference

In order to show the existent works chronologically, Table 2 shows a summary of the works that consider the interpretability for Linguistic FRBSs grouped by quadrants and by publication date: by years and within each year by alphabetical order. This table represents a categorization of the current state-of-the art to assess the interpretability of linguistic FRBSs.

Table 2. Summary of the current state-of-the-art to assess the interpretability of linguistic FRBSs by years

NR = Number of Rules, NC = Number of Conditions, Nmf = Number of membership functions, Nfeat. = Number of Features, Cons. = Consistency, Rfired. = Number of Rules fired at the same time, Coin. = Cointension, Ctr. = Constraints, Meas. = Measures; Clas. = Classification, Reg. = Regression, Ctl. = Control, Am = Absolute Measures, Rm = Relative Measures.

Using

Q1

Q2

Q3

Q4

Authors

Ref.

Year

Type

MOEAs

NR

NC

Nmf

Nfeat.

Cons.

Rfired.

Coin.

Ctr.

Meas.

Ishibuchi et al.

35, 36, 39;

1995, 1997, 1995

Clas.

, ,

Pedrycz et al.

62;

1996

Reg.

Am

Oliveira et al.

59,60;

1999

Ctl.

Am

Jin et al.

44, 45 ;

2003, 1999

Ctl.

Cheong et al.

15, 16 ;

2000, 2003

Ctl.

Espinosa et al.

23;

2000

Reg.

Jin

43;

2000

Ctl.

Cordón et al.

18, 19;

2001

Reg.

Ishibuchi et al.

37;

2001

Clas.

Suzuki et al.

25, 70, 71;

2001, 2003

Reg.

Am

Cordón et al.

20;

2003

Clas.

Guillaume et al.

30;

2003

Clas.

Ishibuchi et al.

41;

2003

Clas.

Ishibuchi et al.

40;

2003

Reg.

Nauck et al.

58;

2003

Clas.

Am

Pedrycz et al.

61;

2003

Reg.

Pena-Reyes et al.

63;

2003

Clas.

Tikk et al.

73;

2003

Clas.

Vanhoucke et al.

74;

2003

Clas.

Guillaume et al.

31;

2004

Clas.

Am

Ishibuchi et al.

42;

2004

Clas.

Casillas et al.

14;

2005

Clas.

Narukawa et al.

57;

2005

Clas.

Mikut et al.

55;

2005

Clas.

Alcalá et al.

2;

2007

Reg.

Alcalá et al.

1;

2007

Reg. \ Ctl.

Alcalá et al.

4;

2007

Reg.

Cococcioni et al.

17;

2007

Reg.

Fazendeiro et al.

24;

2007

Ctl.

Am

Ishibuchi et al.

38;

2007

Clas.

Liu et al.

47;

2007

Clas.

Mencar et al.

52 ;

2007

Clas.

Alonso et al.

9;

2008

Clas.

Pulkkinen et al.

65;

2008

Clas.

Am

Pulkkinen et al.

66;

2008

Clas.

Alcalá et al.

3;

2009

Reg.

Alonso et al.

10;

2009

Clas.

Botta et al.

11;

2009

Reg.

Am

Gacto et al.

27;

2009

Reg.

Pulkkinen et al.

64, 67;

2009, 2010

Clas. \ Reg.

Alonso et al.

6;

2010

Clas.

Alonso et al.

8;

in press

Clas.

Gacto et al.

28;

2010

Reg.

Rm

Márquez et al.

51;

2010

Reg.

Mencar et al.

53;

in press

Clas.

Taking into account the situation depicted in Table, most of the works consider measures included inside the quadrants Q1 and Q2 which are considered as the classic interpretability measures. The number of rules in the quadrant Q1 is one of the more used measures in the literature, for which reason it is possible to consider it as a good measure of complexity at the rule base level. However the total number of conditions seems to be a more complete way since it can consider both the length of the rules and the number of rules, in an simple measure. In the quadrant Q2 most of the works simply impose restrictions on the maximum number of MFs allowed, even though, depending on the problem (particularly in high dimensional problems) decreasing the number of features should be preferred. The large quantity of works published in 2003 was motivated by the following two books (J. Casillas and O. Cordón and F. Herrera and L. Magdalena, "Interpretability issues in fuzzy modeling", vol. 128, 2003 and J. Casillas and O. Cordón and F. Herrera and L. Magdalena, "Accuracy improvements in linguistic fuzzy modeling", vol. 129, 2003) on the interpretability-accuracy trade-off in the field of FRBSs. However, apart from this, the interest of researchers has increased particularly from 2007, giving rise to the appearance of many works from this year to the present. Inside quadrant Q3 there are a few measures but some works propose promising measures such as the consistency of the rules and the more recent number of rules fired at the same time and cointension. In quadrant Q4 there are a lot of works imposing constraints. However, recently new absolute or relative semantic interpretability measures have arisen, which are more suitable to be taken into account for optimization processes. For Q3 and Q4 quadrants, there are still no widely accepted measures, which will arise with their use as happened with the number of rules for Q1. Additionally, the use of Multi-Objective Evolutionary Algorithms (MOEAs) has emerged as a good way to handle interpretability since they allow both complexity and semantic interpretability measures to be optimized together by also taking into account the accuracy of the model.

Related Bibliography and Special Issues focused on Interpretability Issues of FRBSs

This section tries to present a quick snapshot of the Interpretability of FRBs status stressing the following points:

Related Bibliography

Journal Special Issues

For beginners, we present next the related bibliography associated to the books and review papers published in the specialized literature:

1) Books.

J. Casillas and O. Cordón and F. Herrera and L. Magdalena, "Interpretability issues in fuzzy modeling", vol. 128, 2003

J. Casillas and O. Cordón and F. Herrera and L. Magdalena, "Accuracy improvements in linguistic fuzzy modeling", vol. 129, 2003

2) Review papers.

C. Mencar and A.M. Fanelli, "Interpretability constraints for fuzzy information granulation", Information Sciences, vol. 178, no. 24, pp. 4585-4618, 2008 doi

S. M. Zhou and J. Q. Gan, "Low-level interpretability and high-level interpretability: A unified view of data-driven interpretable fuzzy system modelling", Fuzzy Sets and Systems, vol. 159, no. 23, pp. 3091-3131, 2008 doi

Next, we provide a list of the journal special issues devoted to Interpretability of FRBSs, including important contributions. We include a global table of contents for these special issues.

Jose M. Alonso and Luis Magdalena. Special Issue on Interpretable Fuzzy Systems. Information Science Volume 181, Issue 20 (2011), pages 4331-4822. Table of contents

The collection of papers that we can find on this special issue is briefly described in the following:

The special issue of 2011, co-edited by Jose M. Alonso and Luis Magdalena, gives an overview of some of the hottest topics in the field of interpretable fuzzy systems. (M.J. Gacto, R. Alcalá and F. Herrera, Interpretability of linguistic fuzzy rule-based systems: An overview of interpretability measures, Information Science 181:20 (2011) 4340-4360, doi: 10.1016/j.ins.2011.02.021), present a thorough chronological review of papers dealing with interpretability of linguistic fuzzy rule-based systems. The main goal consists of establishing a reference framework for interpretable fuzzy systems with the aim of helping readers to have better and deeper understanding of the topic. (C. Mencar, C. Castiello, R. Cannone and A.M. Fanelli, Design of fuzzy rule-based classifiers with semantic cointension, Information Science 181:20 (2011) 4361-4377, doi: 10.1016/j.ins.2011.02.014), describe a novel methodology for designing interpretable fuzzy rule-based classifiers. The main contribution of this work arises from the fact that the proposed design process is guided by a new interpretability measure grounded on what is named as semantic cointension approach. (Andri Riid and Ennu Rüstern, Identification of transparent, compact, accurate and reliable linguistic fuzzy models, Information Science 181:20 (2011) 4378-4393, doi: 10.1016/j.ins.2011.01.041), tackle the search of a good interpretability-accuracy trade-off in the context of 0-th order Takagi–Sugeno fuzzy systems where priority is usually given to accuracy disregarding interpretability. (Adolfo R. de Soto, A hierarchical model of a linguistic variable, Information Science 181:20 (2011) 4394-4408, doi: 10.1016/j.ins.2011.04.006), introduces a theoretical hierarchical model of linguistic variables paying attention to both syntactic and semantic definitions. (Serge Guillaume and Brigitte Charnomordic,Learning interpretable fuzzy inference systems with FisPro, Information Science 181:20 (2011) 4409-4427, doi: 10.1016/j.ins.2011.03.025), present an open source software for fuzzy inference system design and optimization called FisPro.

Slides for Interpretability of FRBSs Presentations

R. Alcalá (August 2011). A double Axis Classification of Interpretability Measures for Linguistic Fuzzy Rule-Based Systems

The slides of this talk are associated to the paper: M.J. Gacto, R. Alcalá, F. Herrera, Interpretability of Linguistic Fuzzy Rule-Based Systems: An Overview of Interpretability Measures, Information Sciences, 181:20 (2011) 4340–4360 doi: 10.1016/j.ins.2011.02.021  .
These slides were presented in the 9th International Workshop on Fuzzy Logic and Applications (WILF 2011).