Feature construction (also known as constructive induction or attribute discovery) enriches data by adding derived features. These can enrich a data analysis pipeline by capturing relevant relationships within the data that downstream processes are otherwise unable to model or exploit. They may also support explainable AI by making relationships explicit that would otherwise be implicit and difficult to comprehend.
Our pioneering research demonstrated that feature construction can empower machine learning systems to construct more accurate models across a wide range of learning tasks.
Publications
Yip, S.; Webb, G. I.
Incorporating Canonical Discriminate Attributes in Classification Learning
Elio, R. (Ed.): Proceedings of the Tenth Biennial Canadian Artificial Intelligence Conference(AI-94), pp. 63-70, Morgan Kaufmann, Banff, Canada, 1994.
@inproceedings{YipWebb94b,
title = {Incorporating Canonical Discriminate Attributes in Classification Learning},
author = {S. Yip and G. I. Webb},
editor = {R. Elio},
year = {1994},
date = {1994-01-01},
booktitle = {Proceedings of the Tenth Biennial Canadian Artificial Intelligence Conference(AI-94)},
pages = {63-70},
publisher = {Morgan Kaufmann},
address = {Banff, Canada},
abstract = {This paper describes a method for incorporating canonical discriminant attributes in classification machine learning. Though decision trees and rules have semantic appeal when building expert systems, the merits of discriminant analysis are well documented. For data sets on which discriminant analysis obtains significantly better predictive accuracy than symbolic machine learning, the incorporation of canonical discriminant attributes can benefit machine learning. The process starts by applying canonical discriminant analysis to the training set. The canonical discriminant attributes are included as additional attributes. The expanded data set is then subjected to machine learning. This enables linear combinations of numeric attributes to be incorporated in the classifiers that are learnt. Evaluation on the data sets on which discriminant analysis performs better than most machine learning systems, such as the Iris flowers and Waveform data sets, shows that incorporating the power of discriminant analysis in machine classification learning can significantly improve the predictive accuracy and reduce the complexity of classifiers induced by machine learning systems.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper describes a method for incorporating canonical discriminant attributes in classification machine learning. Though decision trees and rules have semantic appeal when building expert systems, the merits of discriminant analysis are well documented. For data sets on which discriminant analysis obtains significantly better predictive accuracy than symbolic machine learning, the incorporation of canonical discriminant attributes can benefit machine learning. The process starts by applying canonical discriminant analysis to the training set. The canonical discriminant attributes are included as additional attributes. The expanded data set is then subjected to machine learning. This enables linear combinations of numeric attributes to be incorporated in the classifiers that are learnt. Evaluation on the data sets on which discriminant analysis performs better than most machine learning systems, such as the Iris flowers and Waveform data sets, shows that incorporating the power of discriminant analysis in machine classification learning can significantly improve the predictive accuracy and reduce the complexity of classifiers induced by machine learning systems.
Yip, S.; Webb, G. I.
Empirical Function Attribute Construction in Classification Learning
Zhang, C.; Debenham, J.; Lukose, D. (Ed.): Artificial Intelligence: Sowing the Seeds for the Future, Proceedings of Seventh Australian Joint Conference on Artificial Intelligence (AI'94), pp. 29-36, World Scientific, Armidale,NSW, Australia, 1994.
@inproceedings{YipWebb94a,
title = {Empirical Function Attribute Construction in Classification Learning},
author = {S. Yip and G. I. Webb},
editor = {C. Zhang and J. Debenham and D. Lukose},
year = {1994},
date = {1994-01-01},
booktitle = {Artificial Intelligence: Sowing the Seeds for the Future, Proceedings of Seventh Australian Joint Conference on Artificial Intelligence (AI'94)},
pages = {29-36},
publisher = {World Scientific},
address = {Armidale,NSW, Australia},
abstract = {The merits of incorporating feature construction to assist selective induction in learning hard concepts are well documented. This paper introduces the notion of function attributes and reports a method of incorporating functional regularities in classifiers. Training sets are preprocessed with this method before submission to a selective induction classification learning system. The method, referred to as FAFA (function attribute finding), is characterised by finding bivariate functions that contribute to the discrimination between classes and then transforming them to function attributes as additional attributes of the data set. The value of each function attribute equals the deviation of each example from the value obtained by applying that function to the example. The expanded data set is then submitted to classification learning. Evaluation with published and artificial data shows that this method can improve classifiers in terms of predictive accuracy and complexity.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The merits of incorporating feature construction to assist selective induction in learning hard concepts are well documented. This paper introduces the notion of function attributes and reports a method of incorporating functional regularities in classifiers. Training sets are preprocessed with this method before submission to a selective induction classification learning system. The method, referred to as FAFA (function attribute finding), is characterised by finding bivariate functions that contribute to the discrimination between classes and then transforming them to function attributes as additional attributes of the data set. The value of each function attribute equals the deviation of each example from the value obtained by applying that function to the example. The expanded data set is then submitted to classification learning. Evaluation with published and artificial data shows that this method can improve classifiers in terms of predictive accuracy and complexity.
Yip, S.; Webb, G. I.
Discriminate Attribute Finding in Classification Learning
Adams, A.; Sterling, L. (Ed.): Proceedings of the Fifth Australian Joint Conference on Artificial Intelligence (AI'92), pp. 374-379, World Scientific, Hobart, Tas., Australia, 1992.
@inproceedings{YipWebb92b,
title = {Discriminate Attribute Finding in Classification Learning},
author = {S. Yip and G. I. Webb},
editor = {A. Adams and L. Sterling},
year = {1992},
date = {1992-01-01},
booktitle = {Proceedings of the Fifth Australian Joint Conference on Artificial Intelligence (AI'92)},
pages = {374-379},
publisher = {World Scientific},
address = {Hobart, Tas., Australia},
abstract = {This paper describes a method for extending domain models in classification learning by deriving new attributes from existing ones. The process starts by examining examples of different classes which have overlapping ranges in all of their numeric attribute values. Based on existing attributes, new attributes which enhance the distinguishability of a class are created. These additional attributes are then used in the subsequent classification learning process. The research revealed that this method can enable relationships between attributes to be incorporated in the classification procedures and, depending on the nature of data, significantly increase the coverage of class descriptions, improve the accuracy of classifying novel instances and reduce the number of clauses in class description when compared to classification learning alone. Evaluation with the data on iris flower classification showed that the classification accuracy is slightly improved and the number of clauses in the class description is significantly reduced.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper describes a method for extending domain models in classification learning by deriving new attributes from existing ones. The process starts by examining examples of different classes which have overlapping ranges in all of their numeric attribute values. Based on existing attributes, new attributes which enhance the distinguishability of a class are created. These additional attributes are then used in the subsequent classification learning process. The research revealed that this method can enable relationships between attributes to be incorporated in the classification procedures and, depending on the nature of data, significantly increase the coverage of class descriptions, improve the accuracy of classifying novel instances and reduce the number of clauses in class description when compared to classification learning alone. Evaluation with the data on iris flower classification showed that the classification accuracy is slightly improved and the number of clauses in the class description is significantly reduced.
Yip, S.; Webb, G. I.
Function Finding in Classification Learning
Proceedings of the Second Pacific Rim International Conference on Artificial Intelligence (PRICAI '92), pp. 555-561, Springer-Verlag, Seoul, Korea, 1992.
@inproceedings{YipWebb92a,
title = {Function Finding in Classification Learning},
author = {S. Yip and G. I. Webb},
year = {1992},
date = {1992-01-01},
booktitle = {Proceedings of the Second Pacific Rim International Conference on Artificial Intelligence (PRICAI '92)},
pages = {555-561},
publisher = {Springer-Verlag},
address = {Seoul, Korea},
abstract = {The paper describes a method for extending domain models in classification learning by deriving new attributes from existing attributes. The process starts by finding functional regularities within each class. Such regularities are then treated as additional attributes in the subsequent classification learning process. The research revealed that these techniques can reduce the number of clauses required to describe each class, enable functional regularities between attributes to be incorporated in the classification procedures and, depending on the nature of data, significantly increase the coverage of class descriptions and improve the accuracy of classifying novel instances when compared to classification learning alone.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The paper describes a method for extending domain models in classification learning by deriving new attributes from existing attributes. The process starts by finding functional regularities within each class. Such regularities are then treated as additional attributes in the subsequent classification learning process. The research revealed that these techniques can reduce the number of clauses required to describe each class, enable functional regularities between attributes to be incorporated in the classification procedures and, depending on the nature of data, significantly increase the coverage of class descriptions and improve the accuracy of classifying novel instances when compared to classification learning alone.