Data Mining: Practical Machine Learning Tools and Techniques, Second Edition

ration. All who have worked on the machine learning project here have con-

tributed to our thinking: we would particularly like to mention Steve Garner,

Stuart Inglis, and Craig Nevill-Manning for helping us to get the project off the

ground in the beginning when success was less certain and things were more

difficult.

The Weka system that illustrates the ideas in this book forms a crucial com-

ponent of it. It was conceived by the authors and designed and implemented by

Eibe Frank, along with Len Trigg and Mark Hall. Many people in the machine

learning laboratory at Waikato made significant contributions. Since the first

edition of the book the Weka team has expanded considerably: so many people

have contributed that it is impossible to acknowledge everyone properly. We are

grateful to Remco Bouckaert for his implementation of Bayesian networks, Dale

Fletcher for many database-related aspects, Ashraf Kibriya and Richard Kirkby

for contributions far too numerous to list, Niels Landwehr for logistic model

trees, Abdelaziz Mahoui for the implementation of K*, Stefan Mutter for asso-

ciation rule mining, Gabi Schmidberger and Malcolm Ware for numerous mis-

cellaneous contributions, Tony Voyle for least-median-of-squares regression,

Yong Wang for Pace regression and the implementation of M5

¢, and Xin Xu for

JRip, logistic regression, and many other contributions. Our sincere thanks go

to all these people for their dedicated work and to the many contributors to

Weka from outside our group at Waikato.

Tucked away as we are in a remote (but very pretty) corner of the Southern

Hemisphere, we greatly appreciate the visitors to our department who play 

a crucial role in acting as sounding boards and helping us to develop our 

thinking. We would like to mention in particular Rob Holte, Carl Gutwin, and

Russell Beale, each of whom visited us for several months; David Aha, who

although he only came for a few days did so at an early and fragile stage of the

project and performed a great service by his enthusiasm and encouragement;

and Kai Ming Ting, who worked with us for 2 years on many of the topics

described in Chapter 7 and helped to bring us into the mainstream of machine

learning.

Students at Waikato have played a significant role in the development of the

project. Jamie Littin worked on ripple-down rules and relational learning. Brent

Martin explored instance-based learning and nested instance-based representa-

tions. Murray Fife slaved over relational learning, and Nadeeka Madapathage

investigated the use of functional languages for expressing machine learning

algorithms. Other graduate students have influenced us in numerous ways, par-

ticularly Gordon Paynter, YingYing Wen, and Zane Bray, who have worked with

us on text mining. Colleagues Steve Jones and Malika Mahoui have also made

far-reaching contributions to these and other machine learning projects. More

recently we have learned much from our many visiting students from Freiburg,

including Peter Reutemann and Nils Weidmann.

x x x

P R E FAC E

P088407-FM.qxd  4/30/05  10:55 AM  Page xxx

---

Ian Witten would like to acknowledge the formative role of his former stu-

dents at Calgary, particularly Brent Krawchuk, Dave Maulsby, Thong Phan, and

Tanja Mitrovic, all of whom helped him develop his early ideas in machine

learning, as did faculty members Bruce MacDonald, Brian Gaines, and David

Hill at Calgary and John Andreae at the University of Canterbury.

Eibe Frank is indebted to his former supervisor at the University of

Karlsruhe, Klaus-Peter Huber (now with SAS Institute), who infected him with

the fascination of machines that learn. On his travels Eibe has benefited from

interactions with Peter Turney, Joel Martin, and Berry de Bruijn in Canada and

with Luc de Raedt, Christoph Helma, Kristian Kersting, Stefan Kramer, Ulrich

Rückert, and Ashwin Srinivasan in Germany.

Diane Cerra and Asma Stephan of Morgan Kaufmann have worked hard to

shape this book, and Lisa Royse, our production editor, has made the process

go smoothly. Bronwyn Webster has provided excellent support at the Waikato

end.

We gratefully acknowledge the unsung efforts of the anonymous reviewers,

one of whom in particular made a great number of pertinent and constructive

comments that helped us to improve this book significantly. In addition, we

would like to thank the librarians of the Repository of Machine Learning Data-

bases at the University of California, Irvine, whose carefully collected datasets

have been invaluable in our research.

Our research has been funded by the New Zealand Foundation for Research,

Science and Technology and the Royal Society of New Zealand Marsden Fund.

The Department of Computer Science at the University of Waikato has gener-

ously supported us in all sorts of ways, and we owe a particular debt of

gratitude to Mark Apperley for his enlightened leadership and warm encour-

agement. Part of the first edition was written while both authors were visiting

the University of Calgary, Canada, and the support of the Computer Science

department there is gratefully acknowledged—as well as the positive and helpful

attitude of the long-suffering students in the machine learning course on whom

we experimented.

In producing the second edition Ian was generously supported by Canada’s

Informatics Circle of Research Excellence and by the University of Lethbridge

in southern Alberta, which gave him what all authors yearn for—a quiet space

in pleasant and convivial surroundings in which to work.

Last, and most of all, we are grateful to our families and partners. Pam, Anna,

and Nikki were all too well aware of the implications of having an author in the

house (“not again!”) but let Ian go ahead and write the book anyway. Julie was

always supportive, even when Eibe had to burn the midnight oil in the machine

learning lab, and Immo and Ollig provided exciting diversions. Between us we

hail from Canada, England, Germany, Ireland, and Samoa: New Zealand has

brought us together and provided an ideal, even idyllic, place to do this work.

P R E FAC E

x x x i

P088407-FM.qxd  4/30/05  10:55 AM  Page xxxi

---