Imagine your children receiving personalised learning through a one-to-one tutor that is available 24/7, that knows what both their skills and capabilities are throughout their lives and provides the adequate high-quality learning for each of them to maximise their potential and learning outcomes. Interested?
What if I tell you that this tutor is virtual and run in a computer program? Still interested?
Intelligent Tutoring Systems (ITS) are computer programs that use Artificial Intelligence techniques and Cognitive Science to enhance and personalise learning through virtual tutors. This concept is not new, as the earliest intelligent tutor was implemented in the 1970s (J. Carbonell, 1970). Intelligent tutors (as master human tutors do) individualise instructions and provide adequate responses to students adapted to each student’s strengths and weaknesses.
In recent years, many ITSs have been developed and tested, either used alone or in combination with traditional learning methods, proving to be effective in helping to learn specific subjects as maths and grammar and improving the learning outcomes of students. Despite that the gap between human tutors and virtual tutors is lessening, ITSs still have many limitations. Therefore there is no intelligent tutor that has the cognitive understanding of an actual human tutor so far. Nevertheless, when the limitations are overcome, they could become either a great alternative or complement to human tutors considering their benefits in cost, time, availability and scale.
Tutoring benefit, most students have the potential to reach a high level of learning
In 1984, Benjamin Bloom, an educational psychologist, discovered that the average student under individual tutoring (1:1 teacher/student ratio) was about two standard deviations (two sigma) above the average of a conventional class. Meaning that the average tutored student was above 98% of the student in a conventional class or that she could perform better than 49 out of every 50 students within a traditional classroom. This was a breakthrough discovery as the tutoring process demonstrates that most of the students do have the potential to reach this high level of learning, specifically for the 80% of students that do relatively poor under conventional instruction as compared with what they might do under tutoring. (B.S. Bloom, 1984)
But the “not-so-good” news is that the one-to-one tutor method is not scalable. To put it simply, there are no enough one-to-one skilled human tutors for each student. To tackle this problem three main approaches have been tested up to date i) to implement methods of group instruction aiming at being as effective as the one-to-one tutoring. Bloom proposed a combination of Mastery learning with complementary teaching methods (in mastery learning each student must achieve mastery in a subject before moving to the next advanced subject, with no time constraint) ii) online platforms that manage and provide one-to-one human mentoring. This optimises the availability of mentors but at some point face the issue of cost efficiency and 24/7 availability worldwide iii) limited deployment of ITSs either for the private tutoring market or in the formal education system (performing trials).
These approaches have provided limited results and therefore, in my view, the definitive solution is to develop ITSs at full potential being accessible, effective, efficient, and affordable to all. What are the challenges to build and implement such type of system? Keep reading…
Artificial Intelligence (AI) and Cognitive Science enable the revolution in learning
Applied in learning, Artificial Intelligence without Cognitive Science is like rock without roll, they do work together. Cognitive Science includes AI as part of its multidisciplinary essence.
Cognitive Science is the science of the mind and intelligence, comprising philosophy, psychology, Artificial Intelligence, neuroscience, linguistics, and anthropology (Stanford Encyclopedia of Philosophy). Therefore, it is an amazing and exciting interdisciplinary study of the mind that benefit from the diversity of outcomes from all those different fields and where great innovation is simply created by combining things that have never been put together before.
The most fundamental assumption of cognitive science is that minds are information processors. The dominant hypothesis is that thinking is best understood as representational structures in the mind (analogous to computer data) that are managed by computational procedures (similar to computational algorithms). Accordingly, the mind would contain mental representations such as rules, concepts, logical propositions, images, and analogies and it would use mental procedures as search, deduction, matching, and retrieval.
Unlike cognitive science, Artificial Intelligence is generally well known because in the last few years everything claims to have “AI inside” such as TVs and cars…despite only a few people understand what AI is. While AI has been around for nearly 60 years, it nevertheless remained a fringe technology until only recently because of sweeping changes in recent years, entailing the abundance of data (big data), economic access to computing power and advances in Machine Learning (the ability to learn from data without being explicitly programmed).
Industry interest has leveraged these advances to extract huge value out of data, but current AI developments seen to focus mostly on data-driven AI. Artificial Intelligence is more than that, AI attempts not just to predict what will happen next based on data but also to build intelligent entities. In my eyes ITSs will become the driver to speed up the development of knowledge-based AI (based on an explicit representation of domain knowledge that a machine reason about), the less mature-missing part to build a complete AI system.
Using Artificial Intelligence in Learning could help to promote personalisation, get better outcomes and help advance collaborative learning. From a business perspective, the greatest potential application for AI in learning is predicted in assessment and language learning. Regarding ITSs, it seems to me that the adoption will come first from the private tutoring market. Nevertheless, the real revolution will come when AI was applied massively in Education (AIED), i.e formal education system in schools, but I am afraid that apart from some testing we will have to wait. AI in education faced many difficulties to grow because education systems around the world are more reluctant to technological changes in their traditional organisation (education is 10 years behind any other industry adopting a new technology).
Building an ITS, basic components
Computers have been used in education since the 1960s under the Computer Assisted Instruction (CAI) concept. These earlier CAI programs, despite having achieved some improvements in students’ scores, have several limitations. They are described as branching programs, recognising only the built-in answers defined by the teacher that builds the program. Unlike CAIs, ITSs (using AI) dynamically analyses the solution history of each student and decide on what to do next, using principles of how to teach. This requires an architecture that integrates different knowledge modules.
Although different types of ITS could have different architectures there are four basic components that the traditional representation of an ITS includes (W. Clancey, 1986) (A. Alkhatlan and J. Kalita, 2018)
- Domain Knowledge. It represents the expert knowledge about the content that will be taught “what to teach“. It includes the knowledge base (the kernel of the system), a formal model of how the problem is to be solved including definitions procedures, facts, and rules.
- Student model. It contains the student’s knowledge and skills, “whom to teach”. It is a description of what the student knows. It includes information about the student as time spent in the problems, possible misconceptions, hints provided, correct and incorrect answers and preferred learning style.
- Tutoring model, “how to teach”. It includes the teaching strategies, the pedagogical knowledge and the decision on how and when to interact with the student.
- User interface with a proper user experience design (UX), “how to interact” represents advanced methods and strategies for communicating between humans and computers leveraging the Human-Computer Interaction (HCI) field of study. It is the “look and feel” for the tutor and the hook for the student’s engagement.
For those ITSs that use all four modules (ITSs differ greatly in their architecture), the teaching-learning cycle would be as follows (in a very simplified way): i) a customised problem is presented to the student. As the student is solving the problem she might be receiving new data ii) at some point she requests some data or makes some questions. As the tutor is always watching, it passes this request and all the information that the student has received about the problem to the expert knowledge module (a problem-solving simulator program), that queries the knowledge base and generates the preferred behaviour of what to do next. iii) this input the student module that relates the student behaviour (that is monitored) to the expert model and gets what the student knows (as a subset of the expert knowledge base) weighted with degrees of belief for each item. iv) this information fed back the tutoring module that decides what to say next (give a piece of advice, make an interruption, give a hint, make some explanation…) and gathers new information from the student’s learning (behaviour, answer, comments..) to start a new teaching-learning cycle.
There are several ITS approaches to develop and represent each component (e.g three types of domain knowledge models used frequently in ITSs are the cognitive model, the constraint-based model, and the inclusion of an expert system). For those interested in further information, I would suggest reading (A. Alkhatlan and J. Kalita, 2018) (B. P. Woolf, 2010)
Key Challenges for ITSs to become revolutionary
As I commented at the beginning, in recent years numerous effective and successful ITS have been built, but they still need to bridge the gap with an actual human tutor and become more efficient and affordable to all. To my mind, these are the challenges and significant areas of research that an ITS need to focus on to reach its full potential:
- ITSs need to implement the main components of learning: Cognition, Motivation, and Emotion. Historically, the acquisition and development of intellectual competences have been focused just in cognition and explained in terms of cognitive capacity. However, recent learning theories have supported an integrative approach that views cognition, motivation, and emotion as intrinsically related so learner’s cognitive ability depends on her emotion and motivation. (D. Yun Dai and R. J. Stenberg 2004)
- Affective Tutoring Systems are ITSs that can recognise human emotions. They use various technologies and techniques to recognise, model, understand and respond to student’s emotion. Last year I had the opportunity to meet some main players in the education sector in China and I could observe first hand the actual application of facial and voice recognition technology to these systems, capable to detect affective states of the learner such as confused, happy, neutral, sad, scared, bored, angry, disappointed and surprised. Although this represents great progress for the actual application of AI, it raises some privacy and ethical concerns that need to be addressed as a key issue in the implementation of AI.
- Succeed in engaging students. ITSs despite their advances and incremented complexity do not always succeed in engaging the student. Motivations are typically conceptualised as processes that drive goal-directed behaviours aimed at achieving desired outcomes and avoiding undesired ones. The student’s motivation has an immense impact on her learning outcomes and ITSs need to be designed including the detection, diagnosis, development, and response to student’s motivation.
- I hold the view that Game-based tutoring systems are the best approach to get a student’s engagement and make them more motivated to use these intelligent systems for a longer time. The novelty of an ITS and its interactive component could help students pay attention for a few hours, but could be boring and monotonous after weeks or months. Students learn better when they are having fun and engaged in the learning process and besides it has been found that educational games have advantages over traditional tutoring approaches (G. T. Jackson and D. MacNamara, 2011). Furthermore, there is a current interest in the design of educational games that use the motivational impact of uncertain rewards to engage learners and to enhance their learning (Yes! uncertainty can be motivating). The addition of AIED techniques to the design of these educational games would enable, for example, the provision of uncertain rewards to be calibrated to a learner’s reaction to a given level of uncertainty.
- And to develop a great game-based affective tutoring system (all of that) a differentiated user experience design is required, providing excellent and effective communication between the virtual tutor and the learner. Most of the ITSs that I have tested, despite having great interactive components, are “no so good looking” and feel as just computers (maybe not passing the Turin test for me). ITSs would leverage the Human-Computer Interaction (HCI) field of study to implement advanced methods for communicating between human and computers, that will make learners more engaged and motivated. Apart from using sophisticated graphic techniques, Natural Language Processing (NLP) is a key component to improve human-machine interaction as it is concerned with both understanding and generating language.
- Internet of Things implementation is key to make ITSs “feel”, i.e detecting what learner is doing and recognising her emotions and motivations by providing identification, sensing, communication, and cloud computational capacity.
- How effective are ITSs? There is no consensus on the effectiveness of ITSs in providing the learning outcomes they claim to provide. There have been some studies and analysis to investigate this issue and although the results indicate that students using an ITS outperform to those receiving traditional class instructions, the improvements don’t reach up to Bloom’s finding of the 2.o multiplicative effect of human tutoring (those studies reported a median effect size up to 0.5, but varies depending on the subject domain and other factors) (A. Alkhatlan and J. Kalita, 2018). The main conclusion is that there is still a lot of research to do and questions to answer about both the critical factors that affect learning in an ITS and the changes to be made to improve them.
- Difficult to build. Developing a new ITS is a hard and challenging task that requires complex reasoning and the collaboration among programmers, teachers, and domain expert (education and technology) and few tools exist to make this job easier. It is estimated that one hour of instruction for an ITS could require up to 200 hours of development time (B. P. Woolf, 2010). In recent years there has been a strong interest in simplifying the process of building an ITS by doing authoring of ITS easier and more affordable to programmers and teachers. Authoring tools enable faster development of tutors, reduce the building effort, and create more ITSs (and more diverse). There are different categories of authoring tools (pedagogy-oriented, performance-oriented, a model-tracing system as cognitive tutor authoring tools…) but the main practical category is those that require programming skills and those that do not (opening the development of an ITS to a wider range of participants). New approaches are being tested in this field and this month researchers at Carnegie Mellon University have published a paper to rapidly develop ITSs with a new method, based on machine learning, claiming that may enable a teacher to create a 30-minute lesson in about 30 minutes.
- Generation and analysis of lifelong learning data, from early stages in children’s education through college graduation and beyond. Nowadays, there is no such database that would enable unique insights improving learning outcomes, using data mining and learning analytics. Ideally, these lifelong learning databases would be the learning records of an individual, integrating data from different learning systems such as her ITS and any educational management information system (EMIS) from the school she is attending (ethics, privacy and security are a must).
- ITSs to develop 21st century competencies beyond tutoring on core academic skills (as maths or language). In my view, Collaborative Problem Solving is the most essential cross-cutting capability for an ITS to be focused on, as it incorporates both the behavioural skills and thinking skills required to effectively solve a problem. This development would be supported by the progress and application of AI to collaborative learning.
- Redefinition of the human teacher role in a joint learning system with ITS and students. The human teacher could act as both an ITS designer (with a proper authoring tool) and a teaching collaborator. Regarding this point, I mainly considered ITSs as an assistant to the human teacher (that had the main role) enabling teachers to become more “productive” having quality time for those students requiring some learning personalisation. However, I changed my mind on my trip to China last year when I observed ITSs as the main tutor for English language learning. They explained to me that they couldn’t find so many native english speaker human tutors for so many millions of Chinese students so they developed an effective virtual tutor. Although ideally, human teachers should generally play the main role in a learning system, some applications make sense for ITSs to step up.
- Develop effective ITSs business models for both the private tutoring market and the education system (governments). Today, ITSs are expensive and some of them (as the main publishers) try to recover in short-term all the investment they made making no case for success. The global private tutoring market size is predicted to reach over hundreds of billion dollars by 2025 (these figures are being updated because of covid-19), so ITSs should aim at growing and increasing its market share.
- Do not let developing countries to fall behind in the implementation of ITSs. As I explained in the paper I published with other experts last year, we need to democratise access to ITSs making them affordable to all and taking into account both the barriers faced entering the developing world and giving them the cultural awareness required to be adapted to a diversity of cultures.
Maybe you have noticed that I have tried to avoid giving specific examples of ITSs. Nevertheless, if I had to highlight just a few players because of either it has become an ITS best practice or their contribution to this field, would be i) an ITS as MATHiau (by Carnegie Learning) for learning maths ii) several companies that have developed and improved expert systems for years as iteNlearning (based on cognitive science expertise) and iii) AI education companies and Tech organisations in China: New Oriental Education, TAL education, Squirrel, ByteDance… In the further reading section that follows, you may find more examples and analysis of Intelligent tutors.
Further reading
J. Carbonell (1970) “AI in CAI: An artificial-intelligence approach to computer-assisted instruction,” IEEE Transactions on Man-Machine Systems, vol. 11, no. 4, pp. 190–202.
B. S. Bloom (1984) “The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring”, Educational researcher, vol. 13, no. 6, pp. 4–16.
W. Clancey (1986)” Intelligent Tutoring Systems: a Tutorial Survey” Department of Computer Science. Stanford University.
A. Alkhatlan and J. K. Kalita (2018) “Intelligent Tutoring Systems: A Comprehensive Historical Survey with Recent Developments”.
B. P. Woolf (2010) “Building intelligent interactive tutors: Student-centered strategies for revolutionizing e-learning”. Morgan Kaufmann.
D. Yun Dai, and R. J. Stenberg (2004) “Motivation, Emotion and Cognition. Integrative perspectives of intellectual functioning and development”.
G. T. Jackson and D. McNamara (2011) “Motivational impacts of a game-based intelligent tutoring system,” in Twenty-Fourth International FLAIRS Conference. Bermudez, J.L (2014). “Cognitive Science. An Introduction to the Science of the Mind”. Cambridge University Press
D. Weitekamp III, E. Harpstead and K. R. Koedinger (2020) “An Interaction Design for Machine Teaching to Develop AI Tutors”. Carnegie Mellon University. F.Pedro, M.Subosa, A. Rivas and P. Valverde (2019) “Artificial Intelligence in Education: Challenges and Opportunities for Sustainable Development”. Unesco.
