Fig. 1. PAT-Java example presentation interface. The problem-solving window with the spreadsheet is now controlled by the example presentation module. The system is ready to present the next problem-solving step to the student.
Abstract. In this paper we show that, with an appropriate component-based architecture, new functionality can be added to an Intelligent Tutoring System (ITS) with minimal effort. In particular, we show that an explanation function can be added to a component-based ITS which was originally designed to support activity in a learning-by-doing environment. We support these two claims by presenting our recent efforts to extend the Java Algebra Tutor, a variant of the PAT algebra tutor, with a generic example explanation module.
A human tutor performs several different activities in the teaching process. For example, when teaching some math topic in the classroom a human tutor may:
For an ITS to approach the flexibility and generality of a human tutor, it will need to support at least these kinds of activities, in ways that adapt to individual students' needs and abilities. Adaptive hypermedia systems [2] can present the core knowledge. Problem solving support systems [6] can help the student in the process of problem solving. Solution analyzers [15] can explain errors and deduce misconceptions. Problem sequencing systems [7] can analyze the student model and select the most relevant activity. However, no existing single ITS can intelligently perform more than a few of the activities listed above.
There is a very practical reason for this lack. Existing "single-purpose" ITSs are a natural result of the traditional approach to ITS development. With this approach, developing an ITS which can support two or three different activities doubles or triples the research and development time. Very few ITS research groups or developers can afford this effort. At the same time, to be useful, ITSs have to support some reasonable part of the teacher's or/and student's work. Otherwise, the benefit from using one more system is not worth the burden of installing and learning it.
This problem was analyzed by Brusilovsky [1] in his talk at the AI-ED'95 conference. The author claimed that the problem can be resolved with a new approach to ITS development and a new architecture based on re-usable interacting components. With such a component-based architecture, new functionality could be added to an ITS with minimal effort through re-use of the existing components in another context. At the same conference, Ritter and Koedinger [12] suggested one promising component-based architecture. This architecture has since been applied to the development of several component-based ITSs [11; 13].
In this paper we present another component-based ITS built along the lines of Ritter and Koedinger's [13] architecture. This system provides both interactive problem solving support and step-by-step example presentation in basic algebra. The example presentation functionality was added to the core problem-solving support engine with minimal programming by re-using existing modules in another context. One of the goals of this paper is to demonstrate how new functionality in the given component-based architecture can be added by re-using existing components. To achieve this goal, we provide some basic background on Ritter and Koedinger's [13] plug-in architecture, briefly present the original problem-solving support system and describe how the example explanation component was built by re-using the modules of the original system.
The Java Algebra Tutor is a variant of the Practical Algebra Tutor (PAT), described in [6]. The tutor assists students in solving word problems described by one or two linear equations. Students solve these problems by completing a spreadsheet and graph describing the problem situation and by solving equations related to the problem situation. The system encourages active learning-by-doing. Students can ask for hints at any step of the process, and the system provides immediate feedback on the appropriateness of each student action. The Java Algebra Tutor currently implements only the spreadsheet portion of the system.
PAT contains an expert system capable of solving the problems that are posed to students. As students take steps to complete the problem (for example, by filling in cells in the spreadsheet), the tutor considers whether or not those steps are consistent with a solution that it would follow. If not, the tutor checks to see if the step is consistent with a common error (or "bug"). In such cases, the tutor is able to provide instruction tailored to that bug. Since the tutor is tracking the student's solution at each step, the tutor is able to give help associated with the student's solution path at any time.
In addition to providing help and identifying errors, the tutor continually assesses the student's progress. Each of the rules in the underlying expert system represents a skill that the student needs to master in order to solve the problem. Each skill is either mastered or not, and the tutor maintains, for each student, the probability that the student has mastered the skill. These probabilities are displayed graphically on the "skillometer," which provides the student with information about his or her progress along each of the component skills in the curriculum.
The Java Algebra Tutor contains a compiled version of the expert system's rules, as they apply to the current problem situation. Instead of pattern matching against rules in the expert system, the tutor simply matches student inputs and branches along predetermined paths. This compiled version of the system provides better response time with a lower memory requirement, while still preserving the entire behavior of the model-tracing system. From the user's point of view, there is no difference between the systems.
While the Java Algebra Tutor provides support for students' problem-solving activities, examination of its initial use made it clear that more specific declarative support for the activity was needed. This is especially important when the instruction is being delivered over the World-Wide Web. Unlike in the classroom context, students accessing the tutor over the World-Wide Web do not have a teacher or other student to guide them in working through a new problem type, and students are much more likely to jump around the curriculum, thus getting less practice on multiple problems of the same format. These students find it tedious to be thrown into the middle of a problem-solving situation, with the only guidance being the ability to ask for help at each step along the way. To provide some declarative instruction, [3] developed a link between an online tutor for algebra and declarative instruction delivered through InterBook [2].
Still, the addition of declarative instruction through InterBook represents only one step towards more flexible presentation of domain knowledge. An obvious next step would be to add the ability for students to follow complete examples of problem solving using the exact tools they will be asked to use to solve problems. Ideally, these examples would be personalized to reflect each student's level of skill. We can distinguish three basic methods of presenting the information required to master a skill: learning from declarative information, learning by doing, and learning by example. These methods of learning may involve different cognitive mechanisms [14] and can be seen as complimentary. They may account for forming different components of a complex skill [8]. An effective tutor (human or computer) should apply all the three in combination.
Despite a growing body of work on learning from examples in the psychological literature [c.f. 5; 16], learning from examples has received relatively minor attention in the ITS area. ITS researchers acknowledge the role of learning from examples in problem solving [9], but existing ITSs rarely support this method. The ones that do offer an example when it is required [4; 10; 17], typically leave the student alone to explore the examples. We do not know any ITS which provides step-by-step examples in a way that is sensitive to individual students' abilities.
In this paper, we show how this step-by-step example presentation can be added with minimal effort to any system which includes problem solving expertise and an appropriate component-based architecture. Moreover, as we will show later, our example presentation module itself could be re-used to present examples in different domains, thus reducing the efforts to implement this important functionality.
To add example presentation to PAT-Java, we have developed a small example-presentation module which performs its duties mainly by communicating with existing components such as the Tutor and the Spreadsheet. The module interacts with the student through a small dialog (Figure 1) which appears when the student select an example to learn.
Fig. 1. PAT-Java example presentation interface. The problem-solving window with the spreadsheet is now controlled by the example presentation module. The system is ready to present the next problem-solving step to the student.
Once the student has selected a problem on which to receive an example solution, the student presses the NEXT STEP button (Figure 1). Each step of the example consists of two substeps. On the first substep, the system prepares the student for the next step. It highlights the next cell to be filled and tells the student what is supposed to be entered on this step. This information is presented using an existing interface component: the help window used to present all help text in the Java Algebra Tutor. This window includes two buttons, labeled >>> and <<<, which allow the student to request more or less detailed explanations of why a particular step is appropriate (Figure 2). At this point, the cell to be filled is still highlighted, but empty.
Fig. 2. The display after the student pushes NEXT STEP, and the system starts to explain a step in the solution.
When the NEXT STEP button is pushed, its text changes to DO IT! To start the second substep, the student has to push the DO IT! button. After that, the system simply performs the problem solving step, i.e., fills the current cell. (Figure 3). At this point, the DO IT! button is converted back to say NEXT STEP, and the system is ready to present the next step to the student.
Fig. 3. After the student pushed the DO IT button, the explained step is performed and all changes are made in the interface. The system is ready to explain next problem-solving step.
By providing a progressively more detailed set of explanations and by separating the demonstration of the action from the explanation itself, we expect to give them a better understanding of each problem-solving step. We also hope to promote students' self-explanations [5] as they go through the problem. That is, we expect students to rely more and more on their own explanations of the actions suggested by the example presentation module and less on the accompanying text.
The sequence of steps presented to the student adapts to the student's knowledge level in the following way. If a student has demonstrated, through prior problem solving, that he or she has the ability to complete a particular step, then the system will immediately perform that step, without the student needing to intervene by pressing the NEXT STEP or DO IT! buttons. For example, if a student has demonstrated the ability to correctly complete the column labels in the spreadsheet, the example presentation module will fill in those labels, without any explanations being provided, before demonstrating to the student how to enter the unit of measure.
Ritter and Koedinger [12; 13] proposed a "plug-in" architecture for intelligent tutoring systems. The primary purpose of the architecture was to allow the incorporation of commercial, off-the-shelf software into a tutoring environment. This was accomplished by enforcing a distinction between the tutoring knowledge and the tools which students use to solve problems. In this context, the "tool" was defined as the program that the user sees (such as a spreadsheet, word processor or simulation environment), and the "tutor agent" was defined as the program that contains the tutoring knowledge and is able to evaluate and offer advice to the student using the tool. The plug-in architecture provides the ability to monitor the student's behavior at an appropriate level and the ability to provide feedback to students in a flexible manner, so that the tutor agent does not need to be aware of many of the details of the tool. Some messages in this "plug-in" architecture are presented in Table 1. In the Java Algebra Tutor and the example presentation module, we are taking advantage of the "plug-in" architecture for a very different purpose. Instead of incorporating off-the-shelf tools, the Java Algebra Tutor uses a custom-built spreadsheet module. Still, the design of the tutor agent component isolates the appropriate elements of the system in a way that allows a novel component, the example presentation module, to use the tutor agent for its own purposes.
This system provides an illustration of how it is possible, in a well-defined component architecture, to re-use the system's knowledge in an unanticipated context. The tutoring component, which is the part of the system responsible for deciding which step to take next, what help text to give for it and which skills are involved in taking that step, is the exact same program that is used in the learning-by-doing system. The example presentation module system is opportunistically taking advantage of some of the features of the tutoring component that result from it following Ritter and Koedinger's [13] plug-in architecture. In this section, we describe some aspects of the architecture that make this possible.
| Message
|
Parameters
|
Handled by
|
Meaning
|
| start problem
|
initial, goal state
|
tutor agents
|
start an activity
|
| process tool action
|
selection, action, input
|
tutor agents
|
student performed some action
|
| process help request
|
selection, input
|
tutor agents
|
student asks for hint
|
| process done
|
tutor agents
|
student indicates problem complete
| |
| get next step
|
tutor agents
|
request for next step in solution
| |
| perform user action
|
selection, action, input
|
tool
|
perform some action
|
| get user value
|
object, property, datatype
|
tool
|
request for some value of tool property
|
Tab. 1. Subset of messages defined in the plug-in architecture in [13].
The plug-in tutoring architecture defines a strict distinction between a tool, which is the component of the system with which the user interacts in order to solve a problem, and the tutor agent (also called tutoring component) which is the part of the system that is responsible for guiding the student in the problem solving process and evaluating the student's abilities in the domain. The plug-in architecture defines a set of messages that allow the tool to communicate the semantics of student actions to the tutor agent and a set of messages that are sent from the tutor agent to the tool in order to give feedback to the student and initialize the state of the tool. The plug-in architecture also defines messages that can be used to query the state of the tool and tutor agent.
The basic algorithm used by the example presentation module is as follows:
Steps 3 to 5 are repeated until the tutor agent indicates that the student has completed the problem. It should be apparent that (with the exception of a few elements of its user interface) the example presentation module is completely general. The algorithm described is in no way specific to the Java Algebra Tutor, algebra, or to tutors using a spreadsheet tool. The example presentation module does not need to interpret the semantics of the recommended student step; it just passes it back on to the tutor agent as the student's action. Similarly, it doesn't need to know the semantics of the help text or the skills affected by student actions; it just needs to be able to display the help text and access student model information for the skills it is told about. Thus, the example presentation module can be used with any tutor agent and tool following the plug-in architecture.
The algorithm used by the example presentation module is similar to one described in [11]. In that paper, a system component queried the tutor agent (one used to teach equation solving) repeatedly to discover a recommended problem solution. However, instead of using that solution path to demonstrate an example to the student, that system used the information to compare the recommended solution to that performed by the student, as a way of identifying places where the tutor agent was deficient. Both of these cases demonstrate that, although tutor agents are written with the primary purpose of providing instruction in a learn-by-doing system, when they are implemented as components with appropriate interfaces, their knowledge can be reused in novel and interesting ways. In this architecture, knowledge re-use is different from knowledge sharing. Different components do not (necessarily) have access to the same knowledge base, as they would in a typical blackboard system. Instead, each component is assumed to maintain its own knowledge, but it is able to communicate aspects of its understanding when queried.
In this paper we have tried to demonstrate that new functionality can be relatively easily implemented in an ITS built with a component based architecture, such as the "plug-in" architecture [13]. In particular, we have tried to show how to implement an example explanation module in a system with problem-solving expertise. We hope that the ideas and the real-world example presented in this paper will encourage other ITS developers to use a component based architecture and to consider an example explanation module as an extension of existing problem solving support systems.
Among our own plans is to continue the work on component architecture for ITSs. We hope to experiment with more components and to develop a tool kit for building component based ITSs with Java on the Internet. This model for example presentation can be made much more adaptive. Currently, the system has two options for explaining each step of the example. If the skills behind the step are mastered, the system will not stop to explain the step at all. If the skills are not yet mastered, the step will be explained in two substeps. There are several ways of improving on that. First, we can manipulate the granularity of a step explanation to a greater extent. For some simple steps already seen before, one substep is enough. For some complex steps which are new for the student, even two sub-steps and brief explanations may be not enough to understand why something was done by the system. Second, the text and vocabulary of the explanation generated for each substep could be adapted to the student's knowledge. Third, the order of showing the steps may be adapted to the student's background knowledge. For example, we might recommend that an algebra novice fill in the formula cell at the very end, so that completed rows in the spreadsheet can support inductive inference of the formula. On the other hand, a student who already understands formulas would be instructed to fill in the formula cell before calculating the answers, so that the system will calculate using the formula and the entered givens (as a spreadsheet does).
This work was supported by the National Science Foundation and the Advanced Research Projects Agency under Cooperative Agreement CDA-940860, and by a James S. McDonnell Foundation grant to Peter Brusilovsky.