Creating more versatile intelligent learning environments with a component-based architecture

Steven Ritter1, Peter Brusilovsky2, and Olga Medvedeva1

1 Department of Psychology, Carnegie Mellon University
Pittsburgh, PA 15213, USA
{sritter, medol}@andrew.cmu.edu

2 School of Computer Science, Carnegie Mellon University
Pittsburgh, PA 15213, USA
plb@cs.cmu.edu

Abstract. In this paper we show that, with an appropriate component-based architecture, new functionality can be added to an Intelligent Tutoring System (ITS) with minimal effort. In particular, we show that an explanation function can be added to a component-based ITS which was originally designed to support activity in a learning-by-doing environment. We support these two claims by presenting our recent efforts to extend the Java Algebra Tutor, a variant of the PAT algebra tutor, with a generic example explanation module.

1 Introduction

A human tutor performs several different activities in the teaching process. For example, when teaching some math topic in the classroom a human tutor may:

For an ITS to approach the flexibility and generality of a human tutor, it will need to support at least these kinds of activities, in ways that adapt to individual students' needs and abilities. Adaptive hypermedia systems [2] can present the core knowledge. Problem solving support systems [6] can help the student in the process of problem solving. Solution analyzers [15] can explain errors and deduce misconceptions. Problem sequencing systems [7] can analyze the student model and select the most relevant activity. However, no existing single ITS can intelligently perform more than a few of the activities listed above.

There is a very practical reason for this lack. Existing "single-purpose" ITSs are a natural result of the traditional approach to ITS development. With this approach, developing an ITS which can support two or three different activities doubles or triples the research and development time. Very few ITS research groups or developers can afford this effort. At the same time, to be useful, ITSs have to support some reasonable part of the teacher's or/and student's work. Otherwise, the benefit from using one more system is not worth the burden of installing and learning it.

This problem was analyzed by Brusilovsky [1] in his talk at the AI-ED'95 conference. The author claimed that the problem can be resolved with a new approach to ITS development and a new architecture based on re-usable interacting components. With such a component-based architecture, new functionality could be added to an ITS with minimal effort through re-use of the existing components in another context. At the same conference, Ritter and Koedinger [12] suggested one promising component-based architecture. This architecture has since been applied to the development of several component-based ITSs [11; 13].

In this paper we present another component-based ITS built along the lines of Ritter and Koedinger's [13] architecture. This system provides both interactive problem solving support and step-by-step example presentation in basic algebra. The example presentation functionality was added to the core problem-solving support engine with minimal programming by re-using existing modules in another context. One of the goals of this paper is to demonstrate how new functionality in the given component-based architecture can be added by re-using existing components. To achieve this goal, we provide some basic background on Ritter and Koedinger's [13] plug-in architecture, briefly present the original problem-solving support system and describe how the example explanation component was built by re-using the modules of the original system.

2 The Java Algebra tutor

The Java Algebra Tutor is a variant of the Practical Algebra Tutor (PAT), described in [6]. The tutor assists students in solving word problems described by one or two linear equations. Students solve these problems by completing a spreadsheet and graph describing the problem situation and by solving equations related to the problem situation. The system encourages active learning-by-doing. Students can ask for hints at any step of the process, and the system provides immediate feedback on the appropriateness of each student action. The Java Algebra Tutor currently implements only the spreadsheet portion of the system.

PAT contains an expert system capable of solving the problems that are posed to students. As students take steps to complete the problem (for example, by filling in cells in the spreadsheet), the tutor considers whether or not those steps are consistent with a solution that it would follow. If not, the tutor checks to see if the step is consistent with a common error (or "bug"). In such cases, the tutor is able to provide instruction tailored to that bug. Since the tutor is tracking the student's solution at each step, the tutor is able to give help associated with the student's solution path at any time.

In addition to providing help and identifying errors, the tutor continually assesses the student's progress. Each of the rules in the underlying expert system represents a skill that the student needs to master in order to solve the problem. Each skill is either mastered or not, and the tutor maintains, for each student, the probability that the student has mastered the skill. These probabilities are displayed graphically on the "skillometer," which provides the student with information about his or her progress along each of the component skills in the curriculum.

The Java Algebra Tutor contains a compiled version of the expert system's rules, as they apply to the current problem situation. Instead of pattern matching against rules in the expert system, the tutor simply matches student inputs and branches along predetermined paths. This compiled version of the system provides better response time with a lower memory requirement, while still preserving the entire behavior of the model-tracing system. From the user's point of view, there is no difference between the systems.

3 Presenting examples while teaching problem solving skills

While the Java Algebra Tutor provides support for students' problem-solving activities, examination of its initial use made it clear that more specific declarative support for the activity was needed. This is especially important when the instruction is being delivered over the World-Wide Web. Unlike in the classroom context, students accessing the tutor over the World-Wide Web do not have a teacher or other student to guide them in working through a new problem type, and students are much more likely to jump around the curriculum, thus getting less practice on multiple problems of the same format. These students find it tedious to be thrown into the middle of a problem-solving situation, with the only guidance being the ability to ask for help at each step along the way. To provide some declarative instruction, [3] developed a link between an online tutor for algebra and declarative instruction delivered through InterBook [2].

Still, the addition of declarative instruction through InterBook represents only one step towards more flexible presentation of domain knowledge. An obvious next step would be to add the ability for students to follow complete examples of problem solving using the exact tools they will be asked to use to solve problems. Ideally, these examples would be personalized to reflect each student's level of skill. We can distinguish three basic methods of presenting the information required to master a skill: learning from declarative information, learning by doing, and learning by example. These methods of learning may involve different cognitive mechanisms [14] and can be seen as complimentary. They may account for forming different components of a complex skill [8]. An effective tutor (human or computer) should apply all the three in combination.

Despite a growing body of work on learning from examples in the psychological literature [c.f. 5; 16], learning from examples has received relatively minor attention in the ITS area. ITS researchers acknowledge the role of learning from examples in problem solving [9], but existing ITSs rarely support this method. The ones that do offer an example when it is required [4; 10; 17], typically leave the student alone to explore the examples. We do not know any ITS which provides step-by-step examples in a way that is sensitive to individual students' abilities.

In this paper, we show how this step-by-step example presentation can be added with minimal effort to any system which includes problem solving expertise and an appropriate component-based architecture. Moreover, as we will show later, our example presentation module itself could be re-used to present examples in different domains, thus reducing the efforts to implement this important functionality.

4 Example presentation in PAT-Java

To add example presentation to PAT-Java, we have developed a small example-presentation module which performs its duties mainly by communicating with existing components such as the Tutor and the Spreadsheet. The module interacts with the student through a small dialog (Figure 1) which appears when the student select an example to learn.

Fig. 1. PAT-Java example presentation interface. The problem-solving window with the spreadsheet is now controlled by the example presentation module. The system is ready to present the next problem-solving step to the student.

Once the student has selected a problem on which to receive an example solution, the student presses the NEXT STEP button (Figure 1). Each step of the example consists of two substeps. On the first substep, the system prepares the student for the next step. It highlights the next cell to be filled and tells the student what is supposed to be entered on this step. This information is presented using an existing interface component: the help window used to present all help text in the Java Algebra Tutor. This window includes two buttons, labeled >>> and <<<, which allow the student to request more or less detailed explanations of why a particular step is appropriate (Figure 2). At this point, the cell to be filled is still highlighted, but empty.

Fig. 2. The display after the student pushes NEXT STEP, and the system starts to explain a step in the solution.

When the NEXT STEP button is pushed, its text changes to DO IT! To start the second substep, the student has to push the DO IT! button. After that, the system simply performs the problem solving step, i.e., fills the current cell. (Figure 3). At this point, the DO IT! button is converted back to say NEXT STEP, and the system is ready to present the next step to the student.

Fig. 3. After the student pushed the DO IT button, the explained step is performed and all changes are made in the interface. The system is ready to explain next problem-solving step.

By providing a progressively more detailed set of explanations and by separating the demonstration of the action from the explanation itself, we expect to give them a better understanding of each problem-solving step. We also hope to promote students' self-explanations [5] as they go through the problem. That is, we expect students to rely more and more on their own explanations of the actions suggested by the example presentation module and less on the accompanying text.

The sequence of steps presented to the student adapts to the student's knowledge level in the following way. If a student has demonstrated, through prior problem solving, that he or she has the ability to complete a particular step, then the system will immediately perform that step, without the student needing to intervene by pressing the NEXT STEP or DO IT! buttons. For example, if a student has demonstrated the ability to correctly complete the column labels in the spreadsheet, the example presentation module will fill in those labels, without any explanations being provided, before demonstrating to the student how to enter the unit of measure.

5 Example presentation in a component-based architecture

5.1 Lightweight tutoring agents: a component-based architecture

Ritter and Koedinger [12; 13] proposed a "plug-in" architecture for intelligent tutoring systems. The primary purpose of the architecture was to allow the incorporation of commercial, off-the-shelf software into a tutoring environment. This was accomplished by enforcing a distinction between the tutoring knowledge and the tools which students use to solve problems. In this context, the "tool" was defined as the program that the user sees (such as a spreadsheet, word processor or simulation environment), and the "tutor agent" was defined as the program that contains the tutoring knowledge and is able to evaluate and offer advice to the student using the tool. The plug-in architecture provides the ability to monitor the student's behavior at an appropriate level and the ability to provide feedback to students in a flexible manner, so that the tutor agent does not need to be aware of many of the details of the tool. Some messages in this "plug-in" architecture are presented in Table 1. In the Java Algebra Tutor and the example presentation module, we are taking advantage of the "plug-in" architecture for a very different purpose. Instead of incorporating off-the-shelf tools, the Java Algebra Tutor uses a custom-built spreadsheet module. Still, the design of the tutor agent component isolates the appropriate elements of the system in a way that allows a novel component, the example presentation module, to use the tutor agent for its own purposes.

5.2 Example presentation

This system provides an illustration of how it is possible, in a well-defined component architecture, to re-use the system's knowledge in an unanticipated context. The tutoring component, which is the part of the system responsible for deciding which step to take next, what help text to give for it and which skills are involved in taking that step, is the exact same program that is used in the learning-by-doing system. The example presentation module system is opportunistically taking advantage of some of the features of the tutoring component that result from it following Ritter and Koedinger's [13] plug-in architecture. In this section, we describe some aspects of the architecture that make this possible.

Message
Parameters
Handled by
Meaning
start problem
initial, goal state
tutor agents
start an activity
process tool action
selection, action, input
tutor agents
student performed some action
process help request
selection, input
tutor agents
student asks for hint
process done

tutor agents
student indicates problem complete
get next step

tutor agents
request for next step in solution
perform user action
selection, action, input
tool
perform some action
get user value
object, property, datatype
tool
request for some value of tool property

Tab. 1. Subset of messages defined in the plug-in architecture in [13].

The plug-in tutoring architecture defines a strict distinction between a tool, which is the component of the system with which the user interacts in order to solve a problem, and the tutor agent (also called tutoring component) which is the part of the system that is responsible for guiding the student in the problem solving process and evaluating the student's abilities in the domain. The plug-in architecture defines a set of messages that allow the tool to communicate the semantics of student actions to the tutor agent and a set of messages that are sent from the tutor agent to the tool in order to give feedback to the student and initialize the state of the tool. The plug-in architecture also defines messages that can be used to query the state of the tool and tutor agent.

The basic algorithm used by the example presentation module is as follows:

  1. The example presentation module tells the tutor agent to create a new student (whose name is "demonstrator") and to direct all feedback about that student to itself. Although this action is outside of the plug-in architecture as described in [13], this is the same procedure used by the "curriculum manager" (described in that paper) when a student logs in to the system. From the tutor agent's perspective, a new student named "demonstrator" has decided to use the system. There are now two students using the system, the real student and the example presentation module.
  2. The example presentation module selects a problem to demonstrate to the student and sends a "start-problem" message to the tutor agent, indicating that this is the problem that the student named "demonstrator" will be using.
  3. The example presentation module displays a NEXT STEP button to the student. When the student presses that button, the example presentation module sends a "get-next-step" query to the tutor agent. This query returns a description of the semantics of the next action that the user should perform. Specifically, each step is described as a selection, action, input triple (roughly corresponding to a direct object, verb and indirect object). For example, when the example presentation module sends the first "get-next-step" query, the tutor agent might reply with selection="cell R1C1 in the spreadsheet", action="type", input="Time", meaning that it would recommend that the student type the word "time" into cell R1C1 of the spreadsheet. We have expanded the semantics of the "get-next-step" query so that, in addition to the selection, action, input triple, it returns a list of the skills that are involved in this action. In this case, the "get-next-step" query would return the information that the single skill called "Enter column label" would be affected by the recommended action.
  4. If the student has not yet demonstrated mastery of the skills needed to perform the next step (e.g., in this case, the student has not mastered the "Enter column label" skill), the example presentation module sends a "process-help-request" message to the tutor agent, passing along the recommended selection as the item on which to give help. From the tutor agent's perspective, the student named "demonstrator" has just asked for help on that item, so it returns the help text that the student would see in that situation. The example presentation module displays that help text to the real student.
  5. The NEXT STEP button changes to a DO IT! button. When the user presses this button, the example presentation module sends a "process tool action" message to the tutor agent, passing along the recommended selection, action and input that it received in step 3. From the tutor agent's perspective, this message is an indication that the student has performed the action in the tool. In our example, the tutor agent believes that the student just typed the word "Time" into cell R1C1 of the spreadsheet. Since the student has not actually interacted with the spreadsheet, the example presentation module also sends a message to the spreadsheet ("perform-user-action"), telling it to change the value of cell R1C1 to "Time."
    If the student had already demonstrated mastery of the skills required to perform the step recommended in step 3, the example presentation module would automatically send the "process tool action" step to the tutor agent (and the "perform-user-action" message to the tool), without any student intervention. This would have the effect of automatically filling in those steps that the student is presumed to already know how to perform. In this way, the system adapts to the student's skill level, only demonstrating those steps that the student does not yet understand.

Steps 3 to 5 are repeated until the tutor agent indicates that the student has completed the problem. It should be apparent that (with the exception of a few elements of its user interface) the example presentation module is completely general. The algorithm described is in no way specific to the Java Algebra Tutor, algebra, or to tutors using a spreadsheet tool. The example presentation module does not need to interpret the semantics of the recommended student step; it just passes it back on to the tutor agent as the student's action. Similarly, it doesn't need to know the semantics of the help text or the skills affected by student actions; it just needs to be able to display the help text and access student model information for the skills it is told about. Thus, the example presentation module can be used with any tutor agent and tool following the plug-in architecture.

The algorithm used by the example presentation module is similar to one described in [11]. In that paper, a system component queried the tutor agent (one used to teach equation solving) repeatedly to discover a recommended problem solution. However, instead of using that solution path to demonstrate an example to the student, that system used the information to compare the recommended solution to that performed by the student, as a way of identifying places where the tutor agent was deficient. Both of these cases demonstrate that, although tutor agents are written with the primary purpose of providing instruction in a learn-by-doing system, when they are implemented as components with appropriate interfaces, their knowledge can be reused in novel and interesting ways. In this architecture, knowledge re-use is different from knowledge sharing. Different components do not (necessarily) have access to the same knowledge base, as they would in a typical blackboard system. Instead, each component is assumed to maintain its own knowledge, but it is able to communicate aspects of its understanding when queried.

6 Summary and future work

In this paper we have tried to demonstrate that new functionality can be relatively easily implemented in an ITS built with a component based architecture, such as the "plug-in" architecture [13]. In particular, we have tried to show how to implement an example explanation module in a system with problem-solving expertise. We hope that the ideas and the real-world example presented in this paper will encourage other ITS developers to use a component based architecture and to consider an example explanation module as an extension of existing problem solving support systems.

Among our own plans is to continue the work on component architecture for ITSs. We hope to experiment with more components and to develop a tool kit for building component based ITSs with Java on the Internet. This model for example presentation can be made much more adaptive. Currently, the system has two options for explaining each step of the example. If the skills behind the step are mastered, the system will not stop to explain the step at all. If the skills are not yet mastered, the step will be explained in two substeps. There are several ways of improving on that. First, we can manipulate the granularity of a step explanation to a greater extent. For some simple steps already seen before, one substep is enough. For some complex steps which are new for the student, even two sub-steps and brief explanations may be not enough to understand why something was done by the system. Second, the text and vocabulary of the explanation generated for each substep could be adapted to the student's knowledge. Third, the order of showing the steps may be adapted to the student's background knowledge. For example, we might recommend that an algebra novice fill in the formula cell at the very end, so that completed rows in the spreadsheet can support inductive inference of the formula. On the other hand, a student who already understands formulas would be instructed to fill in the formula cell before calculating the answers, so that the system will calculate using the formula and the entered givens (as a spreadsheet does).

Acknowledgments

This work was supported by the National Science Foundation and the Advanced Research Projects Agency under Cooperative Agreement CDA-940860, and by a James S. McDonnell Foundation grant to Peter Brusilovsky.

References

  1. Brusilovsky, P.: Intelligent learning environments for programming: The case for integration and adaptation. In: Greer, J. (ed.) Proc. of AI-ED'95, 7th World Conference on Artificial Intelligence in Education, Washington, DC, AACE (1995) 1-8, available online at http://www.contrib.andrew.cmu.edu/~plb/papers/AIED-95.html
  2. Brusilovsky, P., Eklund, J., and Schwarz, E.: Web-based education for all: A tool for developing adaptive courseware. Computer Networks and ISDN Systems. 30, 1-7 (1998) 291-300
  3. Brusilovsky, P., Ritter, S., and Schwarz, E.: Distributed intelligent tutoring on the Web. In: du Boulay, B. and Mizoguchi, R. (eds.) Artificial Intelligence in Education: Knowledge and Media in Learning Systems. IOS, Amsterdam (1997) 482-489
  4. Brusilovsky, P. L.: Intelligent Tutor, Environment and Manual for Introductory Programming. Educational and Training Technology International 29, 1 (1992) 26-34
  5. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., and Glaser, R.: Self-explanations: How students study and use examples to solve problems. Cognitive Science 13 (1989) 145-182
  6. Koedinger, K. R., Anderson, J. R., Hadley, W. H., and Mark, M. A.: Intelligent tutoring goes to school in the big city. In: Greer, J. (ed.) Proc. of AI-ED'95, 7th World Conference on Artificial Intelligence in Education, Washington, DC, AACE (1995) 421-428
  7. McArthur, D., Stasz, C., Hotta, J., Peter, O., and Burdorf, C.: Skill-oriented task sequencing in an intelligent tutor for basic algebra. Instructional Science 17, 4 (1988) 281-307
  8. McKendree, J. and Anderson, J. R.: Effect of practice on knowledge and use of basic Lisp. In: Carroll, J. M. (ed.) Interfacing Thought: Cognitive Aspects of Human-Computer Interaction. Bradford Books/MIT Press, Cambridge (1987)
  9. Pirolli, P. L. and Anderson, J. R.: The role of learning from examples in the acquisition of recursive programming skills. Canadian Journal of Psychology 39 (1985) 240-272
  10. Recker, M. M. and Pirolli, P.: Student strategies for learning programming from a computatioinal environment. In: Frasson, C., Gauthier, G. and McCalla, G. I. (eds.) Intelligent Tutoring Systems. Springer-Verlag, Berlin (1992) 382-394
  11. Ritter, S.: Comminication, cooperation and competition among multiple tutor agents. In: Boulay, B. d. and Mizoguchi, R. (eds.) Artificial Intelligence in Education: Knowledge and Media in Learning Systems. IOS, Amsterdam (1997)
  12. Ritter, S. and Koedinger, K. R.: Towards lightweight tutoring agents. In: Greer, J. (ed.) Proc. of AI-ED'95, 7th World Conference on Artificial Intelligence in Education, Washington, DC, AACE (1995) 91-98
  13. Ritter, S. and Koedinger, K. R.: An architecture for plug-in tutor agents. Journal of Artificial Intelligence in Education 7, 3/4 (1996) 315-347
  14. chmalthofer, F. and Kuehn, O.: The psychological process of constructing a mental model when learning by being told, from examples, and by exploration. In: Ackermann, A. and Tauber, M. (eds.): Informatics and Psychology. Schuerding Proceedings (1989)
  15. Sleeman, D. H.: An attempt to understand student understanding in basic algebra. Cognitive Science 8 (1984) 387-412
  16. VanLehn, K., Jones, R. M., and Chi, M. T. H.: A model of the self-explanation effect. Journal of the Learning Sciences 2, 1 (1992) 1-60
  17. Weber, G.: Individual selection of examples in an intelligent learning environment. Journal of Artificial Intelligence in Education 7, 1 (1996) 3-31