FREE ELECTRONIC LIBRARY - Abstracts, online materials

Pages:   || 2 | 3 | 4 | 5 |   ...   | 6 |

«Far East Journal of Experimental and Theoretical Artificial Intelligence Volume 1, Issue 2, 2008, Pages 87-125 Published online: August 12, 2008 This ...»

-- [ Page 1 ] --

Far East Journal of Experimental and Theoretical Artificial Intelligence

Volume 1, Issue 2, 2008, Pages 87-125

Published online: August 12, 2008

This paper is available online at http://www.pphmj.com

 2008 Pushpa Publishing House




MICHAEL S. GLASS and MARTHA W. EVENS Department of Mathematics and Computer Science Valparaiso University Valparaiso, IN 46383, U. S. A.

e-mail: michael.glass@valpo.edu Computer Science Department Illinois Institute of Technology Chicago, IL 60616, U. S. A.

e-mail: evens@iit.edu Abstract We have constructed a new module to process student natural language input to CIRCSIM-Tutor, an intelligent tutoring system designed to help medical students learn to solve problems involving the negative feedback process that regulates blood pressure in the human body.

CIRCSIM-Tutor spends most of its time engaging the student in a natural language-based dialogue. The new input understander uses an information extraction approach that is robust enough to handle freeform student input. We describe an evaluation of CIRCSIM-Tutor by forty-two students at Rush Medical College, with particular emphasis on the performance of the input understander.

Keywords and phrases : natural language understanding, intelligent tutoring system, information extraction, finite state parsing.

This research was partially supported by the Cognitive Science Program of the United States Office of Naval Research under Grant No. N00014-94-1-0338 to Illinois Institute of Technology and by the National Science Foundation under award No. 0634049 to Valparaiso University. The content does not reflect the position or policy of the government and no official endorsement should be inferred.

Communicated by Shun-Feng Su Received January 25, 2008; Revised May 2, 2008 MICHAEL S. GLASS and MARTHA W. EVENS Introduction We have constructed a new module designed to understand natural language student input to CIRCSIM-Tutor, an intelligent tutoring system that helps medical students learn to solve problems involving the negative feedback loop that regulates blood pressure in the human body.

It has been tested on forty-two students in a standard physiology laboratory at Rush Medical College.

CIRCSIM-Tutor is a language-based tutor. In fact, the designers of this system were motivated by the belief that the use of language enhances learning, that learning new scientific concepts is inextricably enmeshed with learning the language of that science, and that trying to explain new phenomena in words is the best way to understand them.

Thus the system primarily engages the student in dialogue. It begins by describing a disturbance to the cardiovascular system, such as a hemorrhage or a pacemaker malfunction; it asks the student to enter qualitative predictions about the responses of seven physiological variables to this perturbation; and then it launches a tutorial dialogue.

The new input understander was designed to be robust enough to handle free-form student input and fast enough that the system can respond to the student in under two seconds. Thus we chose an information extraction approach using a cascade of finite state machines.

The new module was one of several improvements so that this version of CIRCSIM-Tutor became the first to be used with large classes of students. Later in this paper we describe the evaluation of our system in a regularly scheduled class laboratory exercise in November, 1999.

Interactive dialogue-based tutoring systems are less common and harder to build than other kinds of computer-aided instruction. But there is scientific evidence for endeavoring to create a dialogue-based system.

There is evidence in the education literature that human tutoring is highly effective compared to other forms of instruction, and there are studies that show that putting theories into words and giving explanations aids learning.

Studies demonstrating the value of tutoring

–  –  –

one-on-one tutoring and make it clear that it is an effective kind of teaching. Cohen et al. [14] conducted a metastudy of 65 controlled evaluations of school tutoring programs, concluding that tutored students outperform control subjects to a high degree of statistical significance.

Bloom [5] reported good results comparing classes of elementary school students taught in a normal classroom using mastery learning methods against students tutored individually or in groups of two or three together. Bloom found that the average tutored student performed two standard deviations above the average normal classroom student.

Michael and Rovick [53] demonstrated that one-on-one tutoring is effective even for bright and motivated adults like those at Rush Medical College. Students tutored for one hour showed significantly more improvement between pre-test and post-test than a control group that spent the same time on task reading carefully chosen relevant text. A recent paper by VanLehn et al. [69] shows that tutoring produces greater improvement than reading under a variety of circumstances.

Studies demonstrating the importance of language in learning

Several recent studies have demonstrated the importance of natural language in learning. There is reason to believe that merely making students talk or write has value. That is, independent of whether the system understands what they say, making students articulate ideas improves retention and understanding. Chi et al. [13] demonstrated experimentally what they call the “self-explanation effect”, where learners are prompted to explain back what they just learned to a neutral listener. Based on this notion, Aleven and Koedinger modified a geometry tutor to prompt for student explanations of their proofs [1, 2]. Their experiments showed that encouraging self-explanation by students improved learning outcomes. However without two-party dialogue--students did not receive feedback on their explanations from the geometry tutor---students left the majority of the explanation boxes empty. Aleven et al. later produced a tutor [3] that attempts to understand student input and to provide feedback to improve the quality of that input. In the realm of tutors that talk but do not listen, Di Eugenio et al. [18, 19] have demonstrated that more sophisticated natural language tutorial utterances produce better outcomes than the same knowledge expressed in template-constructed unsophisticated language.

MICHAEL S. GLASS and MARTHA W. EVENS Graesser [34, 36] dissected 66 one-hour tutoring sessions with untrained human tutors. Analysis of pre-tests and post-tests showed that tutoring was effective despite the fact that these tutors rarely used sophisticated tutoring strategies. The tutors asked many deep questions;

the students also asked deep questions at the rate of eight per hour, which is much more often than in ordinary classroom teaching. Graesser et al. [37] concluded that it is these deep questions and the attempts to state them and answer them that account for much of the learning.

There is also evidence that dialogue itself is an important factor in learning. In an ingenious series of experiments, Fox Tree [22] has shown that people learn more from overhearing a dialogue than from overhearing a monologue with the same content.

Other experiments in adding natural language interaction to tutoring systems From the beginning of research on intelligent tutoring systems, many researchers believed that natural language was essential to tutoring.

Carbonell’s [9] SCHOLAR geography tutoring system asked questions and produced the first mixed-initiative dialogue with a computer. Collins and his collaborators [15, 16] continued to stress natural language interaction as this project grew. Brown and Burton’s SOPHIE system [7, 8], which tutored students in basic concepts of electricity, was also based on natural language interaction. Wilensky’s Unix Consultant [70], which is more of a coach than a tutor, also used natural language interaction extensively, based on a semantic grammar for Unix concepts.

The most pronounced difficulty with SCHOLAR and other early dialoguebased tutoring systems was a lack of multi-turn coherence to the dialogue. Typically a question would not be related to the previous or following question.

Cawsey [10] studied tutoring in the domain of electrical circuits and focused on the interactive nature of tutor explanations. In building a system to emulate this kind of tutoring she created a number of dialogue plans for tutoring. Cawsey’s perception of the interactive nature of explanations in tutoring helped us to recognize the same kind of behavior in transcripts of human tutors, prompting the development of multiturn planning in CIRCSIM-Tutor [64].


Interest in natural language based tutoring systems has increased in the last decade. While ITS’96 had no papers on tutoring systems based on natural language dialogue except Freedman and Evens [26] from the CIRCSIM-Tutor project, the situation has since changed. Both the Conference on Intelligent Tutoring Systems and the Conference on Artificial Intelligence in Education regularly feature multiple sessions and ancillary workshops on natural language processing in tutoring systems. The center of much of this new activity was the NSF sponsored Circle Project, joint between the University of Pittsburgh and Carnegie Mellon University. VanLehn directed development of the Atlas Project [27, 59, 60, 68]. Atlas-Andes [41] is a physics tutor that combines a model tracing approach with natural language dialogue with the goal of improving the student’s ability to solve problems in qualitative reasoning.

Atlas uses Freedman’s Atlas Planning Environment (APE) [23, 24, 25], a hierarchical, opportunistic, reactive planner, to plan natural language dialogue. Rosé’s CARMEL parser [59, 60], which serves as the input understanding component of Atlas-Andes and other Circle projects, is intended to handle a wide range of student input, especially studentgenerated explanations. Lane and VanLehn [46] have built a dialoguebased tutor for program design. Moore’s emphasis on the role of planning in generation, especially in the generation of dialogue, has been of great importance in text generation in general [55], with significant contributions to tutorial dialogue generation especially. Her Beetle system [17] illustrates what good planning can contribute to naturalsounding dialogue. Heffernan and others [38] have done significant work building a framework for more easily creating dialogue-based tutoring.

Graesser and others have produced another set of dialogue based tutoring systems [35]. The input understanding in his AutoTutor uses latent semantic analysis [45] for robust processing of complex student utterances. Student answers that are not easily recognized by more conventional means (usually the longer ones) are evaluated by their similarity to a collection of archetypal answers to the question. Some of the archetypes contain known misconceptions or partially correct answers. AutoTutor was originally designed as a computer literacy tutor, but it has now been ported to other domains such as physics [69].

MICHAEL S. GLASS and MARTHA W. EVENS There are now dialogue-based computer tutors that use spoken input and output, for example SCoT at Stanford [56] and ITSPOKE at Pittsburgh [51]. Litman’s system, ITSPOKE, which is essentially AtlasAndes with spoken input and output, has been shown to produce better and faster learning outcomes than Atlas-Andes. These systems still need the kind of language understanding and generation mechanisms developed by CIRCSIM-Tutor and other typed-language systems, as well as the kinds of dialogue strategies used in human tutoring.

How the CIRCSIM-Tutor project got started

Our goal from the beginning was to emulate a subset of expert human tutoring. Our primary source of data on both tutor and student behavior is a set of human-tutored keyboard-to-keyboard tutoring transcripts that the CIRCSIM-Tutor project has accumulated. These tutoring sessions were held with the student and the tutor in different rooms communicating only through the computer keyboard and screen in order to match the conditions under which the students would interact with the computer tutor; this process also simplified the collection of the session transcripts. We have 75 transcripts from expert tutor sessions, mostly one hour in length [33, 42, 43, 44]. In this paper we use raw transcript extracts, so that the reader can understand the kind of input that the system encounters. We have sometimes cleaned up the portion typed by the human tutor to make it easier to read. References to transcript abstracts are of the form “K51-tu-17-3” meaning transcript 51, turn 17, sentence 3. It is the tutor’s turn. In all 75 transcripts the tutors are Joel Michael and Allen Rovick, professors of physiology at Rush Medical College; the students are first-year medical students. In addition to the data from human tutoring we also use examples from logs of medical students using various revisions of CIRCSIM-Tutor Version 2.

Before this project started, in addition to tutoring students face-toface themselves, Michael and Rovick had already built several CAI systems for students of physiology: CIRCSIM, ABASE, and GASP [50, 61, 62, 63]. These systems have been very successful and are still being widely used, but Michael and Rovick decided that natural language input and output are essential to making better systems.

This paper outlines the problems we set out to solve, explains our solution, and describes an evaluation with a class of medical students. In


the next section we discuss the operation of CIRCSIM-Tutor. Then we describe the operation of the new input understanding component, showing the types of phenomena it is designed to address. We describe the results of evaluating CIRCSIM-Tutor with regard to learning gains and with regard to the performance of the input understander. Finally we outline further input understanding phenomena we might address in the future.

The Operation of CIRCSIM-Tutor

Pages:   || 2 | 3 | 4 | 5 |   ...   | 6 |

Similar works:

«Padre Beacon October 2014 Volume 1, Issue 1 This month of Padre Beacon is full of fun activities, health hints, and class news. We hope you enjoy. In this Issue Mrs. Vikki 1 Mrs. Vicki Principal’s Corner 2 Helpful Healthy Hints 3 Mrs. Vicki is the first person you see when you enter DeSmet. She is the smiling face that What’s New? / Survey 4 everyone sees when you walk in the front door. Class News 5 She is an amazing secretary that loves the Resource Room 7 atmosphere at the school. This...»

«Comments on ‘‘Does Paranormal Perception Occur in Near-Death Experiences?’’ Bruce Greyson, M.D. Department of Psychiatric Medicine, University of Virginia ABSTRACT: Keith Augustine’s critique of studies of veridical perception in near-death experiences is based on unsubstantiated speculation from the popular media, rather than on supportive data or peer-reviewed literature. Nevertheless, addressing the issues he raises would improve the methodology of near-death research and...»

«Share. Care. Cure. Assessment Manual and Technical Toolbox for Independent Assessment Bodies September 2015 This draft version of the Assessment Manual and Technical Toolbox for Independent Assessment Bodies has been made available to the public with the sole aim of providing information to interested parties. This version is subjected to further changes and additions and should not be considered an official document of the EC This report has been developed in the framework of the service...»

«D052 HANDBOOK ON HeAltH WOrKfOrce PlANNING MetHODOlOGIeS AcrOSS eU cOUNtrIeS edited by Annalisa Malgieri, Paolo Michelutti, Michel Van Hoegaerden RELEASE 1 Funded by the Health Programme ISBN 978-80-89825-00-4 of the European Union Joint Action Healh Workforce Programming & Forecasting Funded By the Health Programme of the European Union Deliverable D052 “Handbook on Health Workforce Planning Methodologies across EU Countries” E.B. APPROVED (06.03.2015) Published by the Ministry of Health...»

«Investigation of the Brain Regions Involved in Improvisation Richard Falco1, Frederick Bianchi1, and Karl G. Helmer2,3, Department of Humanities and Arts, Worcester Polytechnic Institute Athinoula A. Martinos Center of Biomedical Imaging, Massachusetts General Hospital Department of Radiology, Harvard Medical School. Background and Significance Improvisation in music is the generation of a new melody (with or without the accompanying harmony) in response to a written melody or harmonic...»

«The Independent Influence of Aerobic Fitness and Running Economy on Thermoregulatory Responses during Treadmill Running Jovana Smoljanić B.Sc., University of Ottawa, 2011 THESIS Submitted to the Faculty of Graduate and Post-Doctoral Studies In partial fulfillment of the requirements for The Degree of Master’s of Science in Human Kinetics School of Human Kinetics Faculty of Health Sciences University of Ottawa © Jovana Smoljanić, Ottawa, Canada, 2014 ACKNOWLEGMENTS First of all, I would...»

«MEDICAL COVERAGE GUIDELINES ORIGINAL EFFECTIVE DATE: 07/25/11 SECTION: SURGERY LAST REVIEW DATE: 04/30/13 LAST CRITERIA REVISION DATE: 04/30/13 ARCHIVE DATE: VARICOSE VEINS, VENOUS INSUFFICIENCY AND LEG ULCER TREATMENTS Coverage for services, procedures, medical devices and drugs are dependent upon benefit eligibility as outlined in the member's specific benefit plan. This Medical Coverage Guideline must be read in its entirety to determine coverage eligibility, if any. The section identified...»

«2014 Coventry Health Care of Illinois, Inc. PROVIDER MANUAL TABLE OF CONTENTS V2014.2 I — INTRODUCTION A. Company Overview E. Self-Administered Injectables B. Roles of Provider Relations and Customer Service F. Non-covered Medications G. Pharmacy Network II — PLAN MEMBER INFORMAT ION H. Express Scripts Mail Order A. Plan Member ID Cards I. Maintenance Drug Program B. Verification of Plan Member Eligibility J. Appeal Rights C. Plan Member Copayments K. Injectable Medications D. Out-of-Pocket...»

«Expiring Payment Policies Could Mean Cuts to Primary Care and Other Internists’ Services 2014 Leadership Day on Capitol Hill Background Where Things Stand The federal government enacted into law two critical programs in 2010 to increase payment levels temporarily for primary care services under Medicare and Medicaid. Unfortunately, both programs will soon expire, which not only puts access to primary care services in jeopardy for so many Medicare and Medicaid beneficiaries but it could also...»

«Effects on Boys of Priest Abuse 1 Effects on Boys of Priest Abuse Richard B. Gartner, Ph.D. Published in the Psychologist-Psychoanalyst, 22:15-17 (2002) I write this in early September, 2002, in New York City, where I live and work. As people ponder the first anniversary of September 11th, ideas about the nature of trauma abound in the media, in private discourse, and in my consulting room. It is futile and divisive to think about whether trauma from an attack on a city (and country) is...»

«TOOLBOX OF ASSESSMENT METHODS© ABMS A Product of the Joint Initiative ACGME Outcomes Project Accreditation Council for Graduate Medical Education American Board of Medical Specialties (ABMS) Version 1.1 September 2000 TOOLBOX OF ASSESSMENT METHODS© A Product of the Joint Initiative ACGME Outcomes Project Accreditation Council for Graduate Medical Education American Board of Medical Specialties (ABMS) Version 1.1 September 2000 © Copyright 2000 Accreditation Council for Graduate Medical...»

«Prince George County Parks and Recreation PRINCE GEORGE COMMUNITY CENTER 11100 OLD STAGE ROAD PRINCE GEORGE, VA 23875 OFFICE (804) 458-6164 FAX (804) 458-5021 ACTIVITY HOTLINE (804) 458-5186 WWW.PRINCEGEORGEVA.ORG Welcome “To enhance the quality of life of Prince George County residents by promoting healthy lifestyles and providing enriching leisure services.” The Prince George Parks and Recreation Department invites you to join in the fun!! Youth will enjoy summer programs that include...»

<<  HOME   |    CONTACTS
2017 www.abstract.dislib.info - Abstracts, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.