BestChoice, a Model for Interactive Web-Based Teaching
Sheila
Woodgate and David Titheridge
The
BestChoice is an open-access interactive web site
(www.bestchoice.che.auckland.ac.nz) that was developed initially to support
learning in large first year Chemistry classes at The University of Auckland.
More recently BestChoice
has been expanded to provide learning opportunities for a wider ranger of
users. The model underpinning BestChoice learning activities is simulation of the
interchange of a student with an experienced teacher. Thus student responses on
BestChoice
question pages generate instant assessment and feedback. BestChoice is innovative in its
emphasis on teaching both concepts and problem-solving strategies by guiding
students in ways that promote their understanding.
View a selection of BestChoice
Question Pages, by clicking on Live
(during 2006).
View screen shots of BestChoice
Question Pages by clicking on For Archive
(after 2006).
The discussion below considers the
· design
of BestChoice
· features of BestChoice
· evaluation of students' perception of BestChoice
· incorporation of BestChoice into first year
university and high school courses
· evaluation
of student usage of BestChoice
Design
considerations
Facilitating both teaching and learning
using web-based activities
Teaching Chemistry to
large classes at first year university is largely transmission of information
to students who adopt a passive role. However, for quality learning to occur,
the student must assume an active role and engage with the content through problem-solving.
Often the subject expert is not readily accessible to offer guidance to the
student during the learning process. This may cause
- uncertain learners to become discouraged
and to conclude that chemistry is "too hard".
- confident learners to solve problems
algorithmically without developing full understanding.
BestChoice activities have been
designed to support the learning process by acting as a bridge between
"hearing about it in lectures" and "doing it yourself". The
authors' aim was
- to construct activities that
students like to use and which help them to learn
- to develop methods for tracking
how students use the activities in order to detect how better to support
learning
The process by which we
have achieved these objectives is described below.
Establishing a connection between web-based
learning activities and other study resources
Traditionally, students
study using print materials. They solve problems using pen and paper. Many
students prefer to have model answers readily accessible.
If BestChoice
activities are to be valued as learning tools, students should be able to
relate these activities to other study tools. The authors have established this
connection by
- constructing BestChoice
pages that are visually appealing and use conventional symbolism in
display of mathematical and chemical concepts
- developing activities that help
students learn how to solve problems that appear on written assessments
- giving users ready access to correct
answers and to background material required.
Overcoming
some of the limitations of teaching over the web
BestChoice has been designed to
present information in a way that prevents cognitive overload. The emphasis is
on requiring users to answer questions. Content is then exposed in small
quantities in the feedback generated by the user's response.
Waiting
for downloads or for the computer to respond is frustrating.
BestChoice pages are text-based. Graphics and animations are used only
where necessary. This allows BestChoice to be used on a dial-up connection.
Common
web browser answer-input devices (text-boxes, dropdown lists) do not support
subscripts and superscripts.
This limitation has been
overcome by developing answer input devices having support for both images and
formatted text.
Constructing BestChoice
modules to simulate a one-on-one tutorial
An
experienced teacher would, in a one-on-one tutorial, mix discussion of concepts
with questions to probe the student's level of understanding. Concepts are
presented as Review Pages in BestChoice. These are typically brief and highlight
only major principles.
Some examples of Review
Pages can be viewed by clicking Live
(during 2006).
Screen shots of Review Pages are available on clicking For Archive (after 2006).
Most modules begin with a Review Page. Concepts are then developed
systematically through the
- questions on Question Pages that follow
each Review Page.
- feedback displayed when users answer
questions on Question Pages.The feedback either
relates to the previous Review page by reiterating principles or presages
the next Review Page by applying and extending the principles.
Thus, during completion
of a BestChoice
module, learning is scaffolded by the constant feedback
and guidance provided by the system. Some students struggle to understand what
the question requires or may not be able to enter the correct answer. These BestChoice
users have the option for the system to show them the correct answer so that
they can overcome this barrier and move on to complete the problem.
Connecting the BestChoice
activities with the student's course of study
Teachers
choose course resources to help students study. In order for BestChoice
learning activities to be accepted as a course resource, we constructed
groupings of modules for target groups both in first year Chemistry courses at
The University of Auckland and in Chemistry classes at
Implementation of
BestChoice
BestChoice for the web 2002
The BestChoice web site went live in
2002, with a total of 1900 pages in 60 modules available. The content was taken
from a DOS version of BestChoice
that had been developed by one of us. 90% of the pages were Question Pages;
thus the emphasis was, as intended, on users entering responses that generate
feedback.
The use of BestChoice
was recommended to students in two first-year courses at The University of
Auckland in Semester 2, 2002. An end-of-semester survey in one of these courses
endorsed the approach taken in BestChoice modules. The survey asked “What
feature of BestChoice did you like best?”.
The two most common responses to the question were “combination of
questions and review pages” and “feedback".
While we had put in place the structure described above and had some validation
of the BestChoice
approach to supporting learning, BestChoice 2002 was limited to one answer per
question page; thus multistep problems extended over
several pages. Furthermore, the lack of control over the positioning of the
answer fields and their associated feedback was a severe hindrance in
authoring.
BestChoice for the web 2003-2006
Both the BestChoice
content and the BestChoice
system have been upgraded and expanded on a continual basis during the last
three years. BestChoice
currently has 2500 different pages in more than 100 modules with 6000
possibilities for interaction that result in instructive feedback. A variety of
new features have been developed so that the author can
- construct Question Pages having any
number of answer fields.
- control the positioning of the answer
fields and their associated feedback on Question Pages.
- use a variety of types of answer
fields on the same Question Page.
- use correct answers to one part of
a Question Page to trigger the appearance of a second part.
The BestChoice Question Pages accessed
using the links below show that we have made significant progress toward
mimicking stepwise problem-solving for a variety of problem types.
To see actual BestChoice
Question pages, click on Live
(during 2006)
To access screen shots of BestChoice Question pages, click on For Archive (after 2006).
Evaluation of BestChoice using an on-line survey
Establishing mechanisms for on-line
evaluation of BestChoice by students
We wanted our student
users to indicate what they liked about BestChoice and how BestChoice
could be improved. Therefore we incorporated on-line evaluation mechanisms into
the BestChoice
system, in order to get a continual feedback from our users.
In BestChoice
2002 students could enter feedback using a text form accessed by clicking a Comments button that appears at the top
of every page. Very few comments were received. As a consequence, for 2003 we
added the survey below on the last page of each module. This allows the user to
enter comments and to rate the modules on a six-point scale. Each user could
enter one rating and one comment per module.
The comments entered
in the survey indicate the student perception of BestChoice
This
on-line survey has been a very rich source of student comment, most of which is
pertinent to teaching and learning. 2100 complimentary comments were entered
during 2003-2005. A few unedited examples are given below.
User 1: Wonderful! Sooooo helpful! Best feature is that you can read the
theory, then do the quiz! (3 Nov 03)
User 2:This
was fun and helped me understand much easier. I was not placed under pressure
at all, because I was allowed to make mistakes. I really enjoyed this. (22 Mar
04)
User 3: This has
made so much that I didnt understand much clearer.
The little amounts of information followed by heaps of questions makes it
really easy to absorb!(04 Aug 03)
User 4: Man, you
have no idea how helpful Bestchoice is....the
questions on them are relavent to what we are
learning and to make it even more helpful....the hint and give up function help
alot....thank you so much! (15 Jun 03)
User 5: This
site is AWESOME!!! im really
glad theres something like this up and running to
help students who are willing to study at home. it
gives a break from just reading and doing examples from books and really
helps!!! im sure my marks
are going to improve!!!! THANKS!!! (30 Apr 03)
User 6: i learnt alot, & it went
smoothly, gradually getting harder & making me learn much more efficiently.
it is very helpful thank you (9 May 03)
The comments entered during 2003-2005 have been assigned to categories. Three
of these categories (complimentary, suggestion and critical) are relevant to
teaching and learning. 78% of comments have been assigned to one of these. 22%
of comments either do not pertain to BestChoice or highlight typographical
errors and bugs that have been fixed.
Many of the comments relevant to teaching and learning have multiple threads.
Any comment that includes a compliment is assigned to Compliment. Critical comments that
include a suggestion are assigned to Suggestion.
The table shows the distribution of comments in these three categories. The
trend is toward a lower percentage in the Criticism
category and higher percentages in the Compliment
and Suggestion categories as a
result of enhancement both of the system and the content.
|
Comments entered in on-line survey |
|||||||
|
Year |
Total |
Compliment |
|
Suggestion |
|
Criticism |
|
|
2005 |
1173 |
68% |
15% |
17% |
|||
|
2004 |
1134 |
67% |
14% |
19% |
|||
|
2003 |
401 |
64% |
12% |
24% |
|||
These data show that the learners perceive that BestChoice is beneficial to them and helps them to
learn. Furthermore users suggest ways in which BestChoice can be improved (even
through critical comments). One advantage of a web-based system is that
problems identified can be rectified and the revised version made available
within a short time frame.
The
module ratings (out of 6) indicate that most students find BestChoice
modules helpful
Students indicate how helpful they have found a module by choosing one of the
six radio buttons on the end-of-module survey form. These
range from not at all
(1) to fantastic (6). The
average response rate to this part of the survey over all modules and all
cohorts of users is 30%. As shown below, the radio buttons corresponding to 4,5 and 6 ratings are much more commonly chosen than those
corresponding to 1,2 and 3 ratings. The most frequent choice in each year is fantastic (6).
|
Module ratings using a 6 point scale |
||||||||
|
Year |
Total responses |
Average |
1 |
2 |
3 |
4 |
5 |
6 |
|
2005 |
6866 |
4.64 |
253 |
275 |
717 |
1383 |
2034 |
2204 |
|
2004 |
6442 |
4.71 |
186 |
266 |
570 |
1309 |
2019 |
2092 |
|
2003 |
2010 |
4.53 |
83 |
106 |
223 |
437 |
578 |
583 |
Using BestChoice as part of course materials
BestChoice for Chemistry students at The University of Auckland
In 2003, use of BestChoice was compulsory in two first year
Chemistry courses at The University of Auckland. In 2006 use of BestChoice
is compulsory in five first year Chemistry courses. In total this involves
approximately 2000 users, some of whom are enrolled in more than one course.
Although one of the authors has had extensive experience teaching first year
Chemistry, neither of us is currently lecturing in any of the first year
Chemistry courses. Therefore, support of course coordinators and teachers is
crucial. These academic staff
- introduce their class to BestChoice, providing details of the
registration process.
- promote BestChoice activities as
learning experiences.
- encourage students to use BestChoice
to support their learning during the entire course. One lecturer has
recently referenced relevant BestChoice modules in the
course notes.
What form does the
compulsory usage take?
BestChoice
modules are currently used as assignments in four courses.
Cohort: Students in one of these
courses are science majors. Students in the other three courses are studying
Chemistry at pre-university level.
Contribution to overall assessment:
All courses have a maximum of 100 marks. Ten modules at 0.5 marks each are
compulsory in the course for science majors and in one of the pre-university
courses. Five modules at 1 mark each are compulsory for the other two
pre-university courses.
One of these courses was
the first to use BestChoice
as an assessment component. Based on that experience, course coordinators
specify the modules to be completed and set completion deadlines. This ensures
that students begin to use BestChoice early in the course.
BestChoice modules are used in one course as both
pre-lab activities and assignments
Cohort: In semester 1 these
students are intending Health Science students and science majors (including
Chemistry). In Semester 2 they are science majors.
Assessment of prelabs:
Evidence for completion of the module is submission of a printout
of the last page of the module with the lab script. For some experiments, the
last page is part of the background information/data required for the
experiment.
Contribution of assignments to overall
assessment: The course maximum is 100 marks. Four modules at 0.5
marks each are compulsory.
The BestChoice pre-labs replace
written pre-labs that were marked by supervisors at the beginning of the lab
session. Because BestChoice
pages are marked as they are completed, the supervisors now have more time to
help students with their practical work at the beginning of the lab. The course
coordinator also believes that there is less "blind copying" and more
opportunity for learning when the students complete pre-lab exercises on line
This course coordinator has decided, for both the pre-lab and assignment
modules, to disable the Give Up
and Show Next options that
allow BestChoice
users to view the correct answer. Despite this, these modules have high student
ratings, and students do not comment on the absence of these features in the
on-line survey.
BestChoice
sessions in a computer lab were also a feature of our four day catch-up course
in 2006.
Each day 80 students
used BestChoice
modules that reinforced lecture material.
BestChoice for New
Zealand
Evaluating BestChoice through usage data
Usage of BestChoice
has grown steadily
BestChoice collects a variety of
data for each user. This includes demographic details (user-entered),
registration date, number of logins, and a count of pages on which answers have
been entered as well as how many attempts were required to get the correct
answer and the time interval over which the question was answered. The table below shows that there has been a
steady increase in the number of registrations per year over 2003-2005. The
increase in 2005 is largely due to
|
Year |
Number of registrations |
Number of Active Users |
Logins |
|
2005 |
7 560 |
4 000 |
50 190 |
|
2004 |
5 983 |
3 200 |
37 916 |
|
2003 |
3 380 |
1 400 |
15 581 |
There has also been growth in the number of active
users. These users have answered questions on more than 20 pages.
In 2003 an active user entered at least 20 answers. In 2004-2005 an active user
will have have entered at least 100 answers. There is
a correlation between the percentage increase in either logins or registrations
over an entire year and the percentage increase in active usage.
Active usage data highlights the importance
of the connection to the user's course of study
Users are asked on registration to identify themselves with one of
the institutions shown in the table below. The table also shows the number of
active users associated with each institution for 2005.
|
Institution |
Active users in 2005 |
|
The |
2230 |
|
NZ high schools |
1670 (from 210 schools) |
|
Other NZ Universities |
88 |
|
Outside NZ |
12 |
The largest groups of active users are those for whom specific courses have
been created. The difference between the two large cohorts is that while use of
BestChoice
is compulsory for most university users,
The timing of logins gives evidence for how
these two cohorts use BestChoice
The
graph below shows the number of logins by students at The University of
Auckland for each month during 2005. This indicates that usage of BestChoice during the semester (March-June and then
July - November) is fairly consistent and that usage drops during the
inter-semester period. Comparison with data from 2003 and 2004 shows that, due
to the support of the course coordinators, students in each subsequent year use
BestChoice
both earlier in the semester and more consistently throughout the semester.
The graph above for
In conclusion
What have we found?
The
BestChoice
project has shown that web-based activities can facilitate learning by
requiring the user to take an active role, with the system then responding to
the user's input. The guidance offered by BestChoice makes successful
completion of the problem more accessible for more learners who, over time,
should gain the skills to answer the questions without guidance. Consider the
unedited comments.
User 7: The main thing I liked is
that it gave me the opportunity to practice and learn from my mistakes. (31 Mar
06)
User 8: liked
this topic. Being able to see and work everything out myself really helped me
understand this. Thanks (31 Mar 06)
User 9: It
helped very much. It was much easier to understand than my notes, textbook and
lecturer! The reviews are so concise and great. Thanks! (31 Mar 06)
BestChoice activities also provide capable learners with
an enhanced learning experience as insights are revealed during problem-solving
and appear in the form of feedback just as the user has entered the answer that
they thought was correct.
User 10: Very good, now I
understand alot more little things I wouldve never thought of asking. (19 Mar 06)
User 11: it lead us to think critically and logically, and not just know
facts that have already been provided for us in the course notes. (19 Mar 04)
The BestChoice
project has also shown that users can inform the design of systems to support
their learning. The extent and quality of feedback entered by users has been
overwhelming.
User 12: BC
helps me understand chemistry in a way that is both faster and more convenient
than tedious text book exercises - almost like having a personal tutor watching
over my shoulder. Thank you and keep up the good work! (2 Apr 06)
User 13: Bestchoice is simply the best learning technique i ever came across. It walks me through the important steps
to follow inorder to solve a question and makes the
theory more simpler. (26 Mar 06)
How does BestChoice
compare to other learning tools?
A paper survey asking
"What feature of the course most helps you to learn?" has been
circulated in some courses where use of BestChoice is compulsory.
(Science majors) 100 surveys, 62 responses to
this question, BestChoice
47; Handouts 37; Labs 21; Lectures 5
(Science majors) 170 surveys, 111 responses, BestChoice 37; Handouts 44; Labs 25; Lecturers 20
(Summer pre-university; no-lab course) 80 surveys, 63 responses, BestChoice
25; Lectures/lecturers 28; Handouts 13
The student responses indicate that they view BestChoice as an effective
learning tool that complements traditional learning tools. BestChoice is not, and was never
intended to be, a replacement for these.
It is a challenge for lecturers to accommodate the variety of learning styles
and academic backgrounds in the large classes at first year university level.
Web-based activities like those in BestChoice can be an addition to
the suite of tools available to help meet this challenge. Unlike the most other
learning tools, use of BestChoice
by students generates data that can provide information on how to support
learning more effectively and efficiently.
Where to now?
The most frequent
suggestion entered in our survey is "more questions". New Question
Pages are currently being added on a continual basis. An important aspect of
the writing of these pages is that new ways being explored to present content
interactively, including working toward enabling the user to choose the extent
of guidance offered on a BestChoice
page.
The discussion above is a broad analysis of data pertinent to student
perception and usage. It is now appropriate to extend these investigations by
considering student response data relevant to individual modules and questions
to determine whether areas of difficulty for students can be identified. Once
barriers to learning have been identified, the focus would be on improving
support for learning in these areas.
In conclusion, it is evident that a large number of users believe that BestChoice
is very effective in helping them learn Chemistry. Furthermore, once a system
like BestChoice
is in place, it thrives on student usage because usage creates data that
highlights where the system needs improving.
BestChoice
(www.che.auckland.ac.nz/bestchoice)
is open access. Use Demo mode (no registration, no marks stored) or register.
In either case, we would appreciate your feedback.
Acknowledgement: We acknowledge Drs David
Salter and Judy Brittain, and Ian Torrie.
David and Judy coordinate Chemistry courses that use BestChoice. Ian advertises BestChoice to high school teachers. We thank the
Chemistry Department and the Teaching Improvement Grants Committee of The
University of Auckland for financial support.