Using Voice Recognition with Students with Learning Disabilities

Peel District School Board, Massey Street Public School
Science Technology Action Research (S.T.A.R.)
By Craig Stuve, Massey Street P.S., May 2002

Research Question

How can the provisions of alternate recording methods (voice recognition software) assist students with learning disabilities to communicate and demonstrate scientific knowledge clearly and accurately?

Background Information

I teach in a K-5 school in Brampton, Ontario. I work mostly with learning disabled students in Grade 4 and 5. I also have some time in my schedule set aside to assist the school with the integration of computers into the curriculum. I have always had an interest in new technologies and for the past 8 to 10 years have been actively exploring the use of computer technology in the school system and at home. This is my third year in the STAR project (Science and Technology Action Research) offered through the Peel District School Board in which I teach.

While attending a computer conference (ECOO Conference) in April 1999, I had an opportunity to attend a short workshop on Speech recognition or Voice recognition software. I was very impressed by technology and looked forward to exploring the software for the learning disabled students in my school. In the fall of 2001 the opportunity to work with the software became available as our special education resource teacher arranged for our school to get a copy of Dragon NaturallySpeaking5 (Preferred Version).

I was very pleased and decided to explore the software in the context of the STAR Project. I was very impressed with the software and its ability to correctly record voice to text. The software was very user friendly and worked very well even after a short training session. The following week I began creating voice files for students and decided to aim my research at students in grades 3, 4 & 5.

Hardware and Technical Issues

The computer I am using has Celeron Processor 433MHz and 256MB of memory and an upgraded sound card from Creative Labs. The microphone is a Labtec-ClearVoice Headset, model LVA-8450. It is a directional microphone that is designed to work effectively in a room with some background noise.

The computer is set up in the computer lab, which is a room off the school library. It is placed on a table with wheels so that it can travel to individual classrooms. It is also hooked into the school network so that it can print to networked printers in the school.

Loading the Software

The first step was to load the software, review the user manual and hook up the microphone. The software was very easy to load and the on screen instructions were clearly written. The software leads you through a series of steps including a volume check and microphone check. Once this was complete I was able to create my own voice file by reading a series of paragraphs into the microphone. This took about 10 minutes and once complete I began to practice reading into the microphone. I was pleasantly surprised by the accuracy of the program after such a short training period.

An Overview of Speech Recognition

There is not a great deal of research on the use of speech recognition for students with learning disabilities; however, I did find an informative Internet site titled, Speaking to Write: Realizing the Potential of Speech Recognition For Secondary Students with Disabilities.

Speaking to Write, is a three year research project at Children's Hospital in Boston. It is exploring the use of speech recognition for secondary students with learning disabilities. I found that this site provided an excellent overview of speech recognition, Spotlight on Speech Recognition, and it answered many of the common questions such as those listed below. The questions are grouped under the following headings.

What are the skills and characteristics needed by a student in order to use speech recognition software?

This is a very complex question and involves aspects of the following characteristics, cognitive skills, speech abilities, reading abilities, spelling abilities, language processing abilities, self-monitoring skills, perseverance, and interest in writing. In my research, I selected students with learning disabilities who were motivated to write, had at least average ability and average oral language skills. Success in using the program also depends on the amount of training and support that is available for the student. Some students are successful in using the program with very limited training while others require ongoing training and support to make effective use of the software. In using continuous speech recognition there is a need for good language processing skills including the ability to speak in complete sentences or at least long phrases.

It may be more valuable for some students with learning disabilities to use discrete speech software as it is designed to take word-by-word input rather than speaking naturally in longer sentences.

Selection of Students for Research

All the students selected for training in the software had been assessed by a psychologist and declared to be Exceptional (Learning Disabled) by the Identification, Placement and Review Committee. Our In School Review Committee (ISRC), also played a role in the selection of the students. Teachers who wanted their students trained were encouraged to forward the student's name to the ISRC for discussion. We selected students with ability in the average range and strong verbal skills. I selected five students for this research but am reporting on the first three only.

Student 1, Grade 3, David

Creating a voice file

The first student I trained was in grade 3. He has a clear speaking voice and above average oral language skills. He shows an excellent understanding of the concepts taught in all subject areas but finds it very difficult to complete written assignments. He has been assessed by a school psychologist as a student with learning disabilities in visual-motor integration and written language areas. Since the reading level of the text in the training process in the grade 5/6 which is about 2 years higher than his independent reading level modifications to the training process were required. To overcome this difficulty I printed a hard copy of the text for him to take home and practice reading orally.

The following day, I began training. As he was reading the passages I sat behind and to the right. This allowed me to assist him in managing the training process. I showed him the proper location of the microphone. The location of the microphone is a very important factor in using the program successfully. The software allows you to Pause during the training process. This proved to be a very important feature when creating a voice file. I could select the Pause icon and allow the student to orally practice reading short passages of text. Then he could read the text with a high degree of accuracy even though it was above his independent reading level. Since the proper creation of voice file is very important we decided to complete the training a second time. This went quite well and following the second training session we began to use the program. He had immediate success with the program and he proudly created a few sentences.

Using the software to complete science journals

Session 1, Student 1

The first session David brought his science journal with some pictures and words on the topic of Ways That Plants Help.

A scan of the 
  mind map that David sketched.
Sample of David's mind map for his Ways that Plants Help report

Below is a copy of the report David created during his first session.

Ways that Plants Help

Trees
Trees provide shade.
Trees provide food for animals.
Trees provide would for making canoes.
Trees provide wood for making canoes.
Trees provide wood for making canoes.
Plants provide oxygen.
Trees provide wood so that we can build homes.
We can make paper from trees.
We can make wooden toys from trees.

Session 2, Student 1

During this session, he brought some notes he had created while watching and discussing a video in class on the topic of magnets. He creating a series of key words around the topic of magnets using a mind mapping strategy introduced by his classroom teacher.

An image of the 
  mind map David drew to complete his report
Sample of classroom writing, David's mind map for his report on magnets

Below is an example of his classroom handwriting on the topic. It took 20 minutes to create the short passage of writing.

An image of David's handwriting 
  in his science journal
This is an extract of David's science journal created with some teacher assistance.

Magnets

Aluminum and copper are not attracted to magnets.
You can play in many games with magnets.
You can play a fishing pond game.
If you can put coins under the sand and then catch them with a magnet.
The earth is the biggest magnet in the world.
Rockets use many magnets to help them fly.
A compass uses magnets to direct the arrows.
Plastic is not attractive to magnets.
Some magnets are so strong they can go through copper.
Farmers sometimes put magnets down cows throats so the magnets go to the cows stomach. The magnet attracts any metal in the cows stomach. The magnet stays with the cow for the rest of its life.

Notes: Learning to read the screen and make corrections. Developing good editing skills. Voice is quite clear and having about 90% plus success.cgs

Student 2, Grade 5, Taylor

The second student I trained was a grade 5 student named Taylor. He has a clear speaking voice and average oral language skills and ability in the average range. He finds it very difficult to complete written assignments and has had difficulty learning to read. Taylor was unable to have a psychological evaluation but was declared Learning Disabled by the IPRC based on assessment information gathered by the Special Education Staff at school.

A scan of Taylor's handwriting 
  in his daily notebook
This graphic shows an example of daily writing selected from Taylor's class notebook.

During this session Taylor brought some jot notes he had created while researching the topic of Lightning. He created the first draft nearly independently and I assisted him to edit and revise his work. Below is his final edited product.

Lightning

Lightning is a very strange thing and it can even hit in the same spot twice.
Lightning is a very big charge of energy. The energy goes into the ground or between clouds. Lightning often hits the tallest object in the area.
Lightning usually hits metal objects like buildings and cars.
Lightning often hits water so you should never go fishing or swimming during a storm. Lightning can still strike anywhere so be careful.
Lighting sometimes comes when there is a storm or in the summer when it is humid. Some people think lightning is purple or yellow but it is actually white. Lightning travels faster than the speed of sound.
Lightning can travel up to 60 miles per second.
And can be 50,000 degrees Celsius hot.

Student 3, Grade 4, Andrew

The third student I trained was Andrew, a grade 4 student. He has a moderate to severe learning disability. His oral language skills are in the superior (gifted) range and his reasoning skills are above average. His weaknesses include an attention deficit disorder with hyperactive behavior symptoms. Andrew also has speech difficulties (poor articulation). He has had an individual psychological assessment and been declared Learning Disabled. He has a great deal of difficulty completing any written work.

Andrew was the most challenging student I trained. One problem was his need to keep moving. He had trouble sitting still on the chair and would touch nearby items or play with the microphone cord. When speaking into the microphone he would often use a very expressive voice and make articulation errors. He required a great deal of individual attention to create a voice file. When using the program he also had to continue to train individual words to assist the program with word recognition.

Andrew was able to make use of the program with individual assistance.

An image of Andrew's 
  handwriting and drawings in his class notebook
This is an example of writing selected from Andrew's classroom journal.

I selected the following example for Andrew even though it was not from Science. Andrew completes a home reading assignment each week in a high interest, controlled vocabulary. He especially enjoys reading these stories as they are usually related to Science and Social Studies. I had Andrew discuss the story and point out the important or interesting parts. Once our discussion was complete he successfully used the software.

Summary of Story by Andrew

I read a book about spelunking. Spelunking is exploring caves.
I learned that you should never go with less than four people.
You should not have less than four people when you both splunking.
If one person gets hurt, one person can stay behind and the other two can go for help.
A good form of light is if you have a light attached your hard helmet. This allows your hands to be free for climbing. You should wear coveralls when you go splunking. This allows you to wear clothes underneath that stay clean. On your feet you should wear boots not shoes.
Shoes will rip and boots will not.
I learned a lot from the story.

Summary:

Can the software assist students demonstrate their scientific knowledge?

The answer is a resounding yes. Once the students had tried the program they were highly motivated to continue training and wanted to use the program to complete written work in all subject areas.

Can students develop the skills necessary to work independently with the software?

This depends on the individual student. I found that a number of the students I trained are capable of using the software with only minimal support. The editing and revising process does require teacher assistance.

Recommendations:

  1. Do take the time to train yourself in the use of the program.
  2. Introduce the software to the classroom teacher and encourage them to create their own file. This gives them a good understanding of the program and its potential.
  3. Take time to create a good voice file. This may mean that training takes place in three or four sessions rather than one session.
  4. Introduce the program to the parents. Many are willing to purchase the software and use to complete homework assignments.
  5. Do visit the Guide To Assistive Technologies as it provides an excellent overview of assistive technologies for learning disabilities in a simple and easy to understand format.
  6. It is very important that students are prepared to use the program. The most important part is giving the students time to understand the concepts they are writing about and plenty of opportunities to discuss the topic before using the program.
  7. Students need to come to the program with jot notes, mind maps, or other pre-writing strategies.
  8. The software is an excellent tool for teachers to create their voice file which can then be used to scribe for students. Students can orally explain their understanding of a science concept and the teacher can record their ideas.

Copyright 1999-2010 Next Generation Technologies Incorporated

Jump to: site navigation | page content

Highlighted Products and Services

Site Navigation