Introduction
Our university is a special one which admits only visually impaired or hearing impaired students. The division for the hearing impaired students has a department of synthetic design and the department has many lectures concerned with operating graphic design software, such as Adobe Photoshop or Illustrator. In these lectures, our teaching staffs are struggling every day to teach how to operate the software using sign language, power point materials, video resources et cetera. In general, the graphic design software requires a variety of combination keys and tricky mouse actions to operate it. Teaching such complicate operation to hearing impaired difficult. The main problem is that the students cannot read the instruction text (or the sign language) and the mouse cursor movement at the same time when they watch the screen which reflects the teaching staff’s operation. Imagine watching someones operation of graphic design software without any voice information. It might be hard to understand what the operator does even if there is an instruction text on the corner of the screen.
Of course using video materials or giving precise preliminarily instruction is available when hearing impaired students learn something [1, 2] and our teaching staffs utilized such kind of education materials in usual case. Also speech recognition system [3] or summary scribing service is valuable to support hearing impaired students in these days, however, it seems they can not solve the problems of teaching computer operation because the instructing text information of these systems is not in the same place and not in the same time to the computer operation. Summary scribing service in usual case and its display area is separated. The delay and separation between the text information and the real time computer operation makes the students confuse especially in the complex operation mentioned above. Besides such background, one of the authors was just against this problem in his lecture class, and then, he thought up an idea to solve it.
Basic Idea
The idea is really simple. It is to show icons of mouse clicks and pressed combination keys and to show instruction text in real time aside of the mouse cursor. We started to discuss new supporting software based on this idea. The most important point is that the positions of each indicating components placed around the mouse cursor not far from it. Because the hearing impaired students cannot pay attention to the separated area at the same time. At first, we surveyed software which displays mouse clicking and pressing modified keys [4, 5]. However, they are not appropriate for our purpose. Then, we decided to develop an original software tool and named it as “Synchronized Key points Indication Tool: SZKIT.”
Figure 1 shows an overview of SZKIT. It is composed of several icons and an instruction text area. To explain its function, the first figure shows most of components. Preparing icons are mouse click indicator and modified key indicators (Shift, Control, and Alt Key ) and normal key indicator which appears in case of pressing it after modified key (it is not shown in this figure). Each icon is hiding under normal condition and the clicks mouse button or presses the modified key. On the other hand, the instruction text area is displayed all the time basically and the content of the text can be changed on appropriate timing by the operator. When the instruction text area is not needed, it can be hided arbitrarily by pressing Windows key + Ctrl key. To re-display this text area, the only thing the operator has to do is pressing Windows key + Shift key.
Other figures are example of display instruction texts. The first example shows “Shift Key + Left Click” without the text area. The next figure is an example of “Ctrl Key + Right Click” with the text area. The difference of right click and left click is not only the shape of the icon but also its color. The last example is “Ctrl Key + Alt Key + A.” As shown in this example, every combination key with modified keys is displayed.
[Fig. 1] An overview of SZKIT and examples when it works.
Functions of showing in struction text
Addition to the basic function of showing icons related to the actions which operator makes, SZKIT has a function to display a series of instruction text blocks sequentially. The text contains explanations how to operate target ed graphic deInternational sign software and is prepared by the teaching staff in advance. Each text block should be separated by a blank line in the text file and SZKIT displays from the first text block after loading all contents of the file.
Here, this text blocks should be changed in appropriate timing during the lecture like subtitles in the movie film. To implement this changing function, SZKIT should accept triggered key action which causes forwarding or rewinding the text blocks. We were annoying this point because SZKIT runs simultaneously to the target graphic design software and the focus of the operating system is not on SZKIT, but on the targeted one. It means that the key action to control SZKIT must not be the same as anz operation of the software. In usual, graphic design software has so many combination keys and it is difficult to find a combination which no software use. After several discussions and try and errors we finally decide to implement hooking of space bar. As shown in Figure 2, “Space bar + S” causes forwarding and “Space bar + W” causes rewinding the in struction text blocks. Besides the assignment, “Space bar + number” is implemented for changing the series of text files which file name starts with numbers. Space bar works normally when the operator doesn’t press any key after pressing space bar, therefore this key hooking doesn’t disturb normal operation of space bar using this key assignment, the operator can change the instruction text block in appropriate timing without being out of focus on the targeted software. He/she can also show to hearing impaired students mouse click and modified key operations without any special actions as a matter of course.
In addition, it is possible to modify or fix the instruction text after pressing “Windows + Shift Key”. The fixed instruction text is automatically saved when the file changes or SZKIT ends. With help of this function, the teaching staff can make improvements in the instruction text during the lecture.
[Fig. 2] Key assignment of changing instruction text blocks.
Estimation
In the autumn semester of 2011, one of the authors uses SZKIT in his class and it is now continuing. All students of the class have hearing impairment, including severe one. We have a plan to take an interview and some questionnaire to the students at the end of the semester. Therefore it is not clear the results of estimation still now but he reported he feels surely that the students understand the graphic design software well compare with last lecture without SZKIT. The tangible effect is that there is less question about operations from students and less miss operations. More precise estimation results will be reported in camera ready paper.
Figure 3 is an example of screen shot which shows operation of Adobe Illustrator with SZKIT. It is a case of dragging an object with pressing Alt key. It is clear what operation is done by the teaching staff and it is easy to get information for the hearing impaired students.
[Fig. 3] A screen shot which shows operation of Adobe Illustrator with SZKIT.Every operation of mouse clicking and pressing modified key is displayed in real time.
Conclusion
To solve the problems of teaching computer operation to hearing impaired students, we developed a support software tool which displays mouse clicking and modified keys and instruction texts. With this software, instruction texts and its operation is clearly displayed on the screen in the same time.
Moreover, this software is also available for e-learning as a by-production. Only recording the operation using this software makes e-learning contents because all required information is into the screen. In this manner, we think software is not only for the for elderly person or novice person when they learn the computer operation.
References
[1] Cruse, E. Using educational video in the classroom: Theory, research and practice. Library Video Company. 2007. http://www.libraryvideo.com/articles/article2 6.asp.
[2] Debevc, M., Peljhan, Z. The role of video technology in on-line lectures for the deaf. Disability and Rehabilitation. 2004, Vol. 26, No. 17.
[3] Wald, M. Captioning for Deaf and Hard of Hearing People by Editing Automatic Speech Recognition in Real Time. Computers Helping People with Special Needs Lecture. Notes in Computer Science. Springer, 2006. p. 683–690.
[4] Target: http://download.cnet.com/Target/3000–2075_4–10915620.html.
[5] osdHotkey: http://www.romeosa.com/osdHotkey/help.html.
