Paper details

1 Introduction

The Support Centre for Students with Special Needs at Masaryk University, Czech Republic, provides hearing impaired students (typically, hard of hearing students that comprehend written Czech) as well as visually impaired students, with services in educational setup.

One of the major services for students with hearing impairment is the speech-to-text reporting service, when a reporter records the speech (a lecture or a seminar in a university environment or even any similar event outside of a university environment) as a text on a computer by means of a simultaneous typing, and several students reading the report on one common screen at the same time.

As students with visual impairment are concerned, one of the major problems is the accessibility of the visual documents and other materials presented by a lecturer via a presentation screen. Distance to the presentation screen as well as light conditions or the presentation layout creates readability barriers and obstructs the students to perceive presented information.

2 Visualisation of the Synchronous Transcription (Speech-To-Text Reports)

As the number of students who are dependent on that service and who attend classes together has been increasing, Masaryk University tried to find a satisfactory solution to situations when groups of three or more students needed to follow a transcript at once and provide them with a comfortable access to the report.

2.1 Requirements for a Solution

From this experience we gathered several key points which the new system should fulfill. The requirements for a new solution were the following:

• simple and quick technical preparation;

• independence on a local technical infrastructure;

• speech-to-text report must be synchronized with a reporter’s typing;

• the look of the reported text should be customizable;

• a possibility of two-way communication;

• support for Czech and English language.

2.2 Analysis of Comparable Existing Systems

When analyzing existing systems comparable, we intentionally skip those which provide speech-to-text transformation based on automatic speech-to-text recognition. Systems which have been analyzed involves a human speech-to-text reporter:

• Google Documents (http://drive.google.com)

• Bee Communications (http://www.bee-communications.com)

• eScribe (http://www.escribe.cz)

Their common denominator is dependency on Internet connectivity in the location of text reporting. We consider this crucial requirement to be the most disadvantageous because utilization of the systems becomes highly dependent on local technical infrastructure. Additionally, most of the systems have quite complicated process of initialization and fail in the requirement of document layout setup independent among users.

2.3 Polygraf – Own Developed System

Defined requirements and analysis led us to the developing of our own system – Polygraf and practising it. The system meets all the requirements listed above and presents a technical solution to all those issues. It is based on a technology which can be used independently and which considers common technical parameters of an environment such as a wireless network signal coverage. The only requirement is access to electricity. All those make it quite a universal transcript visualisation system.

The key benefits and features of Polygraf visualisation of transcripts:

• It consist of three main hardware parts only, therefore it can be easily transported and prepared for work in a very short time.

• It does not require Internet connection, initial network set-up, or its own server.

• It is compatible with Microsoft Word environment commonly used among Czech speech-to-text reporters.

• It uses handheld devices (e.g. smart phones or tablet) as a text display to ensure the basic user comfort.

• The reports are available in the environment of native applications (Windows, iOS, Android) and are also accessible via standard Internet browsers.

• It enables two-way communication: reporter to user and also user to reporter by simply typing a short message which will then appear in the text-reporters screen.

• The software solution also enables to show a report as dynamic captions on a speaker's computer and thus immediately supplement the speaker's presentation when there is one.

2.3.1 Hardware Components of the System

1 The computer for the reporter - desktop or a laptop computer with Microsoft Windows operating system and Microsoft Word in version 2003 or later installed, and equipped with Wi-Fi network interface.

2 The communication bridge - A Wi-Fi access point to create a wireless network connecting all devices together and transfer the text between them. There is no need of Internet connectivity for this wireless network.

3 The display for the user - Any device with the iOS, Android or Windows operating system. Typically, an smart phone or tablet computer with Wi-Fi interface. There can be many of those user’s devices used as a handheld displays to follow a transcript.

2.3.2 Software Components of the System

1 The application for the reporter’s computer – Microsoft Word AddIn. This is installed separately alongside the Word, and its purpose is to “share” the text out of the Word documents to the users’ devices.

2a The application for the handheld devices handles the reception of a text report and its continuous updating and displaying according to a custom setting of the user (font type, font size, colors, line-spacing etc.).

2b The application for the presenter’s computer is an alternative viewing application, which displays the reported text in the form of continuously updated subtitles of up to four lines displayed as a floating caption, which can be attached to a speaker's presentation.

3 Presentation Screen Mirroring Enhancing the Readability for Users with Visual Impairment

Many lectures and presentations are supplemented by presentation slides using a projection. Speeches of the presentations often contain only part of information and referring of the speaker to his/her visual documents projected is very common. While blind users may work with accessible formats and have the material at their disposal, visually impaired users try to follow the mainstream visual content. Visually impaired people, but not only them, have to struggle with bad visualization conditions on the spot as well as with slides prepared inproperly. To reduce the difficulties and to offer the possibility to magnify the content by the user, Masaryk University has extended its Polygraf application to provide a screen mirroring of the presenter’s computer.

3.1 Requirements for a Solution

Observing students in lectures at the university and participants at presentations on conferences we gathered a list of key features required to such a system:

• simple and quick technical preparation;

• independence on presentation technologies;

• speech-to-text report must be possible to be added;

• magnification of the content must be possible;

• compatible with handheld devices of users;

• nearly real-time update of the mirroring without consuming too much battery power.

3.2 Analysis of Comparable Existing Systems

When analyzing existing systems we encountered software solutions providing the active screen sharing including control handling as well as screen casting solutions.:

• TeamViewer (http://www.teamviewer.com)

• Quick ScreenShare (http://quickscreenshare.com/)

• Skype Screen Sharing (http://www.skype.com)

The common disadvantage on all software solutions is the time needed for the preparation. While it is possible to prepare a presentation computer within a lecture room, it is impossible to maintain the installations and provide the service with personal devices (such as laptops) or, at a conference, to manage a situation when speakers are exchanging together with their presentation computers.

Additionally, the software we tested is quite complicated in processes of initialization of sharing and partially it is limited in number of participating users. None of the software offered the possibility to include a transcript.

3.3 Polygraf – Continous Developed System on Universal Access

For those reasons mentioned above, Masaryk University considered to continue the development of the Polygraf application, adding the function of screen mirroring. It is implemented utilizing external hardware without needs of adapting presenter’s software environment. The combination of screen mirroring and the speech-to-text services should be provided.

The key benefits and features of Polygraf screen mirroring:

• It consist of three main hardware parts only, therefore it can be easily transported and prepared  for work in a very short time.

• It does not require Internet connection, initial network set-up, or its own server.

• It is platform independent on the side of presenters

• It uses handheld devices (e.g. smart phones or tablet) as a display to ensure the basic user comfort.

• The mirroring is available in the environment of native applications (Windows, iOS, Android)

• The software solution also enables to show a screen mirroring in combination with a speech-to-text; it offers the magnification of the screen capturing and the transcript visualisation can be displayed color-inverted.

3.3.1 Hardware Components of the System

1 A screen capturing device for the presenter – a broadcasting device (e.g. Epiphan VGA Broadcaster) capturing the VGA signal and providing it, apart from the output for data projectors, as a broadcasted motion JPEG through network interface.

2 The communication bridge – a Wi-Fi access point to create a wireless network connecting all devices together and to transfer the text between them. There is no need of Internet connectivity for this wireless network.

3 The display for the user – any device with the iOS, Android or Windows operating system. Typically, a smart phone or tablet computer with Wi-Fi interface. There can be many of those user’s devices used as a handheld displays to follow a screen mirroring.

3.3.2 Software Components of the System

The only software required for the visualisation of screen mirroring is an application for the handheld devices which handles the reception of a screen capturing and it is continuously updating and displaying it according to a custom setting of the user.

4 Usage and Practical Experience

Polygraf system was used throughout the last three academic years in lectures of Masaryk University in total amount several hundreds of them. Commonly, there were 3–6 students attending each of the lectures. There was several sessions to adapt the reporters to the new system, which they were supposed to get familiar with.

Polygraf system was also used for the text-reporting and subtitling of presentations in the International Conference Universal Learning Design in Brno, Czech Republic, 2011. The text transcript was available to the audience through LCD displays situated on both sides next to the speakers podium as well as to any user of the iPad handheld devices which were available to borrow. At ULD conference, four text-reportings were running simultaneously – one report for each of two languages in each of two conference rooms. Additionally, during the last year, the usage of Polygraf system at conferences has been enhanced. The service was realized at INSPO Conference, Prague, Czech Republic in 2011 and 2012 as well as at the International conference ICCHP 2012 in its Universal Learning Design Track.

During the next academic year and the 3rd Universal Learning Design conference in February 2013 the screen mirroring will be introduced to users and a wider user feedback will be available by the time of the conference.