Semiautomated relay method and apparatus

Patent metadata

Field	Value
Publication number	US-10878721-B2
Application number	US-201514953631-A
Country	US
Kind code	B2
Filing date	Nov 30, 2015
Priority date	Feb 28, 2014
Publication date	Dec 29, 2020
Grant date	Dec 29, 2020

Abstract

A hearing user's device for communicating with a hearing impaired assisted user using an assisted user's device that includes a speaker and a display screen for broadcasting a hearing user's voice signal and presenting captioned text associated with the hearing user's voice signal to the assisted user, respectively, the hearing user's device comprising a microphone for receiving the hearing user's voice signal as spoken by the hearing user, a display screen and a processor linked to the microphone and the display screen, the processor transmitting the hearing user's voice signal to the assisted users device, the processor presenting a quality indication of the captioned text presented to the assisted user via the assisted users device for consideration by the hearing user.

First claim

What is claimed is:

1. A hearing user's device for communicating with a hearing impaired assisted user using an assisted user's device that includes a speaker and a display screen for broadcasting a hearing user's voice signal and presenting captioned text associated with the hearing user's voice signal to the assisted user, respectively, the hearing user's device comprising:
a microphone for receiving a hearing user's voice signal as spoken by a hearing user;
a display screen; and
a processor linked to the microphone and the display screen, the processor transmitting the hearing user's voice signal to the assisted user's device, the processor presenting a quality indication of the captioned text presented to the assisted user via the assisted user's device for consideration by the hearing user, wherein the processor presents a call assistant activator via the hearing user's device that can be selected to change a process used to generate the captioned text associated with the hearing user's voice signal.

2. The device of claim 1 wherein the quality indication includes the captioned text presented to the assisted user.

3. The device of claim 1 wherein the quality indication includes a calculated value that represents a confidence factor indicating likelihood that the captioned text presented to the assisted user is accurate.

4. The device of claim 1 wherein a system processor transcribes the hearing user's voice signal to captioned text and transmits the captioned text to the assisted user's device for display.

5. The device of claim 4 wherein the system processor also transmits the captioned text to the hearing user's device for display.

6. The device of claim 5 wherein the system processor is remotely located from the hearing user's device.

7. The device of claim 1 wherein the hearing user's device processor also presents a text transcription associated with an assisted user's voice signal on the hearing user's device display screen.

8. The device of claim 7 wherein a system processor uses an automated voice recognition (AVR) engine to generate the text transcription associated with the assisted user's voice signal.

9. The device of claim 8 wherein the AVR engine is located in one of the assisted user's device and the hearing user's device.

10. The device of claim 1 wherein an initial process used to generate the captioned text associated with the hearing user's voice signal includes an automated voice recognition (AVR) engine and wherein a process used to generate the captioned text associated with the hearing user's voice signal after the activator is selected includes a call assistant.

11. The device of claim 1 wherein the hearing user's device links directly to the assisted user's device during a communication and wherein the hearing user's device links directly to a relay when a call assistant is required for at least some portion of a captioning service.

12. A hearing user's device for communicating with a hearing impaired assisted user using an assisted user's device that includes a speaker and a display screen for broadcasting a hearing user's voice signal and presenting captioned text associated with the hearing user's voice signal to the assisted user, respectively, the hearing user's device comprising:
a microphone for receiving a hearing user's voice signal as spoken by a hearing user;
a display screen; and
a processor linked to the microphone and the display screen, the processor transmitting the hearing user's voice signal to the assisted user's device, the processor receiving information indicating a quality of the captioned text presented to the assisted user from a system processor and presenting a quality indication of the captioned text presented to the assisted user via the assisted user's device on the hearing user's device display screen for consideration by the hearing user.

13. The device of claim 12 wherein the hearing user's device receives the information indicating a quality from the assisted user's device.

14. The device of claim 12 wherein the captioned text is generated at a relay that includes the system processor, the system processor generating the information indicating a quality.

15. The device of claim 12 wherein the quality indication includes the captioned text presented to the assisted user.

16. The device of claim 12 wherein the quality indication includes a calculated value that represents a confidence factor indicating likelihood that the captioned text presented to the assisted user is accurate.

17. The device of claim 12 wherein the system processor is remotely located from the hearing user's device.

18. The device of claim 17 wherein the system processor is remotely located from the assisted user's device.

19. The device of claim 12 wherein the hearing user's device processor is further programmed to provide an option on the hearing user's device display screen enabling the hearing user to change a text captioning process.

20. A method enabling captioned communication between a hearing user using a hearing user's device and an assisted user using an assisted user's device wherein the assisted user's device includes a processor, display screen, a microphone, a speaker and a caption activator, the method comprising:
the assisted user's device processor programmed to perform the steps of:
(i) establishing a first link to the hearing user's device for transmitting voice signals between the assisted user's device and the hearing user's device;
(ii) broadcasting a hearing user's voice signal to the assisted user via the speaker; and
(iii) upon receiving a signal indicating that the caption activator has been selected:
(a) causing the hearing user's device to link directly to a relay via a second link and to transmit the hearing user's voice signal to the relay via the second link while the hearing user's voice signal is continually broadcast via the speaker;
(b) receiving transcribed text from the relay; and
(c) presenting the transcribed text to the assisted user via the display screen.

21. The method of claim 20 wherein the transcribed text from the relay is received via a third communication link.

22. The method of claim 21 further including the step of, upon receiving a signal indicating that the caption activator has been selected, transmitting voice signals between the assisted user's device and the hearing user's device through the relay and the second and third links and discontinuing the first link.

23. The method of claim 22 wherein the step of causing the hearing user's device to link directly to a relay includes transmitting a signal to the hearing user's device to initiate the direct link.

24. The method of claim 20 wherein the assisted user's device processor is further programmed to use an automated voice recognition (AVR) engine to transcribe an assisted user's voice signal to text.

25. The method of claim 24 wherein the assisted user's device processor transmits the text generated by the automated voice recognition (AVR) engine to at least one of the relay and the hearing user's device.

26. The method of claim 25 wherein the relay includes a call assistant station that includes a display screen and wherein the text generated by the AVR engine is presented to a call assistant via the display screen.

27. The method of claim 25 wherein the hearing user's device includes a display screen and whe

Assignees

• Ultratec Inc

Inventors

• Engelke Robert M
• Colwell Kevin R
• Engelke Christopher

Classifications

• G10L15/26 Primary

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• H04W4/12

  Messaging; Mailboxes; Announcements · CPC title

• G10L15/07

  to the speaker · CPC title

• G10L2015/221

  Announcement of recognition results · CPC title

• H04M2201/18

  Comparators · CPC title