User-adaptive restaurant management system

Patent metadata

Field	Value
Publication number	US-11030678-B2
Application number	US-201816221943-A
Country	US
Kind code	B2
Filing date	Dec 17, 2018
Priority date	Dec 17, 2018
Publication date	Jun 8, 2021
Grant date	Jun 8, 2021

Abstract

A user-adaptive order processing terminal includes a display, a microphone, a configuration manager, and a motion sensor. The display displays electronic menu items in a first area for selection by a user, where the electronic menu items are accessed via selection of one or more sub-menu items. The microphone detects audio spoken by the user. The configuration manager captures the audio from the microphone, transmits the audio via first messages to a backend server, receives second messages from the backend server providing text that corresponds to the audio, accesses suggested menu items that correspond to the text, and modifies a second area of the display to present the suggested menu items for selection, where the suggested menu items would otherwise be presented in the first area through selection of the one or more sub-menu items. The motion sensor subsequently detects distance to and motions performed by the user, where the configuration manager captures movements of the user from the motion sensor, transmit the motions via third messages to the backend server, receives fourth messages from the backend server providing 3-dimensional (3D) gestures that correspond to the movements, and adds one or more of the suggested menu items to an electronic order in accordance with the 3D gestures.

First claim

What is claimed is:

1. A user-adaptive order processing terminal, comprising:
a display, configured to display electronic menu items in a first area for selection by a user, wherein said electronic menu items are displayed when said user selects one or more sub-menu hyperlinks on said display;
a microphone, configured to detect speech spoken by said user;

a configuration manager, coupled to said display and said microphone, configured to capture said speech from said microphone, to transmit said speech via first messages to a backend server, to receive second messages from said backend server providing one or more keywords that correspond to said speech, to access suggested menu items that correspond to said one or more keywords, and to modify a second area of said display to present said suggested menu items for selection, wherein said suggested menu items would otherwise be presented in said first area through selection of said one or more sub-menu hyperlinks, wherein said backend server is not on-premise with the terminal; and
a motion sensor, coupled to said configuration manager, configured to subsequently detect distance to and movements performed by said user, wherein said configuration manager captures said movements from said motion sensor, transmits said movements via third messages to said backend server, receives fourth messages from said backend server providing 3-dimensional (3D) gestures that correspond to said movements, and to access and execute commands corresponding to said 3D gestures to add one or more of said suggested menu items to one of a plurality of electronic orders, wherein said 3D gestures do not require said user to have physical contact with said motion sensor.

2. The user-adaptive order processing terminal as recited in claim 1 , further comprising:
a local gesture table, coupled to said configuration manager, that provides a mapping from said 3D gestures to said commands executed by the terminal, wherein said configuration manager executes said commands via data sent to said display.

3. The user-adaptive order processing terminal as recited in claim 2 , wherein said display comprises a touchscreen display, and wherein the terminal is employed as a self-ordering kiosk, and wherein said user is not required to touch said display in order to create said electronic order.

4. The user-adaptive order processing terminal as recited in claim 2 , wherein said terminal further comprises:
a state processor, configured to queue state changes in order queues that correspond to said one of said plurality of electronic orders; and
an order processor, coupled to said state processor and within which said configuration manager is disposed, configured to generate said state changes, and configured to access and transmit said state changes in each of said order queues to said backend server, from oldest to youngest, when operably connected to a network, wherein said order processor comprises current order state fields corresponding to all of said plurality of electronic orders.

5. The user-adaptive order processing terminal as recited in claim 4 , wherein the terminal operably connects to said network via one or more Wi-Fi access points.

6. The user-adaptive order processing terminal as recited in claim 1 , wherein said configuration manager modifies text size of said one of said plurality of electronic orders on said display as a function of said distance.

7. The user-adaptive order processing terminal as recited in claim 1 , further comprising:
a local keyword table, coupled to said configuration manager, that provides a mapping of said one or more keywords to said suggested menu items.

8. A user-adaptive order processing terminal, comprising:
a display, configured to display electronic menu items in a first area for selection by a user, wherein said electronic menu items are displayed when said user selects one or more sub-menu hyperlinks on said display;
a microphone, configured to detect speech spoken by said user;
an order processor, coupled to said display and a motion sensor, configured to maintain current states of all of a plurality of electronic orders, said order processor comprising:
current state fields, each maintaining a current state for a corresponding one of said plurality of electronic orders; and

a configuration manager, coupled to said display and said microphone, configured to capture said speech from said microphone, to transmit said speech via first messages to a backend server, to receive second messages from said backend server providing one or more keywords that correspond to said audio, to access suggested menu items that correspond to said one or more keywords, and to modify a second area of said display to present said suggested menu items for selection, wherein said suggested menu items would otherwise be presented in said first area through selection of said one or more sub-menu hyperlinks, wherein said backend server is not on-premise with the terminal; and
said motion sensor, coupled to said order processor, configured to subsequently detect distance to and movements performed by said user, wherein said configuration manager captures said movements from said motion sensor, transmits said movements via third messages to said backend server, receives fourth messages from said backend server providing 3-dimensional (3D) gestures that correspond to said movements, and accesses and executes commands corresponding to said 3D gestures to add one or more of said suggested menu items to said corresponding one of a plurality of electronic orders, wherein said 3D gestures do not require physical contact with said motion sensor.

9. The user-adaptive order processing terminal as recited in claim 8 , further comprising:
a local gesture table, coupled to said order processor, that provides a mapping from said 3D gestures to said commands to be executed by the terminal, wherein said configuration manager executes said commands via data sent to said display.

10. The user-adaptive order processing terminal as recited in claim 9 , wherein said display comprises a touchscreen display, and wherein the terminal is employed as a self-ordering kiosk, and wherein said user is not required to touch said display in order to create said electronic order.

11. The user-adaptive order processing terminal as recited in claim 9 , wherein said terminal further comprises:
a state processor, configured to queue state changes in order queues that correspond to said plurality of electronic orders, wherein said order processor generates said state changes, and to accesses and transmits said state changes in each of said order queues to said backend server, from oldest to youngest, when operably connected to a network.

12. The user-adaptive order processing terminal as recited in claim 11 , wherein the terminal operably connects to said network via one or more Wi-Fi access points.

13. The user-adaptive order processing terminal as recited in claim 8 , wherein said configuration manager modifies text size of said corresponding one of said plurality of electronic orders on said display as a function of said distance to said user.

14. The user-adaptive order processing terminal as recited in claim 8 , further comprising:
a local keyword table, coupled to said configuration manager, that provides a mapping of said one or more keywords to said suggested menu items.

15. A user-adaptive order processing method, comprising:
displaying electronic menu items in a first area for selection by a user, wherein the electronic menu items are displayed when the user selects one or more sub-menu hyperlinks on th

Assignees

• Toast Inc

Inventors

• Panigrahi Toshit
• Vick Ronald

Classifications

• G06Q20/18

  involving self-service terminals [SST], vending machines, kiosks or multimedia terminals · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06Q30/0641 Primary

  utilising user interfaces specially adapted for shopping · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title