Computer network, computer-implemented method, computer program product, client, and server for natural language-based control of a digital network

Patent metadata

Field	Value
Publication number	US-9590818-B2
Application number	US-201213413373-A
Country	US
Kind code	B2
Filing date	Mar 6, 2012
Priority date	Mar 7, 2011
Publication date	Mar 7, 2017
Grant date	Mar 7, 2017

Abstract

The present application relates to a computer network, a computer-implemented method, a computer program product, a client, and a server for natural language-based control of a digital network. In one aspect, the computer network for natural language-based control of a digital network may comprise: a digital network operable to provide sharing of access to a network between a plurality of devices connected in the digital network; a client installed in the digital network and operable to provide a unified natural language interface to a user to control the digital network using natural language; a server connected to the client over the network and operable to process a user request of the user performed through the unified natural language interface; and one or more software agents operable to execute at least one action on at least one of the plurality of devices based on the processed user request.

First claim

What is claimed is:

1. A method comprising:
providing, by one or more processors of a client device and for display, a unified natural language interface;
receiving, by the one or more processors and using the unified natural language interface, a mode selection comprising one of an active mode or a passive mode;
selectively operating, by the one or more processors, in the one of the active mode or the passive mode based on the mode selection,
the one of the active mode or the passive mode being used to receive an instruction for performing a device action,
the active mode occurring prior to receiving the instruction for performing the device action, and when the mode selection identifies the active mode,
the passive mode occurring prior to receiving the instruction for performing the device action, and when the mode selection identifies the passive mode,
when operated in the active mode, the unified natural language interface being operable to interpret verbal information from a user input, and to control at least one device based on the verbal information interpreted from the user input, and
when operated in the passive mode, the unified natural language interface being operable to continuously listen for a user request, to extract non-verbal information and verbal information from the user request, and to control the at least one device based on the non-verbal information and the verbal information extracted from the user request;

receiving, by the one or more processors, the instruction for performing the device action;
transmitting, by the one or more processors, based on selectively operating in the one of the active mode or the passive mode, and using a network, information based on the mode selection and the instruction for performing the device action to a server device,
the network being operable to provide communication between the client device, the server device, and a plurality of devices,
each of the plurality of devices being capable of performing a particular device action based on receiving a particular instruction from the client device,

the client device processing the mode selection of the active mode by performing a first set of communications,
the server device processing the mode selection of the active mode by performing a second set of communications,
the client device processing the mode selection of the passive mode by performing a third set of communications,
the third set of communications being different than the first set of communications, and

the server device processing the mode selection of the passive mode by performing a fourth set of communications,
the fourth set of communications being different than the second set of communications;

receiving, by the one or more processors and based on transmitting the information regarding the mode selection, response information from the server device; and
instructing, by the one or more processors and based on receiving the response information from the server device, a device, of the plurality of devices, to perform the device action.

2. The method of claim 1 , further comprising:
receiving a verbal request from a user;
formatting the verbal request into speech signal information; and
transmitting the speech signal information.

3. The method of claim 1 , further comprising:
selecting the device, of the plurality of devices, based on analyzing a list of tags included in the response information.

4. The method of claim 1 , further comprising:
providing, based on instructing the device to perform the device action, an audio message indicating a result of the device action.

5. A non-transitory computer-readable medium storing instructions, the instructions comprising:
one or more instructions which, when executed by one or more processors of a client device, cause the one or more processors to:
provide, for display, a unified natural language interface;
receive, using the unified natural language interface, a mode selection comprising of one of an active mode or a passive mode;
selectively operate in the one of the active mode or the passive mode based on the mode selection,
the one of the active mode or the passive mode being used to receive an instruction for performing a device action,
the active mode occurring prior to receipt of the instruction for performing the device action, and when the mode selection identifies the active mode,
the passive mode occurring prior to receipt of the instruction for performing the device action, and when the mode selection identifies the passive mode,
when operated in the active mode, the unified natural language interface being operable to interpret verbal information, from a user input, and to control at least one device based on the verbal information interpreted from the user input, and
when operated in the passive mode, the unified natural language interface being operable to continuously listen for a user request, to extract non-verbal information and verbal information from the user request, and to control the at least one device based on the non-verbal information and the verbal information extracted from the user request;

receive the instruction for performing the device action;
transmit, based on selectively operating in the one of the active mode or the passive mode, and using a network, information based on the mode selection and the instruction for performing the device action to a server device,
the network being operable to provide communication between the client device, the server device, and a plurality of devices,
each of the plurality of devices being capable of performing a particular device action based on receiving a particular instruction from the client device,

the client device processing the mode selection of the active mode by performing a first set of communications,
the server device processing the mode selection of the active mode by performing a second set of communications,
the client device processing the mode selection of the passive mode by performing a third set of communications,
the third set of communications being different than the first set of communications, and

the server device processing the mode selection of the passive mode by performing a fourth set of communications,
the fourth set of communications being different than the second set of communications;

receive, based on transmitting the information regarding the mode selection, response information from the server device; and
instruct, based on receiving the response information from the server device, a device, of the plurality of devices, to perform the device action.

6. The non-transitory computer-readable medium of claim 5 , where the instructions further include:
one or more instructions which, when executed by the one or more processors, cause the one or more processors to:
receive a verbal request from a user; and
format the verbal request into speech signal information; and
transmit the speech signal information.

7. The non-transitory computer-readable medium of claim 5 , where the instructions further include:
one or more instructions which, when executed by the one or more processors, cause the one or more processors to:
select the device, of the plurality of devices, based on analyzing a list of tags included in the response information.

8. The non-transitory computer-readable medium of claim 5 , where the instructions further include:
one or more instructions which, when executed by the one or more processors, cause the one or more processors to:
provide, based on instructing the device to perform the device action, an audio message indicating a result of the device action.

9. A client device comprising:
a memory to st

Assignees

• Capuozzo Giuseppe
• Mammoliti Fabio
• D'Errico Daniele
• Borghini Marco
• Accenture Global Services Ltd

Inventors

• Capuozzo Giuseppe
• Mammoliti Fabio
• D'Errico Daniele
• Borghini Marco

Classifications

• H04L12/2818 Primary

  from a device located outside both the home and the home network (access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; arrangements for transmitting signals characterised by the use of a wireless eletrical link G08C17/00; telephonic communication systems adapted for combination with remote control systems H04M11/007) · CPC title

• G08C2201/31

  Voice input · CPC title

• H04L2012/285

  Generic home appliances, e.g. refrigerators · CPC title

• G10L2015/223

  Execution procedure of a spoken command · CPC title

• G10L15/22

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title