Natural language web browser

Patent metadata

Field	Value
Publication number	US-11599709-B2
Application number	US-201615159048-A
Country	US
Kind code	B2
Filing date	May 19, 2016
Priority date	May 19, 2016
Publication date	Mar 7, 2023
Grant date	Mar 7, 2023

Abstract

Described are natural language web browsers configured to provide a natural language interface to an existing web browser or a newly created web browser. That is, users interact with the web browser employing speech (or text) as input and output instead of using a mouse and keyboard as input and an electronic display screen as output. Embodiments of a natural language web browser converts user questions, statements, and/or commands into actions, reads the resulting HTML and converts the HTML into natural language descriptions to provide to the user.

First claim

What is claimed is:

1. A natural language web browsing method, comprising:
accepting by a web browser a first natural language input, the web browser configured to browse a computer network including a plurality of websites having web pages;
finding a best match to the first natural language input based on a match to the same, or conceptually similar, language;
parsing the best match to the first natural language input into at least one of a question, a statement, and a command;
operating the web browser in accordance with the first parsed natural language input, wherein the first parsed natural language input that the web browser is operated in accordance with includes one of a first navigation command, a first read command, a first summarize command, a first describe command, a first click command, a first type command, a first find command, and a first search command;
returning at least one result from the operating the web browser in accordance with the first parsed natural language input, wherein the operating the web browser is accomplished by use of at least a digital processing device;
accepting by the web browser a second natural language input corresponding to one of the at least one result of the first natural language web browsing;
parsing the second natural language input into at least one of a question, a statement, and a command;
operating the web browser in accordance with the second parsed natural language input, wherein the second parsed natural language input that the web browser is operated in accordance with includes one of a second navigation command, a second read command, a second summarize command, a second describe command, a second click command, a second type command, a second find command, and a second search command; and
returning at least one result from the operating the web browser in accordance with the second parsed natural language input, wherein the operating the web browser is accomplished by use of at least a digital processing device.

2. The method of claim 1 further including:
determining that the first natural language input is ambiguous;
in response to the determination that the first natural language input is ambiguous:
generating a query asking what was meant by the first natural language input;
accepting, by the web browser, the best match to the first natural language input in response to the query.

3. The method of claim 2 comprising the operating of the web browser in accordance with at least one of the first parsed natural language input and the same or conceptually similar natural language input, without limiting access to web sites on the world wide web.

4. The method of claim 3 wherein the operating includes navigating directly to a particular web page on the world wide web not restricted by the method to a preset number of web sites.

5. The method of claim 1 further including providing information related to actions of the web browser that occurred during the natural language web browsing.

6. The method of claim 5 further including providing information in response to a determination that a current focus of the web browsing has changed.

7. The method of claim 1 wherein the parsing includes:
enumerating web elements of a current web page that has been identified in a current web browsing session;
generating potential references for each enumerated web element;
finding a second best match between segments of the first natural language input and the generated potential references;
where finding the best match to the first natural language input based on a match to the same, or conceptually similar language includes finding a best match between the segments of the first natural language input and known question, statement, and command types; and
creating an ambiguity in response to a determination that more than one best match of the finding step is possible.

8. The method of claim 7 wherein:
the enumerated web elements include at least one of a header section, a first section, and a second section; and
the first natural language input is received from a device that does not have an electronic display.

9. The method of claim 1 wherein the navigation command is performed by:
directing the web browser to visit a destination named by a universal resource locator (URL) when the destination of the navigation command is a URL destination;
setting a focus to the URL destination; and
describing the focus to a user.

10. The method of claim 9 wherein visiting the URL destination includes:
giving the URL destination to the web browser;
loading a selected web page corresponding to the URL destination; and
restructuring the selected web page to improve an ability to describe the selected web page.

11. The method of claim 10 wherein restructuring the selected web page includes:
creating a tree structure, having tree elements, for the loaded web page based on HTML of the tree structure;
finding labels for tree elements in the HTML of the loaded web page;
replacing at least one unlabeled HTML document division element with children of the at least one unlabeled HTML document division element; and
for a tree element having more than N children, creating a more element with the remainder of children elements.

12. The method of claim 1 wherein the describe command further includes:
enumerating children of a given web element;
generating a simple reference for each child;
generating a reference for the given web element;
identifying that the given web element reference has a generated list of references; and
in response to the given web element being a URL, identifying a title of the web page.

13. The method of claim 12 further comprising:
determining that two or more adjacent children have a same reference, and
in response to the determination that two or more adjacent children have the same reference, replacing the two or more adjacent children with a group reference.

14. The method of claim 1 wherein the read command further includes:
recursively describing a web element; and
including the text of any web element encountered that has a text associated with the encountered web element.

15. The method of claim 1 wherein the first natural language input includes a question, statement, or command that is specific to the web site currently being visited.

16. The method of claim 1 wherein the first click command further includes:
directing the web browser to click on a web element based on the first parsed natural language input;
performing a navigation operation when the web element is a link; and
otherwise identifying how the web page changed.

17. The method of claim 1 wherein the first click command is a click command executed by the web browser, on a first web page, based on the first parsed natural language input; and
the second click command is a click command executed by the web browser, on a second web page, based on the second parsed natural language input, wherein the first web page and second web page can be the same.

18. The method of claim 1 , wherein the first click command is digitally executed by the web browser.

Assignees

• Palo Alto Res Ct Inc

Inventors

• Maxwell Iii John T
• Dent Kyle D
• Bobrow Daniel G

Classifications

• G06F16/338

  Presentation of query results · CPC title

• G06F40/205

  Parsing · CPC title

• G06F16/2246

  Trees, e.g. B+trees · CPC title

• G06F16/3344

  using natural language analysis · CPC title

• G06F40/143 Primary

  Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD] · CPC title