Multi-user search system with methodology for bypassing instant indexing

Patent metadata

Field	Value
Publication number	US-2016055143-A1
Application number	US-201414555180-A
Country	US
Kind code	A1
Filing date	Nov 26, 2014
Priority date	Aug 21, 2014
Publication date	Feb 25, 2016
Grant date	—

Abstract

Multi-user computer search system with methodology for bypassing instant indexing of documents. In some embodiments, for example, a system for bypassing instant indexing includes a token store storing a set of token for a current version of a document and a tokenizer server configured to tokenize a new version of the document and to generate a set of tokens for the new version of the document. The system further includes an instant indexer server configured to determine tokens to index the document by based on identified differences between the set of tokens for the new version of the document and the set of tokens for the current version of the document, to determine whether the new version of the document is a bypass document version or non-bypass document version, and to generate an index mutation for the new version of the document including a specification of whether the generated index mutation is a bypass index mutation or a non-bypass index mutation based on whether the new version of the document is a bypass document version or non-bypass document version. The system also includes an index mutation server configured to provide the generated index mutation to an index server, if the generated index mutation is a non-bypass index mutation, or not provide the generated index mutation to the index server, if the generated index mutation is a bypass index mutation.

First claim

1 . A system comprising:
a token store storing a set of token for a current version of a document; a tokenizer server configured to tokenize a new version of the document and to generate a set of tokens for the new version of the document; an instant indexer server configured to determine tokens to index the document by based on identified differences between the set of tokens for the new version of the document and the set of tokens for the current version of the document, to determine whether the new version of the document is a bypass document version or non-bypass document version, and to generate an index mutation for the new version of the document including a specification of whether the generated index mutation is a bypass index mutation or a non-bypass index mutation based on whether the new version of the document is a bypass document version or non-bypass document version; and an index mutation server configured to provide the generated index mutation to an index server, if the generated index mutation is a non-bypass index mutation, or not provide the generated index mutation to an index server, if the generated index mutation is a bypass index mutation.

2 . The system of claim 1 , wherein the instant indexer server is further configured to:
store the generated index mutation in an index mutation journal, if the generated index mutation is a non-bypass index mutation; and not store the generated index mutation in the index mutation journal, if the generated index mutation is a bypass index mutation.

3 . The system of claim 1 , wherein the instant indexer server is further configured to determine tokens to no longer use to index the document based on identified differences between the set of tokens for the new version of the document and the set of tokens for the current version of the document.

4 . The system of claim 1 , wherein the index mutation server is further configured to receive a network request from the index server and to provide the generated index mutation from an index mutation journal to the index server in response to the network request, if the generated index mutation is a non-bypass index mutation.

5 . The system of claim 1 , wherein the instant indexer server is further configured to append the generated index mutation to an index mutation journal, if the generated index mutation is a non-bypass index mutation.

6 . The system of claim 1 , wherein the tokenizer is further configured to send a message to the instant indexer server responsive to tokenizing and generating the set of tokens for the new version of the document, the message comprising an identifier of the document, an identifier of the new version of the document, and a specification of whether the new version of the document is a bypass document version or a non-bypass document version.

7 . The system of claim 1 , wherein the instant indexer is further configured to determine the timestamp to associate with the generated index mutation in an index mutation journal, if the generated index mutation is a non-bypass index mutation.

8 . The system of claim 1 , wherein the tokenizer is further configured to store the generated set of tokens for the new version of the document in the token store.

9 . The system of claim 1 , wherein the instant indexer server is further configured to store the generated index mutation in an index mutation journal, if the generated index mutation is a non-bypass index mutation, and to store the set of tokens for the new version of the document in the token store responsive to storing the index mutation in the index mutation journal.

10 . The system of claim 1 , wherein the index server is configured to update an in-memory delta index responsive to receiving the generated index mutation from the index mutation server.

11 . The system of claim 1 , wherein the generated index mutation further comprises an identifier of the document.

12 . A method comprising:
a token store storing a set of tokens for a current version of a document; tokenizing a new version of the document and generating a set of tokens for the new version of the document; determining tokens to index the document by based on identified differences between the set of tokens for the new version of the document and the set of tokens for the current version of the document; determine whether the new version of the document is a bypass document version or non-bypass document version; generating an index mutation for the new version of the document including a specification of whether the generated index mutation is a bypass index mutation or a non-bypass index mutation based on whether the new version of the document is a bypass document version or non-bypass document version; and providing the generated index mutation to an index server, if the generated index mutation is a non-bypass index mutation, or not providing the generated index mutation to the index server, if the generated index mutation is a bypass index mutation; wherein the method is performed by one or more computing devices.

13 . The method of claim 12 , further comprising:
storing the generated index mutation in an index mutation journal, if the generated index mutation is a non-bypass index mutation; and not storing the generated index mutation in the index mutation journal, if the generated index mutation is a bypass index mutation.

14 . The method of claim 11 , further comprising:
determining tokens to no longer use to index the document based on identified differences between the set of tokens for the new version of the document and the set of tokens for the current version of the document.

15 . The method of claim 11 , further comprising:
receiving a network request from the index server; and providing the generated index mutation from an index mutation journal to the index server in response to the network request, if the generated index mutation is a non-bypass index mutation.

16 . The method of claim 11 , further comprising:
appending the generated index mutation to an index mutation journal, if the generated index mutation is a non-bypass index mutation.

17 . The method of claim 11 , further comprising:
adding a message to a message queue responsive to tokenizing and generating the set of tokens for the new version of the document, the message including an identifier of the document, an identifier of the new version of the document, and a specification of whether the new version of the document is a bypass document version or a non-bypass document version.

18 . The method of claim 11 , further comprising:
determining the timestamp to associate with the generated index mutation in an index mutation journal, if the generated index mutation is a non-bypass index mutation.

19 . The method of claim 11 , further comprising:
storing the generated set of tokens for the new version of the document in the token store.

20 . The method of claim 11 , further comprising:
storing the set of tokens for the new version of the document in the token store responsive to storing the index mutation in an index mutation journal, if the generated index mutation is a non-bypass index mutation.

21 . The method of claim 11 , further comprising:
the index server updating an in-memory delta index responsive to obtaining the generated index mutation.

22 . The method of claim 11 , wherein the generated index mutation further comprises an identifier of the document.

Assignees

• Dropbox Inc

Inventors

• Goel Samir
• Chastagnol Franck
• Agrawal Abhishek

Classifications

• G06F16/22

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/134 Primary

  Distributed indices · CPC title

• H04L63/101

  Access control lists [ACL] · CPC title

• G06F16/313

  Selection or weighting of terms for indexing · CPC title

• G06F16/319

  Inverted lists · CPC title