Systems, devices, and methods for content indexing and querying in a search system

Patent metadata

Field	Value
Publication number	US-2020372011-A1
Application number	US-202016991452-A
Country	US
Kind code	A1
Filing date	Aug 12, 2020
Priority date	Oct 19, 2017
Publication date	Nov 26, 2020
Grant date	—

Abstract

Systems and methods for indexing content in a personal search system are disclosed. In one embodiment, the method comprises receiving a content item from a data source; extracting, from the content item, at least one document appearing in the content item, the at least one document including a first timestamp; storing the document in a content index; retrieving an event associated with the document from an event index, the event including a second timestamp and one or more attributes; generating an updated document based on the one or more attributes, the updated document including the second timestamp and the one or more attributes; and replacing the document stored in the content index with the updated document if the second timestamp is later than the first timestamp.

First claim

1 - 20 . (canceled)

21 . A method comprising:
initiating indexing of historic content and real-time content at a first time point, the historic content comprising content items stored prior to the first time point and the real-time content comprising content items received after the first time point; executing backlog jobs in parallel with real-time processing jobs, the backlog jobs indexing the historic content in a content index and the real-time processing jobs indexing the real-time content in the content index; receiving a query from a client device, the query received during execution of the backlog and real-time processing jobs; returning a partially responsive result to the client device, the partially responsive result based on a current state of the backlog jobs.

22 . The method of claim 21 , wherein executing the backlog jobs comprises processing batches of the historic content beginning at an initial time point and terminating at the first time point, the initial time point occurring before the first time point.

23 . The method of claim 21 , wherein executing the backlog jobs comprises processing batches of the historic content beginning at the first time point and terminating at an initial time point, the initial time point occurring before the first time point.

24 . The method of claim 21 , wherein indexing the real-time content comprises performing an atomic update of an index for the real-time content.

25 . The method of claim 21 , further comprising:
receiving a second query from the client device after the backlog jobs have completed; and returning a full result to the client device, the full result comprising all responsive items in the historic content and real-time content.

26 . The method of claim 21 , wherein returning the partially responsive result comprises returning all responsive results from the real-time content and at least one responsive result from the historic content.

27 . The method of claim 21 , further comprising updating a historic content item in the content index based on the real-time processing jobs.

28 . A non-transitory computer-readable storage medium for tangibly storing computer program instructions capable of being executed by a computer processor, the computer program instructions defining the steps of:
initiating indexing of historic content and real-time content at a first time point, the historic content comprising content items stored prior to the first time point and the real-time content comprising content items received after the first time point; executing backlog jobs in parallel with real-time processing jobs, the backlog jobs indexing the historic content in a content index and the real-time processing jobs indexing the real-time content in the content index; receiving a query from a client device, the query received during execution of the backlog and real-time processing jobs; returning a partially responsive result to the client device, the partially responsive result based on a current state of the backlog jobs.

29 . The computer-readable storage of claim 28 , wherein executing the backlog jobs comprises processing batches of the historic content beginning at an initial time point and terminating at the first time point, the initial time point occurring before the first time point.

30 . The computer-readable storage of claim 28 , wherein executing the backlog jobs comprises processing batches of the historic content beginning at the first time point and terminating at an initial time point, the initial time point occurring before the first time point.

31 . The computer-readable storage of claim 28 , wherein indexing the real-time content comprises performing an atomic update of an index for the real-time content.

32 . The computer-readable storage of claim 28 , further comprising:
receiving a second query from the client device after the backlog jobs have completed; and returning a full result to the client device, the full result comprising all responsive items in the historic content and real-time content.

33 . The computer-readable storage of claim 28 , wherein returning the partially responsive result comprises returning all responsive results from the real-time content and at least one responsive result from the historic content.

34 . The computer-readable storage of claim 28 , further comprising updating a historic content item in the content index based on the real-time processing jobs.

35 . A device comprising:
a processor; and a storage medium for tangibly storing thereon program logic for execution by the processor, the stored program logic comprising:
logic, executed by the processor, for initiating indexing of historic content and real-time content at a first time point, the historic content comprising content items stored prior to the first time point and the real-time content comprising content items received after the first time point;
logic, executed by the processor, for executing backlog jobs in parallel with real-time processing jobs, the backlog jobs indexing the historic content in a content index and the real-time processing jobs indexing the real-time content in the content index;
logic, executed by the processor, for receiving a query from a client device, the query received during execution of the backlog and real-time processing jobs;
logic, executed by the processor, for returning a partially responsive result to the client device, the partially responsive result based on a current state of the backlog jobs.

36 . The device of claim 35 , wherein executing the backlog jobs comprises processing batches of the historic content beginning at an initial time point and terminating at the first time point, the initial time point occurring before the first time point.

37 . The device of claim 35 , wherein executing the backlog jobs comprises processing batches of the historic content beginning at the first time point and terminating at an initial time point, the initial time point occurring before the first time point.

38 . The device of claim 35 , wherein indexing the real-time content comprises performing an atomic update of an index for the real-time content.

39 . The device of claim 35 , the stored program logic further comprising:
logic, executed by the processor, for receiving a second query from the client device after the backlog jobs have completed; and logic, executed by the processor, for returning a full result to the client device, the full result comprising all responsive items in the historic content and real-time content.

40 . The device of claim 35 , wherein returning the partially responsive result comprises returning all responsive results from the real-time content and at least one responsive result from the historic content.

Assignees

• Oath Inc

Inventors

• Bortnikov Edward
• Shacham Ohad
• Basin Dmitry
• Sheffi Gali
• Gottesman Yonatan

Classifications

• G06F16/2365 Primary

  Ensuring data consistency and integrity · CPC title

• G06F16/93

  Document management systems · CPC title

• G06F16/2228

  Indexing structures · CPC title

• G06F16/24575

  using context · CPC title

• G06F16/248

  Presentation of query results · CPC title