Visual Recognition and Sensor Fusion Weight Detection System and Method

Patent metadata

Field	Value
Publication number	US-2025225846-A1
Application number	US-202519089345-A
Country	US
Kind code	A1
Filing date	Mar 25, 2025
Priority date	Dec 21, 2018
Publication date	Jul 10, 2025
Grant date	—

Abstract

Disclosed are visual recognition and sensor fusion weight detection system and method. An example method includes: tracking, by a sensor system, objects and motions within a selected area of a store; activating, by the sensor system, a first computing device positioned in the selected area in response to detecting a presence of a customer within the selected area: identifying, by the sensor system, the customer and at least one item carried by the customer; transmitting, by the sensor system, identifying information of the customer and the at least one item to a computing server system via a communication network; measuring, by the first computing device, a weight of the at least one item; transmitting, by the first computing device, the weight to the computing server system via the communication network; and generating, by the computing server system, via the communication network, transaction information of the at least one item.

First claim

1 . A method, performed by a computer system comprising a processor and a non-transitory computer-readable medium, comprising:
detecting motions of objects within an area based on sensor data captured by a plurality of sensors of a computing device; determining that an object is within the area based on the detected motions of objects; responsive to determining that the object is within the area, capturing one or more images of the object using one or more cameras of the computing device; weighing an item using the computing device; identifying the item based on the weight of the item and the one or more images captured by the one or more cameras of the computing device; and transmitting identifying information of the item to a computing server system of the of the computing device via a communication network.

2 . The method of claim 1 , wherein determining that the object is within the area comprises:
determining that the object is within a threshold distance of the computing device for a threshold period of time.

3 . The method of claim 1 , wherein the area is a selected area within a store.

4 . The method of claim 3 , wherein the selected area within the store is a visually-designated area within the store.

5 . The method of claim 1 , wherein the plurality of sensors comprises a weight sensor or an infrared sensor.

6 . The method of claim 1 , wherein detecting motions of objects within an area comprises:
applying a visual recognition algorithm to an image captured by a camera.

7 . The method of claim 1 , wherein the computing device is a device that is coupled to a shopping cart.

8 . The method of claim 1 , further comprising:
identifying a user associated with the object based on the sensor data captured by the plurality of sensors.

9 . The method of claim 8 , wherein identifying the item comprises:
identifying a motion of the user based on the sensor data.

10 . The method of claim 1 , further comprising:
transmitting the weight of the item to the computing server system.

11 . A non-transitory computer-readable medium storing instructions that, when executed by a computer system, cause the computer system to perform operations comprising:
detecting motions of objects within an area based on sensor data captured by a plurality of sensors of a computing device; determining that an object is within the area based on the detected motions of objects; responsive to determining that the object is within the area, capturing one or more images of the object using one or more cameras of the computing device; weighing an item using the computing device; identifying the item based on the weight of the item and the one or more images captured by the one or more cameras of the computing device; and transmitting identifying information of the item to a computing server system of the of the computing device via a communication network.

12 . The computer-readable medium of claim 11 , wherein determining that the object is within the area comprises:
determining that the object is within a threshold distance of the computing device for a threshold period of time.

13 . The computer-readable medium of claim 11 , wherein the area is a selected area within a store.

14 . The computer-readable medium of claim 13 , wherein the selected area within the store is a visually-designated area within the store.

15 . The computer-readable medium of claim 11 , wherein the plurality of sensors comprises a weight sensor or an infrared sensor.

16 . The computer-readable medium of claim 11 , wherein detecting motions of objects within an area comprises:
applying a visual recognition algorithm to an image captured by a camera.

17 . The computer-readable medium of claim 11 , wherein the computing device is a device that is coupled to a shopping cart.

18 . The computer-readable medium of claim 11 , the operations further comprising:
identifying a user associated with the object based on the sensor data captured by the plurality of sensors.

19 . The computer-readable medium of claim 18 , wherein identifying the item comprises:
identifying a motion of the user based on the sensor data.

20 . The computer-readable medium of claim 11 , the operations further comprising:
transmitting the weight of the item to the computing server system.

Assignees

• Maplebear Inc

Inventors

• Gao Lin
• Yang Shiyuan

Classifications

• G06Q20/208

  Input by product or record sensing, e.g. weighing or scanner processing · CPC title

• G06V20/44

  Event detection · CPC title

• G06V20/68

  Food, e.g. fruit or vegetables · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title

• G07G1/0072

  with means for detecting the weight of the article of which the code is read, for the verification of the registration (weighing combined with recording means, e.g. in weighing scale, G01G19/415) · CPC title