Computer systems with handheld controllers

Patent metadata

Field	Value
Publication number	US-12524088-B2
Application number	US-202318348943-A
Country	US
Kind code	B2
Filing date	Jul 7, 2023
Priority date	Jul 11, 2022
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

A system may include an electronic device such as a head-mounted device and a handheld controller for controlling the electronic device. The handheld controller may have a housing with an elongated shaft extending between first and second tip portions. The handheld controller may include a touch sensor for gathering touch input, a button, rotating scroll wheel, or other input device for gathering other user input, a force sensor for gathering force input, an inertial measurement unit for gathering motion data, a camera for capturing images of the environment, and/or one or more haptic output devices such as an actuator for providing haptic output. The control circuitry may send control signals to the head-mounted device based on the sensor data and user input (e.g., force input, touch input, motion input, voice input, etc.) gathered with the handheld controller.

First claim

What is claimed is:

1 . A handheld controller for controlling an electronic device, comprising:
a housing having an elongated shaft extending between first and second tip portions; a force sensor in the first tip portion configured to gather force input; a camera in the first tip portion configured to capture images; a motion sensor configured to gather motion data; an actuator configured to provide haptic output in response to the force input and the motion data; and control circuitry in the housing configured to send control signals to the electronic device based on the force input, the motion data, and the captured images.

2 . The handheld controller defined in claim 1 wherein the first tip portion is removable from the elongated shaft.

3 . The handheld controller defined in claim 1 wherein the elongated shaft has a flat surface and a curved surface.

4 . The handheld controller defined in claim 3 further comprising a button on the flat surface of the elongated shaft.

5 . The handheld controller defined in claim 3 further comprising a touch sensor on the curved surface of the elongated shaft.

6 . The handheld controller defined in claim 1 further comprising a power receiving coil configured to receive wireless power.

7 . The handheld controller defined in claim 1 wherein the actuator comprises a linear resonant actuator that is configured to impart directionality to the haptic output.

8 . The handheld controller defined in claim 1 further comprising a scroll wheel that rotates relative to the elongated housing.

9 . The handheld controller defined in claim 1 wherein the elongated housing comprises first and second housing portions operable in a first mode in which the first and second housing portions are mated with one another and a second mode in which the first and second housing portions are separated from one another and independently operated.

10 . The handheld controller defined in claim 1 wherein the control signals include instructions to select a user interface element on a display in the electronic device.

11 . The handheld controller defined in claim 1 wherein the control signals include instructions to draw a line on a display in the electronic device.

12 . The handheld controller defined in claim 1 wherein the control signals include instructions to type a word on a display in the electronic device.

13 . The handheld controller defined in claim 1 further comprising:
a touch sensor on the elongated shaft configured to gather touch input.

14 . The handheld controller defined in claim 13 wherein the housing has a length between 130 mm and 160 mm and a diameter between 10 mm and 15 mm.

15 . The handheld controller defined in claim 13 wherein the camera comprises a one-pixel camera.

16 . The handheld controller defined in claim 13 wherein the control signals include instructions to scroll content on a display in the electronic device.

17 . A handheld controller for controlling an electronic device, comprising:
a housing having an elongated shaft extending between first and second tip portions; a force sensor in the first tip portion configured to gather force input; a motion sensor configured to gather motion data; an actuator configured to provide haptic output in response to the force input and the motion data; control circuitry in the housing configured to send control signals to the electronic device based on the force input and the motion data; and visual markers located on the elongated shaft, wherein the visual markers are configured to be captured by a camera in the electronic device and wherein the electronic device tracks a location of the handheld controller relative to the electronic device using the visual markers and the motion data.

18 . The handheld controller defined in claim 17 wherein the control signals include instructions to manipulate computer-generated display content on the electronic device.

19 . The handheld controller defined in claim 18 wherein the instructions are based on the motion data.

20 . The handheld controller defined in claim 19 wherein the instructions to manipulate the computer-generated display content comprise instructions to manipulate content selected from the group consisting of: virtual keyboard keys and a virtual writing implement.

Assignees

• Apple Inc

Inventors

• Dey Stephen E
• Bosch Erin M
• Pan Yuhao
• Colahan Ian P
• Ewy Christopher K

Classifications

• G06F1/1694

  the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer · CPC title

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06F3/0485

  Scrolling or panning · CPC title

• G06F3/03545

  Pens or stylus · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title