Method for displaying image combined with playing audio in an electronic device

Patent metadata

Field	Value
Publication number	US-9743033-B2
Application number	US-201414162175-A
Country	US
Kind code	B2
Filing date	Jan 23, 2014
Priority date	Mar 18, 2013
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

A method for generating an image combined with audio, and image display and audio output includes displaying an image, when a first image object within the image is selected, outputting a first audio object corresponding to the first image object and, when a second image object within the image is selected, outputting a second audio object corresponding to the second image object.

First claim

What is claimed is:

1. A method of an electronic device, the method comprising:
acquiring an image via at least one camera in the electronic device;
acquiring audio for the image via at least one microphone in the electronic device;
identifying at least one image object included in the acquired image
identifying at least one audio object included in the acquired audio;
generating mapping data including information for indicating a relationship between the identified at least one image object and the identified at least one audio object; and
generating integrated data based on image data for the acquired image, audio data for the acquired audio, and the generated mapping data,
wherein the audio data comprises information regarding a processed audio, and
wherein the processed audio is generated by removing at least one of a mute duration, a noise, or a background shutter sound caused by the acquiring the image from the acquired audio.

2. The method of claim 1 , wherein the audio is acquired during a duration determined based on a timing initializing to acquire the image.

3. The method of claim 1 , wherein,
if the image is a still image, the image is acquired at a timing within a duration acquiring the audio, and
if the image is a moving image, the image is acquired within the duration acquiring the audio.

4. The method of claim 1 , wherein identifying the at least one image object comprises identifying a closed area on the acquired image indicating a specific subject or a specific movement as the at least one image object.

5. The method of claim 1 , further comprising:
identifying a face of a character in the image; and
determining a characteristic of the character by comparing the identified face of the character with reference face images of characters,
generating the image data comprising information for the characteristic,
wherein the characteristic comprises gender or ages.

6. The method of claim 1 , further comprising separating a voice signal, by speaker, from the acquired audio.

7. The method of claim 6 , wherein separating the voice signal by speaker comprises:
identifying a voice signal included in the audio; and
identifying a voice signal of each speaker by comparing a speaker-related feature with a characteristic of the identified voice signal.

8. The method of claim 7 , wherein the speaker-related feature is determined through analysis of an image, analysis of an audio signal collected at a phone call, interpretation of a literal meaning through voice recognition, or direction determination.

9. The method of claim 6 , wherein separating the voice signal by speaker comprises:
identifying a plurality of voice signals from the audio; and
separating the plurality of voice signals according to a direction in which the identified plurality of voice signals are received.

10. The method of claim 1 , wherein generating the mapping data comprises:
generating the information for indicating the relationship between the identified at least one object and the identified at least one audio object, by respectively mapping, to the at least one object, the at least one audio object having a characteristic corresponding to a characteristic of the at least one object; and
generating the mapping data including the generated information for indicating the relationship.

11. The method of claim 1 , wherein generating the mapping data comprises:
generating the information for indicating the relationship between the identified at least one object and the identified at least one audio object by respectively mapping, to the at least one image object, the at least one audio object that is received while the at least one image object is moving; and
generating the mapping data including the generated information for indicating the relationship.

12. The method of claim 1 , wherein generating the mapping data comprises:
generating the information for indicating the relationship between the identified at least one object and the identified at least one audio object by respectively mapping, to the at least one image object, the at least one audio object having a reception direction corresponding to a direction of the at least one image object; and
generating the mapping data including the generated information for indicating the relationship.

13. The method of claim 1 , wherein generating the mapping data comprises:
displaying a UI for setting the relationship between the identified at least one image object and the identified at least one audio object;
generating the information for indicating the relationship based on an input of a user's instruction on the displayed UI; and
generating the mapping data including the generated information for indicating the relationship.

14. The method of claim 1 , wherein generating the mapping data comprises:
determining a combination relationship between the at least one image object and the at least one audio object that are identified from the image and the audio acquired at different time points, as the relationship; and
generating the mapping data including the information for indicating the combination relationship.

15. The method of claim 1 , wherein a form of the integrated data is one of a first form in which the image data is inserted into the audio data, a second form in which the audio data is inserted into the image data, a third form in which the image data is constructed as a video track and the audio data is constructed as an audio track, and a fourth form in which the image data, the audio data, and the mapping data separately exist.

16. An electronic device, comprising:
at least one processor;
at least one camera operatively coupled to the at least one processor; and
at least one microphone operatively coupled to the at least one processor,
wherein the at least one processor is configured to:
acquire an image via at least one camera in the electronic device and acquire audio for the image via at least one microphone in the electronic device;
identify at least one image object included in the acquired image and identify at least one audio object included in the acquired audio;
generate mapping data including information for indicating a relationship between the identified at least one image object and the identified at least one audio object; and
generate integrated data based on image data for the acquired image, audio data for the acquired audio, and the generated mapping data,
wherein the audio data comprises information regarding a processed audio, and
wherein the processed audio is generated by removing at least one of a mute duration, a noise, or a background shutter sound caused by the acquiring the image from the acquired audio.

17. The device of claim 16 , wherein the audio is acquired during a duration determined based on a timing initializing to acquire the image.

18. The device of claim 16 , wherein,
if the image is a still image, the image is acquired at a timing within a duration acquiring the audio, and
if the image is a moving image, the image is acquired within the duration acquiring the audio.

19. The device of claim 16 , wherein the at least one processor is configured to:
identify a closed area on the acquired image indicating a specific subject or a specific movement as the at least one image object.

20. The device of claim 16 , wherein the at least one processor is further configured to:
identify a face of a character in the image;
determine a characteristic of the character by comparing the ide

Assignees

• Samsung Electronics Co Ltd
• Samsung Electronics Co Ltd

Inventors

• Kim Hyun-Jung
• Cheong Cheol-Ho
• Kim Jin-Woo
• Lee Ki-Huk

Classifications

• G11B27/34

  Indicating arrangements  {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} · CPC title

• G11B27/28

  by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)} · CPC title

• G11B27/11 Primary

  by using information not detectable on the record carrier · CPC title

• H04N5/93 Primary

  Regeneration of the television signal or of selected parts thereof · CPC title

• G11B20/10

  Digital recording or reproducing · CPC title