3

AN OVERBROAD PATENT ON Recognizing objects in images - This application from Sony seeks to patent the idea of...Identifying objects through image process and registering content information about those objects! 10 minutes of your time can help narrow US patent applications before they become patents. Follow @askpatents on twitter to help.

QUESTION - Have you seen anything that was published before 11/14/2011 that discusses:

  • Object recognition combined with sensor data (e.g. GPS, Inertial Sensors, etc.)

If so, please submit evidence of prior art as an answer to this question.. We welcome multiple answers from the same individual.

EXTRA CREDIT - A reference to anything that meets all of the criteria to the question above AND ALSO uses environmental data (e.g. inertial sensor or GPS) or positional data about the object to be recognized.

TITLE: Recognizing objects in images and environmental data

Summary: [Translated from Legalese into English] A camera-type device which recognizes objects in images and attaches content to the recognized objects, which content could include environmental data (e.g. GPS, orientation of camera, etc.)

  • Publication Number: US20130124518 A1
  • Application Number: US 13/670,705
  • Assignee: Sony
  • Prior Art Date: Seeking prior Art predating 11/14/2011
  • Open for Challenge at USPTO: Open through 11/12/2013

Claim 1 requires each and every step below:

  1. an object information acquiring unit that acquires object information obtained by performing processing for detecting and identifying an object for image data; and

  2. a content information registering unit that registers target content information in a database in association with the object information.

In English this means:

A device for recogizing objects in images, comprising:

  1. a unit that detects and identifies an object in the image data; and

  2. a unit that associates content information with the object and registers that in a database.

Good prior art would be evidence of a system that did each and every one of these steps prior to 11/14/2011

You're probably aware of ten pieces of art that meet this criteria already... separately, the applicant is claiming Object recognition involving environmental data. Examples of environmental data cited by the applicant include:

  • temperature, atmospheric pressure, and humidity,
  • orientation and acceleration,
  • proximity,
  • illuminance,
  • noise,
  • date,
  • attitude (six axes)
  • user's vital data (complexion, body temperature, body mass index (BMI),
  • user's operation (commuting, playing golf, having dinner, cleaning, etc.)

"an information registration system from the Applicant"

What is good prior art? Please see our FAQ.

Want to help? Please vote or comment on submissions below. We welcome you to post your own request for prior art on other questionable US Patent Applications.

Improve this question

7 Answers 7

Reset to default
1

University of Calgary reported, in July 2011, on "AUTOMATED URBAN FEATURES CLASSIFICATION AND RECOGNITION FROM COMBINED RGB/LIDAR DATA"

http://www.ucalgary.ca/engo_webdocs/NES/11.20345_HassanEid.pdf

This is a doctoral thesis by Hassan Elsaid Elhifnawy Eid. Quoting from the abstract:

Although a Red, Green and Blue (RGB) image provides rich semantic information for different features, it is difficult to extract and separate features which share similar texture properties. The data provided by a LIght Detection And Ranging (LIDAR) system contain dense spatial information for terrain and non-terrain objects, but feature extraction poses difficulties in separating different features sharing the same height information. The thesis objective is to introduce an automated urban classification technique using combined semantic and spatial information leading to the ability to extract different features efficiently.

Using GPS in stead of LIDAR seems to me a trivial extension to this idea...

Improve this answer
1

I have in my possession a copy of Computer Power and Human Reason by Joseph Weizenbaum, first published in 1976 although my copy is from a 1993 reprint of a 1984 edition - the edition apparently only adding a fresh preface. ISBN 0-14-017911-9.

On page 194 he introduces as "current research" (presumably for the mid-1970s) a "hand-eye machine" (ie. a simple robot) built by MIT's AI Lab. This was evidently capable of recognising (and then manipulating) objects with simple geometric shapes, such as boxes and pyramids, and determining their colour. That, by itself, takes care of Step 1.

Step 2 is also covered because the robot maintains a database of information about the objects it recognises, including their location and the manipulations the robot has done on them. The following quotes from pp. 194-5 illustrate this:

It can be questioned about the current configurations of blocks on the table, about the events that have gone on during the discussion, and to a limited extent about its reasoning. It can be told simple facts which are added to its store of knowledge for use in later reasoning.

...

Human: Find a block which is taller than the one you are holding and put it into the box.

Robot: By "it," I assume you mean the block which is taller than the one I am holding.

H: OK.

R: (The hand sets down the block it is holding and picks up a taller one.)

H: What does the box contain?

R: The blue pyramid and the blue box.

H: What is the pyramid supported by?

R: The box.

...

H: Why did you clear off [the red] cube?

R: To put it on a large green cube.

H: Why did you do that?

R: To stack up the red cube, a large red block, and a large green cube.

H: Why did you do that?

R: Because you asked me to.

Improve this answer
1

This sounds like "data fusion", which is defined as:

the act or process of combining or associating data or information regarding one or more entities considered in an explicit or implicit knowledge framework to improve one’s capability (or provide a new capability) for detection, identification, or characterization of that entity

-- Open Geospatial Consortium, OGC Fusion Standards Study, Phase 2 Engineering Report, December 13 2010

This functionality was already present in systems designed for "Geospatial Intelligence":

The R&D group at Lockheed-Martin Canada has developed a target identifier function called ID Box. This computer program ...

  • ... transforms the sensor attribute input into a few contact ID declarations...

  • ... enables the ID Box to fuse attribute data from almost all kinds of sensor or information sources that may be found on large warships or patrol aircraft. These attributes are ...

    • ... acoustical signature from sonar system ...
    • ... dimensional data from imaging systems ...
    • ... dynamical parameters from positional trackers ...

-- M.A. Simard, P. Valin, F. Lesage, The ID Box: a Multisource Attribute Data Fusion Function for Target Identification, April 24 2000

Improve this answer
1

This sounds like a response to DARPA's Visual Media Reasoning (VMR) program which goes back to August 2011. There's a sample video from Wikipedia's computer vision article.

Let me know if this is relevant.

Improve this answer
1

Object Recognition? Does a vehicle numberplate count as an object? Stored in a Database? No Problem Environmental Data? Datetime and physical location

http://en.wikipedia.org/wiki/Police-enforced_ANPR_in_the_UK

plus many other sites

Improve this answer
0

A Pokedex

Said device describes a system which performed all of the mentioned steps: in (1) detecting and identifying an object in an image through a camera or other electromagnetic sensor system along with (2) registering target objects to a database and providing environmental data (such as location, sound, time, and vital data)

This prior art has appeared in several different forms as it was widely featured in various media in the Pokemon franchise including software, books, and television. Broadcast of a television show in the United States featuring said prior art occurred no later September 9, 1998 with other forms of media also extant before 11/14/2011.

Improve this answer
2
  • Can you provide links or details of this prior art that clearly shows when it was published?
    – Ron J.
    Commented Aug 15, 2013 at 12:02
  • en.wikipedia.org/wiki/Pok%C3%A9mon it celebrated its 10th anniversary in 2006, and Pokemon red and Pokemon blue was published in 1996. By Nintendo.
    – Daniel
    Commented Sep 2, 2014 at 16:34
0

Google has a bunch of pattents listed under image Object recognition before that date, such as ''Method and system for object detection in digital images'' and ''Method for warped image object recognition'' which was awarded in 2005, and seems to be the same thing as what Sony published.

Also this looks like Augmented Reality.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .