CALL FOR PRIOR ART: (Nintendo) Object transparency based on distance in virtual environment

Question

TITLE: Computer-Readable Storage Medium, Information Processing System, And Information Processing Method

• App Number: 13/028,329
• Publication Number: US 2012/0146993 A1
• Assignee: Nintendo Co., Ltd.
• Prior Art Cutoff Date: predating December 09, 2010

Summary: This invention sets the transparency of a virtual object based on the distance between the viewpoint and the virtual object in a virtual environment generating both left eye and right eye images. Note: transparency in this context could be interpreted to mean several related concepts, such as fading and alpha level adjusting.

Claim 1 (as filed) is:

A computer-readable storage medium having stored therein a display control program which is executed by a computer of a display control apparatus that displays a predetermined virtual space on a display apparatus capable of stereoscopic display, the display control program causing the computer to function as:

• object placement means for placing a predetermined object in the virtual space;

• transparency degree setting means for, in accordance with a distance specified between (1) the predetermined object placed in the virtual space, and (2) a viewpoint position based on the position of a virtual camera used for virtually shooting the virtual space, setting the degree of transparency of a part or the entirety of the predetermined object such that the longer the distance is, the higher the degree of transparency is; (emphasis added)

• image generation means for generating an image for a right eye and an image for a left eye by shooting the virtual space with a right virtual camera and a left virtual camera, respectively, so that the predetermined object, which is included in each of the image for a right eye and the image for a left eye, has the degree of transparency set by the transparency degree setting means; and

• display control means for displaying the image for a right eye and the image for a left eye generated by the image generation means, on the display apparatus.

Sample Figure

WHY IT MATTERS: Object transparency based on distance between a viewpoint and an object appears to have been done before, such as in the area of ray tracing.

QUESTION: Have you seen anything (published before December 09, 2010) that describes the transparency degree setting means in the manner recited in claim 1?

If so, please submit evidence of that prior art as an answer below. Please submit only one piece of prior art per answer below. We welcome multiple prior art proposals from the same individual; please create separate answers for each one. This is so the community can vet each individual piece of prior art independently.

For details about what makes good prior art, please see our FAQ. Once you have submitted prior art, check back soon to see if the Ask Patents community has chosen your prior art to be submitted to the United States Patent & Trademark Office.

If you'd like to contribute in another way, please vote or comment on submissions made below. And we welcome you to post your own request for prior art if you know of another questionable patent or patent application.

Thanks for participating!

Answer 1

Addressing the "transparency degree setting means" issue: I believe prior art is very clearly shown by Blender's Mist feature, which uses this technique and is documented here. That page is marked as modified in 2011, but the history goes back earlier; the use of adjusting transparency based on the distance is first mentioned in this version of the page from the history (published 4 September 2007):

Mist can greatly enhance the illusion of depth in your rendering. To create mist, Blender makes objects farther away more transparent (decreasing their Alpha value) so that they mix more of the the background color with the object color. With Mist enabled, the further the object is away from the camera the less it's alpha value will be.

This directly equates to "the longer the distance is, the higher the degree of transparency is".

Addressing the issue of generating "an image for a right eye and an image for the left eye": The same software (Blender) also has a plugin that does exactly this, by shooting the virtual space with a right and a left virtual camera. The plugin is described on this page. Relevant illustration from the page:

The last bullet point ("display control means for displaying the image...") is also addressed on the same page:

You can use Blenders Node Editor to control the left and right camera render results and combine them to a Side-by-Side, Interlaced or Anaglyph image

I believe that the first point in the claim ("object placement means for placing a predetermined object in the virtual space") is obvious, but for completeness this is described in Blender's documentation here:

Each object has a center or origin point. The location of this point determines where the object is located in 3D space.

Answer 2

Prior art for this would appear to be OpenGL. Any version of OpenGL. Yes, even OpenGL 1.0, published in 1994. Let's take these in order:

• object placement means for placing a predetermined object in the virtual space;

Yes, OpenGL can be used to do that.

• transparency degree setting means for, in accordance with a distance specified between (1) the predetermined object placed in the virtual space, and (2) a viewpoint position based on the position of a virtual camera used for virtually shooting the virtual space, setting the degree of transparency of a part or the entirety of the predetermined object such that the longer the distance is, the higher the degree of transparency is; (emphasis added)

Even ignoring shader-based techniques, OpenGL's fixed-function fog is perfectly capable of this. It's simply a matter of setting the fog color alpha to a value other than 1 and turning on blending.

The code would look something like this:

glFogi(GL_FOG_MODE, GL_LINEAR);
glFogf(GL_FOG_START, 0.8f);
glFogf(GL_FOG_END, 1.0f);
GLfloat fogColor[4] = {0.0f, 0.0f, 0.0f, 0.0f};
glFogfv(GL_FOG_COLOR, fogColor);
glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA); //Pre-multiplied transparency

This will fade the object based on the distance from the virtual camera in the virtual space. The degree of transparency of part of the object will indeed be such that "the longer the distance is, the higher the degree of transparency".

This code would work on OpenGL 1.0 (PDF). Obviously, shaders could do the same thing, but having it as an explicit feature of the API makes this more relevant as prior art. These six lines, written in 1994, do what Nintendo is claiming to patent.

• image generation means for generating an image for a right eye and an image for a left eye by shooting the virtual space with a right virtual camera and a left virtual camera, respectively, so that the predetermined object, which is included in each of the image for a right eye and the image for a left eye, has the degree of transparency set by the transparency degree setting means; and

• display control means for displaying the image for a right eye and the image for a left eye generated by the image generation means, on the display apparatus.

OpenGL has supported stereoscopic rendering from day one. Every version of OpenGL has had separate back and front buffers for left and right eyes (GL_BACK_LEFT, GL_FRONT_LEFT, GL_BACK_RIGHT, GL_FRONT_RIGHT).

Generally speaking, only professional hardware exposes stereoscopic rendering through OpenGL. But the API can do it.

Answer 3

This shows how to calculate fog/mist with distance, such that objects merge with the background the further away you go:

“Visual simulation of atmospheric haze”, P J Willis, Computer Graphics Forum, 6, 1, Jan 1987, pp 35-43.

Answer 4

Minecraft added its initial test support on 2009-08-13 for anaglyph 3D (anaglyph 3D is a form of stereoscopic 3D, as per the claims), and the fog that fades objects out as their distance from the camera increases predates at least 2009-05-20, since an option to modify the distance the fog is at was added on that date.

This YouTube video of Minecraft, dated 2009-12-24, clearly demonstrates both the fog that fades objects to different degrees as their distance from the camera changes, as well as the anaglyph 3D feature.

Answer 5

Magic Carpet by Bullfrog used the same technique in 1994 to hide the limited rendering-powers of Pentium pcs... It was a cool way to make the more distant scenery disappear.

I found the following review from 1994:

The texture-mapped polygons are expertly depth cued and shade off into a beautiful mist effect in the near distance – this not only softens the whole view, making it more realistic...

Answer 6

The Source engine has support for fading "props", 3d models in levels, based on the distance. See the BaseFadeProp section of https://developer.valvesoftware.com/wiki/Prop_static for an example.

It has publicly released support for stereographic rendering since earlier this year. This by the use of the Oculus devkit. The free to play game "Team Fortress 2" is a concrete example of these both features being used together.

Answer 7

The technique of fading an object based on distance has been used in videogames and 3d applications for decades. See the "LOD Blending" section of the Wikipedia article on "Popping (computer graphics)": http://en.wikipedia.org/wiki/Popping_(computer_graphics)

Answer 8

This looks similar to Patent US6361438 B1, Video game transparency control system for images. The patent is owned by Konami.

http://www.google.com/patents/US6361438

Answer 9

"the longer the distance is, the higher the degree of transparency" Isn't this just saying "We want to take coding for fog and and patent making the numbers go down instead of up with distance" Would that fail for obviousness? Might also investigate prior art for the treatment of mirrors.

Answer 10

Re "a display apparatus capable of stereoscopic display", my first guess at where you can find prior art is the Virtual Boy game console, but I am not familiar with whether people could tune the brightness of the LEDs to give a sense of "depth" to the background. If somebody could recall a game which did this, then that would be a stellar example that the effect has been done since the late 1990s.

Wikipedia also kindly offers a list of stereoscopic video games to examine for the effect. Is anyone familiar with any of these? Did any of them tweak luminance or transparency to make things appear more distant?

UPDATE: tunnels in Descent (1995), like this screenshot shows, were darkened based on how far away they were. Descent was a stereoscopic game.

Answer 11

This paper shows how to combine colour, position and alpha in any way you want:

Projective alpha colour, Eurographics Conference 2006, Vol 25 number 3, pp. 557-566 Author Philip Willis

Answer 12

This paper shows various ways of modifying colour etc by combining colour, position and alpha:

“ARTcams: Attributed Rational Tensor Cameras”, Chuan Li, Peter Hall and Philip Willis Proc. Computational Aesthetics in Graphics, Visualization, and Imaging, pp. 73-81, 2009. O. Deussen and P. Hall (Editors

Answer 13

WP-Cumulus is a WordPress plugin that uses this basic concept to render a rotatable sphere of tags from blog posts. Here is a description and demo dating from 2008:

http://www.roytanck.com/2008/03/06/wordpress-plugin-wp-cumulus-flash-based-tag-cloud/

Answer 14

The OpenGL Programming Guide, 5th edition (published in 2006), describes use of "fog" in great detail between pages 261 and 271. It begins:

Computer images sometimes seem unrealistically sharp and well defined. Antialiasing makes an object appear more realistic by smoothing its edges. Additionally, you can make an entire image appear more natural by adding fog, which makes objects fade into the distance...

When fog is enabled, objects that are farther from the viewpoint begin to fade into the fog color.

Answer 15

Silent Hill: Downpour (2012) is an implementation that supports both stereoscopic rendering and fog, which constitutes prior art: http://en.wikipedia.org/wiki/Silent_Hill:_Downpour. This game was built using Unreal Engine, which uses OpenGL and DirectX for rendering.

This video is taken from the game. Fog (distance-based object transparency changes) is clearly visible at the beginning: http://youtu.be/nbEI4B_rmiU

Answer 16

Sounds exactly like "soft particles" where particles are made more transparent based on their distance in the scene from the camera. Here's NVidia's demo for this from 2007 for DirectX: http://developer.download.nvidia.com/whitepapers/2007/SDK10/SoftParticles_hi.pdf

Answer 17

Well, I used this technique in a project described on my website. Since I haven't published any papers about this project, I'm not sure it counts (and frankly the "fake fog transparency" trick never struck me as novel since I was, myself, copying it from numerous earlier examples even then in 2004), but the web does represent a historical record of sorts. At least, it establishes that this technique is quite old.