| Author | Message | ||
     clyde Rating: N/A |
Hi All, I only recently viewed the converted videos from AnotherWorld. Needless to say I was impressed, and even on initial scrutinity, it's apparent to me that this is'nt done via depth mapping or object segmentation on a frame by frame basis. I have a very "Stong" hunch that it has something to do with techniques that are much more simpler, and using some sort of global algorithm with only a minimal Human intervention on hard to do frames/scenes. Plainly looking at the conversions of matrix and MI-2, it shows lots of nice roundness even in extreme closeups of faces... This got me thinking.. there was a post I think on this board about Depth Cue extraction from 2d images using principles based on camera and lens optics. I think some had even rebuffed the person from belgium who had posted this ( i cant find the posts now, neither the diagrams that i had seen here). Anyways I did searches on subjects like Depth from Focus, Depth restoration +2d, etc and Im amazed by the results. I hope others join in this discussion, but from what i've gathered... - You can actually judge the "depth" of an entire scene, still or in motion, by finding the point of focus of that scene.. i.e the sharpest area of the scene is where the camera would be logically focussing. - with that info it should be a workable task to "assign" variable postive and negative parallax to the rest of the areas of the scene. - This could be done for example by "projecting" the scene onto some geometry in a CGI 3D software, or 3D compostiting sofwtare from 2 "projection cameras" , the stereobase and convergence/toe-in governed by the sharpest part of the image. in theory for a "motion scene" say you sampled two consecutive frames of video... the point of focus (or zero parallax) objects would have no blur and other objects would have eitehr a positive or negative horizonal shift of their pixels. - there could be some formula to then "map" the video onto a plane in a 3d compositing software using different convergence angles for left/right views. This is all very hard to explain here in print but heres a few good links: http://www.sonic.net/~cooptown/jcooper/stereo_3d_tutorial_site.html http://iris.usc.edu/Vision-Notes/bibliography/shapefrom402.html http://www.cise.ufl.edu/~lok/teaching/dcvef03/Seeing%203D%20from%202D%20Images.ppt#260,7,Retinal Disparity http://www.cs.sfu.ca/people/Faculty/Li/papers-on-line/Jian-SPIE2000.pdf ----- Lets see where this gets us and if any workable solution emerge Regards Clyde | ||
| clyde Rating: N/A |
Im not giving up on my theory that you could extract almost "all" depth information in a scene from just the focus point in the scene. There is lots of software and plugins for manipulating depth of field. one such brilliant one is this http://www.richardrosenman.com/dofpro.htm In the above software by generating just one depth map manually, you can then selectively assign the camera focus to that area and the rest of the scne is blurred in accordance to depth of field rules. so what if someone were to "reverse" this process? ie, study which part of a scene was "in focus" and then selectively generate a greyscale depthmap for the entire scene, based on the blurriness of the other pixels in the scene? How would u measure how deep - or- blur the other ojects in a secne were? the same way algorithms can focus or sharpen images in photoshop for example. in short, is it possible to generate a greyscale depthmap by assigning greyscale... (depth) (white = in front ,black =extremely inward and varying greys as inbetween values) ...to different parts in a scene based on a reference point of focus of that scene (ie zero depth?) I should think so, but will investigate further. Regards Clyde | ||
| Puppet Kite Kid Rating: N/A |
Check this out... you are looking at three 2D images, layered together: Parallel view. Use StereoPhoto Maker to view in in any stereoscopic format. Sorry... I don't have time right now to convert these demos to other formats. http://www.puppetkites.net/stereo3Dtech/2Dto3D-1_P.jpg I repeat... these are three 2D images. There are multiple copies of them layered together. There are no "stereoscopic images" used as the source images... only 2D images. The background was "left 2D" by just making two copies and shifting the horizontal parallax to set it back into the the depth of the composition, to become the apparent far point. The toy wagons were converted to stereoscopic 3D simply by horizontally stretching a copy of it by 1 to 3% (depending on the size of the wagon in the composition). More stretch will give it more depth, but notice it only took 1 to 3% stretch to give it all the depth it needed. More than that and you start getting severe image distortion. People studying stereoscopic deviation, take note ;-) When you stretch a copy, you do not introduce vertical parallax, so there is no stereographic artifacting by doing that. It only introduces deviation into the new pair of 2D images. Bingo :-) *Real* stereo 3D from three 2D images. I figured this out by mistake (as usual) by studying the differences between vertical and horizontal keystone distortion... of which there is none of in this 2D to 3D conversion :-) -- P. K. Kid Stereoscopic 3D images and movie clips (all G-rated): http://www.puppetkites.net | ||
| JT Rating: N/A |
Please check my abstract of a paper that was presented at Stereoscopic Displays and Applications 2005. http://www.spie.org/app/publications/index.cfm?fuseaction=toc&volume=5664&view=1#104 | ||
| clyde Rating: N/A |
JT , this is VERY encouraging. Although I'm nowhere close to being able to delve and fully understand the math behind such a process, Im sure that someone can apply a set of rules in software to "extract" the depth info from bluriness caused due to focus and depth of field and thus generate a depth map for use in anything from conventional 2 view stereoscopy to the needed in-between views for autostereoscopic displays. This can even go one step further then, and a user can "change" the depthmap to "focus" on different subjects in a scene, thus giving a 2d-3d conversion *creator* full control over composition of a 3d scene. Regards Clyde | ||
| Puppet Kite Kid Rating: N/A |
As a follow up [I have posted this information to three forums: Photo3D and 3DTV (Yahoo groups) and the Stereo3D forum], here are a series of stereo images that shows the effects of horizontally stretching (or squeezing or warping) one copy of a 2D image to create *real* stereoscopic images. This is only to show the very basics. Since I had never done this before yesterday, I am obviously not an expert ;-) Parallel views. Use StereoPhoto Maker to convert these to other stereo formats: http://www.puppetkites.net/stereo3Dtech/2Dto3D-horizstretch1_P.jpg http://www.puppetkites.net/stereo3Dtech/2Dto3D-horizstretch2_P.jpg http://www.puppetkites.net/stereo3Dtech/2Dto3D-horizstretch3_P.jpg I used the term "bulging, here, when I probably should have use "warping": http://www.puppetkites.net/stereo3Dtech/2Dto3D-horizstretch4_P.jpg ...and my first attempt at converting a 2D cube to stereo 3D: http://www.puppetkites.net/stereo3Dtech/2Dto3D-horizstretch5_P.jpg The good news is this process can be *relatively* quick, at least probably compared to some methods, such as copying and pasting rows of individual pixels... the bad news is you really need a "Bezier Warp" tool to do this. You have to avoid vertical parallax at all times! "Bezier Warp" is the only tool I have found that does that (use horizontal guidelines when warping). I haven't looked for all available sources of that tool, but Adobe After Effects Professional Edition has it :-) In After Effects, it has a temporal (keyframe) option, which means this effect can be animated! Hey, hey, hey :-) -- P. K. Kid Stereoscopic 3D images and movie clips (all G-rated): http://www.puppetkites.net | ||
| JT Rating: N/A |
What PKK did was "warping." It is part of the technique of image-based rendering. DIBR (Depth Image Based Rendering) is warping based on depth maps. That is, if PKK did the stretching (not by eye or experience but) based on the magnitude of the depth provided by the depth map, it would be DIBR. That is, the larger the depth the larger the % magnitude of stretching (in PKK's words). One can get more information by googling: "image based rendering" and "depth image based rendering." | ||
| clyde Rating: N/A |
..lets get back on track here.. What our goal should be is global depthmap extraction or generation via info presented in a 2d scene, from depth of field in photography or videography. 3D stereography generation or more precisely, second view generation via depth map is the easy part... ..creating the depthmap from "cues" held in depth of field or Stereo cuses from Focus/defocus is an area that needs to be expored. Regards Clyde | ||
| JT Rating: N/A |
Well, if you want to start a discussion... The two major drawbacks of trying to generate depth maps from blur information given a single 2D image are: (1) Arguably, in most images (such as HDTV video)there is not much blur to start out with, and (2) There is a confound of depth order in that an object beyond the focus plane and that closer than the focus plane can have the same extent of blur. | ||
| Puppet Kite Kid Rating: N/A |
This is my first attempt at doing this, so don't expect to see professional work, here :-) Bezier warping can really be convincing, although I didn't have to use it for this wagon, since there isn't much internal depth variation in the wagon itself. All I really needed to do was squeeze the right image to make it look like the back side of the wagon moved back in depth. Stretching it would do the opposite. Bezier warping would vary the depth internally, depending on what has to happen. Here's a very quick and dirty demo. Most people just repeat pixels to fill in the gaps. You can see the repeating pixels and also plenty of imperfections in this demo. This demo was done with masks (actually more like garbage mattes in this case), including one standard mask and one subtractive mask, and those masks can be animated to *follow* the imagery. Chromakeying is also a very crucial step in this process, if you are dealing with animations, since cutting out every object of every frame would be very impractical (time-consuming). Parallel images (use SPM to convert to other stereo formats). http://www.puppetkites.net/stereo3Dtech/2Dto3D-steps1_P.jpg http://www.puppetkites.net/stereo3Dtech/2Dto3D-steps2_P.jpg http://www.puppetkites.net/stereo3Dtech/2Dto3D-steps3_P.jpg http://www.puppetkites.net/stereo3Dtech/2Dto3D-steps4_P.jpg This example only used horizontal squeezing to add consistent depth to the wagon. You would have to use "Bezier Warping" to internally change the amount of depth in the wagon object itself. I never said this was easy to do :-) Even with professional tools, it is very time-consuming :-) -- P. K. Kid Stereoscopic 3D images and movie clips (all G-rated): http://www.puppetkites.net | ||
| Richard Scullion Rating: N/A |
Clyde, I'm not convinced that I agree the 'Another World' clips couldn't be done by depthmapping. I have produced similar length and quality clips by using the tools in 3DCombine, though I will admit I had a little more trouble getting clean edges. It is just very time consuming. In particular I was struck by the feeling that lots of the objects were just flat, when the should be round. The lamp on the table during the Mask clip when he's watching Diaz is a classic example. Clearly just a cardboard cut-out lamp. If you keep the shifts very low (and they have) then your eye will tend to unflatten the image. | ||
| Anonymous Rating: N/A |
Hi Richard, have you considered adding gradient painting tools via a circle, oval, or rectangular selection tool to give some objects that less cardboard cutout look when converted |
