| Author | Message | ||
     Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 1 Registered: 1-2008 Rating: N/A |
Hi, I've been intrested on this subject for some years now. I've been playing around with After effects to create 3d stereo clips with quite good results. I recently downloaded 3DCombine and ran some tests with it. The result effect is quite good, but I think the final result could be better. Automatic video conversion seems to compare next or previous frames and displace the right image according to this data. I managed to get very similar results with AE7. What I've done in AE7 is just take the saturation of the image (Channel combiner->saturation multiplied->luminance) and use this bw image as displacement map for left and right view. This method work quite well due the fact that saturation decreases according the distance. But to get even better results the method which 3DCombine uses could be blended with this Chroma method I've tested. Here comes the problem; The depthmap created from previous/next frames does not give the directiondata; this leads one direction elements (left or right) to be wrong way round. Does anyone know where to find a filter (for Avisynth, Virtual Dub or After Effects ) which could color motion vector blocks according their directions? For example; if movement is to right, the color could be red and going to left it could be green. Keying these colors and using it as a additional mask with chroma would give really good results. I've tested this with one direction and it works well. Br, mcmount | ||
| Larry Elie (Ldeliecomcastnet) Member Username: Ldeliecomcastnet Post Number: 46 Registered: 10-2006 Rating: N/A |
Could you use the motion vector blocks from something like deshaker (http://www.guthspot.se/video/deshaker.htm)? I thought After Efects had a similar motion vector detection module, but I'm not sure it gives you the between stage of vectors that deshaker has. | ||
| Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 2 Registered: 1-2008 Rating: N/A |
It doesn't do what is needed; It just gives motion vector lines, where I would need real mask created from this data. Closest I've found so far is Avisynth MVTools, where you can choose motion vectors or colored blocks. These colored blocks are too big and it's impossible to get any color info according the direction with it. | ||
| Larry Elie (Ldeliecomcastnet) Member Username: Ldeliecomcastnet Post Number: 47 Registered: 10-2006 Rating: N/A |
I'm afraid you are going to have to either write the module yourself, or find someone to do it for you. Perhaps such a module could be incorporated into 3DCombine. | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 1 Registered: 4-2009 Rating: N/A |
Hello Jari,.. thanks much for your unique technique description. It makes sense but i've found no one else using saturation distance-deminishment to extract depth. You'd written "..in AE7..just take the saturation of the image (Channel combiner->saturation multiplied->luminance) and use this bw image as displacement map for left and right view." Is this done in the Combiner using a single frame or current frame with a previous frame as source? How can this be done in AE6 since its Channel Combiner does'nt have a Luminance option? My results don't produce a usable displacement map, perhaps i'm missing steps? Thanks for any help, Jeff Cottle | ||
| Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 3 Registered: 1-2008 Rating: N/A |
Hi Jeff, I'm sorry I can't help you with AE6 about this. However, you get same chroma map using VirtualDub and trevlacs colourscale -> greyscale filters. I was having long discussions about this chromamap method with Richard Scullion (3DCombine) and we noticed best result will be get by using chromamap layer only with 20% opacity with the final displacement map. I've got best results with motion vector based analysis + chromamap. This way chromamap gives some depth with static images and motion analysis kicks in whenever something moves. Motion analysis is done with avisynth MVTools, then piped to VirtualDub where result is smoothened (temporal smoother + motion blur). Hope this help, Jari | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 2 Registered: 4-2009 Rating: N/A |
Thanks Jari,.. really appreciate your reply and instructions. I'm going to install VirtualDub today and have downloaded trevlacs ColourScale.rar plugin. I'm a novice and a bit confused, 1) does colourscale produce a displacement map or a chroma map, 2) would i use the movie color image layer at 20% opacity with the final displacement (greyscale) map? 3) could you tell me a bit about using the Motion analysis functions in MVTools?, or if not asking too much a small code example? Thanks again, Jeff | ||
| Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 4 Registered: 1-2008 Rating: N/A |
Basically you will build 2 comps, a Comp1 in AE which will be your displacement map. Then you make another comp with rgb image and use Comp1 as displacement map. Comp1 is has at least 2 layers: motion mask based and chroma based maps. 1. Virtual Dub colourscale->greyscale gives you bw image, which will be the chroma map layer with opacity of 20% (topmost layer). With colourscale, use the one which sets luma to black and leaves rgb/hue unchanged. 2. No, you use the one which you build with VD. 3. MVTools motion analysis is really painfull to set up for this purpose. Basically you get bw image with lots of random pixels and you need to clean this with Virtual Dub motion blur and temporal smoother. I can try to find some Avisynth scripts I made for this. Best Regards, Jari | ||
| Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 5 Registered: 1-2008 Rating: N/A |
Here is the avisynth script, Change blksize, blksizeV, overlap and overlapv for tweaking, read from manual. Read the avisynth output to Virtual Dub and apply motion blur + temporal smoother filters. Now you should have your motion mask layer ready for Comp1. ******************************************* source = AviSource ("D:\yourclip.avi").convertToYV12 vectors = source.MVAnalyse(isb = true, delta = 4, blksize = 8, blksizeV = 8, truemotion = true, lambda = 0, lsad = 100, pnew = 0, plevel = 2, overlap = 4, overlapv = 4) source.MVMask(vectors, ml=265, kind=0, gamma=.15, ysc = 50).greyscale.Levels(100, 1, 255, 0, 235) *********************************************** **Edit: I suggest to scale the luminance values of chromamap by using the acadsee filter. The final Comp1 should also be blurred a bit before using it as displacement map. Br, Jari (Message edited by mcmount on April 27, 2009) | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 3 Registered: 4-2009 Rating: N/A |
Hello Jari,.. thanks again for your very helpfull instructions. Below is the code i've written sofar in an attempt to display the results of MAnalyse and MMask beside the original movie frames in Media Player Classic after which i'll continue with your valued AE dual comp instructions.(really looking forward to that) I had to add "MSuper(source)." to your MAnalyse line or i'd get the error msg:#Script error: Invalid arguments to function "MAnalyse"... LoadPlugin("MVTools.dll") # MVToolsV2.21 left = DirectshowSource("ep2_clone_war_p640.mpg") left = converttoRGB(left) source = left.convertToYV12 vectors = MVSuper(source).MVAnalyse(isb = true, delta = 4, blksize = 8, blksizeV = 8, truemotion = true, lambda = 0, lsad = 100, pnew = 0, plevel = 2, overlap = 4, overlapv = 4) source.MVMask(vectors, ml=265, kind=0, gamma=.15, ysc = 50).greyscale.Levels(100, 1, 255, 0, 235) right = BilinearResize(source,640,272).converttoRGB stackhorizontal(left,right) ...i tried to display MVMask adding source2 = MVMask like in this code segment: source2 = source.MVMask(vectors, ml=265, kind=0, gamma=.15, ysc = 50).greyscale.Levels(100, 1, 255, 0, 235) right = BilinearResize(source2,640,272).converttoRGB stackhorizontal(left,right) ...but this resulted in the error: #CAVIStreamSynth: System exception - Illegal Instruction at 0x101012e8 Could you please show me how to display the results of MVMask? I owe you, Jeff | ||
| Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 6 Registered: 1-2008 Rating: N/A |
Hi Jeff, MVTools I use is at the version 1.11.4.5, download it from http://avisynth.org.ru/mvtools/mvtools.html (bottom of the page). Avisynth is 2.5. You don't have to load plugins in your script, just put the plugin to avisynth plugin folder. Define the full path to your clip as well. If stacking, your video dimensions (height-stack horizontal) or (width - stackvertical) must match. I suggest to start script as simple as possible, don't do unnecessary colorspace conversions. For example: source = DirectshowSource("D:\full path\to your\file.mpg").convertToYV12 vectors = source.MVAnalyse(isb = true, delta = 4, blksize = 8, blksizeV = 8, truemotion = true, lambda = 0, lsad = 100, pnew = 0, plevel = 2, overlap = 4, overlapv = 4) left=source.MVMask(vectors, ml=265, kind=0, gamma=.15, ysc = 50).greyscale.Levels(100, 1, 255, 0, 235) stackhorizontal(left,source) This creates motion mask and the original side by side. Hope this helps, Jari | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 4 Registered: 4-2009 Rating: N/A |
Thanks Jari,.. really appreciate the syntax and procedural corrections. Although my system is still producing errors with my code below from your example: 1) without "LoadPlugin("MVTools.dll")" causes Script error: there is no function named "MVAnalyse" 2) using "left=source.MVMask(..." still causes error CAVIStreamSynth: System exception - Illegal Instruction at 0x1b81300. I don't understand why this happens since theres many examples of assigning MVMake or MMake to a variable. MY CODE: #mvtools v1.11.4.5: no MVSuper LoadPlugin("MVTools.dll") source = DirectshowSource("H:\Best Realtime 2d to 3d with DDD Tridef Media Player using instant Depthmap\channel combiner-luminance-after effects\ep2_clone_war_p640.mpg").convertToYV12 vectors = source.MVAnalyse(isb = true, delta = 4, blksize = 8, blksizeV = 8, truemotion = true, lambda = 0, lsad = 100, pnew = 0, plevel = 2, overlap = 4, overlapv = 4) left=source.MVMask(vectors, ml=265, kind=0, gamma=.15, ysc = 50).greyscale.Levels(100, 1, 255, 0, 235) stackhorizontal(left,source) Its almost working, looking forward, Jeff | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 5 Registered: 4-2009 Rating: N/A |
My mistake Jari,.. the MVTools Sources folder was not in the same directory as mvtools.dll. Once it was the same code stopped producing errors but now Media Player Classic plays a blank screen? Should i use ConvertToRGB on a WinXP system and how, or what else would you suggest? Thanks for your patience, Jeff CODE: #mvtools v1.11.4.5: no MVSuper LoadPlugin("MVTools.dll") source = DirectshowSource("H:\Best Realtime 2d to 3d with DDD Tridef Media Player using instant Depthmap\channel combiner-luminance-after effects\ep2_clone_war_p640.mpg").convertToYV12 vectors = source.MVAnalyse(isb = true, delta = 4, blksize = 8, blksizeV = 8, truemotion = true, lambda = 0, lsad = 100, pnew = 0, plevel = 2, overlap = 4, overlapv = 4) left=source.MVMask(vectors, ml=265, kind=0, gamma=.15, ysc = 50).greyscale.Levels(100, 1, 255, 0, 235) stackhorizontal(left,source) | ||
| Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 7 Registered: 1-2008 Rating: N/A |
Hello Jeff, I haven't tried to use wmp classic for reading avs files, I always use virtual dub for this. Colourspace can be YV12, actually it can be what ever in xp. The scripts I wrote will work as they are without anything else or any modification. You just need to verify that the player you are using is supporting avisynth (Virtual dub does). Best regards, Jari | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 6 Registered: 4-2009 Rating: N/A |
Hi Jari,... I took your advice, started VirtualDub 1.1.8, opened your last script (below) via File > Open video file... Jari.avs, played the file and recieved Avisynth read error: CAVIStreamSynth: System exception - Illegal Instruction at 0x10101300 (and 0x101012e8 using mvtools v2.3.1 & 2.4.2). When i disabled the MV lines the script played fine. Could it be a difference in VirtualDub, Avisynth or ffdshow versions, OS's, something else? I'm using WinXP SP2,VDub 1.1.8, Avisynth 2.5.8 RC5 [081227](AviSynth_081227.exe) and ffdshow revision 1909 Mar24/07. Which of these are you using?, or is there something else or missing that can cause these MVTools errors? Appreciate your patience so close to success. YOUR CODE: LoadPlugin("MVTools.dll") source = DirectshowSource("H:\Best Realtime 2d to 3d with DDD Tridef Media Player using instant Depthmap\channel combiner-luminance-after effects\ep2_clone_war_p640.mpg").convertToYV12 vectors = source.MVAnalyse(isb = true, delta = 4, blksize = 8, blksizeV = 8, truemotion = true, lambda = 0, lsad = 100, pnew = 0, plevel = 2, overlap = 4, overlapv = 4) left=source.MVMask(vectors, ml=265, kind=0, gamma=.15, ysc = 50).greyscale.Levels(100, 1, 255, 0, 235) stackhorizontal(left,source) | ||
| Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 8 Registered: 1-2008 Rating: N/A |
Hi, Your error message tells you where the problem is; ...0101300 (and 0x101012e8 using mvtools v2.3.1 & 2.4.2)... You still seem to have mvtools version 2.x.x. installed in your system. Please use the older version I suggested, uninstall the new branch. Best regards, Jari | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 7 Registered: 4-2009 Rating: N/A |
Hi Jari,.. i should have written "...0101300 USING MVTOOLS V1.11.4.5 (and 0x101012e8 using mvtools v2.3.1 & 2.4.2)..." Since then i've also tried MVTools_V1.9.6.7.rar, MVTools-v0.9.8.1.zip to MVTools-v0.9.8.5.zip and MVTools-v0.9.9.zip, MVTools-v0.9.9.1.zip. I tried these after a GOOGLE search with "evaluate: system exception - access violation" MVAnalyse" indicated the problem may be an incompatible MVTools processor version: i'm using a dual 533 cpu computer. Using the following code this time on WindowsNT SP6a i recieve the error shown at the end indicating line 2 (MVAnalyse) as the cause... source = DirectshowSource("ep2_clone_war_p640.mpg").convertToYV12 vectors = source.MVAnalyse(isb = true, delta = 4, blksize = 8, blksizeV = 8, truemotion = true, lambda = 0, lsad = 100, pnew = 0, plevel = 2, overlap = 4, overlapv = 4) left=source.MVMask(vectors, ml=265, kind=0, gamma=.15, ysc = 50).greyscale.Levels(100, 1, 255, 0, 235) stackhorizontal(left,source) #"evaluate: system exception - access violation" (C:\WINNT\Profiles\Administrator\Desktop\TriDef\channel combiner-luminance-after effects\Jari.avs, line 2) I'm not sure what to do next. I have a 1.3ghz system here to try and perhaps i should write MVTools. What type of system are you running?, and perhaps you have a suggestion to solve this last problem? Thanks again, Jeff | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 8 Registered: 4-2009 Rating: N/A |
Jari,.. you'd written "..uninstall the new branch", perhaps this is a basic problem and i have'nt installed MVTools, Avisynth and ffdshow properly or fully. Could you please tell me how you install and configure them? thanks again, Jeff | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 9 Registered: 4-2009 Rating: N/A |
Hello Jari,.. SUCCESS!!, finally. I found sites explaining the install/configure of MVTools and MT 0.7 in system32 and Avisynth plugins folders (avisynth-2-6-0i.exe) and that configured in FFDShow's Avisynth script and other pages (ffdshow revision 1503 Oct4/07). Then i pasted your script in ffdshow's Avisynth page after adding lines: setmemorymax(1024) source=ffdshow_source() And finally drag/dropped my project movie ep2_clone_war_p640.mpg into Media Player 6.4 on WinNT SP6a I hope this will help achieve the same success in Virtualdub since i'll need to pipe the motion analyses to VirtualDub for temporal smoothing and motion blur as you instructed. And i especially hope i did'nt exhaust your invaluable help and patience on mere software configuration. For instance MVAnalyse and MVMake accurately depict greyscale depth according to movement but 1) theres a great deal of noise and.. 2) it can confuse lighting and reflections for motion. Is temporal smoothing and motion blur the answer for the noise or are there others? Is there a way to desensitize MV to lighting or reflection changes? | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 10 Registered: 4-2009 Rating: N/A |
Jari Sorry, i forgot to ask... You'd written to "Read the avisynth output to Virtual Dub", how do i save the Avisynth MV output? you also wrote "scale the luminance values of chromamap by using the acadsee filter", is this filter in VirtualDub or AE? Its really exciting to see such impressive results, i can't thank you enough for sharing you experience. At every oppourtunity in my work i'll happily give you credit. Jeff | ||
| Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 9 Registered: 1-2008 Rating: N/A |
Hi Jeff, 1. Create text file, copy-paste the script sample in to it, save it as test.avs 2. Open this test.avs with Virtual Dub, that's it. You don't have to do any fancy ffdshow modding. AVS files behaves as AVI files for the applications. Best regards, Jari | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 11 Registered: 4-2009 Rating: N/A |
Hello Jari,.. FINALLY! have the MV script working in VirtualDub. You wrote "text file..paste the script..save it as test.avs..Open..with Virtual Dub, that's it. no "fancy ffdshow modding. AVS..behaves as AVI.." I really! wish it were that simple on my system but it took these steps to work on WinNT, which crashes VirtualDub on WinXP: 1) Install avisynth-2-6-0i.exe,(accept all application associations, esp. Media Player 6.4) 2) install VirtualDub-1.8.8.zip (folder named VirtualDub to C:\Program Files\) 3) install FFDShow (rev.1503 on NT, rev.2912 on XP,accept all application associations) 4) paste C:\Program Files\ffdshow\ffvdub.vdf to C:\Program Files\VirtualDub\plugins\ffvdub.vdf 5) VirtualDub, files > Open video file: ep2_clone_war_p640.mpg, 6) VirtualDub, Video > Filters\Add..ffvdub, OK (button), hit play button to reveal ffdshow Avisynth page. Then paste script requiring LoadPlugin function for MVTools.dll and MT.dll else "no such function" errors for MV calls) - ENABLE: AviSynth, Add ffdshow video source, YV12, YUY2, Buffer back/ahead = 2/0, 3:2 Pulldown = Ignore Pulldown, 7) OK (button). But the point is its working. PROBLEMS: 1) theres a great deal of noise and.. 2) it can confuse lighting and reflections for motion. QUESTIONS: Last questions before moving on to AE compositing... 1) I've tried temporal smoothing and motion blur but its hard to tell if their helping. Are there MV settings or different methods the clean or eliminate noise? 2) Is there a way to desensitize MV to lighting or reflection changes? Thanks again for all your help, Jeff | ||
| Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 10 Registered: 1-2008 Rating: N/A |
Hi Jeff, You seem do this very complex way. On Win2000 and XP it only requires Avisynth and Virtual Dub to do the job. Maybe in NT it's a different story. There is a lots of noise with resulting video, that's why you need to use the following filters in VD (in following order): ffvdub.vdf (in your case) motion blur Temporal smoother (max value) The difference is really big with filtering. Motion analysis is just motion analysis, it doesn't understand difference with reflections or movements. You can tweak the blocksizes of MV as I wrote April 27, 2009. Tweaking MV is REALLY painfull and long process, it took weeks to get it to work with these parameters. Hope this helps, Jari | ||
| Jeff C (Danceman) New member Username: Danceman Post Number: 12 Registered: 4-2009 Rating: N/A |
Hi Jari,.. 1) would MaskTools EdgeMask's help MVAnalyse and improve MVMask results? If you tried combining these how were the results? 2) Had some good results hole filling MVMask's with DeSpot so am going to try some AviSynth noise filters and temporal smoothers i've read used with MVAnalyse: RemoveGrain, TemporalDegrain TempGaussMC plus stmedianfilter, temporalcleaner, fluxsmooth and nomosmooth. Have you used these or any others that made a difference? | ||
| Jari Vuoristo (Mcmount) New member Username: Mcmount Post Number: 11 Registered: 1-2008 Rating: N/A |
Hi Jeff, 1. I haven't tried to combine these 2. No I haven't. Please post if you manage to improve the results with the methods you described. Br, Jari |
