Interactive Matting

Publish in

Documents

47 views

Please download to get full document.

View again

of 37
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Share
Description
Interactive Matting. Christoph Rhemann Supervised by: Margrit Gelautz and Carsten Rother. Matting and compositing. Matting and compositing. Outline. Talk Outline: Introduction & previous approaches Our matting model Evaluation strategy. Matting is ill posed. =. +. ●. ●.
Transcript
Interactive Matting Christoph Rhemann Supervised by: Margrit Gelautz and Carsten Rother
  • Matting and compositing
  • Matting and compositing
  • Outline
  • Talk Outline:
  • Introduction & previous approaches
  • Our matting model
  • Evaluation strategy
  • Matting is ill posed
  • = + ● ● Cr,g,b= αFr,g,b+ (1 - α)Br,g,b ● ● Inverse process of compositing: Determine: F, B, α Given: C
  • Matting is ill posed
  • = + ● ● Cr,g,b= αFr,g,b+ (1 - α)Br,g,b ● ● Cr = αFr + (1 - α)Br Cg = αFg+ (1 - α)Bg Cb = αFb+ (1 - α)Bb Underconstrained problem: 7 Unknowns in only 3 Equations
  • User interaction
  • Unknown Trimap Scribbles Foreground Background Unknown Background Foreground
  • Previous approaches
  • C= α F + (1 – α)B ● ● Recall compositing equation:
  • Previous approaches
  • C= α F + (1 – α)B ● ● Recall compositing equation: Closed Form Matting [Levin et al. 06] B R G
  • Previous approaches
  • C= α F + (1 – α)B ● ● Recall compositing equation: Closed Form Matting [Levin et al. 06] Assumption: F and Bcolors in a local window lie on color line B R G
  • Previous approaches
  • C= α F + (1 – α)B ● ● Recall compositing equation: Closed Form Matting [Levin et al. 06] Assumption: F and Bcolors in a local window lie on color line
  • Analytically eliminate F,B.
  • Alpha can be solved in closed form
  • B R G
  • Previous approaches
  • Result of Closed Form Matting [Levin et al. 06]:
  • Result imperfect: Hairs cut off
  • Problem: Cost function has large solution space
  • True Solution Input image + Trimap Result of [Levin et al 06]
  • Segmentation – based matting
  • What are the reasons for pixels to be transparent? Defocus Blur
  • Lens and defocus
  • Point Spread Function Lens’ aperture Camera sensor Lens Point spread function Focal plane Slides by Anat Levin
  • Lens and defocus
  • Point Spread Function Lens’ aperture Camera sensor Object Lens Point spread function Focal plane Slides by Anat Levin
  • Segmentation – based matting
  • What are the reasons for pixels to be transparent? Defocus Blur Motion Blur PSF forMotion Blur
  • Segmentation – based matting
  • What are the reasons for pixels to be transparent? Defocus Blur Motion Blur Discretization
  • Segmentation – based matting
  • What are the reasons for pixels to be transparent?  Observation: Apart from translucency mixed pixels are caused by camera’s Point Spread Function (PSF) Defocus Blur Motion Blur Discretization Translucency
  • Model for alpha
  • Basic idea: Model alpha as convolution of a binary segmentation with PSF Approach taken [Rhemann et al. 08]: Use this model as prior in framework of [Levin et al. 06] Input image + Trimap Binary segmentation PSF Observed alpha Mattingprocess Input image Iterate a few times Initial alpha using [Wang et al. ´07] (Resultisimperfect) Initialize PSF/ deblur alpha Deblured (sparse) alpha Binarized (sparse) alpha using gradient preserving MRF prior Mattingprocess Segmentation prior Final alpha Binarized (sparse) alpha using gradient preserving MRF prior Groundtruth
  • Comparison
  • Input image Result for [Levin et al. ’06] Input image + trimap
  • Comparison
  • Input image Result of [Wang et al. ’07] Input image + trimap
  • Comparison
  • Input image Result of [Rhemann et al. ’08] Input image + trimap
  • Comparison – Close up
  • Inputimage+ trimap [Levin et al. ’07] [Levin et al. ’06] [Wang et al. ’07] [Rhemann et al. ’08] Ground truth alpha Evaluation ofmattingalgorithms
  • How to compare performance of algorithms?
  • Showing some qualitative results
  • OR
  • Quantitative evaluation using reference solutions
  • Evaluation ofmattingalgorithms
  • Key Factors for a good quantitative evaluation
  • Ground truth dataset
  • Online evaluation
  • Perceptual error functions
  • Groundtruthdataset
  • 35 naturalimages
  • High resolution
  • High quality Triangulation Matting [Smith, Blinn 96] - Photograph object against 2 different backgrounds  True solutiontomattingproblem
  • Input image Ground truth Zoom in Online evaluation Data andevaluationscripts online Advantages:
  • Investigateresults
  • Upload novelresults
  • www.alphamatting.com Perceptuallymotivatederrorfunctions Motivation: Simple metrics not alwayscorrelatedwithvisualquality Input image Zoom in Result 1 SAD: 1215 Result 2 SAD: 806 Perceptuallymotivatederrorfunctions Develop error measures for two properties:
  • Connectivity of foreground object
  • Gradient of the alpha matte
  • Input image Zoom in Result 1 SAD: 312 Result 2 SAD: 83 Perceptuallymotivatederrorfunctions User Study:
  • Goal: Infer visual quality of image compositions
  • Task: Rank to according to how realistic they appear
  • Gradient artifacts Connectivity artifacts Perceptuallymotivatederrorfunctions Correlationoferrormeasurestoaverageuserranking Conclusions
  • Model for alpha  overcomes ambiguities
  • Model-based algorithm: Performs better than competitors
  • Perceptual motivated evaluation
  • Message to you: Evaluation of your algorithm is important
  • Use ground truth data to make quantitative comparisons
  • Use a large dataset
  • Use a training / test split
  • Previous approaches
  • C= α F + (1 – α)B ● ● Recall compositing equation: Data driven approaches (e.g. [Wang et al. 07])
  • Model color distribution of F and B (from the user defined trimap)
  • Observed color more likely under F or B model?
  • Use likelihood in framework of [Levin et al 06]
  • B Model of F Model of B R Observed color G
  • Previous approaches
  • Result of data driven approaches [Wang et al. 07]:
  • Hair is better captured
  • Many artifacts in the background
  • True Solution Input image + Trimap Result of [Levin et al 06] Result of [Wang et al 07]
    Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks