Blue screen matting

Abstract

A classical problem of imaging—the matting problem—is sepa- ration of a non-rectangular foreground image from a (usually) rectangular background image—for example, in a film frame, extraction of an actor from a background scene to allow substitu- tion of a different background. Of the several attacks on this diffi- cult and persistent problem, we discuss here only the special case of separating a desired foreground image from a background of a constant, or almost constant, backing color. This backing color has often been blue, so the problem, and its solution, have been called blue screen matting. However, other backing colors, such as yellow or (increasingly) green, have also been used, so we of- ten generalize to constant color matting. The mathematics of con- stant color matting is presented and proven to be unsolvable as generally practiced. This, of course, flies in the face of the fact that the technique is commonly used in film and video, so we demonstrate constraints on the general problem that lead to solu- tions, or at least significantly prune the search space of solutions. We shall also demonstrate that an algorithmic solution is possible by allowing the foreground object to be shot against two constant backing colors—in fact, against two completely arbitrary backings so long as they differ everywhere.