Accelerated rendering and compositing

http://trac.webkit.org/wiki/Accelerated%20rendering%20and%20compositing

Outline

• Tree mania:
  • RenderObject
  • RenderLayer
  • z-order tree
  • Graphicslayer tree
  • PlatformLayer tree
• Special considerations:
  • overlap
  • clipping
  • reflections
  • video/webgl/plugins/etc.
  • intersection
• Accelerated drawing and compositing

Notes

Compositing: identify elements in the web page that benefit from rendering to bitmap and using gpu to composite it with other elements in the page.

This is done through RenderLayer tree. RenderLayer started as convenience for rendering certain objects. Create a RenderLayer for elements with their own coordinate system: overflow:scroll, transforms, and now video.

Render tree with hierarchy of RenderObjects. Certain RenderObjects have a RenderLayer off of them. There is a RenderLayer tree that mirrors RenderObject tree.

Painting: walk through RenderLayer tree, starting with the root, then go through child RenderLayer. Use RenderLayer for clipping and overflow.

Hit testing: reverse walk through child RenderLayers. It is done in software in RenderLayer code, and so should be same for all platforms.

There is another hierarchy of RenderLayers which is important, and that is the z-order tree. This tree provides back-to-front ordering, and is used to support the z-index CSS property. Before painting, compute z-order tree by going through RenderLayer tree. Find stacking contexts. Painting and hit testing use this z-order tree, not the RenderLayer tree.

Compositing: We can identify RenderLayers that want to be composited, and create a backing store for them. We render to the backing store, then give it to the GPU. Probably rendered using OpenGL, on top of the regular content on page.

We have to ensure that any elements in front of composited layers are also composited.

A RenderLayerCompositor, per RenderView. manages hierarchy of layers and decides what needs to be a RenderLayer.

RenderLayer backing for each composited layer. RenderLayer backing creates at least 1 GraphicsLayer, which is the glue between rendering and platform-compositing code.

Each RenderLayer backing can create another GraphicsLayer; see clipping/masks/negative z indices. In these cases, you slice the GraphicsLayer into multiple planes and create an order for clipping.

GraphicsLayers are in their own tree. Used by platform code (like CoreAnimation) to create PlatformLayers. GraphicsLayers can have more than one platform layer; see masks and reflections. GraphicsLayers have a way of dumping its tree, can get it from gdb.

Overlap testing: RenderLayerCompositor contains a list of rects of things that have been composited. Makes other layers composited if they intersect that list of rects.

Clipping: imagine a div with overflow:hidden, child with transform or video. Implement clipping via platform layer. On Mac: CALayer add masks to apply. If a layer is composited through clipping, it doesn't have its own backing store, paints into something behind it.

Reflections: clone layers, track clones. It's tricky.

Some RenderLayers are special because it contains content that benefits from compositing. Example: RenderVideo gets RenderLayer, becomes composited. The Mac media player hands PlatformLayer to compositing code.

Layer intersections: with 3d transforms, we paint into a GL texture. If a child element intersects with its parent element, the spec now states that the child element's layer needs to break into subtextures along intersection.

Overlap map is updated when there is a layout or style change that changes the compositing layer tree.

Special casing CSS animations: when a transform or opacity animation is running, we don't updating clipping during animations. Update layout at start and end, not during. Don't update scrollbars during, for example. It's a cheat, but gives performance.

Overlap test is used if real (not identity) 3d transforms are not present in page.

Platforms have their own accelerated drawing implementations. On Mac, but compositing still has value when there is accelerated drawing, but it depends on content. There is feedback from platform code to tell RenderLayerCompositor when it's valuable to do compositing.

Plugins like Flash force compositing mode.

Text: on Mac there is a limitation on graphics system where rendering text on transparent background, doesn't do subpixel antialiasing. So the text looks "skinnier". There is no good solution to this problem right now. If something is composited, but with opaque bg-color, then it's fine.

Limitations on compositing: not smart when applying scale trasnforms. On Mac, scale:2 just blows up bitmap, and looks fuzzy. MSIE10 uses Direct2D so it doesn't have this issue.

If we redid WebKit, would it be based on compositing primitives instead of bitmap primitives? No, it probably has to live under the platform layer. MSIE10 has thrown everything out and rebuilt using Direct2D, with good performance gains.

Debugging

1. In Safari, "show compositing borders".
   • Every composited layer gets a border.
   • The number represents the number of times the layer has been repainted.
   • Yellow, now green, boxes have no css properties, so the layer is used just for geometry, and doesn't need a backing store.
   • Red represents layers that need a backing store.
   • Blue outline: composited because it has overflow:hidden/scroll on it.
2. Do you see popping? Something popping in front is probably composited.
3. In gdb, debug only "call showLayerTree" any RenderLayer tree. This shows it as a z-order tree. Also prints whether RenderLayer is composited, and why. Also shows which GraphicsLayers are associated with which RenderLayers.
4. In inspector, turn on/off css properties that affect compositing; see overflow, z-order, transforms.
5. Gmail bug anecdote: copy the DOM into a fresh document, to give you a snapshot. Although there was no CSS styling, the bug still repro'd.
6. Via inspector, start deleting nodes.

The future

WebCore lacks logic where knowing if elements up front obscure stuff behind it. Right now it just repaints everything behind it from back-to-front.

SVG compositing? Currently doesn't use RenderLayer at all. Probably need to split up RenderLayer with a separate compositing piece which SVG could use.