Dev with Jonathan

Debugging Core Animation on OS X

Mon, 09 Nov 2015 00:00:00 +0000

Have you ever looked longingly at the Core Animation template in Instruments and wished it worked for OS X apps? Sadly that hasn’t happened yet, but workarounds exist. In this post, we’ll be exploring ways to debug Core Animation on the Mac so you can make your app as optimized as possible.

Since we don’t have access to the Core Animation template, we’re going to have to resort to two different tools for debugging and profiling.

Profiling FPS

In order to see what FPS our application is achieving, we need to use the Quartz Debug application. Unfortunately, it’s not bundled by default with Xcode. It requires a separate download. At the time of writing, it’s available in Apple’s developer downloads as part of the Graphics Tools package.

The tool is relatively straightforward. You run Quartz Debug, then launch your application. Start an action that requires the application to perform a drawing update (such as scrolling), and watch the FPS & CPU meter on Quartz Debug. Sadly, this is about all of the utility this tool provides. Other debugging tools can be located in Window > Quartz Debug Settings, but unfortunately they are limited in practice. However, it’s still valuable to know the framerate of your application, as it’s handy to determine if your app needs more optimization.

Debugging Core Animation

This is one of the areas that’s been lacking severely on OS X. What most people don’t know is that there’s an entirely hidden way to perform Core Animation debugging in a very similar fashion to iOS.

Does this coloring look familiar? If you’ve utilized Instrument’s Core Animation template for iOS, you might recognize the coloring of opaque views. So how do we get this on the Mac?

Before 10.9, it was possible to utilize undocumented Core Animation environmental variables to enable various Core Animation debugging modes. Unfortunately this does not work in 10.9. Why? Layers in 10.9 are actually rendered out-of-process, meaning it effectively broke the debugging capability that previously existed. But thanks to another undocumented magic environmental variable (h/t Matt Sarnoff), we can achieve the old behavior. Here’s the flag:

CA_LAYER_SURFACE=1

There isn’t any information anywhere about this flag, so utilize it with care. From what I have best been able to determine, it forces the application to render layer trees in-process. This will not cause any harm to your system, but it is something that can potentially impact your performance measurements. As a result, it should not be enabled unless debugging is taking place.

If you’re unfamiliar with how to add environmental variables, you can do so in Xcode in Product > Scheme > Edit Scheme. Select the Run target, click on the Arguments tab, and enter the environmental variable in the list near the bottom. It’ll look like this:

Now that we’ve enabled in-process rendering, we can start using other environmental variables to force Core Animation to enable various debugging modes. Let’s take a look at some of the most useful ones. More can be found here.

Color Opaque

Variable: CA_COLOR_OPAQUE

One of the most significant hits to performance can occur as a result of blending. Transparent layers force the GPU to composite layers all the way up the tree until it reaches an opaque layer. This blending can potentially be expensive. Often it is unnecessary, and can be avoided with just a few changes.

Here’s an example. This is a layer that is non-opaque. Notice that it’s highlighted in red.

By setting it to opaque, it becomes green.

In most applications, it’s not possible to remove all blending. After all, sometimes it’s useful to have blending. However, by removing as much blending as possible, rendering time can be reduced.

Tips:

If the background is a solid color, simply set a background color on the layer that matches that of the layer behind it. This allows your layer to be opaque, but still have the same effect.

The amount of blending occurring is signified by the shade of overlaied red. A light red color indicates little blending. If removing blending isn’t possible, try to make the red color as light as possible.

Color Offscreen

Variable: CA_COLOR_OFFSCREEN

In certain situations, Core Animation is forced to render layers twice: once off-screen and one on-screen. This is simply known as off-screen rendering.

This type of rendering can be triggered by numerous different situations. Here are some of them:

CALayer with rasterization enabled (shouldRasterize = YES)
CALayer with a shadow
CALayer with a corner radius
CALayer with a mask

Here’s an example. The black square has a shadow applied to it.

CALayer *layer = [CALayer layer];
layer.shadowRadius = 5;
layer.shadowOpacity = 1;

With the environmental variable enabled, it looks like this:

In order to remove this offscreen rendering pass, we can set a value on the shadowPath property of our layer. Although the intricacies of shadowPath our outside of the scope of this article, it essentially bypasses a complete traversal of the layer’s sublayers when attempting to figure out the path that the shadow should be cast from. By skipping out on dynamic shadows and providing this ourself, we can usually increase performance quite dramatically. Let’s see what setting a shadow path does:

CGMutablePathRef path = CGPathCreateMutable();
CGPathAddRect(path, NULL, layer.bounds);

layer.shadowPath = path;

CGPathRelease(path);

Much better. On a side note, it seems like if the content view of the window is layer-backed, it is rendered off-screen. I’m not exactly sure why.

Tips:

A common case for setting a border radius on a layer is to get a circle avatar effect. This is rather expensive to do through layers due to off-screen rendering. A good alternative is to pre-render the images clipped to a circle in a drawing context. That circle image can then safely be used as the content for the avatar layer, without it needing any corner radius. Although this has an up-front performance cost, we can avoid the cost per-frame, which is a worthy tradeoff.
In certain situations, layers can be rasterized (shouldRasterize = YES) in order to minimize off-screen rendering. Sometimes this can be relatively beneficial. If the layer’s content does not ever change, it is worth the initial cost to rasterize the layer, as layer rendering will be easy. However, if the content of the layer changes frequently, it will be even more costly than if it was disabled.

Color Copied Images

Variable: CA_COLOR_COPY

If you have an application that is image-intensive, minimizing Core Animation image copies is extremely important. Before we discuss how to fix the issue, let’s talk about why it occurs.

Before images can be displayed by Core Animation, they must first be decompressed. Most images are stored on disk in a compressed format, so they must first be decompressed in order to be usable. This is done automatically by Core Animation when it is time to render. This decompression is not a fast process.

Additionally, even when an image is decompressed it might not be in the right format. Core Animation has a very specific byte-alignment for images, and if the image does not match this, it will be copied on the main thread at render time. As expected, this is very slow. In an application that requires fast image loading due to scrolling, this can be a huge bottleneck.

We can detect some of this through the use of the environmental variable shown above. If an image is copied, it might be displayed in blue. Something like this:

By using proper byte-alignment when decompressing, it should display without any tint:

Note that this instrument does not seem to always work as expected. Although it’s an easy way to check at a glance if there are any issues with image decompression or copying, it’s not completely accurate. The most accurate way to detect if copying is occurring is to use Instruments with a time profiler. You’ll want to be looking for CA::Render::copy_image in your stack trace.

So how can we get proper byte-alignment for our images so Core Animation doesn’t need to do a copy? We need to forcibly decompress the images using Core Animation’s pixel format by redrawing it in a new context. Here’s an example category on NSImage that can accomplish this. It’s recommended to generate this image on a background thread in order to avoid the very problem that we’re trying to prevent.

@implementation NSImage (JNWDecompression)

static CGContextRef JNWCreateOpaqueGraphicsContext(CGSize size) {
 size_t width = size.width;
 size_t height = size.height;
 size_t bitsPerComponent = 8;
 size_t bytesPerRow = 4 * width;

 CGColorSpaceRef colorSpace = NSScreen.mainScreen.colorSpace.CGColorSpace;
 if (colorSpace == nil) {
 colorSpace = (CGColorSpaceRef)CFAutorelease(CGColorSpaceCreateDeviceRGB());
 }

 CGBitmapInfo bitmapInfo = (CGBitmapInfo)kCGImageAlphaNoneSkipLast;
 CGContextRef ctx = CGBitmapContextCreate(NULL, width, height, bitsPerComponent, bytesPerRow, colorSpace, bitmapInfo);

 return ctx;
}

- (NSImage *)jnw_decompressedImage {
 CGImageRef CGImage = [self CGImageForProposedRect:nil context:nil hints:nil];

 CGRect rect = (CGRect){
 .size.width = CGImageGetWidth(CGImage),
 .size.height = CGImageGetHeight(CGImage)
 };

 if (rect.size.width == 0 || rect.size.height == 0) {
 return nil;
 }

 CGContextRef context = JNWCreateOpaqueGraphicsContext(rect.size);

 CGContextDrawImage(context, rect, CGImage);

 CGImageRef nativeCGImage = CGBitmapContextCreateImage(context);
 NSImage *image = [[NSImage alloc] initWithCGImage:nativeCGImage size:self.size];

 CGImageRelease(nativeCGImage);
 CGContextRelease(context);

 return image;
}

@end

CA Instrument

To help make instrumentation easier, I’ve put together a small app.

Here’s how to use it:

Select an app to instrument from the drop-down list in the titlebar. You can either select an app manually, or you an select an app that is already running.
Enable the various instruments that you’d like to utilize. It’s usually best to only use one at a time.
Click instrument. The app will be (re)launched with the correct environmental flags enabled.

The app can be downloaded here.

A short guide to OS X animations

Mon, 28 Apr 2014 00:00:00 +0000

Let’s talk about the current state of animations on OS X. We’ll go through the best approaches for getting the best performance out of standard animations, and the hacks needed to go beyond basic animations.

No layers

The most basic of animations on OS X is driven through the use of NSAnimationContext with no layer-backed views. Here’s how we’d animate our view to the right:

[NSAnimationContext runAnimationGroup:^(NSAnimationContext *context) {
 context.duration = 2.f;
 view.animator.frame = CGRectOffset(view.frame, 100, 0);
} completionHandler:nil];

This is not ideal for many reasons. Since the view is not layer-backed, AppKit will drive this animation on the main thread by repeatedly calling -setFrame:, or in this case -setFrameOrigin:. Animation performance for larger numbers of views using this approach degrades extremely quickly. Your animations will not be smooth. Don’t do this.

Layer-backing

So we don’t want our animations to be driven on the main thread. What’s the alternative? Well we can simply turn on Core Animation by telling our view we want it to have a layer.

However, this might not be enough. It’s extremely important to note that NSView has a property called layerContentsRedrawPolicy that dramatically effects the quality of your animations. This property specifies the behavior for view redrawing when animating, and was introduced in 10.6. In OS X 10.7 there were changes under the hood, and the value of the property actually affected how views are animated. The documentation for this property is fantastic, and it would be a good idea to give it a quick read before we continue.

The most important fact to understand about this property is as follows: if your layer contents redraw policy is set to NSViewLayerContentsRedrawDuringViewResize, which is the default, your animation performance will likely suffer. In fact, it might even approach the same level of performance you get from not having a layer, as AppKit is still going to call -setFrame: on your view to drive the animation. Depending on how you want your view to redraw, a good redraw policy to use is likely NSViewLayerContentsRedrawOnSetNeedsDisplay. If you set it to the aforementioned redraw policy, the view will be animated using Core Animation, which means the animation will be driven on a dedicated background thread. This will produce smooth animations in most cases.

Note that some NSView subclasses (such as NSTextField) will set the contents policy to the correct redraw policy by default. Check the documentation for more details on whether this is the case, or simply do some introspection on a view in question to see what it’s redraw policy is.

// Tell the view to create a backing layer.
view.wantsLayer = YES;

// Set the layer redraw policy. This would be better done in the initialization method of a NSView subclass instead of here.
view.layerContentsRedrawPolicy = NSViewLayerContentsRedrawOnSetNeedsDisplay;

[NSAnimationContext runAnimationGroup:^(NSAnimationContext *context) {
 context.duration = 2.f;
 view.animator.frame = CGRectOffset(view.frame, 100, 0);
} completionHandler:nil];

Alternatively, since we now have a layer we can drop down a bit further past implicit animations to interact with Core Animation directly.

// ... same setup as above

CGRect frame = CGRectOffset(view.frame, 100, 0);

CABasicAnimation *animation = [CABasicAnimation animationWithKeyPath:@"position"];
animation.fromValue = [NSValue valueWithPoint:view.frame.origin];
animation.toValue = [NSValue valueWithPoint:frame.origin];
animation.duration = 2.f;

[view.layer addAnimation:animation forKey:animation.keyPath];
view.layer.position = frame.origin;

However, using Core Animation directly to animate common properties has some significant issues.

The problem with layer-backed views

If you want to perform animations on the layer beyond setting common NSView cover methods (such as frame, or frame origin) the situation starts to rapidly escalate into a catastrophe. Apple has specified the following properties as unsafe to modify:

geometryFlipped, bounds, frame (implied), position, anchorPoint, transform, shadow*, hidden, filters, and compositingFilter

Yes, you read that correctly. The frame, bounds, position, anchor point, and transform properties of a NSView’s backing layer are controlled by the view itself. In other words, the view contains the state of the layer, meaning these properties must be set through the use of the appropriate cover methods on NSView in order to keep the state in sync. This stands in stark contrast to UIView, which is simply wraps up much of the state in the layer itself, meaning either the UIView cover methods or the layer properties themselves can safely be modified.

Some of the cover methods do exist properly in NSView, such as the standard frame setter. However, NSView simply doesn’t offer a way to modify the transform! This means that even the smallest attempt at a somewhat complex animation that involves transformations is technically unsupported. It turns out this is one of the largest issues with animations (or even layers) on OS X at the moment and is not easy to solve properly without the use of hacks.

Why modifying the transform is difficult

Before we continue, let me state that the transform property can technically be modified, and the view will properly display with the modified transform. However, there are numerous issues with applying transforms onto views.

The issue is that the view doesn’t consider the transform at all when it needs to perform any calculations that involve the view state (such as event handling). An example of when this would be an issue is with hit testing. A translation transform that moves a view offscreen will do nothing to the event handling of the view, which means it will still receive clicks and all other events it would normally receive if it was at its original transform.

Not only does NSView ignore the transform entirely for state calculations, it also decides to reset the transform every time the view state changes. As of 10.9, there is an internal method, _updateLayerGeometryFromView, which is called every time the view’s state has a change which would affect layer geometry in any way. This method syncs up the current state of the view to the layer, meaning it resets properties that are critical to moderately complex layer modifications, such as transform or anchorPoint.

This resetting of state is understandable, since it is the only way for the view to manage its layer (since the state is kept in the view), but it is unacceptable since it destroys any custom modifications applied to the layer.

So what can we do?

How we can hack custom layer transforms

The good news is that we can temporarily work around NSView’s state applications while we have custom transforms applied to a layer. The bad news is that we cannot fix the event handling issues, amongst other issues with the view state. Fixing those issues would require some fairly extreme workarounds.

Here’s the cleanest approach I could think of to fix this problem. When a NSView is told that it needs a layer, it will call -makeBackingLayer on the view. Since NSView backing layers use a private class (_NSBackingLayer), we can’t just create a subclass of it, so we need to swizzle a couple of methods on the instance returned by super’s implementation of this method. The benefit of approaching it this way is that it is relatively future-proof and shouldn’t break even if the class changes in the future. We swizzle -setAnchorPoint: and -setAffineTransform:, since these are the two most significant properties that are reset every time the view changes state. We then modify the implementations in our swizzled methods to check a flag before passing through, meaning we can toggle a bypass on and off for both of the setters.

This approach is still dirty, and I don’t want to minimize that. But it does allow for the temporary setting of an anchor point and a transform for an animation without the fear of NSView rushing in to ruin the party. Again, do keep in mind that hit testing (amongst other things) will not know anything about the transform you apply.

Why is transform so important?

You might be thinking that this ranting about the need for transform animations is overblown and unnecessary. On the contrary, I believe that animating the transform is the one true way to create outstanding animations. It is an incredibly powerful and performant way to modify many aspects of the visual state of the layer. Especially for animations that bypass Core Animation completely and utilize a display link, calling -setFrame: on the view will not give decent animation performance due to a significant number of side effects in the setter. Instead, the custom animation can repeatedly call -setTransform: on the layer and still achieve optimal performance.

Shiny things like this can be done in just a few lines of code:

[IBNSpringInterpolator runInterpolationWithState:state handler:^(CGFloat distance, CGFloat progress) {
 CGFloat rotation = M_PI * progress;
 view.layer.transform = CATransform3DMakeRotation(rotation, 0.f, 0.f, 1.f);;
 view.layer.position = (CGPoint){ .x = distance, .y = view.frame.origin.y };
} completion:nil];

And yes, I do plan on opening sourcing that spring interpolator eventually.

Summing it all up

Here’s what you need to know to get great performance:

Use layer-backed views. There is no reason not to.
Use a layer contents redraw policy of NSViewLayerContentsRedrawOnSetNeedsDisplay to allow your animations to use Core Animation.
Don’t modify the layer transform unless you know the side effects.
Custom animations should use the transform if possible for optimal performance.

Great animations are much more difficult to create on OS X than they are on iOS. They require a significant amount of trial and error, but the results are rewarding.

A Core Animation Manifesto

Tue, 07 May 2013 00:00:00 +0000

Core Animation is an advanced compositing and animation framework for iOS and OS X. Not only does it open up the ability to perform incredible animations with just a few lines of code, but it also includes powerful objects called layers, which are extremely lightweight objects that contain some type of visual content which can be easily manipulated by transforms. This article will explore various aspects of Core Animation, beginning with layers.

Core Animation was initially named Layer Kit. What are layers? Layers are a lightweight data structures that are similar to views. They can be arranged in a hierarchy to create a user interface. The parent layer (superlayer) can contain a set of layers (sublayers), forming a layer tree. The sublayers have a coordinate system that is relative to their superlayer.

The real power of layers is their ability to be transformed by transform matrices. Although layers are primarily designed for 2D interfaces, they do exist in a 3D coordinate system and can therefore be transformed in three dimensions. Each instance of a layer has its own coordinate system. Any sublayers added to a layer will be positioned relative to that layer. Layer coordinate systems actually differ between platforms. On iOS, the origin is the top-left corner, whereas on OS X the origin is the bottom-left corner. The remainder of this article will use the OS X coordinate system.

Layers contain multiple properties which affect their geometry. The first is the position. The position is just a struct of type CGPoint, which contains an x and y coordinate, which are relative to the origin of the superlayer. The bounds of a layer is a combination of a struct containing the size of a layer (CGSize), and the origin of the graphics context, which is usually {0, 0}. The anchorPoint is a CGPoint that has valid values in the range of 0.0 – 1.0. The anchor point directly affects how the position of the layer relates to the bounds. For example, a layer with an anchor point of {0.5, 0.5} will have its position set relative to the exact center of the layer. The anchor point also affects transforms, which will be discussed later. Layers also have a frame property, which is derived from the anchor point, position, and bounds.

Layers can be transformed using transform matrices. These transforms can be represented as CATransform3D structs. CATransform3D structs merely contain a 4x4 matrix of floats that represent a transform matrix. By default, a layer has its transform property set to CATransform3DIdentity, which unsurprisingly is the identity matrix. Applying this identity matrix to the layer will revert any transforms applied, and will reset the layer to its default position, scale, rotation, and perspective. There are many ways to transform a layer. Although a transform matrix can be created manually, it is easier to use the built-in functions to easily create a transform with a specific purpose. CATransform3DMakeTranslation creates a transform matrix which is translated by x, y, and z. CATransform3DMakeScale returns an identity matrix scaled by x, y, and z. CATransform3DMakeRotation is a transform matrix which rotates by x, y, and z in radians. A complex transform matrix can be formed by combining these transforms into one matrix. This can be accomplished by using CATransform3DConcat, which takes in two transform matrices and concatenates the second onto the first (a * b). Alternatively, variants of the transformation functions can be used to directly apply one transform onto an existing transform. These are CATransform3DTranslate, CATransform3DScale, and CATransform3DRotate, which take in the same arguments as their counterparts, with the addition of a pre-existing transform. These transforms can be used to create some stunning effects, and are most useful when combined with animation, which will be discussed in a later section.

A layer can be provided with content in multiple ways. The first and easiest method is to simply set the contents property of the layer to an image. The image can either be of type NSImage on the Mac, or CGImageRef on iOS and Mac. The current backing store of the layer is replaced with the content of the image. This behavior can be useful when focusing on optimization, and will be explored in further detail later. Another way to provide content to the layer is through drawing. Layers provide access to the drawing context through the delegate method -drawLayer:inContext:, and the overridden -drawInContext: method for subclasses. In either case, the context is provided as a CGContextRef, which can be used to draw graphics as desired. The drawing is then performed on the CPU, then the context is converted into a texture optimized for the GPU. Behind the scenes, Core Animation ties in directly with Open GL ES. Layers are really just wrappers around GPU textures, because the content is rasterized into a texture before being displayed on the screen.

Core Animation has multiple types of layers built-in. The main class, CALayer, is the parent class for all other types of layers. It provides all the basic features that layers need. Core Animation also includes specialized subclasses of CALayer that add on additional features, or modify behavior in some cases. CATiledLayer is designed to display content with massive dimensions that would be impractical and impossible to load normally. As its name implies, CATiledLayer splits up its content into smaller tiles. As transforms are applied to zoom the layer, tiles are dropped and redrawn at the requested detail asynchronously. CAScrollLayer is optimized for scrolling and displaying a portion of a layer. CATextLayer makes it easy to create a layer with a content of text. There are also additional classes that are only available for specific platforms. On OS X there are CAOpenGLLayer, QCCompositionLayer, QTMovieLayer, and QTCaptureLayer, whereas on iOS there is CAEAGLLayer.

Although layers by themselves are incredibly useful for creating performant interfaces, they are complemented with a powerful ability to tie in with the animation aspect of Core Animation. Apple’s implementation of animations is fantastic. On a superficial level, animations can be performed by creating an instance of an animation and adding it to a layer. Once an animation is added, Core Animation handles the rest. A dedicated secondary thread is used by Core Animation to ensure that regardless of whether the GPU can keep up with a full 60 frames per second, the animation will finish precisely on time. If needed, Core Animation will drop frames automatically to guarantee this behavior.

A brief aside must be taken to discuss the basics of a technology in the Objective-C language known as Key-Value Coding, or KVC. KVC is a method of accessing a property on an object through the use of strings, instead of directly accessing the object’s instance variables or accessors. At first this might seem useless, but in reality it can be incredibly useful, especially when combined with Core Animation. Accessing a KVC-compliant property using KVC is as simple as calling -valueForKey:, passing in the name of the property as a NSString. Use of this technology can dramatically simplify code, such as in the case where identifiers are mapped to property names. Another way of accessing values is through the use of key paths. A key path is a string that in essence “drills down” through KVC-compliant properties until the desired value has been found. Specific Core Animation classes, namely CAAnimation and CALayer extend the NSKeyValueCoding protocol, and add support for key paths, even for structs such as CGPoint and CATransform3D. This ability allows for Core Animation to provide an incredibly simple API for animating specific values.

To get started with a basic animation, Core Animation includes a standard object, CABasicAnimation, that provides all the necessary tools to create an animation which interpolates from one value to another. It is a subclass of CAPropertyAnimation, which in turn is a subclass of CAAnimation. The first step is to create an instance of CABasicAnimation, which can be done by using the designated property initializer, +animationWithKeyPath:. Alternatively, the standard CAAnimation initializer can be used by calling +animation and setting the keyPath property afterwards. The key path is the property or value which needs to be animated. Not all of CALayer’s properties are animatable, but those that aren’t are explicitly listed in Apple’s documentation. Next, the animation needs to know what you want to interpolate from, and what you want to interpolate to. This can be done by setting the fromValue and toValue, which are both of type id. As a consequence of the type being id, scalar types must be wrapped into objects. Floats and other scalar numbers can be wrapped into NSNumbers. Other structs such as CGRect can be wrapped into NSValues. For example, suppose you would like to animate the position. The position in this case would be a CGPoint, which is a struct. We need to wrap this in a NSValue, which can be performed by calling NSValue’s +valueWithCGPoint: method. The final property worth mentioning in CABasicAnimation is byValue, which is optional. If set, Core Animation will attempt to use the value as an increment between interpolation steps. The other inherited properties (such as duration) are optional as well, and will be set to their default values if omitted.

Somewhat more complex animations can be formed by using CAKeyframeAnimation. Keyframe animations in Core Animation seem simple at a first glance, but they can be used to create some surprisingly complex animations. Instead of taking control over the entire interpolation as CABasicAnimation does, CAKeyframeAnimation allows manipulation of intermediate points along the interpolation, which allows for far more control over the animation. There is only one property which is required to be set before the animation is able to be used, namely the values array, which is of type NSArray. The array of values needs to contain at minimum the ending value, but it may contain as many values as required to create the specific animation. There is also an optional property, keyTimes, which is an array of numbers ranging from 0 to 1 which correspond directly to their related values in the values array. If the count of both arrays is not identical, or if invalid numbers are specified in keyTimes, it will be ignored. Since the times range from 0 to 1, the numbers could be thought of as percentages of the total duration of the animation. If the animation needs to move along a specific path, CAKeyframeAnimation has an optional property that accepts a CGPathRef. This path will take precedence over the values array (if provided), and will be used along with the key times (if set) to create a motion along the path. There are several other properties on CAKeyframeAnimation that can be used to further modify the behavior of the animation, but they are outside the scope of this article. However, there is one property, timingFunctions, which serves a unique purpose of allowing for different timing functions for each of the values in the animation.

The ability to set timing functions is also present in CABasicAnimation. By default, animations will use a mostly linear curve, meaning the object will move from one value to another without much variance in the speed. For some purposes this can be useful, but when creating user interface animations this can make the animations appear “dead”. This can be easily remedied by using the timingFunction property declared in CAAnimation, from which CABasicAnimation inherits. In the case of CAKeyframeAnimation, the timing function can be set through the the related array of timingFunctions. Both cases require an object of type CAMediaTimingFunction. A timing function object is usually created by using +functionWithName:, passing in one of the timing function constants. These constants are kCAMediaTimingFunctionLinear, kCAMediaTimingFunctionEaseIn, kCAMediaTimingFunctionEaseOut, kCAMediaTimingFunctionEaseInEaseOut, and kCAMediaTimingFunctionDefault. For most purposes kCAMediaTimingFunctionEaseInEaseOut provides the most natural feeling for animations. Custom timing functions can also be created by using the -initWithControlPoints:::: method on CAMediaTimingFunction. Although this offers a good degree of flexibility, the shape of the resulting bezier path is still cubic, and multiple points are unable to be specified. However, CAKeyframeAnimation was designed with this in mind, and can be an excellent substitute if the timing functions do not fit the needed style of animation.

Although layers can be extremely performant, there are some bottlenecks that can cause unwanted lag. The first area of optimization is blending. When a layer does not have any transparent areas it is opaque. Opaque layers are not blended with any layers beneath them. When a layer is non-opaque and is placed above another layer, the GPU is forced to blend the two layers together, which is quite an expensive operation. Using the Core Animation instrument (currently only available for iOS), detecting blended layers is as easy as checking the box that is labeled “Color Blended Layers”. The areas that are in red are blended, and should be reduced as much as possible to avoid multiple layout passes.

The second area of optimization is pixel alignment. Core Animation, UIKit, and AppKit all use floating-point numbers to store pixel coordinates. When the content is actually drawn on the screen, the pixels don’t align with the points. This causes anti-aliasing, which can put some degree of strain on the GPU, not to mention the unsightly fuzzy appearance. This issue can be resolved easily by ensuring that coordinate calculations are always truncated or rounded to the nearest integer. Since non-integral coordinates usually occur when layout calculations take place, the recommended approach is to use the floor() or ceil() functions to truncate or round the numbers.

Finally, a large source of performance issues can arise from masking to bounds. If a layer has a corner radius set, for example, masking to bounds must be turned on for the layer to render with the rounded corners. Unfortunately, turning on masking actually causes the layer to perform multiple offscreen render passes, which can absolutely destroy performance. There are multiple ways to work around this issue, but the simplest and arguably the best way to avoid this problem is to avoid masking at all. The corner radius effect can be emulated by clipping the context to a path before drawing. Another way to work around this issue is to rasterize the layer.

Layer rasterizing can sometimes bring fantastic performance boosts to complex layers. Layer rasterization can be turned on by setting the shouldRasterize property on the CALayer in question to YES. Rasterization works by rendering the entire layer as a bitmap, including effects such as masked bounds and shadows. This can potentially cause a tremendous increase in speed when animating. The GPU excels at moving around textures, and rasterized layers are no exception. However, rasterizing layers does not always work in your favor. If the content of the layer changes at all, the entire bitmap cache of the layer is invalidated, rendered, and cached again. Naturally this is quite inefficient, and as a result layers that will have any changes applied to them during animation are advised not to rasterize. Additionally, just because a layer is rasterized does not mean that non-opaque layers with blending issues will have improved performance. The blending will still need to take place. Finally, rasterization requires more memory usage due to the additional bitmap cache of the layer. These allocations are significantly more expensive due to the fact that they are effectively mutable surface textures. They will be paged to another backing store rather than destroyed for resource-constrained scenarios. On OS X this is not much of a worry due to the large amount of memory available to applications, but on iOS devices memory usage can be a potential concern.

Decreasing memory usage is a vital aspect of development. Although layers are lightweight, they still must keep their textures in memory. If the same texture is used repeatedly, the same bitmap can be shared across multiple layers. Suppose a table view had cells that contained the same background for each of them. One way to optimize this would be to draw the background once, and render that background into an image that all the cells have access to. Then, each cell can either set their layer’s contents to this image, or create sublayers using the same image for their content. The end result is the reuse of the single bitmap, which helps to bring some welcome memory improvements.

Animations are fantastic. However, they serve a purpose. An application that includes animations just for the sake of having animations should not have them at all. Animations serve multiple purposes, but their most important purpose is to provide context. Suppose you’re using an email client. You are scrolling through your emails, and you decide you no longer need one of them. You tap or click delete. Without animation serving as contextual guidance, the cell that contained the information about the email you currently selected suddenly disappears. This happens so quickly that your eye notices a flash, and suddenly the list is updated and the old email is nowhere to be found. At this point, you might second-guess yourself, and start to wonder whether you really deleted the right message. This type of behavior does not happen in real life, and therefore it’s hard to relate to it. Now suppose this whole process is animated. You delete the message, and it gently drifts away and fades out, while the messages beneath it slide up to fill the gap left by the deleted message. This action is much more relatable to in real life, and an animation in this case gives the user reassurance that the correct message was indeed deleted.

As it unsurprisingly turns out, interfaces that implement these type of natural animations are far more intuitive than those that do not. On the other hand, interfaces that implement animations that do not exist in real life are quite counterintuitive. Another example of an intuitive animation is on iOS. When navigating to a new level in a view hierarchy, the old view is pushed out to the left while the new view is simultaneously pushed in from the right. A similar situation could be created in the real world if two papers were taped together and put underneath a frame that only showed one single paper. If the paper under the frame were pushed to the left, the paper attached to the right would slide into view. These type of animations and movements are instantly relatable; no explanation is needed. This is how animation should be.

Core Animation is truly a masterpiece of engineering. A short amount of code can produce stunning animations that enhance your interface. Nearly all the complicated work previously needed to create complex transformations are completely unnecessary with Core Animation. With some amount of care, proper use of Core Animation can transform your application into something extraordinary.

Optimized Scrolling in NSTableView

Fri, 11 Jan 2013 00:00:00 +0000

First introduced in OS X Lion, view-based table views are a significant advancement over previous cell-based table views. Not only are views easier to work with and customize, but they’re also trivial to animate. The API is fantastic. So what’s the catch?

Scrolling. A standard, unoptimized view-based table view with a somewhat complex layout will have an extremely tough time achieving buttery smooth scrolling at 60 fps. The good news is that we can fix this. Ready? Let’s get started.

It’s all about the layers

Views without layers require a redraw after every frame change. Layer-backed views, on the other hand, are able to cache the result of drawing into an bitmap cached in the GPU, which can then animate and transform that bitmap with great ease. NSView does not enable layer-backing by default.

One class that is critical in the process of scrolling is NSClipView. A clip view is used to contain the document view of a NSScrollView, clip the document view to its frame, and update the NSScrollView when the document view’s size or position changes1.

Update 09/22/15: The information below was written for OS X 10.7. Scroll views as of 10.8+ seem to perform well without needing to customize the layer class of the clip view. I would recommend against swapping the clip view. Rather, the containing scroll view should have wantsLayer set to YES.

If your window’s content view, or any superview of the clip view contains a layer, it will implicitly get a layer generated for itself as well. By default it will use a _NSViewBackingLayer for the backing layer. Unfortunately, this type of layer isn’t quite ideal for a clip view. A much more ideal subclass of CALayer is CAScrollLayer. A CAScrollLayer, as its name implies, is a subclass of CALayer that is optimized for scrolling, and displaying a portion of a layer. Wouldn’t it be great if we could use this for our NSClipView? Well, we can.

Telling a subclass of NSClipView to use a CAScrollLayer is an easy matter of overriding the designated initializer, setting the layer to a CAScrollLayer, and telling the clip view to use a layer. Since we create and set the layer before telling the clip view it needs a layer, this is a layer-hosting view.

- (id)initWithFrame:(NSRect)frame {
 self = [super initWithFrame:frame];
 if (!self) return nil;

 self.layer = [CAScrollLayer layer];
 self.wantsLayer = YES;
 self.layerContentsRedrawPolicy = NSViewLayerContentsRedrawNever;

 return self;
}

We’re not quite there yet. In order to use this fancy clip view, we’re going to need to replace the existing clip view with our custom clip view at runtime. There are multiple ways to do this. I chose to subclass NSScrollView and swap the clip view out during initialization.

- (id)initWithFrame:(NSRect)frameRect {
 self = [super initWithFrame:frameRect];
 if (self == nil) return nil;

 [self swapClipView];

 return self;
}

- (void)awakeFromNib {
 [super awakeFromNib];

 if (![self.contentView isKindOfClass:CustomClipView.class] ) {
 [self swapClipView];
 }
}

- (void)swapClipView {
 self.wantsLayer = YES;
 id documentView = self.documentView;
 CustomClipView *clipView = [[CustomClipView alloc] initWithFrame:self.contentView.frame];
 self.contentView = clipView;
 self.documentView = documentView;
}

The layer-backed clip view already exists in Github’s Rebel, and I’ve added the custom scroll view to Rebel as well. I highly recommend using it.

Ideally, scrolling should be quite a bit better now. But we’re not done yet.

Auto Layout

The idea behind Auto Layout is fantastic. However, with view-based table cells that require somewhat complex layout, auto layout has the potential to cause quite a bit of lag. The only way to remove all auto layout from the table view is to disable auto layout for your entire window. Yes, it’s not ideal. But once auto layout is triggered for any view in your window, the layout system is enabled for every other view in the window, including your table view.

Now let me reiterate: this is only necessary for cells that require a significant amount of layout customization. If you just have an image and a couple of labels, this might not be necessary. Profiling is the only way to determine for sure if auto layout is your problem.

Drawing

If you want a custom cell appearance, the recommended way to approach this is to subclass NSTableRowView, override one of the drawing methods (such as -drawBackgroundInRect:), and draw the custom appearance.

When a table cell view (or a table row view) is dequeued, it is removed from its current superview, and re-added once it has been appropriately reset and enqueued. This means that while scrolling your table view, -drawBackgroundInRect: is being called repeatedly every time the row view is dequeued.

A better approach would be to cache your background as a stretchable image. This image can then be simply set to the layer’s contents on your row view or cell view. By reusing the same image, you’re using the same bitmap in memory for all the rows, and you’re avoiding expensive drawing of your background.

If you’re using 10.8, the most optimized way to do this is to override -wantsLayer and return YES, which allows AppKit in turn to call -updateLayer. This is the recommended time to set the layer’s contents.

What else?

Listed above are just a few techniques I’ve discovered that help make scrolling more smooth. In a future post I’ll detail how scrolling using the arrow keys can be recreated to provide a smoother animation.

About

Mon, 01 Jan 0001 00:00:00 +0000

I’m a developer and designer. I’m passionate about crafting beautiful apps, building performant animations, and creating delightful UI.

Catch me on Twitter, Mastodon, or check out my GitHub.