Functional Programming in Swift: Chapter 3 – Wrapping Core Image

**Note- I originally took this post down because I wasn’t really happy with how I organized it. I also believe that this series has a larger readership than I originally envisioned and I felt that the assumptions I made about this post were confusing. I am going to repost this post and figure out how to proceed in a way that is useful to everyone who is reading my blog. **

I am writing a series of blog posts analyzing the code and concepts from each chapter in Functional Programming in Swift. I am assuming if you are reading this that you are following along in the book, so I am not references their specific code snippets in my blog.

The purpose of this chapter is to show the reader how to do a very important thing in programming, which is wrap a framework or an API written in another language, such as C.

Being able to integrate lower level code into your applications is a really important skill to be able to master. Over the last month I have had a project where I had to write a few wrapper classes around XML-parsing functionality in the C library libxml2, which I will write about in a later blog post. Learning how to search the sparse documentation for this library and figuring out how to integrate it with Swift was a really interesting and painful learning experience. Even though a lot of times I wanted to hang my head and weep in despair, after I got it working I felt kind of like one of those douchebag programming ninjas that recruiters keep contacting me looking for.

This might not be the best sales pitch for why you should care about this stuff, so without further adieu, I will get to the content.

Core Image

Even though the focus of this chapter wasn’t specifically on learning Core Image, I wanted to take a moment to talk about it because it’s a neat little framework.

Core Image is a framework that allows you to add filters to photos in both iOS and Mac applications. Core Image is very similar to GPUImage, except GPUImage is open source, so you can go in and actually see how all of the shaders were written. Core Image has a few more filters than GPUImage has. In iOS 8, Apple opened up the ability for you to write your own Core Image filters, which wasn’t possible before.

Core Image is, as my coauthor Chris Adamson calls it, “stringly typed.” If you want to use a filter from Core Image, you have to reference it in a string. If you spell the filter name wrong then your project will fail silently. Whereas you can use auto-complete on other parts of your project to ensure that you don’t get felled by spelling errors, there is alas no auto-complete in Core Image.

If you’re interested in how to write shaders for either Core Image or GPUImage, I recently wrote an article about it for Objc-io. There are a lot of other neat photo articles in this issue too, so be sure to go check it out if you want to know more about how to use photos in your iOS/Mac applications.

But enough of my own buzz marketing, on to the actual content of the chapter…

Wrapper Type

Generally speaking, Core Image has a rather repetitive pattern. You take an image, filter it, then you take that result and feed it to the next filter you are using.

If you’ve ever used Core Animation or another framework that affects the output of what goes on your screen, you will know that the order you implement your changes in affects your output. The same thing goes for your image filters. If you add more than one filter to an image, changing the order will change what the output image looks like.

Since you are using a filter chain where you are taking the output of one operation as the input of another, it makes sense to encapsulate this functionality into its own type. In this case, the type is a function that takes a CIImage and returns a CIImage.

CIImage filters all take different types of parameters. Some filters only take an image as a parameter while other ones many more. We want to be able to customize our filter functions to take the correct number of parameters. Regardless of what kind of filter we are making or how many parameters they take, we want each of our new custom filter functions to return a function that conforms to the Filter type alias. Naming this function type makes our code cleaner and safer by assigning a name to it and removing code we would have to write over and over again.

Convenience Initializers and Computed Properties

We want to make it easier to extract the output image so that we can chain this filter’s output image to the input of the next instance of our CIFilter class. We are using a convenience initializer and a computed property to extend the class to customize it for our needs.

The class extension has three pieces we are going to look at: a convenience initializer, a computed property, and a typealias representing a dictionary that holds our Core Image parameters.

Any time we create an instance of a class or a struct, we have to handle initializing it. Back in Objective-C we had our pattern of [[NSString alloc] init] to initialize our instances. We still have to initialize our instances, but like many thing in Swift, this process has been simplified. Instead of the long, verbose way of initialization, we just use “()”, which is a void function call. This initialization function is void by default, but life wouldn’t be very interesting if we couldn’t customize things to suit our needs.

Just because the default initializer is a void function does not mean that it always must be. If we have want to pass in parameters to the initializer, we can do so. In the example used in this chapter, we are trying to customize the CIFilter class. We are doing that by extending the class to include the functionality we need.

The normal CIFilter initializer just takes the name of the filter being used. We want to initialize the filter with the dictionary of parameters along with the name. In order to make it easier and clearer what we are doing, we are typealiasing the dictionary of parameters and passing that type into our convenience initializer. Within the convenience initializer, we are calling CIFilter’s designated initializer, essentially wrapping the base CIFilter initialization within another function that allows us to do more.

The last piece of our extension is a computed property, which is our coveted output image. We are treating an outputImage property that is of type CIImage and instead of just setting it to a CIImage, we are checking the value for the key that we are using (which in this case is the name for the CIFilter we want to use) and returning a CIImage.

Computed properties are interesting. They do not store a value. Instead, they are a getter to find and set properties and values indirectly. It is my impression (which, if wrong, I would really appreciate someone correcting for me) that they are similar to functions but with slightly less overhead. Natasha The Robot says in this post that a computed property is a function disguised as a property. I am on the fence about what the difference is between a function and a computed property, so I am on the fence about how to handle them. Would like to write about them further in a future blog post, so hit me up with any thoughts you have at some point in time.

Compositing Filters

Anyone familiar with special effects or Photoshop might be familiar with compositing. Compositing is the art of layering multiple images and filters together to create a single new image. Compositing encompasses everything from making collages out of multiple images to generating large Photoshop pieces of art that have hundreds of layers.

The way we are using compositing here is that we are taking several simple image filters and we are combining them together to make a larger, more complex image filter. Many of the large, complex image filters in both Core Image and GPUImage are composites of smaller, simpler filters.

Core Image has an entire section of filters that are exclusively used to composite images. If you have a chance, it’s a lot of fun to play with compositing filters. If you work with Photoshop and you’re familiar with the various blend modes, these are represented in this category of Core Image filter. You can get some really awesome effects using compositing and blend modes in your photography and I encourage you to explore them.

The first composite filter we are creating is one that superimposes a color on top of the original image. It is made up of the CIConstantColorGenerator and CISourceOverCompositing. The Color Generator is just creating a blanket color layer whose output is not at all affected by the underlying image. The Source Over Compositing is simply taking one image and placing them over another. In our case, it is placing the Color Generator layer on top of our image we want to filter.

After we create this composite filter, we are chaining it to the output of the blur filter. So, first we are taking our image, applying the blur filter, then taking the output from the blur filter and applying our composited red filter to the top. We are specifying that the opacity on the red filter is only twenty percent so that the the second filter doesn’t completely block out the first filter or the input image.

Adding Functionality

Since being able to compose two or more filters together is an incredibly useful thing to be able to do and something we might need to do many times, it makes sense to write a custom function to simplify this process. Also, considering the number of inputs we are dealing with for this functionality, being able to simplify the code and avoid missed parens is a good use of our time.

The composeFilters() function being defined takes two filters as parameters and returns a new filter. In the code sample, we see that we are setting a variable to hold the output of this function, which is a filter. We are then creating a second variable that will hold the output of the filter we just created. Since the output from the composeFilters function is a filter, when you use the variable you are calling a Core Image function that is filtering the image being used as a parameter. When I read through this at first, it took me a little bit to parse apart the logic.

Infix Operators

Infix, Postfix, and Prefix notation is a complex way of labeling a phenomenon most programmers are familiar with: Where to place an operator.

Infix operators are placed between two operands. Which is an obtuse way of saying you write an equation like this:

X + Y

The operator, in this case the “+” sign, gets placed “in-between” the numbers it is operating on.

Prefix operators are placed, just as they sound, before the operands:

+XY

I’ll bet you can figure out how to write a postfix operator now:

XY+

What do these operators do? Why are there a bunch of them?

Think back to your Algebra class. Remember PEMDAS? That acronym helps us remember the order of operations. If you want to isolate two numbers that you want to add together before the result is multiplied, you had to put parentheses around them to ensure that the product was calculated, not just the first number. Infix, Prefix, and Postfix operators affect the order of operations in mathematical calculations. All equations can be represented and translated in each style, so theoretically you don’t lose any functionality by not understanding how to use each type of operator.

These operators are incredibly common in mathematics. Since Haskell and other functional languages evolved from Lambda Calculus, there are a lot of operators and notations that are foreign and unfamiliar to people like me who flunked calculus and grew up on imperative programming.

Swift lets you write your own custom operators. There is a list of ASCII characters that can be used to write custom operators. You have to specify what type of operator you are defining. If you are defining an infix operator, as we are doing here, you need to specify the direction of associativity. In standard mathematical equations we have associativity from left to right. You can choose left, right, or none. None, which is the default, means that you can’t place your custom operator next to another operator with the same precedence.

I am still trying to figure this part of the chapter out. I don’t know why the custom operator “>>>” was chosen. I don’t know if the character is significant or important. My understanding of the code as written is that instead of using a text label for our composeFilters function, we are using symbols. I know this is a common thing in languages like Haskell and unfortunately I haven’t delved into them enough to be able to fully answer my own questions about how infix operators are being utilized here. Again, if someone has an answer, I would appreciate a ping on Twitter.

Currying

Currying is the process of taking one function that takes multiple parameters and breaking it down into a sequence of functions that each take a single argument. For example, if you were writing a function that took two floats as a length and a width for a rectangle, you could rewrite this function to take the first parameter and then evaluates a sequence of functions, each of which take one parameter.

Currying is found in languages like Haskell. In Haskell, the language only allows you to pass one parameter into a function. Currying is a way to get around this constraint.

Side note: Both Haskell and curried functions are named for Haskell Curry. Curry was a mathematician and logician who hopefully didn’t get his lunch money stolen too often for having such a strange name.

Currying gives us some options for customizing our functions. Currying allows us to seed functions by supplying too few arguments and using it as a basis for fully implementing another function. It allows us to choose if we want to apply the function to all of our parameters.

Since this post is getting rather long, I don’t really want to write a comprehensive explanation of currying and why you want to use it. I will save that for another post. The concept of currying is introduced in this chapter, so I wanted to cover it just a little. I would also like to look over the code from this chapter to get a better grasp of currying before I try to explain it further.

Conclusion

There was a lot of stuff being touched on in this chapter. Core Image, infix operators, and currying. These are complicated frameworks and concepts, so again, don’t get frustrated if you got through this chapter not fully understanding how everything works.

Up next, we have our old functional friends Map, Filter, and Reduce. See you next time!