I was really hoping last week to get a post out about creating a default Metal template, but I have been sick for a few weeks. Last weekend I sat in my chair staring at the TV because that was all I could deal with.

I am also recovering from a migraine that I have had for the past four days.

I was hoping to be able to go through my Metal template and remove all of the bells and whistles that were added by the Apple engineers to make the template do something and not just be a blank slate. I don’t feel comfortable enough to go through this and delete things yet, so I would like to talk about various Metal objects and components necessary to create a baseline Metal project and what purpose each of these serve.

I am posting this project on GitHub. I will be modifying it in the next few weeks to make this into a functional template anyone can grab if they want to start a Metal project but don’t want to deal with deleting boilerplate code. It’s not at that point yet, but hopefully before the end of the year!

The first, and most important, property I want to talk about is the MTLDevice. The documentation for MTLDevice is here.

Why Does Our Code Need an MTLDevice?

The MTLDevice is the software representation of the GPU. It is the way that you are able to interface with the hardware.

You might be wondering why this is something you would need. The iPhone is a cohesive unit that just kind of works when you program it. Why do you need to initialize a variable to represent part of your phone?

Whenever you create software that is going to interact with external hardware, you need to have a way to interface with it in your code.

When I was working at SonoPlot, we were writing control software for robots. We had classes for the robotics system and for the camera we were using.

The camera class had to have functions for every action we needed it to have. It also had a property for each thing that we needed to either get or set. One example is resolution. Some of our cameras had variable resolution and others were fixed. We had to have a property to either retrieve what that resolution was or that would set the resolution on the ones that can be set.

Even though we don’t necessarily think of the iPhone in terms of “external” hardware, it is. There are different parts of the iPhone that you can interface through the Cocoa frameworks. Speaking of cameras, the iPhone has a camera.

The AVCaptureDevice in AVFoundation is the interfacing class that Apple provides to allow you to talk to the camera. It fulfills the same function that our Camera and Robotics classes had in the robotics software.

Just as we needed to create a camera and robotics class to talk to our hardware, you also need to create an object that talks directly to your GPU.

One of the promises of the Metal framework is that you, as the developer, would have far more control over allocating and deploying resources in your code. This is reflected in the methods associated with the MTLDevice protocol.

Protocol, Not Object

The first thing to point out is that MTLDevice is not a class, it’s a protocol. At this point I am not entirely certain why it was designed this way.

The code to create the device is as follows:

let device: MTLDevice = MTLCreateSystemDefaultDevice()!

If you hold option and hover over the device variable, it indicates that it is a MTLDevice object. In order to be an object, it must be an instance of a class. My best guess is that this is an instance of NSObject that conforms to the MTLDevice protocol.

I am fascinated as to why this was implemented in this manner. I plan to look into this further, but this is one of those things that having an understanding of why it was done this way doesn’t necessarily help me get things done, so I am going to try and leave it alone for now.

Functionality

The functionality that the MTLDevice needs to be able to do for you are the following:

Identifying Properties

There are many more Apple devices and chip types that support Metal programming now than there were when it was initially announced. Back when it came out in iOS 8, we just had the iPhone 5S and one of the iPad models with an A7 chip.

Now, we have a lot of devices with a lot of different chip sets that all support Metal programming, including the Mac.

We’re in a similar situation with these chips that I was in with supporting the legacy robotics software at SonoPlot. We had three different types of cameras that all had different properties. You can’t assume that all the GPUs in each device are the same.

So, just as we did with the robotics software, there are several properties that are retrievable from the GPU.

The properties you can access on each GPU are:

• Maximum number of threads per thread group
• Device Name
• Whether this GPU supports a specific feature set
• Whether this GPU supports a specific texture sample count

Looking over this list of properties, it looks like as the chips have progressed, they have more and better capabilities. The A7 chip in my iPhone 5S is not as powerful as the A9X chip in my iPad Pro.

If you want to target less powerful devices, it might be useful to get familiar with the idiosyncrasies of the chips in these devices.

Creating Metal Shader Libraries

As a protege of Brad Larson, I am very interested in exploring shaders in Metal.

One of the big, stupid questions I have with Metal is how the shaders are set up. Every project I have seen (including the template) has had one shader file named “Shaders.metal.” Does that mean that all shaders go into one file? Are these just projects that use one shader? Is this basically set up the same way that OpenGL ES 2.0 is set up?

This says that all of the “.metal” files are compiled into a single default library, so that tells me that I can have more than one shader file in a project.

I am looking forward to exploring the shader libraries in the future, but right now just making a note that this is a responsibility of the MTLDevice.

Creating Command Queues

This is interesting to see that Metal has built in command queues.

I know that Brad utilizes Grand Central Dispatch in GPUImage to make it work as efficiently as possible. OpenGL ES doesn’t have a built-in command queue structure to the best of my knowledge. I know that OpenGL ES can only exist on one thread at any given time and that one of the promises of Metal was multithreading. If you’re going to multithread something you need some way of keeping your threads straight.

Looking forward to exploring the Metal Command Queues further.

Creating Resources

The resources you are creating with the Metal device are buffers, textures, and sampler states.

Buffers should be familiar to anyone who works with graphics or audio. To the best of my memory, in OpenGL ES you have three buffers: One for the frame that is currently being displayed, one for the next frame to be displayed, and one in the wings waiting to be deployed when the top buffer gets popped off the stack.

Textures are also a familiar concept. In GPUImage the photo or video you are filtering is your texture, so this is fairly straightforward.

I have never heard of a sampler state, so I am interested to find out what this does.

Not a lot new or exciting here if you have any familiarity with OpenGL ES.

Creating Command Objects to Render Graphics

This is where you set up your pipeline to render your graphics. In OpenGL ES 1.0 this was a fixed function pipeline without shaders. In OpenGL ES 2.0, the programmable pipeline was introduced along with the ability to use shaders. Shaders gave the ability to really generate shadows and ambient occlusion in graphics on an iOS device. Since we’re not taking a step backwards, there are commands here to set up your programmable pipeline.

Creating Command Objects to Perform Computational Tasks

One of the things that really excited me about Metal was the ability to do general purpose GPU programming (GPGPU) in iOS. I had hoped in iOS 8 to hear that OpenCL would be made available on iOS, so I was rather pleased to hear that this functionality was made available.

Jeff Biggus has been speaking about OpenCL for a few years and using the GPU for something other than just graphics processing. It is one of those things on my giant list of things I am interested in but haven’t had a chance to explore yet. This excites me and I am looking forward to writing a Metal project that utilizes this extra functionality.

Thoughts

I am really excited about all of the functionality I see exposed to me in this protocol.

I know I basically just went through the documentation and didn’t necessarily tell you anything you couldn’t look up on your own, but I do think it helps to have some context about WHY this stuff is in here, not just that it exists.

I will be getting into the command queues and buffers in more depth in my next post because those are absolutely necessary for a minimum viable application. It’s helpful to know that these things are created and controlled by this master object.

I hope that this was interesting and useful. I know in most of the documentation I am reading it just mentions you need to create a MTLDevice without exploring what it’s role is in the application. It’s a really vital part of the application and I hope that you have a better understanding and appreciation of what it does within a Metal program.