Category Archives: Metal

Getting Metal Up and Running: Part One

Posted on by

My current side project is working on trying to write at least one graphics post each week. I noticed that people tend to be asked to speak about things they write about. I also noticed that the vast majority of my posts for the last few months have been about depression and cooking and cleaning my house. I would prefer not to be a lifestyle blogger, so I am going to make a better effort to write more about what I am interested in, namely graphics.

Over the next few months I would like to write about frameworks (like Metal and Scene Kit), 3D mathematics, and shader applications. I am going to try to write something about what I am doing. I might not be able to get a whole sample project up and running and it might take a few weeks to get something done. But I plan to try to write about what I have learned and to chart my progress through my various explorations.

Getting Started with Metal

For a while there was a really good OpenGL ES template available in Xcode. Or so I have been told. Now if you try to make an OpenGL template, first off it’s difficult to even find. Secondly, the template is full of a lot of garbage. It’s similar to the Sprite Kit template including the rocket ship and other various assets. You can’t just open a template that renders a solid color on the screen.

My goal initially is to get a good Metal Template without the garbage that I can post on GitHub and make available to people who just want to get up and running.

Metal By Example

I recently bought Warren Moore’s Metal by Example book. Warren was kind enough to complete this book before abandoning us to go and work for Apple on the Metal frameworks. I didn’t realize he was going back until he did, but he’s been kind enough to answer questions about the book and Metal.

My plan was to go through the first chapter of the book and set up a Metal project, but I ran into some issues.

The books is written in Objective-C. I do not want to be one of those people who can’t or won’t read a book in Objective-C, but it does make things a little difficult when you are not used to it. I have difficulty switching my brain from one language to another, so this was one difficulty for me personally.

I also could not get the code to build. The compiler could not find the CAMetalLayer. I realized I was supposed to import Metal. I forgot how to do this. Either this is already done for you in Swift or I have spent so much time on my robotics project with Brad Larson where we wrote most of our own code that I simply didn’t remember how to link anything and the directions were not included. I found the directions on Warren’s companion web site. So if you are going through the book, I highly recommend looking at the site because it has content that is not in the book.

(I don’t want this to come off as a complaint against Warren. I greatly appreciate his efforts with the book and as an author myself, I can totally see me just wanting to get the damn thing done and have it be gone. I am not saying this is what he did, but if this was my book I totally would have done that. I still highly recommend his book and hope I don’t hurt his feelings.)

Even with this additional help, I could not get my code to build. It had trouble finding the QuartzCore/CAMetalLayer.h file. I imported Quartz and Core Animation, but no dice.

At this point I was frustrated and thought about giving up, but I decided to try and load Apple’s base Metal template.

Apple’s Built-In Templates

The Metal template is hidden in the Game templates options. If you navigate through the game templates, there are four types of templates here: Sprite Kit, Scene Kit, OpenGL ES, and Metal. Just because these are in the Game templates doesn’t mean you can only make games with these!

gameTemplates

metalTemplate

options

So I navigated through and got Metal base project in Swift. Yay! The project had a compiler warning! WTF?!

I was on my second glass of wine and was massively annoyed. Nothing I was doing would even build.

I looked up the compiler warning and realized that Metal still does not build in the iOS simulator. When Metal first came out it did not work in the simulator in Xcode 6. I had forgotten that and assumed it would be fix, but apparently not. If you are working with Metal, bear that in mind if you have sample code that you see that comes preloaded with a compiler warning.

I needed to build on my phone, so I plugged it into the computer. It built, but then it would not run. I got a warning on both the phone and Xcode about there being a permissions/privacy issue. My phone was not set up to run my code because it was an unknown developer.

So that began the great search through the Setting on the phone to grant my developer account permission to load code on my phone.

Here are a series of screen shots of where I found the ability to do this in the Settings. I blocked out the number after my email address because I am not sure if it is something I shouldn’t make public or not. I am sure someone will frantically Tweet me about some proprietary information being in these screenshots that I should not share. BTW: I never check the email address on here, I just use it for my developer account, so please do not spam me there, I am plenty available on Twitter.

IMG_3104

IMG_3105

IMG_3106

I don’t remember running code on a device being this difficult. It’s possible I have never tried running my own app on my phone with my own developer account. I had an educational account in school and then most of my other apps were for other companies. I think it might have something to do with TestFlight, maybe?? I would be interested to hear if things got more difficult in Xcode 7.

Finally, after all of this, I got the code to build and run on the phone! Success!!

IMG_3113

The base template is the usual triangle with a different color at each vertex floating around in space.

At this point I could just deleted that stuff and have my base template that was my goal for this week, but I don’t want to do that yet. I would like to look over this code as is to try and figure out how the vertices are read into the program and how the rotation is applied.

I want to figure out how to import a 3D model from a program like Maya or Blender. One of the big things that freaked me out about 3D graphics programming was the idea that I would have to construct my shapes by hand in the code rather than importing XML file representation of 3D objects.

Probably my goal for next week will be to go over the functionality of how this base generated template works before removing the floating triangle and uploading this to GitHub. I would like to use this template as my starting point for all of my future projects. I would also like to figure out what I forgot to import and connect in Warren’s project for the CAMetalLayer object because it is bothering me.

Didn’t get as much done this weekend as I wanted to. I talked in an earlier post about having a rough week and just getting this written was a bit of a struggle. Hoping future weeks will be more productive. But just doing something is better than not doing anything, I guess.

Final Countdown to CocoaConf Columbus 2014

Posted on by

After months of prep work and a roller coaster of changes, I am in the final day before heading off to the first of my three August conferences.

I have encountered more issues with Metal than I was hoping to find. This is the first time I have had a paid developer account during the beta period. Prior to now I was so busy just trying to establish a foundation that I somewhat ignored the new stuff that was coming out. This is the first time I have participated in the early release of not one but two new groundbreaking technologies on the ground floor.

I had to move more of my GPU programming talk over to OpenGL ES than I was planning to. I don’t think that is a bad thing per se. The most important thing I wanted to do was to answer a very specific question about one aspect of OpenGL programming. The fact that Apple came and changed everything about that made my talk both easier and harder. A lot of time was spend explaining why Metal is necessary and that fit into the parameters I wanted to address.

I will be giving this talk again in December. There will be a golden master of Xcode 6 at that point in time. I hope that it will be stable enough at that point that I can speak more about how to do things in Metal specifically rather than just ambiguously saying “This is how this would work if it were working, but it isn’t.”

I am giving my talks later today at Bendyworks. Bendy has been very kind to let me come and practice my talks there. I have found the feedback I get from them to be invaluable. I have also found that I am far less nervous once I have performed the talk at least once in front of real people and not just my pugs.

Speaking of my pugs, I am not going to see them for a week and I am very sad about that. I am going to miss my little buddies. Such is life.

I still have not packed. I need to pack sometime today. I also have to go to our Swift user group meeting to make arrangements with the people I am carpooling to Ohio with.

So I have a half dozen tasks to do today. Just need to take them one at a time to avoid feeling overwhelmed.

This is really stupid, but I keep forgetting that I do these talks because I love traveling to the conference and meeting new people. It’s hard to remember that this is going to be an amazing and awesome experience because I am putting a lot of pressure on myself to do a good job with my talks. I need to make sure I take some time to chill out and not worry so much about what I am doing.

Don’t panic.

Looking forward to seeing all my peeps at CocoaConf Columbus and That Conference in Wisconsin Dells!!

Code of Conduct

Posted on by

So today there has been some controversy on Twitter about 360|iDev’s Code of Conduct.

I honestly do not understand what the controversy is here. They are clearly stating that they want a diverse conference where everyone will be respected.

Here is my perspective on things.

I am a female programmer. I got into programming later in life. I originally studied video and audio production. I was young and foolish and thought I could succeed purely through sheer force of will. I would say I was about 30 when I really began to learn enough programming to make a go of it. At the last job I had I was not only the only woman, I was also the oldest person by a decade. I also have some health issues that make it impossible to work more than 40 hours a week.

Between being a woman, being older, not having a dozen years of experience, and having health issues, I feel very vulnerable in a community that fetishized boys barely old enough to drink who have been coding for fun since they were ten.

I have been extraordinarily privileged to be given the opportunity to speak at conferences on something that isn’t feminism. 360|iDev will be the fifth conference I have spoken at this year. I am speaking about Apple’s new 3D graphics programming framework.

People like Jim Remsik, Dave Klein, and John Wilker are all throwing me a lifeline to give me the chance to establish myself as a professional and maybe have a career.

I worry sometimes that I only get these opportunities because of “male guilt”. I am very concerned with being seen as someone who only gets to speak because they want more women. I have worked tirelessly to try to prove my cred by tackling difficult programming topics that frighten most people away. I am worried sick about not having a great talk to present at 360|iDev because my reach exceeds my grasp.

Even if I make an idiot of myself, at least I was given the chance. That’s all I ask for. I am being given a time and a place and what I do with it is up to me. A lot of people won’t even do that and I am eternally grateful for being given the chance to do what I can and make what I can of it. I am also grateful for the chance to meet the other people who are speaking and attending. In a world where connections are everything, the connections I have made at my conferences have been absolutely invaluable.

I have absolutely no idea what in the Code of Conduct created this fuss. I do know that I submitted a talk to a conference I could not afford to attend and that the organizers are not only giving me a chance to speak about a topic that is near and dear to my heart, they are also financially making it possible for me to go.

Talk is cheap. If you care so much about getting more women in technology, hire more women. Be willing to train them when they don’t have 5-10 years of experience. Mentor someone. Do something that actually costs you time or money. 360|iDev did. Don’t just go on Twitter and be a douche.

Lexical or Preprocessor Issue

Posted on by

So, today was the day I decided to bite the bullet and start working on my Metal demo for CocoaConf Columbus and 360|iDev.

Since a large focus of my talk is on GPUImage, I am hoping to put together a light Metal version of GPUImage that processes an image using a series of filters. I want to write between three and five filters that are easily stacked on one another that have a GPUImage counterpart in order to test how fast Metal processes images compared to GPUImage.

I went to look at what sample code is available from Apple for Metal. To my delight, I saw that there was an image processing base project. It includes one filter to change an image to black and white and that is hardcoded. I should be able to go into this project, add my filters, and add some UI elements allowing me to add the filter shaders I write.

Today I opened the sample code. Immediately, there was an error.

“Lexical or Preprocessor Issue: QuartzCore/CAMetalLayer.h not found.”

This is why we can't have nice things!!

This is why we can’t have nice things!!

Huh. That is inconvenient.

Did some digging. Refrained from asking this question on Stack Overflow because the last time I asked a question about the betas I got a snide person telling me to go somewhere else. Headed to the Dev Forums and found this thread.

Apparently, for the time being, there is no support for Metal in the simulator. There should be support for Metal if you have an A7 device like the iPhone 5S (which I have) that is running the iOS 8 beta.

I have not yet updated my phone to the beta. I know we are getting close to the point where it will be released, so it isn’t a huge thing to update to the beta, I just feel like I have no guarantee that stuff will work on there properly even after I update to the beta.

I must say that this latest wrinkle is not doing anything to sell me on Metal.

Metal only works on iOS A7 chips and now further won’t even work in the simulator. I usually use the simulator in my talks to demonstrate things I am doing, but now I have to get it on my device. I think I can use Airplay to show what the screen looks like, but that is one more step that can go wrong in my process.

The other things I am noticing in the sample applications is that most of the class implementation files end in “.mm”, which means that they are explicitly telling the compiler that there is going to be C++ code in them.

I have not worked with Swift as much as I should have, but I am wondering if this is going to be a problem with trying to write an app in Swift. I know that theoretically Swift is supposed to behave like Objective-C in that you can include C and C++ code, but I have not tried to write straight C code in a Swift class yet. Can you write C code in a Swift class, or is the support just that I can import a C class into a Swift-based project? How is this going to work with Metal?

At least with OpenGL ES you have the GLKit framework with should work with Swift. I am interested to know more about this, but sadly I don’t believe I will be able to explore these issues before I give my talk in Columbus.

I am also trying to figure out just how much C++ I need to know to fully work with Metal. I thought that I needed to know about the same amount of C++ as you need to know of C to work with GLSL, but after seeing the number of classes that are implementing C++, I am slightly worried that I am going to be out of my depth for a while.

These are things I am going to have to take into consideration and disclose during my talk. I know most of these issues will resolve themselves in the next few years, it is just slightly frustrating to sit on the sidelines trying to figure out how to make it work here and now.

Fortune favors the brave.

Heavy Metal

Posted on by

Hair Force One announcing Metal

Hair Force One announcing Metal

I know that the big new hotness from WWDC 2014 for most people is the Swift programming language. Swift has a large impact on me and on the project I am working on that I can’t publicly announce yet, but that was not the most intriguing thing announced to me. The most interesting thing that captured my attention was Metal.

I have been interested in learning OpenGL ever since I heard about it. I had to make the terrible choice last year of choosing whether to learn OpenGL or Core Audio because it would be complete idiocy to try to learn both at the same time. Since Chris Adamson didn’t write a book on OpenGL, I made the choice last year to learn Core Audio. It was the first programming book I read cover to cover and I got to spend a day with him in Boston at CocoaConf doing Core Audio. That was an amazing experience, but it’s time to move on to the next thing.

I started to learn OpenGL ES in earnest back in March. I had a few books and I have primarily been reading the same materials over and over again hoping that my brain translates them.

GPUImage

GPUImage

One accepted way learn OpenGL ES is to work on the GPUImage framework. There is a great blog post about how to write a custom shader here.

I decided a good way to learn OpenGL ES was to do a talk on GPUImage. Many of the tutorials I have seen on the framework basically just tell you how to plug it into your project and use the built-in filters. I wanted to do a talk about how the framework actually works and how to write your own filters. The creator of the framework, Brad Larson, lives in town. He has been extraordinarily generous with his time and knowledge about OpenGL ES. I pitched this talk and got it accepted at two different conference: CocoaConf Columbus and 360|iDev in Denver. Both of these conferences are in August. I pitched these talks around May. I figured that would be a decent amount of time to figure all this stuff out.

Then, like everyone else, I got slammed by WWDC.

I know that I don’t have to talk about Metal. It’s only been publicly announced for a few months and it only works on a handful of devices. There was no reason I couldn’t just keep my original talk topic. No reason except I had some existential questions I wanted answered.

Every time I heard about GPUImage I heard it was faster than Core Image because it was programmed on the GPU. What does that mean? All of my research on OpenGL ES says to push as much work off the GPU as possible, but they never specify what work the GPU is doing. I read a whole book on OpenGL ES without having any real clue what work is being done on the GPU.

The Defending Champion, OpenGL ES!

The Defending Champion, OpenGL ES!

I really wanted to do a talk on how to optimize OpenGL ES. I also wanted to explore what exactly it was that Metal was doing that was so much better than OpenGL ES. I heard a lot of bemoaning about how slow and inefficient OpenGL ES was, but after talking to Brad about it for a little while, I wondered if the mob was wrong.

I am doing my first talk on Metal three weeks from today. I have exactly one slide from my talk done as of 1:00 this afternoon, but I am in the process of gathering the answers to my questions.

One resource I can’t recommend more highly is the video tutorial series done by Ray Wenderlich. I had a list of questions in my head that I now have answers to because of his series on OpenGL ES. I am a quarter of the way through it and subscribing to his video tutorials is the best money I have spent on tech resources this year. It is my hope that one day he will produce a 3D graphics programming, hopefully after I know it well enough to be able to contribute to it!

So, I am going to take some time, but not too much, cataloging my work on this talk. I also have a debugging talk to complete in three weeks along with some obligations for my unnamed project. I think this is doable if I don’t have a panic attack or get distracted by squirrels.

The Famous Utah Teapot

The Famous Utah Teapot

I am planning to include links in my blog to any resource I have found to be particularly useful.

My goal before going to CocoaConf is to have a working Metal application with a few of the GPUImage filters translated from the OpenGL Shading Language to the Metal Shading Language. I would like to show the performance differences between GPUImage and Metal using the same project. I would also like to be able to intelligently explain GPU programming to people who are coming into this without knowing anything about OpenGL.

Three weeks. Two talks. Git ‘er done!