Metal file as part of an iOS framework

Question

I am trying to create a framework that works with METAL Api (iOS). I am pretty new to this platform and I would like to know how to build the framework to work with .metal files (I am building a static lib, not dynamic). Should they be a part of the .a file, or as a resource files in the framework bundle? Or is there an other way to do that? Thanks.

Update: For those who tackle this - I ended up following warrenm's 1's suggested option - converted the .metal file into a string and calling newLibraryWithSource:options:error:. Although it is not the best in performance it allowed me to ship only one framework file, without additional resources to import. That could be useful to whoever creating framework that uses Metal, ARKit, etc with shader files.

Answer 1

There are many ways to provide Metal shaders with a static library, all with different tradeoffs. I'll try to enumerate them here.

1) Transform your .metal files into static strings that are baked into your static library.

This is probably the worst option. The idea is that you preprocess your Metal shader code into strings which are included as string literals in your static library. You would then use the newLibraryWithSource:options:error: API (or its asynchronous sibling) to turn the source into an MTLLibrary and retrieve the functions. This requires you to devise a process for doing the .metal-to-string conversion, and you lose the benefit of shader pre-compilation, making the resulting application slower.

2) Ship .metal files alongside your static library and require library users to add them to their app target

All things considered, this is a decent option, though it places more of a burden on your users and exposes your Metal shader source (if that's a concern). Code in your static library can use the "default library" (newDefaultLibrary), since the code will be compiled automatically by Xcode into the app's default.metallib, which is embedded in the app bundle as a resource.

3) Ship a .metallib file alongside your static library

This is a good middle ground between ease-of-use, performance, and security (since it doesn't expose your shader source, only its IR). Basically, you can create a "Metal Library" target in your project, into which you put your shader code. This will produce a .metallib file, which you can ship along with your static library and have your user embed as a resource in their app target. Your static library can load the .metallib at runtime with the newLibraryWithData:error: or newLibraryWithURL:error: API. Since your shaders will be pre-compiled, creating libraries will be faster, and you'll keep the benefit of compile-time diagnostics.

Answer 2

As someone looking to include metal shader functions in a SceneKit / ARKit related framework, the available answers led me in the wrong direction. There is a much simpler solution that uses makeDefaultLibrary(bundle: Bundle) (iOS 10+) to access the functions included in a framework's .metal dependencies. Adding here for people in a similar position.

TL;DR, Access a Framework's MTLLibrary like this:

        //Get the framework bundle by using `Bundle(for: type(of: self))` from inside any framework class.
        //Then use the bundle to define an MTLLibrary.
        let frameworkBundle = Bundle(for: type(of: self))
        let device = MTLCreateSystemDefaultDevice()
        do {
            let bundleLib = try device?.makeDefaultLibrary(bundle: frameworkBundle)
            print(bundleLib.functionNames) //we can access our framework's metal functions! No build tricks/workarounds.
        } catch {
            print("Couldn't locate default library for bundle: \(frameworkBundle)")
            print( error )
        }

Xcode creates a default library of shader functions at build time by compiling .metal dependencies. This is true of both framework targets and app targets, so the real question is, how do I access my framework’s default library?

It’s possible to access a framework’s default library using the using the makeDefaultLibrary(bundle: Bundle) method on MTLDevice. The sample code above shows more detail.

For Scenekit/ARKit with SCNProgram

The bundle library can be set as an SCNProgram’s library property, and then fragment and shader functions can be defined just as if the .metal file was included in the main project:

        //The SCNProgram that will use our framework's metal functions
        var program = SCNProgram()

        //Use the framework's bundle to define an MTLLibrary.
        let frameworkBundle = Bundle(for: type(of: self))
        let device = MTLCreateSystemDefaultDevice()
        do {
            let bundleLib = try device?.makeDefaultLibrary(bundle: frameworkBundle)

            //set the SCNProgram's library, and define functions as usual
            program.library = bundleLib
            program.fragmentFunctionName = "yourCustomFrameworkFragmentFunction"
            program.vertexFunctionName = "yourCustomFrameworkVertexFunction"
        } catch {
            print("Couldn't locate default library for bundle: \(frameworkBundle)")
            print( error )
        }

Answer 3

The approach suggested by the questioner could not possibly work (hence, the lack of sample code). A Metal shader (.metal) is just a collection of functions, it does not a MTLLibrary (.metallib) make. Here is working code that compiles a Metal shader from a character (const char *) array (not the same as NSString); it is followed by instructions for converting a .metal file to a .metallib file prior to runtime.

Compiling a Metal Shader during Runtime

The following sample could also be used to provide users with a Shader Editor, and can allow you to update just the shader portion of your app without requiring the user to update the entire app:

NSError* error = NULL;
const char* vshSource =
"using namespace metal;\n"
"typedef struct {\n"
"    packed_float2 position;\n"
"    packed_float2 texcoord;\n"
"} Vertex;\n"

"typedef struct {\n"
"    float3x3 matrix;\n"
"    float3 offset;\n"
"} ColorConversion;\n"

"typedef struct {\n"
"    float4 position [[position]];\n"
"    float2 texcoord;\n"
"} Varyings;\n"

"vertex Varyings vertexPassthrough(\n"
"device Vertex* verticies [[ buffer(0) ]],\n"
"unsigned int vid [[ vertex_id ]]\n"
") {\n"
"   Varyings out;\n"
"   device Vertex& v = verticies[vid];\n"
"    out.position = float4(float2(v.position), 0.0, 1.0);\n"
"    out.texcoord = v.texcoord;\n"
"    return out;\n"
"}\n";

const char* fshSource =
"using namespace metal;\n"
"typedef struct {\n"
    "packed_float2 position;\n"
    "packed_float2 texcoord;\n"
"} Vertex;\n"

"typedef struct {\n"
    "float3x3 matrix;\n"
    "float3 offset;\n"
"} ColorConversion;\n"

"typedef struct {\n"
    "float4 position [[position]];\n"
    "float2 texcoord;\n"
"} Varyings;\n"

"fragment half4 fragmentColorConversion(\n"
                                       "Varyings in [[ stage_in ]],\n"
                                       "texture2d<float, access::sample> textureBGRA [[ texture(0) ]],\n"
                                       "constant ColorConversion &colorConversion [[ buffer(0) ]]\n"
                                       ") {\n"
    "constexpr sampler s(address::clamp_to_edge, filter::linear);\n"
    "return half4(half3(textureBGRA.sample(s, in.texcoord).rgb), 1.0);\n"
"}\n";

id <MTLFunction> vertexProgram;
id <MTLLibrary> vertexLibrary = [_device newLibraryWithSource:[NSString stringWithUTF8String:vshSource] options:NULL error:&error];
if (NULL != vertexLibrary)
{
    vertexProgram = [vertexLibrary newFunctionWithName:@"vertexPassthrough"];
} else {
    NSLog(@"Error compiling vertex program: %@", error.description);
}

id <MTLFunction> fragmentProgram;
id <MTLLibrary> fragmentLibrary = [_device newLibraryWithSource:[NSString stringWithUTF8String:fshSource] options:NULL error:&error];
if (NULL != fragmentLibrary)
{
    fragmentProgram = [fragmentLibrary newFunctionWithName:@"fragmentColorConversion"];
}  else {
    NSLog(@"Error compiling fragment program: %@", error.description);
}

The following are excerpts from an Apple Developer Documentation publication; although the information is relatively rudimentary, use it as a basis for a common framework shared by you and your audience when communicating about its subject matter.

Creating Libraries During the App Build Process

The accepted answer is flat-out wrong for the same reasons; and, it's claims about performance trade-offs are questionable. Here are the only definitive statements that can be made about compiling Metal shaders and creating Metal libraries, followed by actual code:

Functions and Libraries

This chapter describes how to create a MTLFunction object as a reference to a Metal shader or compute function and how to organize and access functions with a MTLLibrary object.

MTLFunction Represents a Shader or Compute Function

A MTLFunction object represents a single function that is written in the Metal shading language and executed on the GPU as part of a graphics or compute pipeline. For details on the Metal shading language, see the Metal Shading Language Guide.

To pass data or state between the Metal runtime and a graphics or compute function written in the Metal shading language, you assign an argument index for textures, buffers, and samplers. The argument index identifies which texture, buffer, or sampler is being referenced by both the Metal runtime and Metal shading code.

For a rendering pass, you specify a MTLFunction object for use as a vertex or fragment shader in a MTLRenderPipelineDescriptor object, as detailed in Creating a Render Pipeline State. For a compute pass, you specify a MTLFunction object when creating a MTLComputePipelineState object for a target device, as described in Specify a Compute State and Resources for a Compute Command Encoder.

A Library Is a Repository of Functions

A MTLLibrary object represents a repository of one or more MTLFunction objects. A single MTLFunction object represents one Metal function that has been written with the shading language. In the Metal shading language source code, any function that uses a Metal function qualifier (vertex, fragment, or kernel) can be represented by a MTLFunction object in a library. A Metal function without one of these function qualifiers cannot be directly represented by a MTLFunction object, although it can called by another function within the shader.

The MTLFunction objects in a library can be created from either of these sources:

• Metal shading language code that was compiled into a binary library format during the app build process.
• A text string containing Metal shading language source code that is compiled by the app at runtime.

Compiling shader language source files and building a library (.metallib file) during the app build process achieves better app performance than compiling shader source code at runtime. You can build a library within Xcode or by using command line utilities.

Using Xcode to Build a Library

Any shader source files that are in your project are automatically used to generate the default library, which you can access from Metal framework code with the newDefaultLibrary method of MTLDevice.

Using Command Line Utilities to Build a Library

Figure 8-1 shows the command line utilities that form the compiler toolchain for Metal shader source code. When you include .metal files in your project, Xcode invokes these tools to build a library file that you can access in your app at run time.

To compile shader source into a library without using Xcode:

1. Use the metal tool to compile each .metal file into a single .air file, which stores an intermediate representation (IR) of shader language code.
2. Optionally, use the metal-ar tool to archive several .air files together into a single .metalar file. (metal-ar is similar to the Unix ar.)
3. Use the metallib tool to build a Metal .metallib library file from IR .air files or from archive .metalar files.

Example: Building a Library File with Command Line Utilities

xcrun -sdk macosx metal MyLibrary.metal -o MyLibrary.air
    xcrun -sdk macosx metallib MyLibrary.air -o MyLibrary.metallib

To access the resulting library in framework code, call the newLibraryWithFile:error: method:

NSError *libraryError = NULL;
NSString *libraryFile = [[NSBundle mainBundle] pathForResource:@"MyLibrary" ofType:@"metallib"];
id <MTLLibrary> myLibrary = [_device newLibraryWithFile:libraryFile error:&libraryError];
if (!myLibrary) {
    NSLog(@"Library error: %@", libraryError);
}