In the third part of the Qt graphics series, we'll look at how shaders are handled in Qt Quick in Qt 5.14 when switching the scene graph to rendering through QRhi and the Qt Rendering Hardware Interface (Qt's hardware rendering interface). Let's cover shader processing before delving into RHI itself, because Qt Quick applications that use ShaderEffects or custom materials need to provide the fragment and/or vertex shader code themselves, and so they need to know (and in Qt 6 go to on) a new approach to processing shaders.
Speaking of Qt 6 (although everything described here only applies to Qt 5.14 and may change in later releases) what is available here will most likely serve as the basis for graphics processing and shader computation in Qt 6, once the few remaining jagged edges are eliminated.
Why something new?
Problem 1
Browsing the qt source tree (i.e. the git repo containing QtQml, QtQuick and related modules) and searching the shader catalog with vertex and fragment shaders for Qt Quick's built-in scene designer materials reveals that Qt Quick already ships with two versions of each GLSL vertex or fragment shader:
Why? This is due to support for kernel profile OpenGL contexts (for version 3.2 and higher). Since OpenGL implementations are not required to support compiling GLSL 100/110/120 shaders in such a context, Qt has no choice but to ship it with two options: one suitable for OpenGL ES 2.0, OpenGL 2.1 and compatibility profiles, and the other (version 150 in practice), which is used only when the context is profile. As described in part 1 of the article series, this is important to allow application developers to decide which OpenGL context to request when their application combines its own OpenGL rendering and Qt Quick-based user interface, whether the context is a compatibility or a main profile, Qt Quick will still be able to display.
This is fine when the number of options is 2. What if Vulkan-compatible GLSL (Vulkan-compatible GLSL), HLSL and MSL might now be needed in addition? Unfortunately, the approach is not very large-scale.
Problem 2
Some new graphics APIs no longer have built-in support for compiling shaders, unlike OpenGL (goodbye glCompileShader). And, even if they do, at least as a standalone library, they may not offer run-time reflection capabilities, which means there is no way to dynamically determine which vertex inputs and other shader resources are vertex, fragment, or compute shaders, and what the location of these resources (for example, what are the names and offsets of elements in a single block).
Problem 3
Inside detail: Qt Quick's batching system relies a bit on vertex shader rewriting for materials that are used with a so-called merged batch (this is what happens when multiple geometry nodes end up generating a single draw call). Rewriting the shader on the fly before passing it to glCompileShader is fine when only one shading language is used, but doesn't scale when we need to implement the same logic for several different languages.
What then?
Looking at the Khronos SPIR page, you see a beautiful and informative picture of the SPIR-V Open Source Ecosystem. Why not try building on it?
Key components that are of interest:
• glslang, a compiler from (OpenGL or Vulkan) GLSL to SPIR-V, an intermediate representation.
• SPIRV-Cross, a library for thinking about SPIR-V and disassembling it into high level languages such as GLSL, HLSL and MSL.
So if you "standardize" one language, such as Vulkan-style GLSL, and compile it to SPIR-V, you get something suitable for Vulkan. If we then run the SPIR-V binary through SPIRV-Cross, we get the necessary reflection information and can generate source code for various versions of GLSL, HLSL, and Metal Shading Language (GLSL is still important because while there are extensions to OpenGL that use SPIR-V, you simply cannot count on this, since such an extension will probably be absent on 90% of Qt target platforms and devices).
Finally, put it all together (including the reflection metadata) into an easily (de)serializable package, and there you have it.
Therefore, the pipeline that is used when running a Qt Quick application with QSG_RHI=1 set is:
Vulkan-flavor GLSL [ -> generate batching-friendly variant for vertex shaders] -> glslang : SPIR-V bytecode -> SPIRV-Cross : reflection metadata + GLSL/HLSL/MSL source -> pack it all together and serialize to a .qsb file
The .qsb extension comes from the name of the command line tool that performs the above steps - qsb , short for Qt Shader Baker (not to be confused with QBS).
At runtime, .qsb files are deserialized into QShader instances. It's a fairly simple container, following standard Qt patterns such as implicit sharing, and containing multiple variations of source code and bytecode for a single shader, along with a QShaderDescription that, unsurprisingly, contains reflection data. Like the rest of the RHI, these classes are private APIs for now.
The graphics layer directly uses QShader instances. The graphics pipeline state objects define a QShader for each active shader stage. The QRhi backends then choose the appropriate shader variant from the QShader package.
Since Qt 5.14, in practice this means searching:
• SPIR-V 1.0 when targeting Vulkan,
• Source HLSL or DXBC for Shader Model 5.0, when targeting D3D11,
• Metal 1.2 compliant MSL source or precompiled metalib when targeting Metal,
• a GLSL source for one of the versions 320, 310, 300, 100, when targeting an OpenGL ES context (in this order of precedence, starting with the highest version supported by the context),
• a GLSL source for one of the versions 460, 450, ..., 330, 150, when targeting an OpenGL kernel profile context (in that order of precedence, starting with the highest version supported by the context),
• A GLSL source for one of the versions 120, 110, when targeting a non-primary OpenGL context (in that order of precedence).
The HLSL and MSL entries in the above list may seem a little odd at first. This is because it is possible to compile HLSL and MSL from source at runtime (the default approach), and some experimentation has also been made to allow precompiled intermediate formats to be included in .qsb packages. In practice, this means calling fxc (no support for dxc yet - this is also in the plans, but will be really relevant only after QT Company starts looking at D3D12) or the Metal command line tools before the *"-> pack it step all together" ("pack everything together") * into the conveyor shown above. The problem here is that such tools are specific to their platform (Windows and MacOS respectively), so qsb can only call them when running on that platform. So, for example, when manually generating a .qsb file on Linux, this is not an option. In the long run, this is likely to become less of an issue, because in the Qt 6 time frame, developers will still need to research better build system integration, so manual tools like qsb will be less common.
Where does this qsb thing come from?
From the Qt Shader Tools module. This provides both the API, QShaderBaker, and the qsb command line tool to perform the compilation, translation, and packaging steps described above.
The qt-labs module does not ship with Qt 5.14 at the moment.
Why? Well, mostly because of 3rd party dependencies like glslang and SPIRV-Cross. There are a few things to research and find out when it comes to being able to compile and run on all Qt Company target platforms, as well as some things related to licenses, etc. If this all sounds familiar, it's because some of the These issues were mentioned in the first part of this article series when we talked about API translation solutions. So, for now, creating a .qsb package involves checking and building that module, and then running the qsb tool manually.
While the solution that comes with Qt is needed, relying on offline shader processing isn't too bad. This is one of the goals for Qt 6 anyway. The idea is to have something that integrates with Qt's build system so that the above shader processing steps are done at application build time ( or libraries). This remains as a future assignment, mainly due to the upcoming qmake -> cmake transition. As soon as the situation stabilizes, you can start building the solution on top of the new system.
So how does Qt Quick do it in Qt 5.14?
Looking at qtdeclarative/src/quick/scenegraph/shaders_ng the answer is obvious by running qsb manually (note the compile.bat) and including the resulting .qsb files into the Qt Quick library via the Qt resource system. This should get a little more complicated later on.
The .vert and .frag files contain Vulkan-compatible GLSL code and are not shipped with the Qt Quick build. The scenegraph.qrc file only lists .qsb files.
The process is well described in the slide below:
Each material has only one pair of vertex and fragment shaders, always written as Vulkan-compatible GLSL, following a few simple conventions (such as using only one unified buffer placed at binding 0).
Each of these files is then run through the shader baking machine, resulting in a QShader package. In this example, the result is 6 versions of the same shader, plus reflection data (which qsb can print as JSON text, however, the .qsb files themselves are compressed binary files and are not human readable). Problems 1 and 2 mentioned above are thus resolved.
Pay attention to the [Standard] tags in the list of shaders. If this were a vertex shader and the -b argument was also given, the number of output shaders would be 12 instead of 6. The 6 extra ones would be marked [Batchable], indicating they were batch friendly. Slightly modified options for the Qt Quick scene graph renderer. This solves problem 3 at the cost of slightly higher storage requirements (but due to the reduced runtime, it's probably worth it).
This covers the core concepts of the new shader pipeline. We should take a look at ShaderEffect and QSGMaterial in a separate article. The main idea (since Qt 5.14) is to pass .qsb filenames instead of shader source strings, but in particular materials need to know a few more things (mainly due to working with uniform buffers instead of single uniforms and because for not having the concept of a per-thread current context where everyone can change states arbitrarily). But about all this another time.
