Graphics, OpenGL, qt, 3D, Quick

The developers have been looking into how to integrate 3D and Qt Quick as deeply as possible, resulting in a new project called Qt Quick 3D which provides a high-level API for creating 3D content for user interfaces from Qt Quick. Instead of using an external engine, which can lead to animation timing issues and several layers of abstraction, Qt Company provides extensions to Qt Quick Scenegraph for 3D content and a renderer for these extended scene graph nodes.

Does this mean that they wrote another 3D Solution for Qt? Not really, because the spatial rendering engine is derived from the Qt 3D Studio renderer. This renderer has been ported to use Qt for platform abstraction and has been refactored to match the coding style of the Qt project.

"San Miguel" test scene running in Qt Quick 3D.

What are the goals of Qt Company? Why another 3D Solution?

Unified graphic history

The most important goal is to unify the graphical history. Qt Company currently offers two end-to-end solutions for creating flexible user interfaces, each with its own toolkit. One of these solutions is Qt Quick for 2D, the other is Qt 3D Studio for 3D. If you limit yourself to using one or the other it usually works out pretty well. However, developers have found that users end up having to mix and match them, which reveals many pitfalls, both in runtime performance and in the developer/designer experience.

Therefore, for simplicity, Qt Company aims to make one runtime (Qt Quick), one general scene graph (Qt Quick Scenegraph), and one design tool (Qt Design Studio). There should be no compromise on features, performance, or developer/designer experience. Thus, there is no need to further divide the development into more products, and thus, they can provide more features and fixes faster.

Intuitive and easy to use API

The next goal for Qt Quick 3D is to provide an API for defining 3D content. An API that is accessible and developer-friendly without having to understand the finer details of the modern graphics pipeline. After all, most users don't need to create custom 3D graphics renderers for each of their applications, but simply need to show some 3D content, often alongside 2D. Therefore, Qt Quick 3D is being developed from this point of view.

That being said, over time, the rendering API will be exposed more and more, allowing for the more advanced use cases needed by power users.

At the time of this writing, Qt Company only provides a QML API, but in the future they also want to provide an open C++ API.

Unified toolkit for Qt Quick

Qt Quick 3D is intended to be the successor to Qt 3D Studio . Currently, Qt 3D Studio will still be developed, but in the long run it will be replaced by Qt Quick and Qt Design Studio .

The developers intend to take the best parts of Qt 3D Studio and port them to Qt Quick and Qt Design Studio. So instead of needing a separate tool for Qt Quick or 3D, you can get both from Qt Design Studio. We are working on the details now and Qt Company hopes to get a preview soon.

For existing users of Qt 3D Studio, they are working on a porting tool to convert projects to Qt Quick 3D. More on this later.

First Asset class of processing pipeline

When working with 3D scenes, the state of the asset system becomes more important because there are now more types of assets in use, and in general they tend to be much larger. Therefore, when developing Qt Quick 3D, the developers tried to figure out how to make it as easy as possible to import your content and convert it into efficient runtime formats for Qt Quick.

For example, at design time you'll want to specify which assets to use based on what your asset creation tools generate (e.g. FBX files from Maya for 3D models, or PSD files from Photoshop for textures), but at runtime you don't want to use an engine for these formats. Instead, you'll want to convert the assets to some efficient runtime format and update them each time the original assets change. The Qt Company wants this to be as automated as possible, and therefore want to build it into the Qt build system and toolkit.

Cross-platform performance and compatibility

Another goal is to support several native graphics APIs using the new Rendering Hardware Interface being added to Qt. Currently, Qt Quick 3D only supports rendering using OpenGL, as do many other components in Qt. However, Qt 6 will use QtRHI as a graphics abstraction and will be able to support rendering through Vulkan, Metal and Direct3D, in addition to OpenGL.

What is Qt Quick 3D?

Qt Quick 3D is not a replacement for Qt 3D, but rather an extension of Qt Quick's functionality for rendering 3D content using a high-level API.

Here is what a very simple project looks like with some helpful notes:

import QtQuick 2.12
import QtQuick.Window 2.12
import QtQuick3D 1.0

Window {
  id: window
  visible: true
  width: 1280
  height: 720

  // Viewport for 3D content
  View3D {
    id: view

    anchors.fill: parent
    // Scene to view
    Node {
      id: scene

      Light {

        id: directionalLight

      }

      Camera {
        id: camera
        // It's important that your camera is not inside 
        // your model so move it back along the z axis
        // The Camera is implicitly facing up the z axis,
        // so we should be looking towards (0, 0, 0)
        z: -600
      }

      Model {
        id: cubeModel
        // #Cube is one of the "built-in" primitive meshes
        // Other Options are:
        // #Cone, #Sphere, #Cylinder, #Rectangle
        source: "#Cube"

        // When using a Model, it is not enough to have a 
        // mesh source (ie "#Cube")
        // You also need to define what material to shade
        // the mesh with. A Model can be built up of 
        // multiple sub-meshes, so each mesh needs its own
        // material. Materials are defined in an array, 
        // and order reflects which mesh to shade

        // All of the default primitive meshes contain one
        // sub-mesh, so you only need 1 material.

        materials: [

          DefaultMaterial {

            // We are using the DefaultMaterial which 
            // dynamically generates a shader based on what
            // properties are set. This means you don't 
            // need to write any shader code yourself.  
            // In this case we just want the cube to have
            // a red diffuse color.
            id: cubeMaterial
            diffuseColor: "red"
          }
        ]
      }
    }
  }
}

The idea is that defining 3D content should be as easy as defining 2D content. There are a few extra things you'll need, like the Lights, Cameras, and Materials concepts, but they're all high-level scene concepts, not graphics pipeline implementation details.

This simple API is, of course, cheaper. While it is possible to customize the scene's materials and content, it is not possible to fully customize how the scene is rendered, unlike Qt 3D with a custom frame chart. Instead, for now there is a fixed front renderer (renderer) where you can define scene properties and how they will be displayed. This is similar to other existing engines which usually have multiple possible rendering pipelines which then render the logic scene.

Camera rotating around car model in skybox with axes and grid lines (note: stutter from GIF 12 FPS).

What can you do with Qt Quick 3D?

It can do many things, but they are created using the following scene primitives:

Node

Node is the base component for any node in a 3D scene. It represents a transformation (transformation) in three-dimensional space, but is not visual. It works similar to how Item type works in Qt Quick.

Camera

The Camera represents how the scene is projected onto a 2D surface. The camera has a position in 3D space (as it is a subclass of Node) and a projection. To render a scene, you need to have at least one camera.

Light (World)

The Light component defines the light source in the scene, at least for materials that use lighting. At the moment there are 3 types of light sources: Directional (directional (default)), Point (point) and Area (covering the area).

Model (Model)

The Model component is the only visual component in the scene. It is a combination of geometry (from mesh) and one or more materials.

The source property of the Mesh component expects a .mesh file, which is the runtime format used by Qt Quick 3D. To get mesh files (mesh), you need to convert 3D models using the asset import tool (asset import tool). There are also several built-in primitives. They can be used by setting the source property to the following values: #Cube, #Cylinder, #Sphere, #Cone, or #Rectangle.

Also added is a programmatic way to define your own geometry at runtime, but it's not yet available in the preview.

Before a Model can be rendered, it must also have a Material. This determines how the mesh is shaded.

Default Materials and Custom Materials

The DefaultMaterial component is an easy-to-use built-in material. All you have to do is create this material, set the properties you want to define, and all the necessary shader code will automatically be generated under the hood. All other properties you set on the stage are also taken into account. There is no need to write graphics shader code yourself (like a vertex or fragment shader).

It is also possible to define so-called CustomMaterials where you provide your own shader code. Qt Company also provides a library of pre-defined CustomMaterials that you can try out by simply adding the following to your QML import:
import QtQuick3D.MaterialLibrary 1.0

Texture

The Texture component represents a texture in a 3D scene, and how it appears in a mesh. The texture source can be an image file or a QML Component.

Sample of available functions

3D views in Qt Quick

To view 3D content in Qt Quick, you need to align it to a 2D surface. For this, the View3D component is used. View3D is the only QQuickItem-based component in the entire API. You can define a scene as a child of the View3D or refer to an existing scene by setting the scene property to the root node (Node) of the scene you want to render.

If you have multiple cameras, you can also specify which camera you want to use to render the scene. By default, it will only use the first active camera defined in the scene.

It's also worth noting that View3D elements don't have to be rendered outside of screen textures before being rendered. You can set one of the 4 following render modes to determine when 3D content is displayed:

one. Texture: View3D is a Qt Quick texture provider and maps content to a texture via FBO.

2. Underlay: The View3D is rendered before the Qt Quick 2D content is displayed directly in the window (3D is always under 2D).

3. Overlay: View3D is rendered after Qt Quick 2D content is rendered directly into a window (3D is always larger than 2D).

four. RenderNode: The View3D is rendered according to the Qt Quick 2D content. However, this can lead to some oddities due to how Qt Quick 2D uses the depth buffer in Qt 5.

2D views inside 3D

You might also want to render Qt Quick content inside a 3D scene. To do this, wherever a Texture is taken as a property value (for example, in the default diffuseMap Material property), you can use a Texture with the sourceItem property set instead of simply specifying the file in the source property. This way the link to the Qt Quick element will be automatically rendered and used as the texture.

Diffuse color textures mapped to cubes are Qt Quick 2D animated elements.

3D QML components

Because Qt Quick 3D is built on top of QML, you can also create reusable components for 3D. For example, if you are creating a Car model that consists of several Models, just save it in Car.qml. You can then create multiple instances of the Car by simply reusing it like you would any other QML type. This is very important because in this way 2D and 3D scenes can be created using the same component model, instead of having to deal with different approaches for 2D and 3D scenes.

Multiple views of the same scene

Because scene definitions can exist anywhere in a Qt Quick project, they can be referenced from multiple View3Ds. If you had multiple cameras in the scene, you could even render from each camera to a different View3D.

4 views of the same teapot scene. Also changes between 3 cameras in perspective.

Shadows

Any Light component can specify that it casts shadows. When enabled, shadows are automatically drawn on the stage. Depending on what you're doing, rendering shadows can be quite expensive, so you can control which Model components cast shadows by setting additional properties on the Model.

Image-Based Lighting

In addition to the standard Light components, you can light up your scene by defining an HDRI map. This Texture can be set either for the entire View3D in the SceneEnvironment property, or for individual Materials.

Animations

Animations in Qt Quick 3D use the same animation system as Qt Quick. You can bind any property to an animator and it will animate and update as expected. Using the QtQuickTimeline module, it is also possible to use keyframe based animations.

Like the component model, this is another important step in bridging the gap between 2D and 3D scenes because it does not use separate, potentially conflicting animation systems.

There is currently no support for rigged animations, but this is planned for the future.

How to try Qt Quick 3D?

The intention is to release Qt Quick 3D as a technical preview with the release of Qt 5.14. Meanwhile, it will be usable now against Qt 5.12 and up.

To get the code, you just need to build the QtQuick3D module found here:
https://git.qt.io/annichol/qtquick3d

What about tools?

The goal is to be able to do everything needed to set up a 3D scene through Qt Design Studio. This means that you can visually mark up the scene, import 3D assets such as meshes, materials and textures, and convert those assets into efficient runtime formats used by the engine.

Demonstration of early integration of Qt Design Studio for Qt Quick 3D.

Import 3D scenes into QML components

Qt Quick 3D can also be used by writing QML code by hand. Therefore, there are also several standalone utilities for converting assets. Once upon a time, such a tool was a balsam asset preparation tool. Now, you can pass an asset to this utility from a 3D asset creation tool like Blender, Maya, or 3DS Max, and it will generate a QML component representing the scene, as well as any textures, meshes, and materials it uses. This tool currently supports creating scenes from the following formats:
• FBX
• Collada (dae)
• OBJ
• Blender (blend)
• GLTF2

To convert the myTestScene.fbx file, you must run:

./balsam -o ~/exportDirectory myTestScene.fbx

This will create a file named MyTestScene.qml along with any required assets. Then you can just use it like any other component in your scene:

import QtQuick 2.12
import QtQuick.Window 2.12
import QtQuick3D 1.0

Window {
 width: 1920
 height: 1080
 visible: true
 color: "black"

 Node {
   id: sceneRoot
   Light { 
   }
   Camera {
   z: -100
   }
   MyTestScene {
   }
 }

 View3D {
   anchors.fill: parent
   scene: sceneRoot
 }
}

Qt Company is working on improving the assets generated by this tool, so improvements are expected in the coming months.

Converting Qt 3D Studio projects

In addition to being able to generate 3D QML components from 3D asset creation tools, a plugin has also been created for the asset import tool to convert existing Qt 3D Studio projects. If you have used Qt 3D Studio before, you will know that it generates XML projects to define the scene. If you give the balsam tool a UIP or UIA project generated by Qt 3D Studio, it will also generate a Qt Quick 3D project based on that. Note, however, that since the runtime used by Qt 3D Studio is different from Qt Quick 3D, not everything will be converted. However, it should give a good approximation or starting point for converting an existing project. Qt Company looks forward to continuing to improve support for this path to smooth the transition for existing Qt 3D Studio users.

An example of a Qt 3D Studio application ported using the Qt Quick 3D import tool (it's not perfect yet).

How about Qt 3D?

The first question Qt Company is looking forward to is why not just use Qt 3D? This is a question that they have been studying for the last couple of years.

One suggestion is that you can just build all of Qt Quick on top of Qt 3D if you mix 2D and 3D. Qt Company got together and started doing this with the 2.3 release of Qt 3D Studio. The powerful Qt 3D API provided a nice abstraction for implementing a rendering engine to recreate the behavior expected by Qt Quick and Qt 3D Studio. However, the Qt 3D architecture makes it difficult to get the performance you need on entry-level embedded hardware. Qt 3D also has some overhead due to its own limited runtime environment, as well as being another layer of abstraction between Qt Quick and the graphics hardware. In its current form, Qt 3D is not ideal for further development if they want to achieve a completely unified graphical history while providing consistent good support for a wide range of platforms and devices ranging from low to high end.

At the same time, there was already a rendering engine in Qt 3D Studio that did exactly what it needed to do and provided a good basis for building additional functionality. This comes with the disadvantage that they no longer have the powerful APIs that come with Qt 3D, but in practice, when you start building a runtime on top of Qt 3D, you end up making decisions about how things should work. work, which in any case leads to a limited ability to customize the framegraph (frame graph). In the end, the most practical solution was to use the existing Qt 3D Studio rendering engine as a base and build on it.

What is the plan for moving forward?

This release is just a preview of what should be. It is planned to provide Qt Quick 3D as a fully supported module with Qt 5.15 LTS. In the meantime, Qt Company is working on further development of Qt Quick 3D as a Tech Preview release with Qt 5.14.

For the Qt 5 series, they are limited in how deeply they can combine 2D and 3D due to promises of binary compatibility. With the release of Qt 6, Qt Company plans to integrate Qt Quick 3D even deeper into Qt Quick to provide an even smoother experience.

The goal is that they want to be as efficient as possible when mixing 2D and 3D content, at no extra cost to users who don't use 3D content at all. They won't do anything drastic, like force all Qt Quick applications to go through a new render, only those that mix 2D and 3D.

Qt 6 will also use the Qt hardware rendering interface to render Qt Quick scenes (including 3D), which should fix many of the current problems Qt Company faces today when deploying OpenGL applications (using DirectX on Windows, Metal on macOS, etc.) d.).

The Qt Company also wants end users to be able to use the C++ Rendering API, which has been created more generically without Qt Quick. The code is now like the private API, but they are waiting for Qt 6 time (and the RHI port) before making the compatibility promises that come with the public APIs.