Qt Quick 3D - Procedural Texture Example | Qt Quick 3D | Qt 6.11.0
Description: Demonstrates how to provide custom texture data from C++ or QML.

- Qt 6.11
- Qt Quick 3D
- Qt Quick 3D Examples and Tutorials
- Qt Quick 3D - Procedural Texture Example
- Qt Quick 3D | Commercial or GPLv3

# Qt Quick 3D - Procedural Texture Example

Demonstrates how to provide custom texture data from C++ or QML.

This demonstrates the various ways to provide texture data generated dynamically at runtime instead of loading it from a static asset. For demonstration purposes this example generates vertical gradient textures from provided start and end colors.

First we define a C++ class for our texture data. We make it a subclass of QQuick3DTextureData. This is not strictly necessary, since there are no virtual functions, but it is much more convenient to have everything in one class. We define the properties we are going to use, and add QML_NAMED_ELEMENT to make it available from QML:

     class GradientTexture : public QQuick3DTextureData
     {
         Q_OBJECT
         Q_PROPERTY(int height READ height WRITE setHeight NOTIFY heightChanged)
         Q_PROPERTY(int width READ width WRITE setWidth NOTIFY widthChanged)
         Q_PROPERTY(QColor startColor READ startColor WRITE setStartColor NOTIFY startColorChanged)
         Q_PROPERTY(QColor endColor READ endColor WRITE setEndColor NOTIFY endColorChanged)
         QML_NAMED_ELEMENT(GradientTexture)
         ...

We add a function to update the texture. It uses setSize, and setFormat to configure the texture, and setTextureData to set the image data:

     void GradientTexture::updateTexture()
     {
         setSize(QSize(m_width, m_height));
         setFormat(QQuick3DTextureData::RGBA8);
         setHasTransparency(false);
         setTextureData(generateTexture());
     }

The function `generateTexture` creates a QByteArray of the correct size, and fills it with image data:

     QByteArray GradientTexture::generateTexture()
     {
         QByteArray imageData;
         // Create a horizontal gradient between startColor and endColor

         // Create a single scanline and reuse that data for each
         QByteArray gradientScanline;
         gradientScanline.resize(m_width * 4); // RGBA8

         for (int x = 0; x < m_width; ++x) {
             QColor color = linearInterpolate(m_startColor, m_endColor, x / float(m_width));
             int offset = x * 4;
             gradientScanline.data()[offset + 0] = char(color.red());
             gradientScanline.data()[offset + 1] = char(color.green());
             gradientScanline.data()[offset + 2] = char(color.blue());
             gradientScanline.data()[offset + 3] = char(255);
         }

         for (int y = 0; y < m_height; ++y)
             imageData += gradientScanline;

         return imageData;
     }

We call `updateTexture` each time a property is changed:

     void GradientTexture::setStartColor(QColor startColor)
     {
         if (m_startColor == startColor)
             return;

         m_startColor = startColor;
         emit startColorChanged(m_startColor);
         updateTexture();
     }

Finally, we can use our new texture from QML:

     Texture {
         id: textureFromCpp

         minFilter: applicationState.filterMode
         magFilter: applicationState.filterMode
         textureData: gradientTexture

         GradientTexture {
             id: gradientTexture
             startColor: applicationState.startColor
             endColor: applicationState.endColor
             width: applicationState.size
             height: width
         }
     }

It is also possible to generate the same texture data in QML. In this case we use the ProceduralTextureData component:

     Texture {
         id: textureFromQML
         minFilter: applicationState.filterMode
         magFilter: applicationState.filterMode
         textureData: gradientTextureDataQML

         ProceduralTextureData {
             id: gradientTextureDataQML

             property color startColor: applicationState.startColor
             property color endColor: applicationState.endColor
             width: applicationState.size
             height: width
             textureData: generateTextureData()

             function linearInterpolate(startColor : color, endColor : color, fraction : real) : color{
                 return Qt.rgba(
                             startColor.r + (endColor.r - startColor.r) * fraction,
                             startColor.g + (endColor.g - startColor.g) * fraction,
                             startColor.b + (endColor.b - startColor.b) * fraction,
                             startColor.a + (endColor.a - startColor.a) * fraction
                             );
             }

             function generateTextureData() {
                 let dataBuffer = new ArrayBuffer(width * height * 4)
                 let data = new Uint8Array(dataBuffer)

                 let gradientScanline = new Uint8Array(width * 4);

                 for (let x = 0; x < width; ++x) {
                     let color = linearInterpolate(startColor, endColor, x / width);
                     let offset = x * 4;
                     gradientScanline[offset + 0] = color.r * 255;
                     gradientScanline[offset + 1] = color.g * 255;
                     gradientScanline[offset + 2] = color.b * 255;
                     gradientScanline[offset + 3] = color.a * 255;
                 }

                 for (let y = 0; y < height; ++y) {
                     data.set(gradientScanline, y * width * 4);
                 }

                 return dataBuffer;
             }
         }
     }

Just like in C++ we fill a QByteArray with image data that reflects the size and format of the texture. When doing this from QML use the ArrayBuffer type to avoid unnecessary type conversion.

For the previous two examples, all of the data for the texture is generated on the CPU side, then uploaded to the GPU for use as texture data. It is also possible to generate the texture data directly on the GPU. In this case we use subclass of QQuick3DTextureProviderExtension instead.

     class GradientTextureProvider : public QQuick3DTextureProviderExtension
     {
         Q_OBJECT
         Q_PROPERTY(int height READ height WRITE setHeight NOTIFY heightChanged)
         Q_PROPERTY(int width READ width WRITE setWidth NOTIFY widthChanged)
         Q_PROPERTY(QColor startColor READ startColor WRITE setStartColor NOTIFY startColorChanged)
         Q_PROPERTY(QColor endColor READ endColor WRITE setEndColor NOTIFY endColorChanged)
         QML_ELEMENT

QQuick3DTextureProviderExtension based classes define render extensions using the Qt RHI API. The subclass must return a QSSGRenderExtension based object when QQuick3DTextureProviderExtension::updateSpatialNode() is called.

     QSSGRenderGraphObject *GradientTextureProvider::updateSpatialNode(QSSGRenderGraphObject *node)
     {
         if (!node)
             node = new GradientTextureProviderNode(this);

         // Update the state of the backend node
         auto gradientNode = static_cast<GradientTextureProviderNode *>(node);
         gradientNode->m_isDirty = true;
         gradientNode->m_width = m_width;
         gradientNode->m_height = m_height;
         gradientNode->m_startColor = m_startColor;
         gradientNode->m_endColor = m_endColor;

         return node;
     }

What actually gets rendered is defined by this QSSGRenderExtension subclass:

     class GradientTextureProviderNode : public QSSGRenderTextureProviderExtension
     {
     public:
         explicit GradientTextureProviderNode(GradientTextureProvider *ext);
         ~GradientTextureProviderNode() override;
         bool prepareData(QSSGFrameData &data) override;
         void prepareRender(QSSGFrameData &data) override;
         void render(QSSGFrameData &data) override;
         void resetForFrame() override;

         bool m_isDirty = false;

         // state
         int m_width = 256;
         int m_height = 256;
         QColor m_startColor = QColor(Qt::red);
         QColor m_endColor = QColor(Qt::blue);

     private:

         QPointer<GradientTextureProvider> m_ext;
         QSSGExtensionId extensionId {};

         std::unique_ptr<QRhiBuffer> quadGeometryVertexBuffer;
         std::unique_ptr<QRhiBuffer> quadGeometryIndexBuffer;

         //
         std::unique_ptr<QRhiTexture> outputTexture; // the final output texture
         std::unique_ptr<QRhiTextureRenderTarget> outputTextureRenderTarget;
         std::unique_ptr<QRhiRenderPassDescriptor> ouputTextureRenderPassDescriptor;

         std::unique_ptr<QRhiBuffer> gradientTextureUniformBuffer;
         std::unique_ptr<QRhiShaderResourceBindings> gradientTextureShaderResouceBindings;
         std::unique_ptr<QRhiGraphicsPipeline> gradientTexture2dPipeline;
     };

By implementing the interfaces of QSSGRenderExtension, the extension can render the texture data directly on the GPU. The prepareDate implementation makes sure that the output texture is created with the correct size and format, as well as registers the output texture as the texture that is being provided.

     bool GradientTextureProviderNode::prepareData(QSSGFrameData &data)
     {
         if (!m_isDirty)
             return false;

         const auto &ctxIfx = data.contextInterface();
         const auto &rhiCtx = ctxIfx->rhiContext();
         QRhi *rhi = rhiCtx->rhi();

         // If there is no available rhi context, then we can't create the texture
         if (!rhiCtx)
             return false;

         extensionId = m_ext ? QQuick3DExtensionHelpers::getExtensionId(*m_ext) : QSSGExtensionId{};
         if (QQuick3DExtensionHelpers::isNull(extensionId))
             return false;

         // Make sure that the output texture is created and registered as the texture provider
         if (!outputTexture ||
             outputTexture->pixelSize().width() != m_width ||
             outputTexture->pixelSize().height() != m_height) {
             outputTexture.reset(rhi->newTexture(QRhiTexture::Format::RGBA8, QSize(m_width, m_height), 1, QRhiTexture::RenderTarget | QRhiTexture::sRGB));
             outputTexture->create();

             outputTextureRenderTarget.reset(rhi->newTextureRenderTarget({ outputTexture.get() }));
             ouputTextureRenderPassDescriptor.reset(outputTextureRenderTarget->newCompatibleRenderPassDescriptor());
             outputTextureRenderTarget->setRenderPassDescriptor(ouputTextureRenderPassDescriptor.get());
             outputTextureRenderTarget->create();

             // Register the output as the texture provider
             QSSGRenderExtensionHelpers::registerRenderResult(data, extensionId, outputTexture.get());

             gradientTexture2dPipeline.reset();
         }

         // If m_isDirty is true than prepareRender and render will actually get called.
         return m_isDirty;
     }

The prepareRender is where the pipelines are created, and the uniform buffers are updated.

     void GradientTextureProviderNode::prepareRender(QSSGFrameData &data)
     {
         const auto &ctxIfx = data.contextInterface();
         const auto &rhiCtx = ctxIfx->rhiContext();
         if (!rhiCtx)
             return;

         QRhi *rhi = rhiCtx->rhi();
         QRhiCommandBuffer *cb = rhiCtx->commandBuffer();
         QRhiResourceUpdateBatch *resourceUpdates = rhi->nextResourceUpdateBatch();

         // Create the pipeline if necessary
         if (!gradientTexture2dPipeline) {
             // 1 quad (2 trianges), pos + uv.  4 vertices, 5 values each (x, y, z, u, v)
             quadGeometryVertexBuffer.reset(rhi->newBuffer(QRhiBuffer::Immutable, QRhiBuffer::VertexBuffer, 5 * 4 * sizeof(float)));
             quadGeometryVertexBuffer->create();

             // 6 indexes (2 triangles)
             quadGeometryIndexBuffer.reset(rhi->newBuffer(QRhiBuffer::Immutable, QRhiBuffer::IndexBuffer, 6 * sizeof(uint16_t)));
             quadGeometryIndexBuffer->create();

             // Uniform buffer: packed into 2 * vec4 (2 RGBA colors)
             const size_t uBufSize = (sizeof(float) * 4) * 2;
             gradientTextureUniformBuffer.reset(rhi->newBuffer(QRhiBuffer::Dynamic, QRhiBuffer::UniformBuffer, uBufSize));
             gradientTextureUniformBuffer->create();

             // Uniform buffer is only bound/used in Fragment Shader
             gradientTextureShaderResouceBindings.reset(rhi->newShaderResourceBindings());
             gradientTextureShaderResouceBindings->setBindings({
                     QRhiShaderResourceBinding::uniformBuffer(0, QRhiShaderResourceBinding::FragmentStage, gradientTextureUniformBuffer.get()),
             });
             gradientTextureShaderResouceBindings->create();

             gradientTexture2dPipeline.reset(rhi->newGraphicsPipeline());
             gradientTexture2dPipeline->setShaderStages({
                     { QRhiShaderStage::Vertex, getShader(QLatin1String(":/shaders/gradient.vert.qsb")) },
                     { QRhiShaderStage::Fragment, getShader(QLatin1String(":/shaders/gradient.frag.qsb")) }
             });
             // 2 Attributes, Position (vec3) + UV (vec2)
             QRhiVertexInputLayout inputLayout;
             inputLayout.setBindings({
                     { 5 * sizeof(float) }
             });
             inputLayout.setAttributes({
                     { 0, 0, QRhiVertexInputAttribute::Float3, 0 },
                     { 0, 1, QRhiVertexInputAttribute::Float2, 3 * sizeof(float) }
             });
             gradientTexture2dPipeline->setVertexInputLayout(inputLayout);
             gradientTexture2dPipeline->setShaderResourceBindings(gradientTextureShaderResouceBindings.get());
             gradientTexture2dPipeline->setRenderPassDescriptor(ouputTextureRenderPassDescriptor.get());
             gradientTexture2dPipeline->create();

             // Upload the static quad geometry part
             resourceUpdates->uploadStaticBuffer(quadGeometryVertexBuffer.get(), g_vertexData);
             resourceUpdates->uploadStaticBuffer(quadGeometryIndexBuffer.get(), g_indexData);
         }

         // Upload the uniform buffer data
         const float colorData[8] = {
             m_startColor.redF(), m_startColor.greenF(), m_startColor.blueF(), m_startColor.alphaF(),
             m_endColor.redF(), m_endColor.greenF(), m_endColor.blueF(), m_endColor.alphaF()
         };
         resourceUpdates->updateDynamicBuffer(gradientTextureUniformBuffer.get(), 0, sizeof(colorData), colorData);

         cb->resourceUpdate(resourceUpdates);
         m_isDirty = false;
     }

The actual rendering for the pass is defined in the render function.

     void GradientTextureProviderNode::render(QSSGFrameData &data)
     {
         const auto &ctxIfx = data.contextInterface();
         const auto &rhiCtx = ctxIfx->rhiContext();
         if (!rhiCtx)
             return;

         QRhiCommandBuffer *cb = rhiCtx->commandBuffer();

         // Render the quad with our pipeline to the outputTexture
         cb->beginPass(outputTextureRenderTarget.get(), Qt::black, { 1.0f, 0 }, nullptr, rhiCtx->commonPassFlags());
         cb->setViewport(QRhiViewport(0, 0, m_width, m_height));
         cb->setGraphicsPipeline(gradientTexture2dPipeline.get());
         cb->setShaderResources(gradientTextureShaderResouceBindings.get());
         QRhiCommandBuffer::VertexInput vb(quadGeometryVertexBuffer.get(), 0);
         cb->setVertexInput(0, 1, &vb, quadGeometryIndexBuffer.get(), QRhiCommandBuffer::IndexFormat::IndexUInt16);
         cb->drawIndexed(6);
         cb->endPass();
     }

The example uses the texture provider in QML like this:

     Texture {
         id: textureFromGPU
         minFilter: applicationState.filterMode
         magFilter: applicationState.filterMode
         textureProvider: GradientTextureProvider {
             startColor: applicationState.startColor
             endColor: applicationState.endColor
             width: applicationState.size
             height: width
         }
     }

Example project @ code.qt.io