This is the first in a series of posts on the HLSL 2021 user-defined templates feature. The goal of this series is to share and learn from the experience of shipping a complicated language feature.
While I’ve worked in Clang’s codebase for years, prior to joining the HLSL team in October 2021, I had no experience with how Clang implements templates. This series is walking through my personal journey learning about how templates work in Clang and the experience of bringing a new language feature into HLSL.
Before HLSL 2021
HLSL does not have a formal specification (although my team and I are drafting one). We did not have a formal process for specifying or evolving HLSL until after HLSL 2021 was feature-complete. The HLSL-Specs process was a direct response to the challenges that came about during the implementation of HLSL 2021.
For many years HLSL has supported template syntax for built-in data types like the HLSL resource types or the matrix and vector types. The template syntax used for built-in types does not rely on C++’s rules for template instantiation. There is no SFINAE or partial specializations. Additionally, HLSL templates with default values for the template arguments can be specified without the empty template argument list that C++ requires. For example:
Texture2D MyTextureF4; // Default template type is `float4`.
// C++ would require writing the type as `Texture2D<>`.
Texture2D<float3> MyTextureF3; // Explicitly `float3`.
Because of the limited uses that HLSL had for templates, it’s more of an implementation detail that they are template types in the AST rather than similar-looking but different grammatical construct.
DXC
The reference compiler for HLSL is the DirectXShaderCompiler, or DXC. DXC is a fork of LLVM and Clang 3.7 stripped down to its core. It supports compiling HLSL code to generate either DirectX Intermediate Language (DXIL) or Standard Portable Intermediate Representation V (SPIR-V).
Many bits of functionality in Clang 3.7 (like C and C++ support) have been disabled in DXC, and parts of Clang have been repurposed to implement HLSL features rather than the C or C++ features they were originally implemented for. In DXC, HLSL is implemented as C++-based with unsupported C++ features removed.
I don’t have the historical context for many of the design decisions in DXC, but I imagine a pretty sound reasoning that goes like: HLSL isn’t C or C++, and it isn’t going to be. Since the HLSL compiler was initially closed source, and since the goal was to create a new HLSL compiler for a raytracing programming model the intent was to match the existing language not to imagine a future for it. As DXC evolved and the goal shifted to replacing FXC different priorities set in and expediency drove decisions.
LLVM & Clang are large codebases that produce large binaries. In the need for compact, fast compilers, pieces of LLVM and Clang were aggressively removed to shrink down and speed up the final binary. Other decisions (like inserting a Windows-specific filesystem abstraction) came logically from the closed source nature of the project origin.
With the benefit of hindsight it can be easy to pick apart individual decisions and question them, but after time working in the codebase I think the decisions are understandable and reasonable given the constraints and goals the team had at the time. None of this is meant to question or shed doubt on those decisions as they were made at the time, even if they haven’t all held to the test of time.
Enter HLSL 2021
HLSL 2021 introduced C++-style templates into the language. The initial implementation was performed by re-enabling previously disabled C++ supporting code paths in DXC. This approach works fine for user-defined types and basic types that are inherited from C, but there were a lot of edge cases that came up when templates were used in conjunction with other HLSL features.
Resolving these issues to make templates a production-ready feature required examining how HLSL features were implemented in DXC and extending, refactoring, or re-implementing those features to work with templates.