Mixins

Motivation

Almost from start of the development Ü has reach metaprogramming mechanisms - templates, static_if, enable_if, type inspection via typeinfo and powerful (but limited) compile-time code execution. But even with such possibilities Ü had some fundamental limitations, which limit some metaprogramming needs. Some of these limitations:

Reference and mutability modifiers of function parameters, return values, struct fields, variables were fixed, even in template code. It wasn’t possible for example to declare a field with its mutability depending on template arguments. But some of these limitations were possible to get around via type templates overloading or enable_if.
It wasn’t possible to create a template method or any function with number of parameters calculated in compilation time. It was possible to create many overloads with different number of parameters, but it wasn’t practical.
It wasn’t possible to create a struct with variable number of fields. All fields were fixed even in templates, no number of files and their names may be configured.
Name lookup was fixed. It wasn’t possible (for example) to call a method or to access a field of a struct if its name isn’t statically known.

These limitations were not so painful, but sometimes forced to write code differently or create too much boilerplate. Because of that i searched for a solution, which allows to solve all these problems.

In search for a solution

Initially i thought to make reference/mutability modifiers syntax more complex - add some constexpr parameter for them in order to make mutability/reference modifier conditional. This might work, but it seems to be too complicated to implement and too boilerplate to use.

Also i have thoughts to implement (somehow) template functions with variable number of parameters. But this idea is also hard to implement and to use.

All these solutions are too partial and can’t solve all problems. Thus i decided to find a perfect solution - which allows to solve all of them.

The most stupid and powerful solution

I decided to investigate, how other programming languages solve such inflexibility. I have found, that the Zig language uses some sort of compile-time code generation for polymorphism, where languages like C++ and Rust use templates. The D language also provides a mechanism to generate code from compile-time generated string, this mechanism is named mixin. After some thinking i decided, that the D solution is almost perfect and is worth to implement it in Ü.

Mixins basics

Generally mixins work pretty simple. Mixin expression is evaluated (in compile time) and given text is parsed as normal Ü program or its fragment. Than parsed syntax elements are added in the place of the mixin usage.

Examples:

mixin( "fn Foo() : i32 { return 42; }" ); // Add a function via mixin.

struct S
{
    mixin( fields ); // Add struct fields using mixin.
    auto& fields= "i32 x; f32 y;";
}

fn Bar()
{
    Foo(); // Call a mixin-generated function.
    var S mut s= zero_init;
    mixin( "s.x= 1; s.y= 0.5f;" ); // Add mixin block elements.
    auto sum= f32(mixin("s.x")) + mixin("s.y"); // Use mixins in expression context.
}

Mixins implementation wasn’t so trivial. For mixins in namespace and class context it was needed to implement mixins evaluation and expansion as separate compilation step - between the names table preparation and later code building. For mixins in block and expression context it’s simpler - result syntax elements may be used directly to build block elements or an expression. Additional complication adds the necessity to parse mixin text properly, using macros available only in the file, where this mixin is used.

Results

After implementing mixins i managed to implement some standard library code, which wasn’t so easy to implement earlier.

Example #1 - homogeneous_tuple

Prior to mixins introduction it was the only way to implement homogeneous_tuple type alias - using type templates overloading.

template</type T/> type homogeneous_tuple</T, 0s/> = tup[];
template</type T/> type homogeneous_tuple</T, 1s/> = tup[ T ];
template</type T/> type homogeneous_tuple</T, 2s/> = tup[ T, T ];
template</type T/> type homogeneous_tuple</T, 3s/> = tup[ T, T, T ];
template</type T/> type homogeneous_tuple</T, 4s/> = tup[ T, T, T, T ];

Now mixins may be used instead, or at least for cases where writing a lot of overloads isn’t practical:

template</type T, size_type size/>
type homogeneous_tuple</T, size/> = mixin( homogeneous_tuple_gen</size/>() );

// Generate a tuple type name like "tup[T,T,T,T,T]"
template</size_type size/>
fn constexpr homogeneous_tuple_gen() : [ char8, 5s + 2s * size ]
{
	// implementation details
}

The second solution isn’t so boilerplate and isn’t limited by size, as manually-written type alias overloads.

Example #2 - zip iterator method.

This method creates zip_iterator, which iterates over two sequences simultaneously and returns pairs of values. Such pairs are represented via a template struct with two fields - first and second. This wasn’t possible to do prior to mixins introduction, since such fields may be values - if underlying iterator produces values, or mutable/immutable references. But with usage of mixins i managed to write such struct template - a mixin is used to generate both these fields.

Possible examples of this struct (after mixin expansion):

// Both results are values.
struct zip_iterator_pair
{
    FirstType first;
    SecondType second;
}

// Both results are immutable references.
struct zip_iterator_pair
{
    FirstType &@("a"c8) first;
    SecondType &@("b"c8) second;
}

// One mutable and one immutable reference.
struct zip_iterator_pair
{
    FirstType &mut @("a"c8) first;
    SecondType &@("b"c8) second;
}

Mixins possibilities

Since code in mixins is generated by another code, anything may be generated. Some examples, what is now possible with mixins:

Accessing arbitrary struct fields in template code. For example - for serialization/deserialization.
Implementing something like std::function from C++ - a template class with () method, which type (number of arguments, their types and value-types) depends on template arguments.
Implementing strings interpolation - generating strings based on a template source string, which directly references variables within its context.
Struct of arrays template - create a struct with arrays for each field of given input struct.
Implementing alternative domain-specific languages within Ü - formatting, regular expressions, SQL, etc.

Mixins disadvantages

Mixins aren’t so ideal. There are some problems with them.

constexpr calculations, which are used to generate mixins, are somewhat limited. For now they do not support unsafe code, including memory allocation and thus almost all standard library containers. This makes writing mixins generation code a little bit hard.

It’s hard to debug compile-time calculations, there is no such thing as debugger for them. The only debug tools are halt and static_assert.

Code produced via mixins is also hard to debug, since there is no location in any source file directly corresponding to mixin code, which may be shown during debugging. It’s even problematic to fix compilation errors within mixin code, since full mixin code can’t be shown in error messages.

So, considering all the problems mentioning above i suggest to limit usage of mixins. If regular template code may be used, it should be used instead of mixins. If code generation is too complicated or slows-down compilation, it may be better to use some external tool for this - some script or proper generator program (like fir GRPC code generation).