December 21, 2014
Hot Topics:

Writing Code that Writes Code

  • July 18, 2014
  • By Bill Hatfield
  • Send Email »
  • More Articles »

Microsoft’s T4 technology makes it possible - even fun - to write programs that, well, write programs! You tell the template how to generate the tedious code and it happily writes and maintains it for you.

When working on a mobile app project, I had a distinct need to automate the generation and maintenance of cookie-cutter code required for every screen. So I went out in search of ways to do it.

The first option I found was a Microsoft technology called CodeDOM (http://msdn.microsoft.com/en-us/library/ms404245(v=vs.110).aspx). This API provides a way for you to generate classes and members. For example the generator code to add a private double field named widthValue to an existing class looks like this.

CodeMemberField widthValueField = new CodeMemberField();
widthValueField.Attributes = MemberAttributes.Private;
widthValueField.Name = "widthValue";
widthValueField.Type = new CodeTypeReference(typeof(System.Double));
widthValueField.Comments.Add(
   new CodeCommentStatement("The width of the object."));
targetClass.Members.Add(widthValueField);

As you can see, it’s quite straightforward but also quite verbose. To generate standard project class would take mounds of code. I didn’t want a solution that was going to be more time-consuming than the original problem!

Fortunately a better technology was available: T4 Templates. T4 is short for “Text Template Transformation Toolkit.” It’s been around since 2005, and was first bundled together with Visual Studio in the 2008 release. Since then it’s been upgraded to improve performance and provide better integration with Visual Studio’s DSL tools.

Microsoft itself uses T4 under the covers to do the code generation work for technologies like ADO.NET Entity Framework, ASP.NET MVC view/controller generation and ASP.NET Dynamic Data.

T4 in Action

Here’s a hello-world-style T4 template example.

<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ output extension=".cs" #>
 
<#@ import namespace="System.Collections.Generic" #>
<# 
var Friends = new List<string>();
// Code here may retrieve friend names from the database and 
// put them into a friends array 
// For this example, we'll just add a few ourselves:
Friends.AddRange(new string[] {"Brad","Dan","Mike","Wayne","Mark"});
#>
using System;
 
class HelloFriends {
<#  foreach (var f in Friends) { #>
  public void Hello<#= f #>() {
     Console.WriteLine("Hello there, <#= f #>, my friend!");
  }
 
<# } #>
}

If you have seen ASP.NET MVC views, classic Active Server Pages, Java Server Pages or similar technologies – this kind of code may look familiar to you.

What you see is two levels of code – template code and output code. Template code is executed when the template is processed. This happens every time the template is modified and saved. (It can also be made to happen whenever the app is compiled.) So the template code is executed at design time to produce the output code. Then the output code executes at runtime.

What is the output code in this case? Glad you asked…

using System;
 
class HelloFriends {
  public void HelloBrad() {
     Console.WriteLine("Hello there, Brad, my friend!");
  }
 
  public void HelloDan() {
     Console.WriteLine("Hello there, Dan, my friend!");
  }
 
  public void HelloMike() {
     Console.WriteLine("Hello there, Mike, my friend!");
  }
 
  public void HelloWayne() {
     Console.WriteLine("Hello there, Wayne, my friend!");
  }
 
  public void HelloMark() {
     Console.WriteLine("Hello there, Mark, my friend!");
  }
 
}

The template code, often called control code, is always within the <# and #> directives and is always executed at design time. When it is executed, it generates the output file.

T4 Anatomy

The first two lines of the template typically look much like the first two lines here:

<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ output extension=".cs" #>

Note that these lines begin with a <#@, indicating they are directives. A directive provides information on how to process the template.

In the first line, a few attributes are set -- most importantly, language. This indicates the language that is used for the template code. Note that this may be different from the language used for the output. That is, you can use a C# template to produce VB code output, if you like. The extension used for the output file is indicated in the second line. So in this case, we’re using a C# template to produce C# output.

The third line is also a directive.

<#@ import namespace="System.Collections.Generic" #>

The import directive identifies a namespace you’d like to use in your template code, like an Imports in VB or a using in C#. Remember this is for the template code – not for the output code. You may have as many of these lines as you need.

Template code is often referred to as control code. That’s what the next section is.

<# 
var Friends = new List<string>();
// Code here may retrieve friend names from the database and 
// put them into a friends array 
// For this example, we'll just add a few ourselves:
Friends.AddRange(new string[] {"Brad","Dan","Mike","Wayne","Mark"});
#>

It is common to have a big block of control code at the top of your template. And usually this is where you retrieve the data you’ll use to generate this template. A template is a way of dynamically creating code, so it must be driven by dynamic data. That data may come from a database, an XML file or virtually any other data source. But whatever it is, you’ll likely need to do some housekeeping to access it and then retrieve the data you’ll use into a an array, list, dictionary or some other internal data structure. This is the spot to do all that.

As the comments indicate I’ve cheated a bit here to keep it simple. I fill a string array with a few friends’ names.

The next bit appears outside the <# and #> delimiters.

#>
using System;
 
class HelloFriends {
<#

Stuff that’s outside the delimiters are called text blocks. Text blocks are simply chunks that will be copied to the output file exactly as is. Here the output file begins with a using statement and then the first line of a class definition. Note that this using applies to the output file, as opposed to the <#@ import namespace…, discussed above, which applies to the template code.

The rest of the template contains a mix of control code and output text.

<#  foreach (var f in Friends) { #>
  public void Hello<#= f #>() {
     Console.WriteLine("Hello there, <#= f #>, my friend!");
  }
 
<# } #>
}

The foreach is within the delimiters, so at design time it loops through all the elements in the Friends list. Note the <# } #> near the end – that’s the bottom of the loop. Most of what’s within the loop is output text that will create a method definition for each friend. However there are a couple of special delimiters used within this text: <#= and #>. These are referred to as expression control blocks. They don’t contain code to execute but rather an expression to be evaluated. The expression will boil down to a single value that will replace the expression block in the final output. In both cases here, that expression is the current string – the name of the friend. This produces the following output for the first element of the list, “Brad”.

  public void HelloBrad() {
     Console.WriteLine("Hello there, Brad, my friend!");
  }

Tricks of the Trade

T4 templates are pretty straightforward, in concept. In practice, however, things can sometimes get confusing. After all, when you run into bugs they may be bugs in the template code or bugs in the output code. You’ll have to debug both at the same time.

Here are a few tips and tricks to make T4 development easier.

Add a Template

You can add a T4 template to any project. When you go to the Add New Item dialog you may or may not see an option for Text Template. If you do, it’ll likely look like this.

Add New Item
Add New Item

If not, don’t worry – you can simply choose Text File and then after it’s created, rename it with the .tt extension. Answer yes to the are-you-sure dialog and you’ve got an empty T4 template.

Get Off on the Right Foot

The first step with a new template is to modify the first lines, or enter them if you started with an empty template.

<#@ template debug="false" hostspecific="false" language="C#" #>
<#@ output extension=".cs" #>

Set the language for the template and the extension for the output file. Note that you can produce an output file with any extension, not just code. You can generate an XML file, a text file or nearly anything else.

Organize Your Template and Output Work

Visual Studio conveniently nests the output page under the template in Solution Explorer.

Solution Explorer
Solution Explorer

The best way I’ve found to see what’s happening when I develop a template is to use a vertically split screen in Visual Studio with the template on the left and the output file on the right.

Vertically Split Screen in Visual Studio
Vertically Split Screen in Visual Studio

As you modify the template, you can save at any point and when you do, the template will be re-run and the corresponding output will appear on the right.

Chase Bugs

When I said you’d be debugging template and output at the same time, I wasn’t kidding. Suppose you forget the semicolon at the end of the AddRange line in the Friends template. Here’s what you’ll see.

Error List
Error List

You get the error you’d expect, but you also get some other confusing errors. In addition, you’ll see that the output file looks like this.

Output File
Output File

If the template code fails, often the template can’t be generated at all. Especially if the failure happens in that first housekeeping/data gathering section. In that case you get a single line as you see here. Notice that the third error is in Friends.cs. Despite failing the template, the compiler still tries to process the output file and finds that single line, thus the #3 error. You’ll see this every time your template fails before generating anything.

But this is just the simplest possible case. If you have an error in an expression or other code that’s mixed up with the output, you may get syntax or other errors in the output. Always focus first on the errors you get in the template. When you get those fixed, the output errors often disappear.

Upgrade Your Editor

You may have noticed that the Visual Studio editor’s support for T4 isn’t what you’d call rich. Even after four releases, T4 just hasn’t gotten the love it needs. Fortunately free third-party add-ons make up the difference.

I use the free Tangible T4 Editor (http://t4-editor.tangible-engineering.com/T4-Editor-Visual-T4-Editing.html). It provides very helpful color coding, highlighting, syntax validation and more.

Tangible T4 Editor
Tangible T4 Editor

I haven’t evaluated it, but another big name for T4 editors and other extensions is Devart (http://www.devart.com/t4-editor/).

Generate at Compile Time

As you’ve seen, templates are re-run every time you make a change and save the template. But since the data source that drives the template is dynamic, you may end up wishing you could trigger a regen at compile time. For some reason, this simple and common requirement isn’t satisfied out-of-the-box. Fortunately, you don’t have to look far to find ways to make it happen.

The Tangible T4 Editor I mentioned above provides a property on each template file named TranformOnBuild. Unfortunately the feature only works if you buy the Pro version of their product (which provides many additional features for a reasonable $130). But for this problem, there’s no reason to pay.

A Google search will turn up solutions you can implement without an add-on that involve modifying your build process to include scripting or batch file code that runs through your project looking for all .tt files and running TextTransform.exe on them.

If, like me, you’re looking for a more install-and-forget solution, you may want to check out the free, open-source Clarius TransformOnBuild  NuGet package (https://github.com/clariuslabs/TransformOnBuild/blob/master/README.md). It worked like a charm for me.

Or, if you prefer, there are many other third-party options that a Google search will turn up.

Conclusion

Code generation can seem like an intimidating prospect at first. It is, virtually by definition, an advanced programming topic. However Microsoft has made big strides in lowering the bar so that simple needs can be met simply and more complex projects are manageable. In this article you got a detailed description of a simple example and a number of real-world tips and tricks to smooth your learning curve with this exciting new technology!


Tags: ASP.NET, database, Microsoft, templates, ADO.NET, T4




Comment and Contribute

 


(Maximum characters: 1200). You have characters left.

 

 


Enterprise Development Update

Don't miss an article. Subscribe to our newsletter below.

Sitemap | Contact Us

Rocket Fuel