Integration Testing with Entity Framework and Snapshot Backups

I've written before about how unit testing Entity Framework is a contentious and sometimes pointless activity. The TL;DR is that LINQ-to-Objects != Linq-to-Entities and so if you want some useful tests around your data tier then integration tests that actually hit a database are what you want.

However hitting an actual database is has serious implications. For a start you need a database server and you need a database. But the real issue lies around cleanup. When you write a test that amends data in the database you need the test to clean up after itself. If it doesn't then the next test that runs may trip over the amended data and that's your test pack instantly useless.

What you want is a way to wipe the slate clean - to return the database back to the state that it was in before your test ran. Kind of like a database restore - except that would be slow. And this is where SQL Server's snapshot backups have got your back. To quote MSDN:

Snapshot backups have the following primary benefits:

• A backup can be created quickly, typically measured in seconds, with little or no effect on the server.
• A restore operation can be accomplished from a disk backup just as quickly.
• Backup to tape can be accomplished by another host without an effect on the production system.
• A copy of a production database can be created instantly for reporting or testing.

Just the ticket.

Our Mission

In this post I want to go through the process of taking an existing database, pointing Entity Framework at it, setting up some repositories and then creating an integration test pack that uses snapshot backups to cleanup after each test runs. The code detailed in this post is available in this GitHub repo if you want to have a go yourself.

We need a database

You can find a whole assortment of databases here. I'm going to use AdventureWorksLT as it's small and simple. So I'll download this and unzip it. I'll drop AdventureWorksLT2008R2_Data.mdf and AdventureWorksLT2008R2_log.LDF in my data folder and attach AdventureWorksLT2008R2 to my database server. And now I have a database:

Assemble me your finest DbContext

Or in English: we want to point Entity Framework at our new shiny database. So let's fire up Visual Studio (I'm using 2013) and create a new solution called "AdventureWorks".

To our solution let's add a new class library project called "AdventureWorks.EntityFramework". And to that we'll add an ADO.NET Entity Data Model which we'll call "AdventureWorks.edmx". When the wizard fires up we'll use the "Generate from database" option, click Next and select "New Connection". In the dialog we'll select our newly attached AdventureWorksLT2008R2 database. We'll leave the "save entity connection settings in App.Config" option selected and click Next. I'm going to use Entity Framework 6.0 - though I think that any version would do. I'm going to pull in all tables / store procs and views. And now Entity Framework is pointing at my database:

Let There be Repositories!

In the name of testability let's create a new project to house repositories called "AdventureWorks.Repositories". I'm going to use K. Scott Allen's fine article on MSDN to create a very basic set of repositories wrapped in a unit of work.

In my new project I'll add a reference to the AdventureWorks.EntityFramework project and create a new IRepository interface that looks like this:

using System;
using System.Linq;
using System.Linq.Expressions;

namespace AdventureWorks.Repositories
{
    public interface IRepository<T> where T : class
    {
        IQueryable<T> FindAll();
        IQueryable<T> FindWhere(Expression<Func<T, bool>> predicate);
        T Add(T newEntity);
        T Remove(T entity);
    }
}

And a new IUnitOfWork interface that looks like this:

using AdventureWorks.EntityFramework;

namespace AdventureWorks.Repositories
{
    public interface IUnitOfWork
    {
        public IRepository<ErrorLog> ErrorLogs { get; }
        public IRepository<Address> Addresses { get; }
        public IRepository<Customer> Customers { get; }
        public IRepository<CustomerAddress> CustomerAddresses { get; }
        public IRepository<Product> Products { get; }
        public IRepository<ProductCategory> ProductCategories { get; }
        public IRepository<ProductDescription> ProductDescriptions { get; }
        public IRepository<ProductModel> ProductModels { get; }
        public IRepository<ProductModelProductDescription> ProductModelProductDescriptions { get; }
        public IRepository<SalesOrderDetail> SalesOrderDetails { get; }
        public IRepository<SalesOrderHeader> SalesOrderHeaders { get; }
        public IRepository<BuildVersion> BuildVersions { get; }

        void Commit();
    }
}

Now for the implementation of IRepository. For this we'll need a reference to Entity Framework in our project. Then we'll create a class called SqlRepository:

using System;
using System.Data.Entity;
using System.Linq;
using System.Linq.Expressions;

namespace AdventureWorks.Repositories
{
    public class SqlRepository<T> : IRepository<T> where T : class
    {
        public SqlRepository(DbContext context)
        {
            _dbSet = context.Set<T>();
        }

        public IQueryable<T> FindAll()
        {
            return _dbSet;
        }

        public IQueryable<T> FindWhere(Expression<Func<T, bool>> predicate)
        {
            return _dbSet.Where(predicate);
        }

        public T Add(T newEntity)
        {
            return _dbSet.Add(newEntity);
        }

        public T Remove(T entity)
        {
            return _dbSet.Remove(entity);
        }

        protected DbSet<T> _dbSet;
    }
}

And we also need the implementation of IUnitOfWork. So we'll create a class called SqlUnitOfWork:

using System;
using System.Linq;
using System.Data.Entity;
using AdventureWorks.EntityFramework;

namespace AdventureWorks.Repositories
{
    public class SqlUnitOfWork : IUnitOfWork
    {
        public SqlUnitOfWork()
        {
            _context = new AdventureWorksLT2008R2Entities();
        }

        public IRepository<ErrorLog> ErrorLogs
        {
            get
            {
                if (_errorLogs == null) _errorLogs = new SqlRepository<ErrorLog>(_context);
                return _errorLogs;
            }
        }

        public IRepository<Address> Addresses
        {
            get
            {
                if (_addresses == null) _addresses = new SqlRepository<Address>(_context);
                return _addresses;
            }
        }

        public IRepository<Customer> Customers
        {
            get
            {
                if (_customers == null) _customers = new SqlRepository<Customer>(_context);
                return _customers;
            }
        }

        public IRepository<CustomerAddress> CustomerAddresses
        {
            get
            {
                if (_customerAddresses == null) _customerAddresses = new SqlRepository<CustomerAddress>(_context);
                return _customerAddresses;
            }
        }

        public IRepository<Product> Products
        {
            get
            {
                if (_products == null) _products = new SqlRepository<Product>(_context);
                return _products;
            }
        }

        public IRepository<ProductCategory> ProductCategories
        {
            get
            {
                if (_productCategories == null) _productCategories = new SqlRepository<ProductCategory>(_context);
                return _productCategories;
            }
        }

        public IRepository<ProductDescription> ProductDescriptions
        {
            get
            {
                if (_productDescriptions == null) _productDescriptions = new SqlRepository<ProductDescription>(_context);
                return _productDescriptions;
            }
        }

        public IRepository<ProductModel> ProductModels
        {
            get
            {
                if (_productModels == null) _productModels = new SqlRepository<ProductModel>(_context);
                return _productModels;
            }
        }

        public IRepository<ProductModelProductDescription> ProductModelProductDescriptions
        {
            get
            {
                if (_productModelProductDescriptions == null) _productModelProductDescriptions = new SqlRepository<ProductModelProductDescription>(_context);
                return _productModelProductDescriptions;
            }
        }

        public IRepository<SalesOrderDetail> SalesOrderDetails
        {
            get
            {
                if (_salesOrderDetails == null) _salesOrderDetails = new SqlRepository<SalesOrderDetail>(_context);
                return _salesOrderDetails;
            }
        }

        public IRepository<SalesOrderHeader> SalesOrderHeaders
        {
            get
            {
                if (_salesOrderHeaders == null) _salesOrderHeaders = new SqlRepository<SalesOrderHeader>(_context);
                return _salesOrderHeaders;
            }
        }

        public IRepository<BuildVersion> BuildVersions
        {
            get
            {
                if (_buildVersions == null) _buildVersions = new SqlRepository<BuildVersion>(_context);
                return _buildVersions;
            }
        }

        public void Commit()
        {
            _context.SaveChanges();
        }

        SqlRepository<ErrorLog> _errorLogs = null;
        SqlRepository<Address> _addresses = null;
        SqlRepository<Customer> _customers = null;
        SqlRepository<CustomerAddress> _customerAddresses = null;
        SqlRepository<Product> _products = null;
        SqlRepository<ProductCategory> _productCategories = null;
        SqlRepository<ProductDescription> _productDescriptions = null;
        SqlRepository<ProductModel> _productModels = null;
        SqlRepository<ProductModelProductDescription> _productModelProductDescriptions = null;
        SqlRepository<SalesOrderDetail> _salesOrderDetails = null;
        SqlRepository<SalesOrderHeader> _salesOrderHeaders = null;
        SqlRepository<BuildVersion> _buildVersions = null;

        readonly DbContext _context;
    }
}

And Now Let's Start Integration Testing!

Let's create a new Unit Test project called "AdventureWorks.Repositories.IntegrationTests". (And just to be clear: this is *not* a unit test project - it is an integration test project.) We'll add a reference back to our AdventureWorks.Repositories project for the repositories and one back to AdventureWorks.EntityFramework for our domain models. And finally you'll need a reference to Entity Framework in your IntegrationTest project as well as well.

We'll copy across the app.config from AdventureWorks.EntityFramework to AdventureWorks.Repositories.IntegrationTests as it contains the database connection details. It'll look something like this:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <configSections>
        <section name="entityFramework" type="System.Data.Entity.Internal.ConfigFile.EntityFrameworkSection, EntityFramework, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" requirePermission="false" />
        <!-- For more information on Entity Framework configuration, visit http://go.microsoft.com/fwlink/?LinkID=237468 -->
    </configSections>
    <connectionStrings>
        <add name="AdventureWorksLT2008R2Entities" 
             connectionString="metadata=res://*/AdventureWorks.csdl|res://*/AdventureWorks.ssdl|res://*/AdventureWorks.msl;provider=System.Data.SqlClient;provider connection string=&quot;data source=.;initial catalog=AdventureWorksLT2008R2;integrated security=True;MultipleActiveResultSets=True;App=EntityFramework&quot;" 
             providerName="System.Data.EntityClient" />
    </connectionStrings>
    <entityFramework>
        <defaultConnectionFactory type="System.Data.Entity.Infrastructure.SqlConnectionFactory, EntityFramework" />
        <providers>
            <provider invariantName="System.Data.SqlClient" type="System.Data.Entity.SqlServer.SqlProviderServices, EntityFramework.SqlServer" />
        </providers>
    </entityFramework>
</configuration>

Now we're ready for a test. We'll add ourselves a class called BuildVersionTests:

using System;
using System.Linq;
using System.Linq.Expressions;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace AdventureWorks.Repositories.IntegrationTests
{
    [TestClass]
    public class BuildVersionTests
    {
        [TestMethod]
        public void BuildVersions_should_return_the_correct_version_information()
        {
            // Arrange
            var uow = new SqlUnitOfWork();

            // Act
            var buildVersions = uow.BuildVersions.FindAll().ToList();

            // Assert
            Assert.AreEqual(1, buildVersions.Count);
            Assert.AreEqual("10.00.80404.00", buildVersions[0].Database_Version);
            Assert.AreEqual(new DateTime(2008, 4, 4), buildVersions[0].ModifiedDate);
            Assert.AreEqual(1, buildVersions[0].SystemInformationID);
            Assert.AreEqual(new DateTime(2008, 4, 4), buildVersions[0].VersionDate);
        }
    }
}

This is as simple as it gets - our test creates a new unit of work and queries the BuildVersions table to see what we can see. All it's really doing is demonstrating that we can now hit our database through our repositories. As a side note, we could have the exact same test operating directly on the DbContext like this:

        [TestMethod]
        public void DbContext_BuildVersions_should_return_the_correct_version_information()
        {
            // Arrange
            var dbContext = new AdventureWorks.EntityFramework.AdventureWorksLT2008R2Entities();

            // Act
            var buildVersions = dbContext.BuildVersions.ToList();

            // Assert
            Assert.AreEqual(1, buildVersions.Count);
            Assert.AreEqual("10.00.80404.00", buildVersions[0].Database_Version);
            Assert.AreEqual(new DateTime(2008, 4, 4), buildVersions[0].ModifiedDate);
            Assert.AreEqual(1, buildVersions[0].SystemInformationID);
            Assert.AreEqual(new DateTime(2008, 4, 4), buildVersions[0].VersionDate);
        }

For the most part we won't be doing this but I wanted to be clear that full power of Entity Framework is available to you as you're putting together your integration tests.

Database Snapshotting Time

Up until this point we've essentially been laying our infrastructure and doing our plumbing. We now have a database, domain models and data access courtesy of Entity Framework, a testable repository layer and finally an integration test pack. What we want now is to get our database snapshot / backup and restore mechanism set up and integrated into the test pack.

Let's add references to the System.Data and System.Configuration assemblies to our integration testing project and then add a new class called DatabaseSnapshot:

using System.Configuration;
using System.Data;
using System.Data.SqlClient;

namespace AdventureWorks.Repositories.IntegrationTests
{
    public static class DatabaseSnapshot
    {
        private const string SpCreateSnapShotName = "SnapshotBackup_Create";
        private const string SpCreateSnapShot =
@"CREATE PROCEDURE [dbo].[" + SpCreateSnapShotName + @"]
    @databaseName        varchar(512),
    @databaseLogicalName varchar(512),
    @snapshotBackupPath  varchar(512),
    @snapshotBackupName  varchar(512)
AS
BEGIN
    SET NOCOUNT ON;

    DECLARE @sql varchar(500)
    SELECT @sql = 'CREATE DATABASE ' + @snapshotBackupName +
                  ' ON (NAME=[' + @databaseLogicalName +
                  '], FILENAME=''' + @snapshotBackupPath + @snapshotBackupName + 
                  ''') AS SNAPSHOT OF [' + @databaseName + ']'
    EXEC(@sql)
END";

        private const string SpRestoreSnapShotName = "SnapshotBackup_Restore";
        private const string SpRestoreSnapShot =
@"CREATE PROCEDURE [dbo].[" + SpRestoreSnapShotName + @"]    
    @databaseName varchar(512),    
    @snapshotBackupName varchar(512)    
AS    
BEGIN    
    SET NOCOUNT ON;

    DECLARE @sql varchar(500)
    SET @sql  = 'ALTER DATABASE [' + @databaseName + '] SET SINGLE_USER WITH ROLLBACK IMMEDIATE'
    EXEC (@sql)

    RESTORE DATABASE @databaseName
    FROM DATABASE_SNAPSHOT = @snapshotBackupName

    SET @sql = 'ALTER DATABASE [' + @databaseName + '] SET MULTI_USER'
    EXEC (@sql)
END";

        private const string SpDeleteSnapShotName = "SnapshotBackup_Delete";
        private const string SpDeleteSnapShot =
@"CREATE PROCEDURE [dbo].[" + SpDeleteSnapShotName + @"]
    @snapshotBackupName varchar(512)
AS
BEGIN
    SET NOCOUNT ON;

    DECLARE @sql varchar(500)

    SELECT @sql = 'DROP DATABASE ' + @snapshotBackupName 
    EXEC(@sql) 
END";

        private static string _masterDbConnectionString;
        private static string _dbName;
        private static ConnectionStringSettings _dbConnectionStringSettings;

        private static ConnectionStringSettings DbConnectionStringSettings
        {
            get
            {
                if (_dbConnectionStringSettings == null)
                    _dbConnectionStringSettings = ConfigurationManager.ConnectionStrings["SnapshotBackup"];

                return _dbConnectionStringSettings;
            }
        }

        /// <summary>
        /// Stored procedures should be executed against master database
        /// </summary>
        private static string MasterDbConnectionString
        {
            get
            {
                if (string.IsNullOrEmpty(_masterDbConnectionString))
                {
                    var sqlConnection = new SqlConnection(DbConnectionStringSettings.ConnectionString);
                    _masterDbConnectionString = DbConnectionStringSettings.ConnectionString.Replace(sqlConnection.Database, "master");
                }
                return _masterDbConnectionString;
            }
        }

        private static string DbName
        {
            get
            {
                if (string.IsNullOrEmpty(_dbName))
                    _dbName = new SqlConnection(DbConnectionStringSettings.ConnectionString).Database.TrimStart('[').TrimEnd(']');

                return _dbName;
            }
        }

        public static void SetupStoredProcedures()
        {
            using (var conn = new SqlConnection(MasterDbConnectionString))
            {
                conn.Open();

                // Drop the existing stored procedures
                SqlCommand cmd;
                const string dropProcSql = "IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[{0}]') AND type in (N'P', N'PC')) DROP PROCEDURE [dbo].[{0}]";
                foreach (var spName in new[] { SpCreateSnapShotName, SpDeleteSnapShotName, SpRestoreSnapShotName })
                {
                    cmd = new SqlCommand(string.Format(dropProcSql, spName), conn);
                    cmd.ExecuteNonQuery();
                }

                // Create the stored procedures anew
                foreach (var createProcSql in new[] { SpCreateSnapShot, SpDeleteSnapShot, SpRestoreSnapShot })
                {
                    cmd = new SqlCommand(createProcSql, conn);
                    cmd.ExecuteNonQuery();
                }

                conn.Close();
            }
        }

        public static void CreateSnapShot()
        {
            var databaseName = new SqlParameter { ParameterName = "@databaseName", SqlValue = SqlDbType.VarChar, Value = DbName };
            var databaseLogicalName = new SqlParameter { ParameterName = "@databaseLogicalName", SqlValue = SqlDbType.VarChar, Value = ConfigurationManager.AppSettings["DatabaseLogicalName"] };
            var snapshotBackupPath = new SqlParameter { ParameterName = "@snapshotBackupPath", SqlValue = SqlDbType.VarChar, Value = ConfigurationManager.AppSettings["SnapshotBackupPath"] };
            var snapshotBackupName = new SqlParameter { ParameterName = "@snapshotBackupName", SqlValue = SqlDbType.VarChar, Value = ConfigurationManager.AppSettings["SnapshotBackupName"] };

            ExecuteStoredProcAgainstMaster(SpCreateSnapShotName, new[] { databaseName, databaseLogicalName, snapshotBackupPath, snapshotBackupName });
        }

        public static void DeleteSnapShot()
        {
            var snapshotBackupName = new SqlParameter { ParameterName = "@snapshotBackupName", SqlValue = SqlDbType.VarChar, Value = ConfigurationManager.AppSettings["SnapshotBackupName"] };

            ExecuteStoredProcAgainstMaster(SpDeleteSnapShotName, new[] { snapshotBackupName });
        }

        public static void RestoreSnapShot()
        {
            var databaseName = new SqlParameter { ParameterName = "@databaseName", SqlValue = SqlDbType.VarChar, Value = DbName };
            var snapshotBackupName = new SqlParameter { ParameterName = "@snapshotBackupName", SqlValue = SqlDbType.VarChar, Value = ConfigurationManager.AppSettings["SnapshotBackupName"] };

            ExecuteStoredProcAgainstMaster(SpRestoreSnapShotName, new[] { databaseName, snapshotBackupName });
        }

        private static void ExecuteStoredProcAgainstMaster(string storedProc, SqlParameter[] parameters)
        {
            using (var conn = new SqlConnection(MasterDbConnectionString))
            {
                conn.Open();
                var cmd = new SqlCommand(storedProc, conn) { CommandType = CommandType.StoredProcedure };
                cmd.Parameters.AddRange(parameters);
                cmd.ExecuteNonQuery();
                conn.Close();
            }
        }
    }
}

The DatabaseSnapshot class exposes 4 methods:

SetupStoredProcedures

This method creates 3 stored procedures on the master database: SnapshotBackup_Create, SnapshotBackup_Restore and SnapshotBackup_Delete. These procs do pretty much what their names suggest and the other 3 methods call these stored procedures when creating, restoring and deleting snapshot backups respectively. You can see the (fairly minimal) SQL for these stored procs at the top of theDatabaseSnapshot class.

CreateSnapShot

This method creates a snapshot backup of the database at this point in time.

RestoreSnapShot

This method restores the database back to state it was in when the snapshot backup was created.

DeleteSnapShot

This method attempts to delete the existing snapshot backup.

In order that we can use the DatabaseSnapshot class we need to add the following entries to our app.config:

<configuration>

    <connectionStrings>

        <add name="SnapshotBackup"
             connectionString="data source=.;initial catalog=AdventureWorksLT2008R2;Trusted_Connection=true;Connection Timeout=200" />

    </connectionStrings>

    <appSettings>
        <add key="DatabaseLogicalName" value="AdventureWorksLT2008_Data" />
        <add key="SnapshotBackupPath" value="C:\DbSnapshots\" />
        <add key="SnapshotBackupName" value="AdventureWorksLT2008R2_Snapshot" />
    </appSettings>
</configuration>

These settings allow have the following purposes:

SnapshotBackup

A connection string that allows DatabaseSnapshot to connect to the database.

DatabaseLogicalName

The logical name of the database you want to backup. (This can be found on the Files tab of the Database Properties in SSMS)

SnapshotBackupPath

The location where the snapshot backup is to be stored. You need to make sure that this exists on your machine.

SnapshotBackupName

The name of the snapshot backup that will be created.

Now to make use of DatabaseSnapshot. Let's add a new class called SetUpTearDown:

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace AdventureWorks.Repositories.IntegrationTests
{
    [TestClass]
    public static class SetUpTearDown
    {
        [AssemblyInitialize]
        public static void TestRunInitialize(TestContext context)
        {
            try
            {
                // Try to delete the snapshot in case it was left over from aborted test runs
                DatabaseSnapshot.DeleteSnapShot();
            }
            catch { /* this should fail with snapshot does not exist */ }

            DatabaseSnapshot.SetupStoredProcedures();
            DatabaseSnapshot.CreateSnapShot();
        }


        [AssemblyCleanup]
        public static void TestRunCleanup()
        {
            DatabaseSnapshot.DeleteSnapShot();
        }
    }
}

At the start of the test run this will create a snapshot in case one doesn't exist already. And at the end of the test run it will be a good citizen and delete the snapshot. We'll also add an extra method to our BuildVersionTests class:

namespace AdventureWorks.Repositories.IntegrationTests
{
    [TestClass]
    public class BuildVersionTests
    {
        // ... 

        [TestCleanup]
        public void TestCleanup()
        {
            DatabaseSnapshot.RestoreSnapShot();
        }
    }
}

This will ensure that after each test runs the database will be restored back to the snapshot created in SetUpTearDown. Now if you re-run your tests, in between each test the restore back to the snapshot is taking place.

Prove it!

Of course the tests we have in place at present don't actually change the data at all. So I could be lying. I'm not. Let's prove it by adding one more class called CustomerTests:

using System;
using System.Linq;
using System.Linq.Expressions;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using AdventureWorks.EntityFramework;

namespace AdventureWorks.Repositories.IntegrationTests
{
    [TestClass]
    public class CustomerTests
    {
        [TestMethod]
        public void Should_change_a_customers_first_and_last_name()
        {
            // Arrange
            var uow = new SqlUnitOfWork();

            // Act
            var customer = uow.Customers.FindWhere(x => x.FirstName == "Jay" && x.LastName == "Adams").First();
            var customerId = customer.CustomerID;
            customer.FirstName = "John";
            customer.LastName = "Reilly";
            uow.Commit();

            // Assert
            Assert.IsNotNull(uow.Customers.FindWhere(x => x.FirstName == "John" && x.LastName == "Reilly" && x.CustomerID == customerId).SingleOrDefault());
        }

        [TestCleanup]
        public void TestCleanup()
        {
            DatabaseSnapshot.RestoreSnapShot();
        }
    }
}

The above test checks that you can look up an existing customer, Mr Jay Adams, and change his name to my name - to John Reilly. If I execute the test above and there was no restore in place then subsequently when I came to exercise this test it should start to fail as it no longer has a Mr Jay Adams to lookup. But with this restore mechanism in place I can execute this test repeatedly without worrying.

Rounding off

And that's us finished - we now have a database snapshot restore mechanism in place. With this we can develop integration tests that thoroughly change the data in our database secure in the knowledge that once the test is complete our database will be restored back to it's initial state.

Obviously there are other alternative approaches for integration testing available to that which I've laid out in this post. But I can imagine that this approach is very useful for applying to legacy applications that you might inherit and need to continue supporting. Also, this approach should fit in well with a continuous integration setup. It would be pretty straightforward to have database that existed purely for testing purposes against which all the integration tests could be set to run at the point of each check in.

Thanks to Marc Talary, Sandeep Deo and Tishul Vadher who all contributed to DatabaseSnapshot. Credit is also due to Google due to the hundreds of articles the team ended up reading on snapshot backups.

Upgrading to TypeScript 0.9.5 - A Personal Memoir

I recently made the step to upgrade from TypeScript 0.9.1.1 to 0.9.5. To my surprise this process was rather painful and certainly not an unalloyed pleasure. Since I'm now on the other side, so to speak, I thought I'd share my experience and cast back a rope bridge to those about to journey over the abyss.

TL;DR

TypeScript 0.9.5 is worth making the jump to. However, if you are using Visual Studio (as I would guess many are) then you should be aware of a number of problems with the TypeScript Visual Studio tooling for TS 0.9.5. These problems can be worked around if you follow the instructions in this post.

Upgrading the Plugin

At home I upgraded the moment TS 0.9.5 was released. This allowed me to help with migrating the Definitely Typed typings over from 0.9.1.1. And allowed me to give TS 0.9.5 a little test drive. However, I deliberately held off performing the upgrade at work until I knew that all the Definitely Typed typings had been upgraded. This was completed by the end of 2013. So in the new year it seemed a good time to make the move.

If, like me, you are using TypeScript inside Visual Studio then you'd imagine it's as simple as closing down VS, uninstalling TypeScript 0.9.1.1 from Programs and Features and then installing the new plugin. And it is if you are running IE 10 or IE 11 on your Windows machine. If you are running a lower IE version then there is a problem.

Regrettably, the TypeScript 0.9.5 plugin installer has a dependency on IE 10. Fortunately TypeScript itself has no dependency on IE 10 at all (and why would it?). This dependency appears to have been a mistake. I raised it as an issue and the TS team have said that this will be resolved in the next major release.

Happily there is a workaround if you're running IE 9 or lower which has been noted in the comments underneath the TS 0.9.5 release blog post. All you do is set the HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Internet Explorer\svcVersion registry key value to 10.0.9200.16384 for the duration of the install.

First hurdle jumped, the upgrade continues simple enough. Then the fun starts...

Declaration Merging is dead... Sort of

Having upgraded my plugin I opened up the project I'm working on in Visual Studio. I used NuGet to upgrade all the Definitely Typed packages to the latest (TS 0.9.5) versions. Then I tried, and failed, to compile. It was the the most obscure error I've seen in a while:

VSTSC : tsc.js(37574, 25) Microsoft JScript runtime error : Unable to get value of the property 'wrapsSomeTypeParameter': object is null or undefined

As you can see there was no indication where in my code the problem was being caused. Fortunately someone had already suffered this particular problem and logged an issue here. Digging through the comments I found a common theme; everyone experiencing the problem was using the Q typings. So what's up with that?

Strangely, if you directly referenced the Q typings everything was okay - which is how the Definitely Typed tests came to pass in the first place. But if you wanted to make use of these typings with implicit referencing (in Visual Studio since TS 0.9.1, all TypeScript files in a project are considered to be referencing each other) - well it doesn't work.

I decided to take a look at the Q typings at this point to see what was so upsetting about them. The one thing that was obvious was that these typings make use of Declaration Merging. And this made them slightly different to most of the other typing libraries that I was using. So I decided to refactor the Q typings to use the more interface driven approach the other typing libraries used in the hope that might resolve the issue.

Roughly speaking I went from:

declare function Q<T>(promise: Q.IPromise<T>): Q.Promise<T>;
declare function Q<T>(promise: JQueryPromise<T>): Q.Promise<T>;
declare function Q<T>(value: T): Q.Promise<T>;

declare module Q {
    //… functions etc in here
}

declare module "q" {
    export = Q;
}

To:

interface QIPromise<T> {
    //… functions etc in here
}
 
interface QDeferred<T> {
    //… functions etc in here
}
 
interface QPromise<T> {
    //… functions etc in here
}
 
interface QPromiseState<T> {
    //… functions etc in here
}
 
interface QStatic {
 
    (promise: QIPromise<T>): QPromise<T>;
    (promise: JQueryPromise<T>): QPromise<T>;
    (value: T): QPromise<T>;
 
    //… other functions etc continue here
}
 
declare module "q" {
    export = Q;
}
declare var Q: QStatic;

And that fixed the obscure 'wrapsSomeTypeParameter' error. The full source code of these amended typings can be found as a GitHub Repo here in case you want to use it yourself. (I did originally consider adding this to Definitely Typed but opted not to in the end - see discussion on GitHub.)

The Promised Land

You're there. You've upgraded to the new plugin and the new typings. All is compiling as it should and the language service is working as well. Was it worth it? I think yes, for the following reasons:

1. TS 0.9.5 compiles faster, and hogs less memory.
   
   
2. When we compiled with TS 0.9.5 we found there were a couple of bugs in our codebase which the tightened up compiler was now detecting. Essentially where we'd assumed types were flowing through to functions there were a couple of occasions with TS 0.9.1.1 where they weren't. Where we'd assumed we had a type of T available in a function whereas it was actually a type of any. I was really surprised that this was the case since we were already making use of noImplicitAny compiler flag in our project. So where a type had changed and a retired property was being referenced TS 0.9.5 picked up an error that TS 0.9.1.1 had not. Good catch!
   
   
3. And finally (and I know these are really minor), the compiled JS is a little different now. Firstly, the compiled JS features all of TypeScript comments in the positions that you might hope for. Previously it seemed that about 75% came along for the ride and ended up in some strange locations sometimes. Secondly, enums are treated differently during compilation now - where it makes sense the actual backing value of an enum is used rather than going through the JavaScript construct. So it's a bit like a const I guess - presumably this allows JavaScript engines to optimise a little more.

I hope I haven't put you off with this post. I think TypeScript 0.9.5 is well worth making the leap for - and hopefully by reading this you'll have saved yourself from a few of the rough edges.