ts-loader Project References: First Blood

So project references eh? They shipped with TypeScript 3. We've just shipped initial support for project references in ts-loader v5.2.0. All the hard work was done by the amazing Andrew Branch. In fact I'd recommend taking a gander at the PR. Yay Andrew!

This post will take us through the nature of the support for project references in ts-loader now and what we hope the future will bring. It rips off shamelessly borrows from the README.md documentation that Andrew wrote as part of the PR. Because I am not above stealing.

TL;DR

Using project references currently requires building referenced projects outside of ts-loader. We don’t want to keep it that way, but we’re releasing what we’ve got now. To try it out, you’ll need to pass projectReferences: true to loaderOptions.

Like tsc, but not like tsc --build

ts-loader has partial support for project references in that it will load dependent composite projects that are already built, but will not currently build/rebuild those upstream projects. The best way to explain exactly what this means is through an example. Say you have a project with a project reference pointing to the lib/ directory:

tsconfig.json
app.ts
lib/
  tsconfig.json
  niftyUtil.ts

And we’ll assume that the root tsconfig.json has { "references": { "path": "lib" } }, which means that any import of a file that’s part of the lib sub-project is treated as a reference to another project, not just a reference to a TypeScript file. Before discussing how ts-loader handles this, it’s helpful to review at a really basic level what tsc itself does here. If you were to run tsc on this tiny example project, the build would fail with the error:

error TS6305: Output file 'lib/niftyUtil.d.ts' has not been built from source file 'lib/niftyUtil.ts'.

Using project references actually instructs tsc not to build anything that’s part of another project from source, but rather to look for any .d.ts and .js files that have already been generated from a previous build. Since we’ve never built the project in lib before, those files don’t exist, so building the root project fails. Still just thinking about how tsc works, there are two options to make the build succeed: either run tsc -p lib/tsconfig.json first, or simply run tsc --build, which will figure out that lib hasn’t been built and build it first for you.

Ok, so how is that relevant to ts-loader? Because the best way to think about what ts-loader does with project references is that it acts like tsc, but not like tsc --build. If you run ts-loader on a project that’s using project references, and any upstream project hasn’t been built, you’ll get the exact same error TS6305 that you would get with tsc. If you modify a source file in an upstream project and don’t rebuild that project, ts-loader won’t have any idea that you’ve changed anything—it will still be looking at the output from the last time you built that file.

“Hey, don’t you think that sounds kind of useless and terrible?”

Well, sort of. You can consider it a work-in-progress. It’s true that on its own, as of today, ts-loader doesn’t have everything you need to take advantage of project references in webpack. In practice, though, consuming upstream projects and building upstream projects are somewhat separate concerns. Building them will likely come in a future release. For background, see the original issue.

outDir Windows problemo.

At the moment, composite projects built using the outDir compiler option cannot be consumed using ts-loader on Windows. If you try to, ts-loader throws a "has not been built from source file" error. You can see Andrew and I puzzling over it in the PR. We don't know why yet; it's possible there's a bug in tsc. It's more likely there's a bug in ts-loader. Hopefully it's going to get solved at some point. (Hey, maybe you're the one to solve it!) Either way, we didn't want to hold back from releasing. So if you're building on Windows then avoid building composite projects using outDir.

Ivan Drago and Definitely Typed

This a tale of things that are and things that aren't. It's a tale of semantic versioning, the lack thereof and heartbreak. It's a story of terror and failing builds. But it has a bittersweet ending wherein our heroes learn a lesson and understand the need for compromise. We all come out better and wiser people. Hopefully there's something for everybody; let's start with an exciting opener and see where it goes...

Definitely Typed

This is often the experience people have of using type definitions from Definitely Typed:

Specifically, people are used to the idea of semantic versioning and expect it from types published to npm by Definitely Typed. They wait in vain. I've written before about the Definitely Typed / @types semantic version compromise. And I wanted to talk about it a little further as (watching the issues raised on DT) I don't think the message has quite got out there. To summarise:

1. npm is built on top of semantic versioning and they take it seriously. When a package is published it should be categorised as a major release (breaking changes), a minor release (extra functionality which is backwards compatible) or a patch release (backwards compatible bug fixes).

2. Definitely Typed publishes type definitions to npm under the @types namespace

3. To make consumption of type definitions easier, the versioning of a type definition package will seek to emulate the versioning of the npm package it supports. For example, right now react-router's latest version is 4.3.1. The corresponding type definition @types/react-router's latest version is 4.0.31. (It's fairly common for type definition versions to lag behind the package they type.)

   If there's a breaking change to the react-router type definition then the new version published will have a version number that begins "4.0.". If you are relying on semantic versioning this will break you.

I Couldn't Help But Notice Your Pain

If you're reading this and can't quite believe that @types would be so inconsiderate as to break the conventions of the ecosystem it lives in, I understand. But hopefully you can see there are reasons for this. In the end, being able to use npm as a delivery mechanism for versioned type definitions associated with another package has a cost; that cost is semantic versioning for the type definitions themselves. It wasn't a choice taken lightly; it's a pragmatic compromise.

"But what about my failing builds? Fine, people are going to change type definitions, but why should I burn because of their choices?"

Excellent question. Truly. Well here's my advice: don't expect semantic versioning where there is none. Use specific package versions. You can do that directly with your package.json. For example replace something like this: "@types/react-router": "^4.0.0" with a specific version number: "@types/react-router": "4.0.31". With this approach it's a specific activity to upgrade your type definitions. A chore if you will; but a chore that guarantees builds will not fail unexpectedly due to changing type defs.

My own personal preference is yarn. Mother, I'm in love with a yarn.lock file. It is the alternative npm client that came out of Facebook. It pins the exact versions of all packages used in your yarn.lock file and guarantees to install the same versions each time. Problem solved; and it even allows me to keep the semantic versioning in my package.json as is.

This has some value in that when I upgrade I probably want to upgrade to a newer version following the semantic versioning convention. I should just expect that I'll need to check valid compilation when I do so. yarn even has it's own built in utility that tells you when things are out of date: yarn outdated:

So lovely

You Were Already Broken - I Just Showed You How

Before I finish I wanted to draw out one reason why breaking changes can be a reason for happiness. Because sometimes your code is wrong. An update to a type definition may highlight that. This is analogous to when the TypeScript compiler ships a new version. When I upgrade to a newer version of TypeScript it lights up errors in my codebase that I hadn't spotted. Yay compiler!

An example of this is a PR I submitted to DefinitelyTyped earlier this week. This PR changed how react-router models the parameters of a Match. Until now, an object was expected; the user could define any object they liked. However, react-router will only produce string values for a parameter. If you look at the underlying code it's nothing more than an exec on a regular expression.

My PR enforces this at type level by changing this:

export interface match<P> {
  params: P;
  ...
}

To this

export interface match<Params extends { [K in keyof Params]?: string } = {}> {
  params: Params;

So any object definition supplied must have string values (and you don't actually need to supply an object definition; that's optional now).

I expected this PR to break people and it did. But this is a useful break. If they were relying upon their parameters to be types other than strings they would be experiencing some unexpected behaviour. In fact, it's exactly this that prompted my PR in the first place. A colleague had defined his parameters as numbers and couldn't understand why they weren't behaving like numbers. Because they weren't numbers! And wonderfully, this will now be caught at compile time; not runtime. Yay!