Defining the Platform Horizon in a Monorepo

At work, I maintain a monorepo of around 100 Node.js packages, 80 of which are internal tools. We consider ourselves maintainers of a developer platform but at the beginning, when we first started the repo, the platform was a bit difficult to define. To become a platform, you need to define integration points that you can control so that each consumer of your platform can leverage the same capabilities and tools. Otherwise, you are just a monorepo with every package doing their own thing.

Bringing the Server and CLI into the Platform

From the beginning, we knew we wanted to leverage Next.js, so many of our internal tools had the following package.json configuration:

{
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start"
  },
  "dependencies": {
    "next": "12.1.0"
  }
}

There were some things we wanted to customize about the Next.js experience. Along side the Next.js dev process, we wanted to add a background TypeScript watcher process to emit type errors to stdout - something that doesn't come out of the box with Next.js. We also wanted to add other things like GraphQL code generation.

In the Next.js server itself, we wanted to add request middleware to integrate with our internal OAuth provider, and add logging, error handling and reverse proxy-ing. However, Next.js does not come with first-class support for Node.js middleware so we had to wrap Next.js with our own custom server. We used Fastify in this instance and delegated React rendering to Next.js using their Node.js APIs. You can learn how we did it in this blog post.

With all these added capabilities, we ended up building our own CLI which starts our custom server. The CLI thinly wraps the build and start commands as well. We named our server Pinternal to make it clear it was for internal web applications at Pinterest.

{
  "scripts": {
    "dev": "pinternal dev",
    "build": "pinternal build",
    "start": "pinternal start"
  },
  "dependencies": {
    "@pinterest-tools/pinternal": "workspace:*"
  }
}

In this example, we symlink the local pinternal monorepo package using the workspace: protocol from PNPM.

Now you can see the platform taking shape. The pinternal CLI and server gives our team levers that we can control. So any improvements to the pinternal CLI or server results in improvements for everyone. We needed to find more of these platform levers to make our team more effective.

Bringing Configuration into the Platform

It didn't take too long for us to recognize we were in configuration hell. It wasn't uncommon for each internal web app to have their own customized set of configs:

.eslintrc.js
.prettierrc.yaml
jest.config.js
tsconfig.json
next.config.js

Some of the configuration such as ESLint and Prettier we could extract and bring to the root of the monorepo so that we could provide the same linting and code formatting experience for everyone. People will inevitably argue for having their own specialized ESLint or Prettier set up, but in my opinion, consistency always trumps taste when it comes to deciding how to set up your linter and formatter.

The other configuration for Jest, Next.js and TypeScript, we can bring into the platform, but we want to offer our developers the ability to customize these configs as we need to be open to the possibility that the platform defaults won't work for every use case so we can't necessarily hide the configs. Instead, we decided to offer APIs that our developers can use to import the boilerplate configs that they can extend if they want to.

This is what our Jest config looks like:

const { withPinternalJestConfig } = require("@pinterest-tools/pinternal");

module.exports = withPinternalJestConfig();

This is what our Next.js config looks like:

const { withPinternalNextConfig } = require("@pinterest-tools/pinternal");

module.exports = withPinternalNextConfig();

And this is our TypeScript config:

{
  "extends": "@pinterest-tools/config/tsconfig.nextjs.json"
}

This gives us more levers we can control to improve the developer experience for Jest, Next.js and TypeScript. For example, if we want to add new setup/teardown code to Jest or a new webpack plugin to Next.js, we can add it for everyone without changing everyone's code.

The base TypeScript config is something I wish we had done earlier. For every unique TypeScript config, you basically have a different programming language that may not even interoperate with each other. Controlling this early will save you headaches later.

Our scaffolding CLI now spits out a web application in our monorepo with all of these configs. Jest, Next.js, and TypeScript are now part of our platform.

Bringing Convention into the Platform

We like the convention over configuration design philosophy of Ruby on Rails and it is something we try to apply to our platform. When our developers deploy their internal tools, we assume each tool follows a series of conventions that makes it possible to build and ship their application without doing any dev ops or writing any configuration.

First, we assume there is a file named Dockerfile at the root of the application. We use this in our CICD pipeline to build a docker image. The name of the docker image we derive from the app's package name. Second, we assume there is a k8s file in the k8s/<appName>.yml path at the root of the application. We read this from the CICD pipeline to deploy to K8s.

We support specifying different K8s manifests for different environments and rely on naming convention to do so. For example, we use the convention of naming our files k8s/<appName>.<envName>.yml to target different deploy environments such as staging while we reserve k8s/<appName>.yml for production.

We also support building multiple docker images automatically too. If the user needs to ship a k8s manifest file with multiple docker images, they can merge different dockerfiles with the <imageName>.Dockerfile convention at the root of their application and we automatically build those images and push to our image registry.

Similarly, if the user needs to deploy a worker cluster for a queue, or a cron job, they can merge different k8s/<serviceName>.yml files and our pipeline will automatically take care of deploying those workloads to k8s.

In the dockerfiles we use base images to expose common functionality across all our images. And we define K8s CRDs to hide common sidecars and configuration boilerplate needed to deploy each internal tool.

Takeaways

A platform doesn't become a platform if you just call it so, otherwise, all you have is lots of code with no common layer and "platform" developers who fail to have any impact at scale. You need to define the horizon that delineates the platform vs. user-land and the horizon needs to manifest itself as code or CI enforced convention otherwise it is just guidance that people will inevitably ignore. This explicitness empowers your users to become creative inside the areas where the platform has no say, ensures enough structure such that your monorepo doesn't become a mess, and it empowers platform developers to introduce improvements that impact everyone.