britrunner

aaron cooper

NEW!

9 Nov 2024

Building a Deno v2 Monorepo (Part 1)

I first heard about Deno about a year or so ago when a colleague told me it was going to be the next NodeJS. I think I was vaguely amused, but didn’t see much in it at the time.

Well, Deno v2.0 recently landed and I thought I’d finally give it a proper look, comparing it to Node and wrapping up with my thoughts about how Deno is now and its potential future.

The plan 📃

I’m going to assess Deno v2 by building a multi-faceted app. More accurately, it’ll be a rebuild of a monorepo I’ve already partly created in Node, with NPM workspaces and Nx. That’ll give me a direct comparison between Deno v2 and Node v20.

This is what I’ll be (re)building:

At each stage I’ll be looking at what Deno offers out of the box, what I can pull from 3rd party modules and what I might copy-paste directly from my Node app.

Act 1: Monorepo setup 🧱

Project init

Getting started with Deno is simple. I installed Deno globally based on the official docs. It’s cool that they provide a few other ways of installing Deno that support installing multiple versions (similar to NVM), which I might try out next time.

Creating a Deno project is simple, too:

Terminal window
deno init llm-tools

And what’s also simple is what you get out of the box:

Terminal window
deno.json
main_test.ts
main.ts

Whoa whoa hol’ up - those are TS files by default, a unit tests file, and there’s no tsconfig. Pure minimalism and I love it. I hope this continues.

As guessed, deno.json is to Deno what package.json is to Node/NPM. It looks like this out of the box:

{
"tasks": {
"dev": "deno run --watch main.ts"
},
"imports": {
"@std/assert": "jsr:@std/assert@1"
}
}

Naturally, deno task dev runs the dev task defined here. It’s nice to see --watch is supported with Deno natively, auto-rerunning the script on file changes.

Workspace setup

As with any package manager worth its salt these days, Deno supports workspaces to make up a monorepo. You tell Deno via the top-level deno.json which subfolders make up your “workspace”:

{
"workspace": ["./core", "./cli", "./ui"],
"imports": {
"@std/assert": "jsr:@std/assert@1"
}
}

Take note if you’re coming from NPM workspaces (plural) that Deno requires you to explicitly pass an array of subfolders, not a glob pattern pointing to any number of folders. This might put off some folks, but I don’t mind.

Creating the subdirs is just a deno init core away (ran from within the project root). You can then safely move any common dependencies into the root-level deno.json (which is just std/assert to begin with).

I made some changes to the child deno.json files based on the official docs:

{
"name": "@alc/llm-tools-core",
"version": "0.1.0",
"exports": "./mod.ts",
"tasks": {},
"imports": {}
}

Finally, I ended up with a project looking like this:

image.png

I might be missing a few things, but I’ll figure them out as I go 😉

Act 2: Building a library 📚

Dependency management

I made a start with the core functionality. Since I already had a bunch of reusable code in my Nx repo, I copy-pasted almost all of it over. What could go wrong?

image.png

Massive exaggeration for the memes. But, as I anticipated, imports were an immediate issue:

image.png

To talk about imports for a sec - Deno v2 can import from 3 different sources:

  1. JSR (JavaScript Registry) - a place for publishing TypeScript-native, web standards-compliant pacakges. Most Deno packages end up here.
  2. HTTPS - import a source from from virtually any HTTPS URL, handy for simple scripts to pull from somewhere like unpkg.
  3. NPM (Node Package Manager) - the largest source of open source packages (typically targeting Node).

I’d like to use JSR as much as possible, given it’s the “Deno way”. When I searched JSR, however, I didn’t find a reliable port of the official openai package. The only “official” sounding one had a 17/100 rating and looked empty. I did see that a fair few packages already published that were similar to what I intended to ship though. That’s something I’ll chew on later and decide if I want to get a bit more creative.

image.png

But! I was very impressed when I swung back over to the official NPM OpenAI package and saw this nugget in the README:

You can import in Deno via:

import OpenAI from 'https://deno.land/x/openai@v4.68.4/mod.ts';

A legit Deno port was already published! 🤩

…BUT! It didn’t work for me. Perhaps it was published pre-Deno v2.0, but this version can’t be resolved by Deno because one of the required files can’t be downloaded.

I finally switched to the vanilla NPM version via: npm:openai@4.68.4 and carried on.

You can specify verbose import sources/versions in import statements directly (in your TS files), or shorten your code imports by adding to imports inside the relevant deno.json :

"imports": {
"openai": "npm:openai@4.68.4"
}

By the way, this is added automatically via: deno add npm:openai@4.68.4 .

This permits the import from "openai" in code, so I don’t have to change my source code at all. You can set the import key to whatever you want, which could allow some very lean import statements.

I added a quick unit test for sanity, and confirmed it’s now initialising correctly with the import statement untouched:

image.png

The built-in deno test command does the trick, or just using the Deno VSCode extension to run the test from the sidebar icon. Tests run incredibly fast.

Personally, I think imports are too complicated. On the surface, this all seems really powerful, being able to import from different sources and specify them via source code import statements or inside the imports block of the Deno config. Compare this to NPM though (which is not flawless, I’ll qualify), 99% of the time you pull packages from NPM, they get added automatically and consistently, and you import them the same way each time. I think Deno opens the floodgates to confusion for developers, who just want to be told to do X by way of Y, and not being offered M or R or Q as options. It’s a minor gripe maybe, but something I hope will have standards better developed for over time.

Having said all that, the fact I’m able to import virtually any NPM package into my Deno app is pretty darn amazing and I know this was a huge step up from v1’s partial support. Let’s move on 🕺

Monorepo dependencies

A big draw of using a monorepo is to promote code reuse, including importing from other packages in the same workspace.

The core package is one I’m intending to do this with, exposing the opinionated LLMChat class for both the CLI and web app to use.

To test how this works, I added some basic code into the cli package to import from core . Back to the confusing imports again - importing from a local package in the same workspace looks like this:

import { LLMChat } from "@alc/llm-tools-core";

The import location should match the name field of the other package. This in itself is fine, but you don’t add anything in the imports of the nearest deno.json.

Yes, that’s yet another slightly different rule when importing something… I’ll have to summarise these in a cheat-sheet.

My cli config looks like this:

{
"name": "@alc/llm-tools-cli",
"version": "0.1.0",
"exports": "./mod.ts",
"tasks": {
"dev": "deno run --allow-env --env-file mod.ts"
},
"imports": {}
}

See: no imports. Also, nice to see Deno supports --env-file to automatically read from a local .env file if one exists. Contrary to the imports fluff, for the most part the Deno devs have considered a large number of different things that most developers need to build an app, and have provided a standard way of doing it.

Running deno task dev runs the placeholder cli script I added, which just imports the LLMChat class and console logs some stuff from it.

image.png

Later I’ll look at how to publish this humble little lib. Now, though, it’s ready for local use, so let’s use it!

Act 3: Command line interface ⌨️

Std packages

As with any good language/framework, Deno has its fair share of standard packages under the @std namespace. Unlike Node’s standard libs like path and fs, Deno requires you to install the ones you need. I’m cool with this if it means a lighter runtime 🪶

I started by installing the CLI package via: deno add jsr:@std/cli. The Deno docs recommend sourcing all @std packages from JSR. Some guides online haven’t been updated yet as they still advise installing from https://deno.land/x/.

The @std/cli package has two main superpowers: prompting the user for secrets, and parsing CLI arguments. The latter is what I want. You know, things like these that you’d usually want to pass to a command line tool:

Terminal window
--config="abc.json" -F --experimental index.ts blablabla

In Node land, yargs tends to be a go-to package for a lot of devs. That’s overkill for simple CLI apps though. @std/cli takes a decidedly simpler approach:

import { parseArgs } from "@std/cli";
const flags = parseArgs(Deno.args, {
string: ["name", "model"],
default: {
name: "openai",
model: "gpt-4o",
},
});

The simplicity here nice. I miss built-in validations from yargs and automatic --help documentation. But I can live without these for now, and if necessary pull in npm:yargs or find a Deno-native alternative later.

Dependencies & security

What caught my eye with Deno near its v1 launch was its approach to consuming third-party modules “safely”. As I said before, I never tried Deno v1, but was keen to see how it worked.

Eagle-eyed readers might have spotted this in a code snippet above: --allow-env. This is an example of allowing the app (and any of its dependencies) access to environment variables. Without this, the app refuses to run, asking the user to explicitly grant the required permission. Other examples include needing to grant permissions for file system and network access. You may also --deny-something for explicitly blocking certain access.

To be honest, I’d expected this to be: a) on a per-dependency basis, and b) more closely related to 3rd-party package security.

A major concern for developers and companies is digital supply chain attacks, a fancy way of saying any software not built in-house is suspect. That’s why big orgs tend to only allow installing dependencies via an internally trusted repository, which is commonly something like a JFrog Artifactory instance.

All this is to say… I was disappointed that Deno doesn’t appear to be tackling this problem exactly. However, I do appreciate what they are offering. This is the first time I’ve witnessed security baked in at this level. I can definitely see myself using --deny-network to shut out any potential 3rd-party code phoning home. I just wish it could be applied in a more granular way - per module or per resource would have been the Chef’s Kiss.

End of Part 1

That’s going to be it for this part. I thought I’d plough through the rest in one go, but this is already a chunky enough article and I want to reflect a little before moving onto the web app and finishing touches.

Thoughts so far on Deno v2?

It’s very nice. I’m liking the fact my project is still looking very clean. Of course, this isn’t the codebase for Google, but if the trend carries on the way I’ve seen it so far, Deno’s going to enable me to keep growing this monorepo and its packages in a cleaner way than I’ve seen before in a Node app.

I did roast Deno a little on import consistency and package security, but they support NPM packages as 1st class citizens now and they’re pushing the bar on per-access app permissions. I just hope in time imports will gain consistency (through docs and/or linting), and that access roles will be made more granular and per-package.

This all said, I’m eager to press on soon and start exploring how UI development goes with Deno 🦖