microgpt.ts

A conversion of Karpathy's MicroGPT to Typescript

Andrej Karpathy gave us MicroGPT (blog, gist), a self contained, impressively minimalist implementation of a GPT.

Not everybody has Python as a preferred language, so, I decided to convert it to Typescript.

As I understand it, Karpathy's purpose was to help make this easier to learn. I'm just trying to bring his work to a wider reading audience.

• 💻 Try it, or
• 🔍 Read the code, or
• 👓 Read the docs, or
• 🏗️ See the builds

How to read this

There are two ways to read this.

1. You could do this Karpathy-style. There is a single-file TS GPT in this repository.
2. You could do this community-style. There is a modular GPT in this repository too.

Or you could

What's the difference?

It's around 450 lines of code. Chewing that down in one piece really suits a lot of people. Other people don't want to think about, say, the random number generator while they're trying to learn Adam, or whatever.

The community style just takes the code and breaks it up into a couple of files - one for the tokenizer, one for the randomizer, one for autograd, et cetera - then adds docblock comments and tests. Community style also adds a single function called test to index, which shows a person how these library calls are made.

How to use this

Well, generally use a production GPT. This is correct, but it doesn't do any of the fancy stuff that gives you real speed. However, for learning purposes, we've included a web GUI that does this live. There's a copy of Karpathy's "tiny shakespeare" to train on in src/html.

Alternately, you can download it and run it locally. If you do, please use npm run local to start a local webserver, then hit the page at https://localhost:4400/usage.html, instead of loading locally from a file url; file:// has CORS consequences, and you won't be able to upload your text trainer.

How to contribute

Patches, features, and bugfixes are accepted graciously. It's easier to accept a PR that has a description, so I know what it does. Please fork the repo and send a PR back from your fork.