Let's Create a Tiny Programming Language

You may already be familiar with one or more programming languages. But have you ever thought about how to create your own programming language? I mean:

A programming language is a set of rules that convert strings into various machine code outputs.

In short, a programming language is just a set of predefined rules. To make it useful, you need something that can understand these rules, such as compilers , interpreters , etc. So we can simply define some rules, and then, to make it work, we can write a program that can understand these rules using any existing programming language, which will become our interpreter.

Compiler

The compiler converts the code into machine code that the processor can execute (such as a C compiler).

Interpreter

The interpreter goes through the program line by line and executes each command.

Want to try it? Let's create a super simple programming language that outputs magenta output in the console. We call it Magenta .

Set up our programming language

I'll use Node.js, but you can learn in any language and the concept remains the same. Let me first create an index.js file and set it up.

 class Magenta {
  constructor(codes) {
    this.codes = codes;
  }
  run() {
    console.log(this.codes);
  }
}

// Currently, we store the code in a string variable called `codes` // Later, we will read the code from the file const codes = `print "hello world"
print "hello again"`;
const magenta = new Magenta(codes);
magenta.run();

What we are doing here is declaring a class called Magenta. This class defines and initializes an object that is responsible for recording text to the console using any text we provide through the codes variable. And, at present, we have defined the codes variable directly using several "hello" messages in the file.

OK, now we need to create a so-called lexical analyzer.

What is a lexical analyzer?

OK, let's talk about English first. Please see the following phrases:

Are you OK?

Here, "Hello" is a greeting, "Is" a supplementary tone, and "you" is a personal pronoun. We have a question mark ("?") at the end. We can divide any sentence or phrase into many grammatical components like this. Another way we distinguish these parts is to divide them into small marks. The program that divides text into tags is our lexical analyzer .

Since our language is very small, it has only two types of tags, each with a value:

1. Keywords
2. String

We can use regular expressions to extract tags from codes strings, but the performance will be very slow. A better approach is to loop through each character of the code string and get the tag. So let's create a tokenize method in the Magenta class - this will be our lexical analyzer.

Complete code

`` javascript class Magenta { constructor(codes) { this.codes = codes; } tokenize() { const length = this.codes.length; // pos 用于跟踪当前位置/索引let pos = 0; let tokens = []; const BUILT_IN_KEYWORDS = ["print"]; // 变量/关键字允许的字符const varChars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_"; while (pos Unknown token: ${word} <code>); } } return tokens; } parse(tokens) { const len = tokens.length; let pos = 0; while (pos Unexpected token: ${token.type}`); } } } run() { const tokens = this.tokenize(); this.parse(tokens); } } }

const codes = print "hello world" print "hello again" ; const magenta = new Magenta(codes); magenta.run();

### Define Rules and Syntax We want to see if our code order matches some rule or syntax. But first we need to define what these rules and syntax are. Since our language is very small, it has only a simple syntax, namely the print keyword followed by a string.

Keyword: Print string

<code>因此，让我们创建一个解析方法，该方法循环遍历我们的标记，并查看我们是否形成了有效的语法。如果是这样，它将采取必要的措施。 ```javascript class Magenta { constructor(codes) { this.codes = codes; } tokenize() { /* tokenizer 的先前代码*/ } parse(tokens) { const len = tokens.length; let pos = 0; while (pos </code>

Look! We already have a working language!

OK, but putting the code in a string variable is not that fun. So let's put our Magenta code in a file called code.m. This way we can separate the magenta code from the compiler logic. We use .m as the file extension to indicate that the file contains code in our language.

Let's read the code from this file:

 // Import file system module const fs = require('fs');
// Import the path module to facilitate path connection const path = require('path');
class Magenta {
  constructor(codes) {
    this.codes = codes;
  }
  tokenize() {
    /* previous code for tokenizer*/
  }
  parse(tokens) {
    /* previous code of parse method*/
  }
  run() {
    /* previous code of run method*/
  }
}

// Read the code.m file // Some text editors use \r\n as a newline instead of \n, so we delete \r
const codes = fs
  .readFileSync(path.join(__dirname, 'code.m'), 'utf8')
  .toString()
  .replace(/\r/g, "");
const magenta = new Magenta(codes);
magenta.run();

Start creating a programming language!

In this way, we have successfully created a mini programming language from scratch. See, programming languages ​​can be as simple as completing a specific thing. Of course, it is unlikely that languages ​​like Magenta here can be useful to be part of a popular framework, but now you have an idea of ​​what it takes to create a programming language.

The sky is the limit. If you want to dig deeper, try following the video I made, which introduces a more advanced example. In this video, I also show how to add variables to your language.

The above is the detailed content of Let's Create a Tiny Programming Language. For more information, please follow other related articles on the PHP Chinese website!