Fourth part of practice 6 of PL: A programming Language

Original repository

Practice repositories

• First part repository
• Second part repository
• Third part repository

First part

• Lexer's regular expressions were created using the XRegExp library, and now they are properly documented.
• The "tokens objects" store the line and the offset.
• Errors messages improved.
• Now the lexer, doesn't destroy the program while it is reading it. The regular expressions use the sticky option.
• Several new tests have been added to check errors.

Second part

• REPL mode.
• Added aliases to some features of the language.
• Added more tests which checks the execution of incorrect programs and the output of correct programs.
• Added negative indices for arrays like in Ruby.
• Added hashes/maps to the language.
• OOP solution, now the different types of nodes of the AST are represented with classes, each node knows how to evaluate itself.
• Rewritten some commented tests.

Third part

For each new functionality several tests and examples programs have been created to test it behaviour.

• Added multiple arguments operations.
• Added array/maps modification using set.
• Added access to array/maps with element or <- functions.
• Added an alias, now : acts as , if is not followed by = (:= is an alias of define).
• Added access to array/maps with ..
• Added bracket notation to access to arrays and maps elements.
• Added access to native JavaScripts methods of array and maps.

Fourth part

For each new functionality several tests and examples programs have been created to test it behaviour.

• Added require functionality, now egg programs could be extended using require.
• Added collect function for arrays.
• Added for loops.
• Added pretty maps keys.
• Added extended regular expressions.
• Added objects.
• Added splat operator.

REPL Mode

Syntactic sugar

Added some aliases:

• := is equivalent to define
• = is equivalent to set
• <- is equivalent to elements
• { is equivalent to (
• } id equivalent to )
• [ is equivalent to (
• ] id equivalent to )

], ) and } must close with the respective symbol, the next code is incorrect:

  do(
    print("Foo"}
  )

Some new tests and example programs have been added to check the aliases behaviour.

Checking errors

Some tests added to verify:

do(
  set(x,9),
  print(x) # ReferenceError: Tried setting an undefined variable: x
)

• Throws a ReferenceError cause x is not defined.

do(
  set(x,9)
)
print(x) #SyntaxError: Unexpected input after reached the end of parsing

• Throws a SyntaxError cause there is input after the end of the block.

do{
  2[12]
}

• Throws an error, numbers can't act as functions.

do(
  define(x, 4),
  define(setx, fun(val,
                   set(x, val)
                  )
        ),
  setx(50),
  print(x)
)

• Produces 50 as output.

Negative indices

Now it is possible to access to an array elements with negative indices like in Ruby:

def(x, array[1,2,3,4])
print([](x, 2))		      #3
print(element(x, -1))   #4
print(<-(x, -4))        #1

Hashes / Maps

Now the language supports maps:

def(x, map{1,1,2,4,3,9,4,16}}
print(x)                      #Map { 1 => 1, 2 => 4, 3 => 9, 4 => 16 }
print(length(x))              #4
print(element(x, 3))          #9
print(<-(x,2))                #4
print({}(x,4))                #16

• To create a map use map or hash function, the first element of each pair arguments will be the key and the second the value.
• Maps responds to the same methods as arrays excepting [] which is typical of arrays(length, element, <-), the equivalent method for maps is {}.

OOP solution

Now the different types of nodes of the AST are represented with classes, each kind of node knows how to evaluate itself.

Multiple arguments operations.

Now the language supports operations like:

do(
  print(+(<(2,3,4,5,6,7), " should be ", "tru", "e")),
  print(+(<(2,3,0,5,6,7), " should be ", "fals", "e")),
  print(+(-(*(2,4),/(64,8),*(0, +(2,2,3,1,3,5,3,4,9))), " should be ", "0")),
  print(+(+(1,2,3,4,5,6), " should be 21"))
)

Special set

Now we can modify an array/map:

do(
  :=(x, arr[1,2,3,arr[4,5,arr[6,7,8]]]),
  =(x,2,-3),
  =(x,3,0,-4),
  =(x,3,2,-1,-8),
  :=(y, "String power"),
  =(x,3,2,-2, y),
  print(x)		#[ 1, 2, -3, [ -4, 5, [ 6, 'String power', -8 ] ] ]
)

Array/maps access

Now we can access to an array or map elements:

do(
  :=(x, arr[1,2,map{1,2,"col",arr[1,2,3]}]),
  element(x,2),         #Map { 1 => 2, 'col' => [ 1, 2, 3 ] }
  element(x,2,1),       #2
  element(x,2,"col"),   #[ 1, 2, 3 ]
  element(x,2,"col",-1) #3
)

:exclamation: Note that {} and [] also works with something like [](x,1,3,4).

Syntactic sugar

• : acts as , if is not followed by = (:= is an alias of define)

do(
  :=(x, map{ "hello": arr[1,2,3,4], "bye": 12, "foo": map{1:2,2:4}}),
  print(element(x, "hello"))		#[ 1, 2, 3, 4 ]
)		

Dot access

Now we can access to an array or map elements using .:

do(
  :=(x, map("a": map("x": array[70, 100]), "b": array[2,3])),
  print(x), # { a: { x: [ 70, 100 ] }, b: [ 2, 3 ] }
  print(x.a),  #  { x: [ 70, 100 ] }
  print(x.b),  # [ 2, 3 ]
  print(x.a.x.-1),  # 100
  print(x.b.1)  # 3
)

Bracket access

Now we can access to an array or map elements using []:

do(
  def(x, array[array[1,4], fun(x,y, +(x,y)),7]),
  print(x[1](5,9)),      # 14
  print(x[0, -1]),       # 4
  print(+(x[0][1], 12))  # 16
)

Access to native methods

Now we can use native JavaScript methods with our egg maps and arrays:

do(
  :=(x, map{1,1,2,4,"mep",map{1,1,2,8,3,27,"mip", map{"mop", "mup"}}}),
  print(x.keys()),
  x.delete(1),
  x.delete("mep"),
  print(x.values())
  def(x, array[array[1,4],5,7]),
  x.push(4),
  print(x),
  print(x.shift()),
  print(x),
)

do(
  def(x, array[array[1,4],5,7]),
  x.push(4),
  print(x),
  print(x.shift()),
  print(x)
)

do(
  :=(z, array[1, 4, "a"]),
  print(z.push(9)),
  print(z["push"](5)),
  print({}(z, "shift")()),
  print(element(z, "push")(8)),
  print(z)
)

Require functionality

Now egg programs could be extended using require:

• Module to include:

do {
  :=(z, map("inc": ->(x,
                        +(x,1)
                     ) # end fun
           ) # end map
    ), # end of :=
  z  # el último valor será exportado
}

• Program including the previous module:

do {
  print("Including module"),
  :=(z, require("examples/modules/incMap.egg")),
  print("Module included"),
  print(z.inc(4))
}

Collect method

Collect function returns a new array with the results of running the given function once for every element in the given array. It expects an array as first argument and a function as second argument:

do(
  def(x, arr[1,2,3,4]),
  def(block,->(element,
               *(element, element)
              )
     ),
  def(z, collect(x, block)),
  print(z),# [ 1, 4, 9, 16 ]
  print(collect(arr[-1,-2,-3,-4],
                ->(element,
                   *(element, element, element)))) # [ -1, -8, -27, -64 ]
)

Collect method only works with arrays.

For loops

For expects four Apply objects:

• Counter expression
• Continue expression
• Increment expression
• Body

do(
  for(:=(x,0), <(x, 9), =(x,+(x, 1)), print(+("Hola ", x)))
)

Pretty map keys

Now it is possible to declare maps following the next syntax:

:=(x, map{1,1,2,4, mep:map{1:1,2:8,3:27, mip: map{mop: "mup"}}})

The "string keys" doesn't need to be between double quotes symbols. :exclamation: Note that with this notation some colateral changes are introduced:

do(
  :=(w, "hi"),
  :=(x, map{w: "bye"}),       
  :=(y, "ho"),
  :=(z, map{y, "bye"}),       
  print(x),               ;Map { 'w' => 'bye' }
  print(z)                ;Map { 'ho' => 'bye' }
)

, and : are not fully equivalent:exclamation::exclamation::exclamation:

Regular Expressions

Now it is possible to create regular expression objects:

• regex/regular_expression/options

Example program:

do {
  :=(d, regex/
        (?<year>  \d{4} ) -?  # year
        (?<month> \d{2} ) -?  # month
        (?<day>   \d{2} )     # day
        /x),
  print(d.test("2015-02-22")),  # true
  :=(m, exec("2015-02-22", d)),  /*  [ '2015-02-22', '2015', '02', '22',
                                     index: 0, input: '2015-02-22',
                                     year: '2015', month: '02', day: '22' ]
                                */
  print(m.year),  # 2015
  print(m.month), # 02
  print(m.day), # 22
  :=(numbers, regex/([-+]?\d*         #Integer number
                        \.?      #Floating point
                        \d+([eE][-+]?\d+)?)/x),
  print(numbers.test("12321.21321-e20")),
  :=(string, regex/["']((?:[^"\\]  |             #Single character excepting double quotes and backslash
                                \\.         #Any character preceded by a backslash
                                   )*)["']/x),
  print(string.test("'hello'")),
  :=(words, regex/([^\s(){}\[\],\"'?]+)/x),
  print(words.test("Word"))
}

The exec method that the regular expresions objects has is the JavaScript native regular expression exec method. If you want to call XRegExp.exec you must invoke it following the next syntax:

:=(m, exec("2015-02-22", d))

So we can access to the named capture groups, for example:

print(m.year),
print(m.month),
print(m.day)

Objects

Now the language supports objects:

do {
  :=(x, Object {
    "c": 0,
		   "gc": ->(this.c),
		"sc": ->(value,
			=(this.c, value)
		)
	}),
	print(x.gc()),
	x.sc(4),
	print(x.gc()),
}

Splat operator

Now the language supports the splat operator:

do(
  def(f1, ->(...elements, collect(elements, ->(element, print(element))))),
  def(f2, ->(first, second, third, print(+(first, second, third)) )),
  def(a, arr[1,2,3]),
  f1(1,2,3),
  f1(a),
  =(a, arr[2,3,4]),
  f2(...a)     ;f(2,3,4)
)

Grammar

expression: STRING
          | NUMBER
          | WORD apply

apply: /* vacio */
     | '(' (expression ',')* expression? ')' apply


WHITES = /^(\s|[#;].*|\/\*(.|\n)*?\*\/)*/;
STRING = /^"((?:[^"\\]|\\.)*)"/;
NUMBER = /^([-+]?\d*\.?\d+([eE][-+]?\d+)?)/;
WORD   = /^([^\s(),"]+)/;

AST

• Expressions of type "VALUE" represent literal strings or numbers. Their value property contains the string or number value that they represent.

• Expressions of type "WORD" are used for identifiers (names). Such objects have a name property that holds the identifier’s name as a string.

• Finally, "APPLY" expressions represent applications. They have an operator property that refers to the expression that is being applied, and an args property that holds an array of argument expressions.

ast: VALUE{value: String | Number}
   | WORD{name: String}
   | APPLY{operator: ast, args: [ ast ...]}

The >(x, 5) would be represented like this:

$ cat greater-x-5.egg
>(x,5)
$ ./eggc.js greater-x-5.egg
$ cat greater-x-5.egg.evm
{
  "type": "apply",
  "operator": {
    "type": "word",
    "name": ">"
  },
  "args": [
    {
      "type": "word",
      "name": "x"
    },
    {
      "type": "value",
      "value": 5
    }
  ]
}

Examples

Some example programs could be found in the examples folder.

Executables

• egg
  • Runs an egg program: bin/egg examples/two.egg compiles the source onto the AST and interprets the AST

$ cat example/one.egg
do(
  define(x, 4),
  define(setx, fun(val,
      set(x, val)
    )
  ),
  setx(50),
  print(x)
)
$ bin/egg one.egg
50

• eggc
  • Compiles the input program to produce a JSON containing the tree: bin/eggc examples/two.egg produces the JSON file examples/two.egg.evm
• evm
  • Egg Virtual Machine. Runs the tree: bin/evm examples/two.egg.evm

$ bin/eggc examples/one.egg
$ bin/evm examples/one.egg.evm
50

Here is the tree in JSON format for the former one.egg program:

$ cat one.egg.evm
{
  "type": "apply",
  "operator": {
    "type": "word",
    "name": "do"
  },
  "args": [
    {
      "type": "apply",
      "operator": {
        "type": "word",
        "name": "define"
      },
      "args": [
        {
          "type": "word",
          "name": "x"
        },
        {
          "type": "value",
          "value": 4
        }
      ]
    },
    {
      "type": "apply",
      "operator": {
        "type": "word",
        "name": "define"
      },
      "args": [
        {
          "type": "word",
          "name": "setx"
        },
        {
          "type": "apply",
          "operator": {
            "type": "word",
            "name": "fun"
          },
          "args": [
            {
              "type": "word",
              "name": "val"
            },
            {
              "type": "apply",
              "operator": {
                "type": "word",
                "name": "set"
              },
              "args": [
                {
                  "type": "word",
                  "name": "x"
                },
                {
                  "type": "word",
                  "name": "val"
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "type": "apply",
      "operator": {
        "type": "word",
        "name": "setx"
      },
      "args": [
        {
          "type": "value",
          "value": 50
        }
      ]
    },
    {
      "type": "apply",
      "operator": {
        "type": "word",
        "name": "print"
      },
      "args": [
        {
          "type": "word",
          "name": "x"
        }
      ]
    }
  ]
}