After a year-and-a-half long hiatus, I am pleased to announce that EndBASIC 0.11.0 is now available! 🥳
This release marks a significant milestone because it addresses the top feature request from you all, namely the ability to define custom functions and subroutines. But it also includes other goodies such as support for an LCD console, a shiny new disassembler, and a faster execution engine.
There is a lot to talk about, but before we get to that, here are the must-visit links:
Without further ado, let’s take a peek into the major changes.
User-defined functions and subroutines
As I mentioned earlier, EndBASIC 0.11 addresses the top feature request of all times: support for user-defined functions and subroutines. EndBASIC 0.10 addressed some of the pain by adding GOTO and GOSUB but there was still a need for structured callables with parameter passing and local variables.
Something like the following is finally possible:
DIM SHARED sum_calls AS INTEGER
FUNCTION sum(a, b)
sum = a + b
sum_calls = sum_calls + 1
END FUNCTION
SUB print_sum(prefix AS STRING, a, b)
PRINT prefix; sum(a, b)
END SUB
print_sum "The sum of 5+7 is:", 5, 7
print_sum "The sum of 1+2 is:", 1, 2
PRINT "sum has been called"; sum_calls; "times"
Easy peasy, right? Indeed it is! But getting here required redoing a huge portion of the compiler and VM internals.
One major piece to resolve this puzzle was to implement a declarative arguments parser for the callables. Up until now, every builtin command and function carried hand-crafted code to parse its arguments. This was not a great design choice due to large amounts of code duplication and the risk of inconsistencies between commands and documentation, but it was the simplest way to get something off the ground a few years back.
I had been meaning to resolve this long-standing “to-do” but, the thing is, BASIC makes it really difficult to come up with a generic arguments parser: every command has its own ad-hoc syntax. I needed a solution though, and after having implemented many commands in the previous 10 releases of EndBASIC, I landed on a handful of generic-enough constructs to serve my purposes… for now.
With this new release, every command and function specifies the structure of its arguments in a declarative fashion. The compiler then uses the generic parser to translate the arguments into call stack values based on what each callable defines. Then, at runtime, every callable just pops typed values off the call stack assuming that they are correctly parsed. Runtime execution of callables is now more efficient than before.
But there was another obstacle to resolve to support user-defined functions, and this was converting the expression evaluator to bytecode.
Bytecode-based expression evaluation
EndBASIC 0.10 shifted the virtual machine from an AST processor to a bytecode evaluator in order to support GOTO and GOSUB. However, the bytecode was really just a trivial lowering of the AST: expressions remained as AST nodes in the bytecode. This was fine for that release, but in order to support calling user-defined functions from within expressions, the expression evaluator needed a way to shift execution from builtin code to interpreted code and viceversa. As other languages call it, addressing “the sandwich problem”.
The way to resolve this was to modify the compiler to emit bytecode for expression evaluation as well, thus turning the bytecode into “real bytecode” not polluted by AST elements. This was easy enough to prototype, but performance obviously suffered: what was previously handled in native code now required executing tens of bytecode instructions.
I had to do significant follow-up work to shift most of the type-checking that happened at runtime to the compiler, thus freeing the VM from complex conditional logic during execution. And the efforts paid off. As a benchmark, I stripped down the Game of Life demo of its UI and made it run 500 iterations on a 200x200 board. EndBASIC 0.10 needs 25 seconds to process this while 0.11 only needs 15 seconds. That’s a 40% improvement, and the optimization work is far from done.
And after doing all of this, I could not resist adding a disassembler. This is just a gimmick right now because the only thing you can do with the disassembled code is peek into the inner workings of the VM, but hey, that’s didactic on its own so it tracks well with the vision for EndBASIC.
Here, witness:
Ready
LIST
a = 3
PRINT 5.2 + a
Ready
DISASM
0000 PUSH% 3 # 1:5
0001 SETV A
0002 PUSH# 5.2 # 2:7
0003 LOAD% A # 2:13
0004 %TO#
0005 ADD# # 2:11
0006 PUSH% 2 # 2:7
0007 CALLB PRINT, 2 # 2:1
Ready
â–ˆ
LCD console
The other big change in EndBASIC has to do with a new backend for its console.
Two years ago, when I was adding GPIO support to EndBASIC, I bought a little LCD for my Raspberry Pi and I envisioned using it to display the EndBASIC console. I did not get to it at the time but, this year, having spent too much time exclusively on Blog System/5, I needed to get back to coding. Working on a driver for the LCD is what sparked my interest to hack EndBASIC again and is what ended up triggering the chain of events that brought you 0.11 today.
I won’t bore you too much with the details about this because I already published a separate blog post that goes into the full story. To recap, let me just show you how this looks like in action:
Known issues
There is one big issue that has come up last minute, and it is that the graphics console does not work on macOS anymore. My understanding is that my SDL driver violates fundamental assumptions of macOS (and possibly Windows as well) by driving the SDL interactions from a secondary thread. I suspect something has changed in macOS that makes this break. I’m not sure if my theory is true though, but even proving the theory is not trivial at all—so I have chosen to ship 0.11 “as is” even if this is broken because 0.10 and earlier are broken too. If you happen to use macOS, I’m sorry, you’ll have to fall back to the web interpreter.
Also, retrofitting bytecode execution into the AST-based evaluator has not been easy and, as a result, the code has accrued significant technical debt. Many things can still be simplified in the codebase and those would yield much faster runtime execution. But once again, I have had to draw a line and call the current implementation “good enough” in order to ship. Rewrites are never a good idea, but I’m at a point where a rewrite of the VM (not anything else!) from scratch would be beneficial.
And to be honest: I need a little break. I have spent all of my free time during the last 5 months on EndBASIC and I have neglected other things I want to do. In particular, Blog System/5 has suffered quite a bit and there are a bunch of topics I want to blog about. So I have had to force the 0.11 release in order to breathe for a little bit. But don’t worry: I’ll be back for more.
In any case, EndBASIC 0.11 should be the best EndBASIC to date. Go forth and play!
Launch interpreter