FIF Isn't Forth

Jim Lawless

Several years ago, I created a Forth-like programming language interpreter called FIF ( FIF Isn’t Forth ). I put FIF aside for a while as other events began to consume my time. I ultimately ended up losing all of the source code that I had built to that point.

I had pondered trying to recreate it, but the work I had done in making FIF opened my eyes to things that I’d want to do differently. I had made FIF too lazily inefficient ( all types are strings, continuous interpretation of text, …etc.)

Although I don’t have the source code, I still have the documentation that I had created. I thought that I would combine the various HTML docs that I had written up into a single post here.

FIF History

I suppose the earliest influence on FIF would have been during the mid-80’s when I downloaded a free implementation of the Forth programming language for my 8-bit microcomputer. I went out and bought the book “Starting Forth” by Leo Brodie, having decided to learn a little about Forth.

The next several days were an absolute joy. The freeware Forth I’d downloaded was much more stable ( and much more fun ) than any of the BASIC / C / Pascal compilers that I’d paid for.

While I never became a rabid Forth programmer, I’d always maintained a fascination with the simplicity, flexibility, and expressive power available in such a small language implementation. Note: To categorize Forth as merely another programming language is a bit of an injustice. Forth can also be considered an entire operating system / operating environment, and interactive development environment.

Over the next twenty or so years, I used a number of programming languages in a professional capacity. Most of these languages stemmed from the C/C++ family.

While using these languages, I still read the occasional article on Forth.

By the mid-90’s, Forth coverage was scarce in the glossy trade magazines, but the Forth community remained alive and well on various Internet portals. The Usenet group comp.lang.forth served as a central message exchange area for those interested in Forth systems.

The comp.lang.forth community’s passion for Forth overwhelmed me. Although I often had little to add to the conversations, I lurked and read, and learned a lot from people who had a long professional history with Forth.

In the late 90’s, one of the programs I support professionally was experiencing difficulties interacting with a new piece of peripheral hardware. The application was written in C++. By this time, the C++ development toolset I used was too large to install on the customer’s PC so that I could debug the problem, so I did the next best thing; I implemented a short postfix language embedded directly into the application. It was very crude … simply an interface to the specific hardware control functions of the program … but it allowed me to call the functions interactively and allowed me to examine their results.

After about 15 minutes of tinkering with the user’s system via my postfix language, I was able to identify the incompatibility with the new hardware and I was able to determine how to make it work within the application.

The speed at which I was able to diagnose and fix my application’s problem amazed me. I had only added about 50 lines of C++ code to my application to implement this little postfix language. Granted, it did not have any requisite safety checks or other niceties, but I found it plausible to implement a low-overhead programming language that could be embedded in an application in a modicum of code.

During this period, I was selling ( and continue to sell ) a series of niche applications. I had intended to add some form of scripting system to some of these applications. After the incident above, my attention turned to possible development of a postfix language as an embedded control language.

In the fall of 1999, I posted this message to comp.lang.forth:

Jim Lawless Oct 10 1999, 3:00 am 
Newsgroups: comp.lang.forth 
From: (Jim Lawless) - Find messages by this author 
Date: 1999/10/10 
Subject: Forth as a scripting / extension language

Greetings, all.

I'd be interested in hearing from anyone who has implemented a Forth-like extension language to a given product ( such as a word-processor or terminal program...)

How well was it accepted by the end-user? ( Did they whine and ask for a BASIC-like language? )

Was the user a technician?

What were the pro's and con's of modeling the extension language after Forth?

Thanks!

Jim Lawless

After a response from a prominent Forther Stepen Pelc, I posted this follow-up message:

Jim Lawless Oct 11 1999, 3:00 am 
Newsgroups: comp.lang.forth 
From: (Jim Lawless) 
Date: 1999/10/11 
Subject: Re: Forth as a scripting / extension language

In article , s...@mpeltd.demon.co.uk says...

:: We've done this many a time, ranging from a set up language for 
:: an uninterruptible power supply (UPS), to configuration tools for 
:: Windows applications.

Thank you for your input, Stephen. Perhaps I should clarify my motives a bit.

I currently sell a few low-cost utility programs and entertainment diversions. I'd like to devise ONE scripting engine to use in all of my products.

Let's say for sake of example that I am producing yet another Windows screen-saver authoring tool. ( Loaded question coming up here...) How well do you think the average user of entertainment software would gravitate to a Forth-like extension language?

I have heard of several variations of C, BASIC, and LISP used successfully as extension languages for end-users, but I don't believe I've really ever heard of a system that specifically provided a Forth-like extension language.

I understand that by its very nature, Forth provides the facilities for self-scripting, but I'm curious as to the acceptance of the language by the end-user.

Jim Lawless

The follow-up messages in the thread and other experiments led me to believe that a postfix language was likely the correct pursuit … but I exhaustively researched other embeddable scripting languages such as LUA, TCL, and Guile.

Shortly after posting this set of messages, my attention became a bit diverted … my time to apply to the postfix language project became scarce. So, the postfix language design stagnated.

You’ll note instances in Usenet history where I seemed to be dipping my toes in the postfix pond … reference to a small C-based stack-language was posted to comp.compilers … I built a scripting language known as Backlash as a Java applet and then rebuilt it as a pure client-side JavaScript system so that it was embeddable in your favorite browser.

Along the way, I found empirical evidence indicating that a number of other kindred spirits had constructed their own Forth-like extension languages:

  • Autodesk ATLAST ( Autodesk Threaded Language Application System Toolkit )
  • UNTIL ( UNconventional Threaded Interpretive Language ) from the book Write Your Own Programming Language Using C++
  • FICL ( Forth-Inspired Command Language )
  • pForth ( Phil Burk’s portable embeddable Forth in C )
  • PISTOL ( Portably Implemented Stack-Oriented Language ) Dr. Dobbs Journal, Feb ‘83
  • STOIC ( STring-Oriented Interactive Compiler )

I hope that FIF fulfills the following goals:

  • FIF should be approachable by non-programmers
  • FIF should allow users of my software to stretch said software beyond my original intentions
  • FIF should allow users of my software to control and manipulate critical aspects of the software
  • FIF should facilitate rapid-development
  • FIF should provide facilities to aid in debugging and long term maintenance of FIF code
  • FIF should be something that I can easily support as a software author
  • FIF should lend itself to a variety of applications

I hope you find FIF to be an interesting control system.

How FIF Differs from Forth

  • Overriding a FIF word changes any invocation of that word.
  • FIF does not emphasize knowledge of the machine as a processor would see it. Instead, FIF emphasizes the use of all string data.
  • FIF does not allow the creation of compile-time words.

FIF Semantics

FIF syntax is intentionally oversimplified.

A FIF program consists of a series of words.

A word is any series of characters that excludes the space-character.

Any number of spaces or empty lines can separate FIF words.

When FIF encounters a word beginning with a digit, it treats the words as a base-10 number and pushes the string representation of that number onto the stack.

When FIF encounters the word s", it takes the remainder of characters ( after the separating space ) up to the next double-quotation-mark character ( " ) and pushes that value as a string onto the stack.

When FIF encounters a system word or user-defined word, it executes the word.

A user may override the behavior of any FIF word, by providing their own implementation of the specific word. ( Flow-control and defining words may not be overriden. )

The following words require one or more words to appear to their right when executing ( these words may not be overridden ):

: Var Synonym Include" S" See

FIF Internals

FIF code is interpreted on-the-fly until a defining-word is encountered. When the defining-word : is encountered, FIF begins compiling the remainder of words up to and including the next ; symbol into an internal program space for later execution.

A “dictionary” containing the word name, the word type ( system word, user-defined word, or variable ) is leveraged.

An array of variable descriptors is used to provide access to user-defined variables. Each decriptor contains a pointer and current maximum size of a variable. If the size of an existing variable is less than or equal to the size of an object which is to be assigned, the object is simply overlaid in the existing memory area. Otherwise, the existing memory area is freed and a new memory area is allocated to contain the new string object.

FIF employs garbage-collection on temporary strings by using a simple ring-buffer as a scratchpad. New strings are allocated at the tail end of the buffer. When we reach the end of the buffer, the pointer to the buffer is moved back to the beginning, consuming older temporary strings.

Permanent strings, such as those used in variable assignments, are allocated dynamically via the C function malloc().

The FIF Interface to C

void fif_register_word(char *w,char mytyp,long myexe,int ovrd); Register a new FIF word. Intended for registering system words only by the external C program.

char *fif_pop(); Pop a string from the FIF data stack.

void fif_push(char *); Push a string onto the FIF data stack.

void fif_eval(char *); Evaluate (execute) a FIF snippet. Definitions may be included. If definitions are included in the EVAL'ed code, they will be compiled into an intermediate form and retained for later execution.

char *fif_newstring(char *); Create a new permanent string.

void fif_err(char *,char *); Issue an error message. The first string is intended to be an error message prefix. The second string is intended to be the object of the error.

void fif_init_words(); Initialize the FIF system.

void fif_interact(); Drop into an interactive console with FIF.

The source code for a simple FIF REPL shell is as follows:

#include <windows.h>
#include "fif.h"

extern int __argc;
extern char **__argv;

/*
int __stdcall WinMain (HINSTANCE hInstance, HINSTANCE hPrevInstance,
                    char * szCmdLine, int iCmdShow) {
*/

void main() {
   fif_init_words();
   fif_eval("s\" FIF v 4.0a\" . cr s\" Copyright 2005 by Jim Lawless\" . cr ");
   fif_eval("s\" (Enter BYE to exit FIF.\" .");
   fif_eval("s\" or enter FIF_SH (filename) to run a script.)\" . cr cr");
   if(__argc>1) {
      fif_load(__argv[1]);
   }
   fif_interact();
}

Note: The above can be compiled as a console or GUI app.

FIF Debugging

FIF debugging can currently be handled by invoking the TRACE word.

Once TRACE is invoked, you’ll see the names of all words as they execute.

In a future version of FIF, one will be able to define a special word that will be called by the tracing engine to allow one to homebrew their own tracer.

NOTRACE turns tracing off.

The FIF Core Word Set

Mathematics

* Multiply top two items on stack. Result is left on stack.

+ Add top two items on stack. Result is left on stack.

- Subtract top two items on stack. Result is left on stack.

/ Divide top two items on stack. Quotient is left on stack.

mod Divide top two items on stack. Remainder is left on stack.

random Consume top of stack item as upper limit and gen random non-negative integer under this limit.

Logic and Control-Flow

< Is less than

= Is equal to

> Is greater than

>= Is greater or equal to

<= Is less than or equal to

!= Is not equal to

if Consume top item from stack. If item is zero, take ELSE or ENDIF branch. Otherwise, follow sequential flow until counterpart ELSE or ENDIF is encountered.

else See use above with IF

fendif See use above with IF

begin Define beginning area for loop construct with UNTIL

until Consume top item from stack. If zero, repeat at word after counterpart BEGIN. Otherwise, continue processing words sequentially.

File I/O

Do not use File I/O!!! These words have not been debugged.

fgetc

fgets

fopen

fputs

fputc

fclose

Console I/O

input Input a line from the console and leave resulting string on top of stack.

. Consume top of stack and display on console.

emit Consume top of stack as a number. Display single character representation of number on console.

cr Emit a newline

Stack Manipulation

dup Duplicate the top item on stack non-destructively

rot Move third item on stack to the top of the stack.

swap Swap top two stack items.

drop Remove top item from stack.

String Manipulation

chrtonum

tolower

toupper

substr

strcmp

strlen

stricmp

numtochr

concat

FIF Common Programming Constructs

( Begin comment. End with ) character.

: Begin definition. End with exit ; character.

; End : definition and/or exit.

include" Include filename on the right of the word INCLUDE" after whitespace. This word appends a file to the active work buffer, then EVAL’s it. I do not yet utilize a mechanism to prevent redundant includes.

synonym Does not consume a stack item. synonym old-word new-word …causes the definition of a new word that points to the definition of an existing word. Useful when overriding an existing word. Can be used to invoke original functionality as needed when overriding a word.

see Interactive. see word-name Show the code comprising a word’s definition if it is a user-defined word.

trace Turn debug-tracing on.

notrace Turn debug-tracing off

var Define a variable. var varname …creates a new dictionary entry for the variable and assigns an index into the list of all variables.

set Consumes top two stack items … one of which is a variable index and sets the variable value.

get Consumes top stack item as variable index and leaves the current value of variable on the top of stack.

eval

Miscellaneous Words

bye Exit FIF

words List words to console.

sleep Consume top stack item a a number of milliseconds and sleep for that interval.

msgbox Display messagebox. More info later.

millitime Get time in milliseconds and leave on top of stack.

system Consumes top of stack as a string and executes it under the current command shell.

Sample FIF Code Snippets

Hello, world!

s" Hello, world!" . cr
Hello, world! in a message box
0 s" Title" s" Hello, world!" msgbox

Add and display 5 and 6

5 6 + .

Display 10 Random Integers, under 100 to the console.

( Test random number generation )
( Let's loop 10 times and gen random numbers from 1 to 100 )
: testrnd
   1 ( put counter on top of stack )
   begin
      dup . s" " .
      100 random . cr
      1 + ( inc counter )
      dup
   10 > until
   drop ;
testrnd

Interact with user and see if the entered number is odd or even. Then, list numbers 1 through 10 and display whether they are odd or even.

( Ask the user to enter a number and tell them
  if it's odd or even. Then, list the first 10 positive
  integers and determine if they're odd or even. )

      ( define word well? that displays either "odd" or "even"
      depending on whether the number at the top-of-the stack
      is odd or even )

   : well? dup ( duplicate top stack item for later display )
      2 mod ( divide by 2 leaving the remainder 1=odd 0=even )
      swap ( swap original number and truth-value from mod )
      . ( display original number )
      s" is " . ( display literal )
      if ( compare against mod value )
        s" odd." .
      else
        s" even." .
      endif
      cr
    ; ( end of word )

   : main
      ( display the message )
   s" Enter a non-negative number." . cr
      ( input the number )
   input
   well? ( test the number and display result )
      ( Now see how 1-10 work )
   cr s" Testing numbers 1 through 10..." . cr
   1 ( push 1 )
   begin
      dup
      well?
      1 + dup
   10 > until
   drop ;
main

Display the lower-case alphabet

  : main
  s" a" chrtonum ( get ASCII value of 'a' onto TOS )
  begin
     dup ( duplicate current value )
     emit ( write the character to the console )
     1 + ( increment to next letter )
     dup ( dup it for comparison )
     s" z" chrtonum ( compare greater than "z" )
     >
  until
  drop ;
main

Display the upper-case alphabet by tricking-out the emit word.

  ( Define new "emit" that only displays in upper case )
  : emit numtochr toupper . ;
  : main
  s" a" chrtonum ( get ASCII value of 'a' onto TOS )
  begin
     dup ( duplicate current value )
     emit ( write the character to the console )
     1 + ( increment to next letter )
     dup ( dup it for comparison )
     s" z" chrtonum ( compare greater than "z" )
     >
  until
  drop ;
main