static compilation [was: port of jolt-burg]

John Leuner <jewel@...> writes:

> I've been working on an experimental port of jolt-burg. The main difference is
> that my implementation language is Common Lisp and that my compiler targets
> relocatable ELF object files.

Interesting!

I was inspired by you, and tried my hand at getting similar changes
into jolt2.  The attached patch should apply cleanly on top of SVN
-r407, followed by my jolt2-fixes.patch that I posted earlier.

Here is the function/examples2/slink/README file (also included in the
attached jolt2-slink.patch) for more information:

SLINK 0.2
*********

Slink is the beginning of a statically-linking compiler extension for
Jolt2 (think "Static LINK" or "System LINK").  It was partly inspired
by John Leuner's Common Lisp port of jolt-burg that creates
relocatable ELF objects.  I'm basically attempting to land the same
kinds of features in COLA proper without changing too much of the
existing compiler machinery.

It has many limitations, but is a proof-of-concept and will probably
evolve (the main direction for evolution is to make the changes less
and less dependent on patching Jolt2's Id sources directly, and
instead accomplish its features at runtime via slink.k and friends).

The centerpiece of this system is slink-compile, which currently can
only generate i386 assembly code.  The following new syntax is
introduced:

(define (slink SYMBOL))
  Import SYMBOL from the system linker.  If running with the dynamic
  compiler, this is equivalent to:

     (active SYMBOL (dlsym "SYMBOL"))

  If under the static compiler (slink-compile), this will cause future
  references to SYMBOL to be resolved at link time rather than
  runtime.

(define (slink SYMBOL) EXPRESSION)
  Create a region of memory containing the results of EXPRESSION.
  SYMBOL is exported to the system linker.  Only expressions resulting
  in integers, strings, floats, and lambdas currently work.

To get a feel for things, run "make test" which exercises slink both
at runtime and compile time.  You'll notice that the results of this
execution are a binary executable (slinktest) with no runtime
dependency on COLA.

IMPLEMENTATION
**************

I did a lot of work creating a bunch of slink-specific methods in the
existing COLA compiler.  Now I'm gradually undoing as much of that
work as possible to leave only the core features in Jolt2, so that the
rest can be just another unprivileged application.

Once you understand what the slink syntax does, it shouldn't be too
hard to follow what I needed to do to get things working.  A "grep -i
slink *.st" is very informative (and half the reason I needed to find
a catchy name that wouldn't be buried in the other identifiers).

Toplevel Jolt expressions are gathered up and attached to the system's
initialisation functions so that they are executed at program start
(for i386, using the same method as C++ constructors).

The syntax and actives systems will need to be revisited, probably to
do something like Scheme-48 where a "load-syntax" or "require-syntax"
will import only the syntax elements into the namespace (thereby
avoiding problems with importing undesired lambdas and toplevel code
evaluation by accident).

It would be really nice to implement a few more CodeGenerator backends
to bypass assembly language entirely and generate binaries.

Slink has a naive three-region model for assembler output.  First
comes a bss section with all the variables, then a data section with
all the strings, then finally a text section with all the compiled
functions.  To my knowledge, this is adequate for the platforms on
which COLA already runs, but of course it would be good to be able to
provide something akin to linker scripts in the future to allow much
more flexible layout.

Have fun,

-- 
Michael FIG <michael@...> //\
   http://michael.fig.org/    \//

John Leuner <jewel@...> writes:

> I've been working on an experimental port of jolt-burg. The main difference is
> that my implementation language is Common Lisp and that my compiler targets
> relocatable ELF object files.

Interesting!

I was inspired by you, and tried my hand at getting similar changes
into jolt2.  The attached patch should apply cleanly on top of SVN
-r407, followed by my jolt2-fixes.patch that I posted earlier.

Here is the function/examples2/slink/README file (also included in the
attached jolt2-slink.patch) for more information:

SLINK 0.2
*********

Slink is the beginning of a statically-linking compiler extension for
Jolt2 (think "Static LINK" or "System LINK").  It was partly inspired
by John Leuner's Common Lisp port of jolt-burg that creates
relocatable ELF objects.  I'm basically attempting to land the same
kinds of features in COLA proper without changing too much of the
existing compiler machinery.

It has many limitations, but is a proof-of-concept and will probably
evolve (the main direction for evolution is to make the changes less
and less dependent on patching Jolt2's Id sources directly, and
instead accomplish its features at runtime via slink.k and friends).

The centerpiece of this system is slink-compile, which currently can
only generate i386 assembly code.  The following new syntax is
introduced:

(define (slink SYMBOL))
  Import SYMBOL from the system linker.  If running with the dynamic
  compiler, this is equivalent to:

     (active SYMBOL (dlsym "SYMBOL"))

  If under the static compiler (slink-compile), this will cause future
  references to SYMBOL to be resolved at link time rather than
  runtime.

(define (slink SYMBOL) EXPRESSION)
  Create a region of memory containing the results of EXPRESSION.
  SYMBOL is exported to the system linker.  Only expressions resulting
  in integers, strings, floats, and lambdas currently work.

To get a feel for things, run "make test" which exercises slink both
at runtime and compile time.  You'll notice that the results of this
execution are a binary executable (slinktest) with no runtime
dependency on COLA.

IMPLEMENTATION
**************

I did a lot of work creating a bunch of slink-specific methods in the
existing COLA compiler.  Now I'm gradually undoing as much of that
work as possible to leave only the core features in Jolt2, so that the
rest can be just another unprivileged application.

Once you understand what the slink syntax does, it shouldn't be too
hard to follow what I needed to do to get things working.  A "grep -i
slink *.st" is very informative (and half the reason I needed to find
a catchy name that wouldn't be buried in the other identifiers).

Toplevel Jolt expressions are gathered up and attached to the system's
initialisation functions so that they are executed at program start
(for i386, using the same method as C++ constructors).

The syntax and actives systems will need to be revisited, probably to
do something like Scheme-48 where a "load-syntax" or "require-syntax"
will import only the syntax elements into the namespace (thereby
avoiding problems with importing undesired lambdas and toplevel code
evaluation by accident).

It would be really nice to implement a few more CodeGenerator backends
to bypass assembly language entirely and generate binaries.

Slink has a naive three-region model for assembler output.  First
comes a bss section with all the variables, then a data section with
all the strings, then finally a text section with all the compiled
functions.  To my knowledge, this is adequate for the platforms on
which COLA already runs, but of course it would be good to be able to
provide something akin to linker scripts in the future to allow much
more flexible layout.

Have fun,

--

-- 
Michael FIG <michael@...> //\
   http://michael.fig.org/    \//