Wednesday, August 29, 2012

Literate Programming

Donald Knuth pioneered Literate programming as a technique for writing structured programs. The literate programming world typically has more descriptive text than code, so rather than "comment out" the descriptive text, they "comment in" the code.

It is very popular in some communities, for example Haskell, where even many blog posts are written in a Literate Haskell style. This is done by creating a .lhs file instead of a .hs file. In it, all lines starting with > are interpreted as code, everything else is considered a comment.

Graham Telfer asked a question on the mailing list if Factor supported literate programming. I thought it might be fun to implement some of the ideas quickly, below I'll share how it works.

The Lexer

Factor uses a lexer object to turn a stream of text into tokens that are used by the parser to turn tokens into Factor objects and definitions. We can extend the lexer system to create a version that skips over any lines that are not prefixed by a > character.

First, we define our literate-lexer sub-class:

TUPLE: literate-lexer < lexer ;

: <literate-lexer> ( text -- lexer ) literate-lexer new-lexer ;

Second, we implement the skip-blank word to skip over all lines that are just comments:

M: literate-lexer skip-blank
    dup column>> zero? [
        dup line-text>> [
            "> " head?
            [ [ 2 + ] change-column call-next-method ]
            [ [ nip length ] change-lexer-column ]
            if
        ] [ drop ] if*
    ] [ call-next-method ] if ;

We can then create a quick syntax word that looks for an end token and parses all the lines between:

SYNTAX: <LITERATE
    "LITERATE>" parse-multiline-string string-lines [
        <literate-lexer> (parse-lines) append!
    ] with-nested-compilation-unit ;

Try It

Now, you can do something like this:

IN: scratchpad <LITERATE
               This is a big wall of text with no Factor code...
               Does this work?
               1 1 + .

               I bet it didn't... maybe the following works:
               > 8675309 .
               Yay!

               You can create functions that span multiple lines
               > : foo ( -- x )
               We interrupt this program to bring you this:
               > 12 ;

               Now, we can run foo:
               > foo .
               LITERATE>
8675309
12

It might be nice to automatically support .lfactor files, but this is a quick prototype to see if it makes sense. Not bad for ten or so lines of code?

It is available now in the development branch of Factor, in the literate vocabulary.

Posted by at No comments:

Thursday, August 16, 2012

Factor 0.95 now available

"The reports of my death are greatly exaggerated" - Mark Twain

I'm very pleased to announce the release of Factor 0.95!

OS/CPUWindowsMac OS XLinux
x86
0.95
0.95
0.95
x86-64
0.95
0.95
0.95

Source code: 0.95

Note: it looks like the Mac OS X binaries unintentionally require 10.8 or greater, due to an upgrade of our build farm. If you want to use it on 10.6 or 10.7 before we make a fix, you can build from source:

git clone https://github.com/slavapestov/factor.git && cd factor && ./build-support/factor.sh update

This release is brought to you with over 2,500 commits by the following individuals:

Alex Vondrak, Alfredo Beaumont, Andrew Pennebaker, Anton Gorenko, Brennan Cheung, Chris Double, Daniel Ehrenberg, Doug Coleman, Eric Charlebois, Eungju Park, Hugo Schmitt, Joe Groff, John Benediktsson, Jon Harper, Keita Haga, Maximilian Lupke, Philip Searle, Philipp Brüschweiler, Rupert Swarbrick, Sascha Matzke, Slava Pestov, @8byte-jose, @yac, @otoburb, @rien

In addition to lots (and lots!) of bug fixes and improvements, I want to highlight the following features:

  • GTK-based UI backend
  • Native image loaders using Cocoa, GTK, and GDI+
  • Sampling profiler replacing counting profiler
  • Code coverage tool to improve unit tests
  • VM and application-level signal handlers
  • ICMP support and an IPv4 "ping" implementation
  • DNS client and "host" implementation
  • Support frexp and log of really big numbers
  • Cross-platform open URL in webbrowser
  • Cross-platform send file to trash (Mac OS X, Linux, Windows)
  • Speedup bignum GCD and ratio operations
  • Speedup in thread yield performance on Mac OS X
  • CSV library is 3x faster
  • XML library is 2x faster
  • JSON library is 2-3x faster
  • Many stability and performance enhancements

Some possible backwards compatibility issues:

  • Change alien references from "<int>" to "int <ref>" and "*int" to "int deref"
  • Removed Windows CE, BSD, and Solaris platform support
  • Natively support binary (0b), octal (0o), and hexadecimal (0x) number syntax
  • Unify "( -- )" and "(( -- ))" stack effect syntax
  • Change prepend to return type of first sequence to match append behavior
  • Change ".factor-rc" to be ".factor-rc" on all platforms
  • Cleanup specialized array syntax to be more generic and consistent
  • Change to quadratic probing instead of linear probing in hashtables
  • Allow dispatching on anonymous intersections/unions

What is Factor

Factor is a concatenative, stack-based programming language with high-level features including dynamic types, extensible syntax, macros, and garbage collection. On a practical side, Factor has a full-featured library, supports many different platforms, and has been extensively documented.

The implementation is fully compiled for performance, while still supporting interactive development. Factor applications are portable between all common platforms. Factor can deploy stand-alone applications on all platforms. Full source code for the Factor project is available under a BSD license.


New Libraries


Improved Libraries:

Posted by at 6 comments:

Monday, August 6, 2012

Echo Server

Factor has nice cross-platform support for network programming. As is fairly typical, I am going to use an "echo server" to demonstrate how the libraries work.

The basic logic for an "echo server" is to read input from a client, write it back to them, and repeat until the client disconnects. Using the input and output stream abstraction, we can build a word which does this in a general manner, recursing until f is read indicating end-of-stream:

: echo-loop ( -- )
    1024 read-partial [ write flush echo-loop ] when* ;

Our "echo server" will use TCP (i.e., connection-oriented networking) and the general server abstraction that comes with Factor with a binary encoding. The server framework uses the name "echo.server" to automatically log client-related messages (such as connect and disconnect events as well as errors) to $FACTOR/logs/echo.server. We specify the echo-loop quotation as the handler for clients:

: <echo-server> ( port -- server )
    binary <threaded-server>
        swap >>insecure
        "echo.server" >>name
        [ echo-loop ] >>handler ;

We can start an echo server on, for example, port 12345:

IN: scratchpad 12345 <echo-server> start-server

Testing this is pretty easy using Netcat or a similar client:

$ nc localhost 12345
Hello, Factor
Hello, Factor

This is available as the echo-server vocabulary.

Posted by at No comments:
Subscribe to: Posts (Atom)