As a diversion today, I added "color tab completion" to the Factor environment. I even made the color names render with their assigned color value:
For the curious, I extended the tools.completion vocabulary to implement a "colors-matching" word and then added a "colors-completion" type to the UI.
We've known for awhile that the Factor user interface is a little bit slow when displaying tables with many rows. I wasn't sure this was a big problem until a detailed bug report was filed (thanks @k7f!) that pointed out the opengl.gl vocabulary (with over 2,000 symbols in it) was "virtually unscrollable".
After poking around at the profiler output, I noticed that some UI gadgets had "baseline alignment" support which was used in the layout process. Some of these calculations were quite expensive and could easily be cached.
After a few changes, and introducing the concept of an "aligned gadget" which provides a mechanism to cache these alignment calculations, the original test case is almost 10 times as responsive as before!
This is available in the master branch on Github.
Function "annotations" are a feature that many languages support. They may take various forms such as decorators in Python or class and method annotations in Java -- having direct (like Python) or no direct (like Java) effect on the code it annotates. Factor has some similar features that I'll demonstrate below.
We can define a simple word to use in this example (that adds 2 to its input):
: foo ( x -- n ) 2 + ;
Using the tools.annotations vocabulary, we can attach a "watch" annotation that prints the inputs and outputs of a word when it is called:
IN: scratchpad \ foo watch
If you print the word definition, you can see how it was modified:
IN: scratchpad \ foo see USING: math tools.annotations.private ; IN: scratchpad : foo ( x -- n ) \ foo entering 2 + \ foo leaving ;
You can call this word (either directly, or indirectly by calling another word which calls it) and see its inputs and outputs. A nice feature of this is that the UI lets you click on these values and see more detail (particularly useful if they are tuples or more complex objects):
IN: scratchpad 3 foo --- Entering foo x 3 --- Leaving foo n 5
You can stop watching a word by calling "reset" on it (or right-clicking on its definition in the UI and choosing "reset") to remove all of its annotations. Currently, a word can only be annotated once.
IN: scratchpad \ foo reset
In addition to "watching", you can also use annotations to track the total running time of a word:
IN: scratchpad \ foo add-timing IN: scratchpad 0 10,000 [ foo ] times . 20000 IN: scratchpad word-timing. foo 0.000594456
It also supports arbitrary annotations, such as adding "before" and "after" logic:
IN: scratchpad \ foo [ "hi" print ] [ "bye" print ] [ surround ] 2curry annotate IN: scratchpad 3 foo . hi bye 5
These annotations can be pretty powerful and were even used to build our code coverage tool.
Donald Knuth pioneered Literate programming as a technique for writing structured programs. The literate programming world typically has more descriptive text than code, so rather than "comment out" the descriptive text, they "comment in" the code.
It is very popular in some communities, for example Haskell, where even many blog posts are written in a Literate Haskell style. This is done by creating a .lhs file instead of a .hs file. In it, all lines starting with > are interpreted as code, everything else is considered a comment.
Graham Telfer asked a question on the mailing list if Factor supported literate programming. I thought it might be fun to implement some of the ideas quickly, below I'll share how it works.
Factor uses a lexer object to turn a stream of text into tokens that are used by the parser to turn tokens into Factor objects and definitions. We can extend the lexer system to create a version that skips over any lines that are not prefixed by a > character.
First, we define our literate-lexer sub-class:
TUPLE: literate-lexer < lexer ; : <literate-lexer> ( text -- lexer ) literate-lexer new-lexer ;
Second, we implement the skip-blank word to skip over all lines that are just comments:
M: literate-lexer skip-blank dup column>> zero? [ dup line-text>> [ "> " head? [ [ 2 + ] change-column call-next-method ] [ [ nip length ] change-lexer-column ] if ] [ drop ] if* ] [ call-next-method ] if ;
We can then create a quick syntax word that looks for an end token and parses all the lines between:
SYNTAX: <LITERATE "LITERATE>" parse-multiline-string string-lines [ <literate-lexer> (parse-lines) append! ] with-nested-compilation-unit ;
Now, you can do something like this:
IN: scratchpad <LITERATE This is a big wall of text with no Factor code... Does this work? 1 1 + . I bet it didn't... maybe the following works: > 8675309 . Yay! You can create functions that span multiple lines > : foo ( -- x ) We interrupt this program to bring you this: > 12 ; Now, we can run foo: > foo . LITERATE> 8675309 12
It might be nice to automatically support .lfactor files, but this is a quick prototype to see if it makes sense. Not bad for ten or so lines of code?
It is available now in the development branch of Factor, in the literate vocabulary.
"The reports of my death are greatly exaggerated" - Mark Twain
I'm very pleased to announce the release of Factor 0.95!
OS/CPU Windows Mac OS X Linux x86 x86-64 Source code: 0.95
Note: it looks like the Mac OS X binaries unintentionally require 10.8 or greater, due to an upgrade of our build farm. If you want to use it on 10.6 or 10.7 before we make a fix, you can build from source:
git clone https://github.com/slavapestov/factor.git && cd factor && ./build-support/factor.sh update
This release is brought to you with over 2,500 commits by the following individuals:
Alex Vondrak, Alfredo Beaumont, Andrew Pennebaker, Anton Gorenko, Brennan Cheung, Chris Double, Daniel Ehrenberg, Doug Coleman, Eric Charlebois, Eungju Park, Hugo Schmitt, Joe Groff, John Benediktsson, Jon Harper, Keita Haga, Maximilian Lupke, Philip Searle, Philipp Brüschweiler, Rupert Swarbrick, Sascha Matzke, Slava Pestov, @8byte-jose, @yac, @otoburb, @rien
In addition to lots (and lots!) of bug fixes and improvements, I want to highlight the following features:
Some possible backwards compatibility issues:
<int>" to "int <ref>" and "*int" to "int deref"( -- )" and "(( -- ))" stack effect syntaxFactor is a concatenative, stack-based programming language with high-level features including dynamic types, extensible syntax, macros, and garbage collection. On a practical side, Factor has a full-featured library, supports many different platforms, and has been extensively documented.
The implementation is fully compiled for performance, while still supporting interactive development. Factor applications are portable between all common platforms. Factor can deploy stand-alone applications on all platforms. Full source code for the Factor project is available under a BSD license.
f (Doug Coleman)resolv.conf files (Doug Coleman)smart-with (John Benediktsson)double>ratio (John Benediktsson)v^, v^n, n^v (John Benediktsson)string-lines, add split* (John Benediktsson)>words (John Benediktsson)Factor has nice cross-platform support for network programming. As is fairly typical, I am going to use an "echo server" to demonstrate how the libraries work.
The basic logic for an "echo server" is to read input from a client, write it back to them, and repeat until the client disconnects. Using the input and output stream abstraction, we can build a word which does this in a general manner, recursing until f is read indicating end-of-stream:
: echo-loop ( -- ) 1024 read-partial [ write flush echo-loop ] when* ;
Our "echo server" will use TCP (i.e., connection-oriented networking) and the general server abstraction that comes with Factor with a binary encoding. The server framework uses the name "echo.server" to automatically log client-related messages (such as connect and disconnect events as well as errors) to $FACTOR/logs/echo.server. We specify the echo-loop quotation as the handler for clients:
: <echo-server> ( port -- server ) binary <threaded-server> swap >>insecure "echo.server" >>name [ echo-loop ] >>handler ;
We can start an echo server on, for example, port 12345:
IN: scratchpad 12345 <echo-server> start-server
Testing this is pretty easy using Netcat or a similar client:
$ nc localhost 12345 Hello, Factor Hello, Factor
This is available as the echo-server vocabulary.
For those of you using terminals, you are probably aware of ANSI escape codes. As a refresher, it allows formatting text with foreground and background colors, underlines, italics, blinking, among other characteristics.
Factor has support for formatted output streams that can apply styles to textual content. It is used extensively by the help system to produce documentation in both the UI and command-line, export to HTML or PDF, and even syntax highlighting.
This morning, I wanted to implement a formatted output stream that can be used in the terminal to produce colored text output easily.
You can look at documentation for the xterm-256color ANSI extensions, but basically you wrap your text with format codes, like so:
Setting foreground to blue:
Setting background to red:
There are varying methods of checking if a terminal supports the 256 color extension, but a simple way is to look at the TERM environment variable:
: 256color-terminal? ( -- ? ) "TERM" os-env "-256color" tail? ;
The specification should tell us how to map RGB color codes to their "256color" code:
RGB ColorsA value between 16 and 231 is used for RGB colors with each color having 6 intensities. Red has a value between 0 and 5 multiplied by 36, Green has a value between 0 and 5 multiplied by 6, and Blue has a value between 0 and 5.
8 to 24 bit color conversion
To display xterm RGB colors as 24 bit RGB colors the following values are suggested for the 6 intensities: 0x00, 0x5F, 0x87, 0xAF, 0xD7, and 0xFF.
Following that guideline, we can build our map of "256color" codes. I also added system colors and grayscale colors, which is left as an exercise for the reader (or you can read the source code):
CONSTANT: intensities { 0x00 0x5F 0x87 0xAF 0xD7 0xFF } CONSTANT: 256colors H{ } ! Add the RGB colors intensities [| r i | intensities [| g j | intensities [| b k | i 36 * j 6 * + k + 16 + r g b 3array 256colors set-at ] each-index ] each-index ] each-index
Given a RGBA Color, we want to convert it to an array of its red, green, and blue components, and then find the "256color" with the smallest "distance" (in this case Euclidean distance) from the desired value.
: color>rgb ( color -- rgb ) [ red>> ] [ green>> ] [ blue>> ] tri [ 255 * round >integer ] tri@ 3array ; : color>256color ( color -- 256color ) color>rgb '[ _ distance ] 256colors [ keys swap infimum-by ] [ at ] bi ;
Converting a color to either a foreground or a background ANSI escape code is now pretty easy:
: color>foreground ( color -- str ) color>256color "\u00001b[38;5;%sm" sprintf ; : color>background ( color -- str ) color>256color "\u00001b[48;5;%sm" sprintf ;
With a basic implementation of the formatted stream protocol, we can now take something like this:
And easily (and automatically!) display it as something like this:
To see more colorful implementation of words:
Or even experiment with various background colors:
This has been committed to Factor's development branch and is available now.
