The TXON, also known as "Text Object Notation", is a proposed format for structured data.
Much less popular than other formats such as JSON, XML, or even INI files - I thought it would still be fun to implement encode and decode words in Factor.
An example TXON might look something like this:
Factor:`
url:`http://factorcode.org`
development:`Started in 2003`
license:`Open source (BSD license)`
influences:`Forth, Lisp, and Smalltalk`
`Encoding
Since TXON uses "`" characters to delimit values, we need to escape them:
: encode-value ( string -- string' ) R" `" "\\`" re-replace ;
To implement encoding in a generic way, we dispatch on the type of object being encoded:
GENERIC: >txon ( object -- string ) M: sequence >txon [ >txon ] map "\n" join ; M: assoc >txon >alist [ first2 [ encode-value ] [ >txon ] bi* "%s:`%s`" sprintf ] map "\n" join ; M: string >txon encode-value ; M: number >txon number>string >txon ;
Decoding
Although the TXON specification includes an EBNF grammar, I am going to show one way to build a parser from scratch. In the tradition of concatenative languages, we will build our decoder from several smaller words.
For symmetry with the encode-value word, we need a way to unescape the ` characters:
: decode-value ( string -- string' ) R" \\`" "`" re-replace ;
Since the TXON format is a series of name:`value` pairs, we can parse the name by finding the separator and then decoding the name (which might contain escaped characters):
: parse-name ( string -- remain name ) ":`" split1 swap decode-value ;
To build a word that finds the first (unescaped) ` character, we will first make a word that looks at adjacent characters, returning true if the second character is an unescaped `:
: `? ( ch1 ch2 -- ? ) [ CHAR: \ = not ] [ CHAR: ` = ] bi* and ;
By grouping the string into adjacent characters, we can find the first unescaped ` (specially handling the case where the first character is an `):
: (find-`) ( string -- n/f ) 2 clump [ first2 `? ] find drop [ 1 + ] [ f ] if* ; : find-` ( string -- n/f ) dup ?first CHAR: ` = [ drop 0 ] [ (find-`) ] if ;
Parsing the value is slightly complicated by the fact that TXON supports values which might themselves be a single value, a sequence of values, or a series of name/value pairs. Basically, that means we need to:
- find the first
`character - checks if the previous character is a
:(indicating a name/value) - parse all name/values if so, otherwise decode the value(s)
That algorithm can be translated into this code:
DEFER: name/values : (parse-value) ( string -- values ) decode-value string-lines dup length 1 = [ first ] when ; : parse-value ( string -- remain value ) dup find-` [ dup 1 - pick ?nth CHAR: : = [ drop name/values ] [ cut swap (parse-value) ] if [ rest [ blank? ] trim-head ] dip ] [ f swap ] if* ;
We want to parse a "name=value" pair, which should be as easy as parsing the name, then the value, then associating into a hashtable:
: (name=value) ( string -- remain term ) parse-name [ parse-value ] dip associate ;
The string might contain a "name=value" pair, or just a single value:
: name=value ( string -- remain term ) [ blank? ] trim ":`" over subseq? [ (name=value) ] [ f swap ] if ;
We finish by building a word to produce all "name=value" pairs, used in the parse-value word earlier.
: name/values ( string -- remain terms ) [ dup { [ empty? not ] [ first CHAR: ` = not ] } 1&& ] [ name=value ] produce assoc-combine ;
Putting all of that together, we can make a word to parse a TXON string, producing "name=value" pairs until exhausted:
: parse-txon ( string -- objects ) [ dup empty? not ] [ name=value ] produce nip ; : txon> ( string -- object ) parse-txon dup length 1 = [ first ] when ;
Try It
You can try this out in the listener:
IN: scratchpad H{ { "a" "123" } } >txon . "a:`123`" IN: scratchpad "a:`123`" txon> . H{ { "a" "123" } }
Can you improve on this? Maybe by using the peg.ebnf vocabulary to create an EBNF parsing word?
The code for this (and a bunch of tests) are on my Github.
