The TXON, also known as "Text Object Notation", is a proposed format for structured data.
Much less popular than other formats such as JSON, XML, or even INI files - I thought it would still be fun to implement encode and decode words in Factor.
An example TXON
might look something like this:
Factor:` url:`http://factorcode.org` development:`Started in 2003` license:`Open source (BSD license)` influences:`Forth, Lisp, and Smalltalk` `
Encoding
Since TXON uses "`" characters to delimit values, we need to escape them:
: encode-value ( string -- string' ) R" `" "\\`" re-replace ;
To implement encoding in a generic way, we dispatch on the type of object being encoded:
GENERIC: >txon ( object -- string ) M: sequence >txon [ >txon ] map "\n" join ; M: assoc >txon >alist [ first2 [ encode-value ] [ >txon ] bi* "%s:`%s`" sprintf ] map "\n" join ; M: string >txon encode-value ; M: number >txon number>string >txon ;
Decoding
Although the TXON specification includes an EBNF grammar, I am going to show one way to build a parser from scratch. In the tradition of concatenative languages, we will build our decoder from several smaller words.
For symmetry with the encode-value
word, we need a way to unescape the `
characters:
: decode-value ( string -- string' ) R" \\`" "`" re-replace ;
Since the TXON format is a series of name:`value`
pairs, we can parse the name by finding the separator and then decoding the name (which might contain escaped characters):
: parse-name ( string -- remain name ) ":`" split1 swap decode-value ;
To build a word that finds the first (unescaped) `
character, we will first make a word that looks at adjacent characters, returning true if the second character is an unescaped `
:
: `? ( ch1 ch2 -- ? ) [ CHAR: \ = not ] [ CHAR: ` = ] bi* and ;
By grouping the string into adjacent characters, we can find the first unescaped `
(specially handling the case where the first character is an `
):
: (find-`) ( string -- n/f ) 2 clump [ first2 `? ] find drop [ 1 + ] [ f ] if* ; : find-` ( string -- n/f ) dup ?first CHAR: ` = [ drop 0 ] [ (find-`) ] if ;
Parsing the value is slightly complicated by the fact that TXON supports values which might themselves be a single value, a sequence of values, or a series of name/value pairs. Basically, that means we need to:
- find the first
`
character - checks if the previous character is a
:
(indicating a name/value) - parse all name/values if so, otherwise decode the value(s)
That algorithm can be translated into this code:
DEFER: name/values : (parse-value) ( string -- values ) decode-value string-lines dup length 1 = [ first ] when ; : parse-value ( string -- remain value ) dup find-` [ dup 1 - pick ?nth CHAR: : = [ drop name/values ] [ cut swap (parse-value) ] if [ rest [ blank? ] trim-head ] dip ] [ f swap ] if* ;
We want to parse a "name=value" pair, which should be as easy as parsing the name, then the value, then associating into a hashtable:
: (name=value) ( string -- remain term ) parse-name [ parse-value ] dip associate ;
The string might contain a "name=value" pair, or just a single value:
: name=value ( string -- remain term ) [ blank? ] trim ":`" over subseq? [ (name=value) ] [ f swap ] if ;
We finish by building a word to produce all "name=value" pairs, used in the parse-value
word earlier.
: name/values ( string -- remain terms ) [ dup { [ empty? not ] [ first CHAR: ` = not ] } 1&& ] [ name=value ] produce assoc-combine ;
Putting all of that together, we can make a word to parse a TXON string, producing "name=value" pairs until exhausted:
: parse-txon ( string -- objects ) [ dup empty? not ] [ name=value ] produce nip ; : txon> ( string -- object ) parse-txon dup length 1 = [ first ] when ;
Try It
You can try this out in the listener:
IN: scratchpad H{ { "a" "123" } } >txon . "a:`123`" IN: scratchpad "a:`123`" txon> . H{ { "a" "123" } }
Can you improve on this? Maybe by using the peg.ebnf
vocabulary to create an EBNF parsing word?
The code for this (and a bunch of tests) are on my Github.
1 comment:
very similar to yaml, but I think yaml is better
Post a Comment