Thursday, January 27, 2011

Wolfram|Alpha using Factor

Wolfram|Alpha is a fun and useful "calculation engine" from the creators of Mathematica. Launched in mid-2009, it is available on the web, mobile devices, and from search engines. Last week, Wolfram announced version 2.0 of their API. As part of that announcement, they made it "free" for up to 2,000 non-commercial requests per month.

Since Factor has a visual REPL, I thought it would make a perfect client for Wolfram|Alpha. Starting today, you can do this:

The Wolfram|Alpha API website has both an API reference and an API explorer that can help you learn how it works. To get started, you need to sign up to receive an API ID.

USING: accessors formatting http http.client images.gif
images.http io kernel namespaces sequences splitting
urls.encoding xml xml.traversal ;

We will store the API ID in a symbol, that you can set:

SYMBOL: wolfram-api-id

The API is queried by creating URLs (from the query input and API ID) and parsing the response as XML.

: query ( query -- xml )
    url-encode wolfram-api-id get-global
    sprintf http-get nip string>xml ;

Inside the XML response are "pods" corresponding to the different types of information returned by Wolfram|Alpha. You can choose to receive the "pods" as plaintext, images, sounds, HTML, or Mathematica values. By default, the API returns plaintext and images.

We can create a wolfram-image. word that queries the API, extracts the "pods", outputs the title and images contained in each "pod":

: wolfram-image. ( query -- )
    query "pod" tags-named [
        [ "title" attr print ]
            "img" deep-tags-named
            [ "src" attr http-image. ] each
        ] bi
    ] each ;

Thats how to integrate with Wolfram|Alpha. Before using, remember to set your API ID:

( scratchpad ) "XXXXXX-XXXXXXXXXX" wolfram-api-id set-global

In addition to this, I made some minor style improvements (the gray titles and indentation in the screenshot), created a wolfram-text. word that prints the response as plaintext, and a wolfram. word that detects if you are running Factor on the command-line or the graphic interface and outputs text or images appropriately.

You can find the code on my Github.

Note: I also needed to make a few minor fixes to Factor's libraries to get this to work. They are not yet merged into the main Factor repository, but should be soon:


Anonymous said...

"Visual REPL" should link here:

Sweet article, man! :-)

tgkuo said...

Hi, great codes.

I tested it, it worked well with wolfram-text. but wolfram-image. showed unknown-image-extension error ( i.e. "" -> no extension).
Is the image fromat changed by wolfram?

mrjbq7 said...

Hi, thanks for the comment! Regarding the "no extension" problem, likely you haven't updated to the latest Factor repository which includes a fix to use the content type from the HTTP headers if specified (since the Wolfram URL's do not end in ".gif").

tgkuo said...

I'm running the mac osx edition, v0.94 on my mac mini machine.
I changed it to deveolopment version but still cann't work. Are there fixes for mac available in the Factor repository? how to get it?

mrjbq7 said...

Hi, tgkuo,

The three patches I linked to at the bottom of my post were committed to the main repository just a few days ago. The development binaries on the website might not have them yet.

If you'd like to try it yourself (starting from 0.94 or the development download), you can apply each change yourself (not too difficult, just click the link and make the same change to your files locally), then press F2 for refresh-all to load the changes and it should work.

Let me know if I can help further!

tgkuo said...
This comment has been removed by the author.
tgkuo said...

Thanks, mrjbq7

It worked after I replace these 3 files to my factor directories.

Thanks for your great work.
Hope more examples to learn. ( btw, are there ways to commit these changes automatically on mac osx system ?)

mrjbq7 said...

Hi tgkuo,

I'm glad you got it working! The Factor system is "image-based", so if you make changes to files that you would like to persist, or USE: vocabularies that you want access to - you can use the "Save Factor Image" menuitem under the "Factor" menu. Next time you launch Factor, it should have persisted those changes.

tgkuo said...

I noticed that factor had problems in parsing http source to xml format, It can not tolerate not well formed one ( e.g. lacking closing tag).
And what more, factor had problems in showing image types of jpeg and bitmap.
The wolfram-alpha gave us the well formed xml file and gif image, so, bypassed these problems.

What I felft is that factor community is small and young so far compared to the mainstream laguages, so many of it's vocabularies are not well developed or had bugs. Am I wrong?

mrjbq7 said...

Hi tgkuo,

Let me try and answer those, one at a time.

1) HTML isn't really XML, although the factor libraries handle it "mostly". A better way would be to implement the HTML5 parsing algorithm into a specific html parsing vocabulary. I know a few people have mentioned it, but am not sure it is being worked on.

2) I'd love to see examples of images that break the image loader. Those bugs should be fixed if possible.

3) The Factor community is fairly young. The language was started in 2003 and has quite a lot of functionality in the various vocabularies, more so than any other concatenative language, but could always use more!