hasen

I was looking at the coffekup html engine, and as I was trying to use it, I realized something: it’s not really extensible. The same thing is true for Haml, and probably some other html DSLs out there.

While it’s interesting that:

.book
   .title = "Something"

compiles into:

<div class="book"><div class="title">Something</div></div>

It’s not awfully useful for large projects, and its only advantage is a nicer syntax.

The same goes for coffeekup. It’s certainly nice that one can write:

ul ->
   li "Item1"
   li "Item2"
   li "Item3"

But ultimately, I want to write something more like:

ulist "item1" "item2" "item3"

or, in s-expression syntax, I want to write:

(ulist "item1" "item2" "item3")

and have it automatically produce:

(ul (li "item1") (li "item2") (li "item3"))

In other words, I want to be able to define custom tags that blend in naturally with the rest of the syntax and don’t look any different; as if they were a natural part of the language:

 (div 'id "menu"
    (list "item1" "item2" "item3"))

The nice things about s-expressions is they’re perfect for building tree structures. But there’s a slight problem in that html trees are not pure trees of nodes only; each node can have attributes.

But I want to write trees with the least amount of ( parenthesis ) possible, so for a good syntax, I propose that symbols should be used to denote attributes, and then whatever follows the 'attr value pairs should be considered as child nodes; like this:

(node-name 'attr value 'attr value 'attr value node node node node)

There’s no ambiguity here, you know what are the attributes and what are the child nodes.

Once we have that, we’re gonna need a mechanism to define custom tags.

Something like mylist above could be defined like this:

(deftag mylist
    (ul (map li children)))

Here, children is implicit, it’s created by the deftag statement. deftag automatically parses the arguments passed to mylist and creates 'attrs and 'children variables. This means that custom tags can also take attributes and use them.

For example, mylist can specify a class for the ul element by using the 'class attribute passed to it:

(deftag mylist
    (ul 'class attr!class (map li children)))

This way, (mylist 'class "menu" "item1" "item2") is a short-hand for:

(ul 'class "menu" (li "item1") (li "item2))

To make this tolerant and robust, we can make nil attributes not count, so that (div 'class nil content) produces the same result as (div content)

These tags like (div ... stuff ..) don’t have to return strings; they can return tag objects. Then these tag objects can be traversed and the final html can be produced by printing to stdout, like the vanilla html.arc does.

I started working yesterday on implementing these ideas, and I already have a basic working version locally.

I can tell it:

arc> (render-html (div 'class "floating" (span "Piece of text") (p "Some paragraph")))
<div class="floating">
  <span>
    Piece of text
  </span>
  <p>
    Some paragraph
  </p>
</div>
nil

And I can define custom tags:

For example, a ul list:

(deftag items
      (e "ul" 'class attr!class
        (map [e "il" 'class attr!itemclass _] children)))

And usage:

arc> (render-html (items 'class "small_menu" 'itemclass "tiny_item" "item1" "item2" "item3"))
<ul class="small_menu">
  <il class="tiny_item">
    item1
  </il>
  <il class="tiny_item">
    item2
  </il>
  <il class="tiny_item">
    item3
  </il>
</ul>
nil

Notice how you can use arbitrary custom attributes.

And here’s a slightly more useful example:

(deftag csslink
    "takes a list of css files and creates tags to include them"
    (map [e "link" 'rel "stylesheet" 'type "text/css" 'href _] (flat children)))

And here it is in action:

arc> (render-html (csslink "s1.css" "s2.css" "s3.css"))
<link rel="stylesheet" type="text/css" href="s1.css">
</link>
<link rel="stylesheet" type="text/css" href="s2.css">
</link>
<link rel="stylesheet" type="text/css" href="s3.css">
</link>
nil

This is all good, but I wanted a way to make it templatize-able. What I mean is, I want to define a page template, that defines the general structure of the page, and put some placeholders in it, which can be populated later. Further more, it should be able to create new templates based on already existing ones.

And as I was thinking about how to do that, it hit me: this is already possible. Templates are nothing more than custom tags! The placeholders are the custom attributes! And since custom tags are based on other tags, then it’s already possible to create a template based on another template.

Here’s a little demonstration:

(deftag page
    (html
      (head (title attr!title)
            (apply jscript attr!js)
            (apply csslink attr!css))
      (body children)))

Here’s an example of how to use it:

(render-html 
     (page 'title "Html template" 
              'js '("first.js" "second.js") 
          (div "This is my content")))

And here’s the output:

<html>
  <head>
    <title>
      Html template
    </title>
    <script type="text/javascript" src="first.js">
    </script>
    <script type="text/javascript" src="second.js">
    </script>
  </head>
  <body>
    <div>
      This is my content
    </div>
  </body>
</html>

Here’s an example of a template based on the “page” template:

(deftag blogpage
    (page 'title "Blog post" 'js "blog.js" (div 'class "blogpost" children)))

(render-html (blogpage "This is my post"))

And the result, as expected:

<html>
  <head>
    <title>
      Blog post
    </title>
    <script type="text/javascript" src="blog.js">
    </script>
  </head>
  <body>
    <div class="blogpost">
      This is my post
    </div>
  </body>
</html>

---

This project is now 3 days old, I’m putting on github, and naming it “sehm”, as in S-Expression Html Markup.

https://github.com/hasenj/sehm

At the end of the arc tutorial there’s an example of a webapp:

(defop hello req (pr "hello world"))
(asv)

(asv) starts the http server (on port 8080 by default). defop defines a request handler (apparently).

You can save this file as “web.arc” for example, and run it from the command line with:

$ arc web.arc   

Note: arc here is a symlink to arc.sh from the Anarki repo.

This will start the web server and you can go to http://localhost:8080/hello

This is a good start for a webapp, but being spoiled by Flask and it’s auto-reloading of python modules, something doesn’t feel quite right.

I want to make changes to “web.arc”, save the file, and have the server automatically reload it.

I asked on the arc forum, and evanmurphy (the guy behind tryarc.org) suggested I could run the server in a thread (thread (asv)) and have the REPL available, this way I can always just (load "web.arc")

Now the problem is, I’m putting the (asv) inside the “web.arc” itself, so if I just reload it, it would start the server again, and that would be bad.

So after some googling, I found you can check if a symbol is bound using

(bound 'symbol)

So instead of:

(= t* (thread (asv)))

We can wrap it in a check:

(if (no:bound 't*)
  (= t* (thread (asv))))

This way we can reload the file without affecting the server thread.

I later expanded this to a more general macro that sets a symbol only once

(mac set-once (s v)
   `(if (no:bound ',s)
      (= ,s ,v)))

(set-once t* (thread:asv))

The second part of the problem is, I wanted the reload to happen automatically when I save the file. It didn’t look like there was any way to do that already, so I had to write one.

We basically want to watch the modification timestamp for changes, and reload the file if it changes. We’ll have to poll the file system for changes.

The function to get the modified timestamp is mtime, which you have to load form “lib/files.arc” (see below for more discussion about that).

My first attempt was kind of a kludge, but then I refactored it.

This function watches the return value of a function func and runs a function on-change when that value changes:

(def watch-fn (func on-change)
     (let init (func)
         (while (is init (func))
            (sleep 2))
     (on-change)))

Ideally I want to watch (mtime file-name) for changes, and have (load file-name) run when that value changes.

I’d have to call:

(watch-fn (fn () (mtime file-name)) (fn () (load file-name)))

There is a bit too much (fn () (...)) boilerplate, so we can macroize it:

; watch the value-exp expression for changes
; and run on-change-exp when it changes
(mac watch (value-exp on-change-exp)
     `(watch-fn (fn() ,value-exp) (fn() ,on-change-exp)))

Now we can:

(watch (mtime "web.arc") (load "web.arc"))

But we’re repeating “web.arc” here, so, one more layer:

(def watch-file (file-name deleg)
     (watch (mtime file-name) (deleg file-name)))

Now we can:

(watch-file "web.arc" load)

But we can still add one more layer:

(def auto-reload (file-name)
     (prn "w/auto-reloading " file-name)
     (watch-file file-name load))

And now we have:

(auto-reload "web.arc")

And it’s working! This feels so cool :D

Now, I mentioned something earlier about mtime being defined in “lib/files.arc”. This is not a problem if your webapp is in the same directory that arc is installed to. But in my case, it wasn’t, so I wanted a way to import that file from anywhere.

As I mentioned, the arc command I’ve used is actually a symlink to arc.sh from the Anarki repo. In that file, arc_dir is used as a variable to hold the directory where arc is installed. If we export it, it will be available to the arc runtime.

At first I was worried that if we export it it would pollute the command line environment, but it won’t. Executing .sh files that have export commands doesn’t actually affect the environment variables in your shell (at least not in bash). It would only pollute your environment if you source the file.

So with that aside, we can grab the environment variable ‘arc_dir’ and use it to build the full path to “lib/files.arc”.

Now, how do we get these environment variables? Well, looking at the definition of mtime, it just calls $.file-or-directory-modify-seconds, and you know what, file-or-directory-modify-seconds is a library function in MzScheme.

(def mtime (path)
  " Returns the modification time of the file or directory `path' in
    seconds since the epoch. "
  ($.file-or-directory-modify-seconds path))

In other words, since Arc is implemented in MzScheme, the standard library is available to Arc. Browsing a bit through it reveals getenv and build-path, so we can get the full path to “lib/files.arc” using:

($.build-path ($.getenv "arc_dir") "lib/files.arc")

Or generalize that as:

(def import (path) (load ($.build-path ($.getenv "arc_dir") path)))

Now we can (import "something.arc") from any where, and it will load “something.arc” relative to the directory where Arc is installed.

Now, this hack is a kludge, so be a bit careful with it if you ever decide to use it.

For general programming, I don’t know if Arc will hold up against python.

But one thing that Arc has the advantage of is that macros almost eliminate the need for special-purpose libraries that implement some DSL such as html templating.

This is a sketch of an html page I’m wanting to write

(html 
    (head
        (title "What's up")
        (script "mystuff.arc")
        (jscript "/javascript/native.js")
        (css "mycss.arc"))
    (body
        (form 
            (text-field 
                (name "what") (label "Remind me to")
                (gray-hint "Pay my phone bill") 
                (hover-hint "This will be the subject of your email"))
            (fuzzy-time-field "time")
            (nice-button "Remember to remind me, go!"))
        (footer "Find the source code on " (link "gitbut" github-url*))
        ))

It’s just a form on a page; nothing really that fancy. Let’s see if I can implement the lower levels of this so that such a code would render the page exactly the way I’d expect it to.