What's up with Lift

I've been using the Lift web framework for a class project. Lift is written in Scala, which is a statically typed functional language.

The philosophy in these languages tends to be that you should use the type system to prove as much of the consistency of your program as possible about your program so that you don't need to check as much at runtime.

Unfortunately, in web programming your data regularly passes through clients that you have no control over, wiping out all the consistency you so carefully checked. Most of our code is spent tediously pulling unstructured data back into a structured form -- in other words, we don't get much benefit from static types.

But what also surprised me was how many parts of the framework used reflection. Now, all in all the parts using reflection are a tiny fraction of the whole Lift code, but the thing about reflection is it is very noticeable, because it deviates from how the language is supposed to behave. I have decided something: if you want to make Scala into a really great language, find all the places Lift uses reflection, and change the language so that reflection is no longer tempting in those cases.

Managing object lifetime

A rendering snippet like

  1  class FooSnippet extends StatefulSnippet {
  2      val p = S param "p" openOr ""
  3  
  4      def render = "p *" #> p
  5  }

gets instantiated per-request, and has access, when it is instantiated, to the request environment in S. I'm not sure if this one is really Scala's fault. It would have been as easy to do

  1  renderingRules.addStatefulRenderer("FooSnippet") { in =>
  2      val p = S param "p" openOr ""
  3      in |> { "p *" #> p }
  4  }

I guess what Lift is really going for using reflection here is it lets you split the rendering rules across multiple files.

But that's not so bad because it's just managing the lifetime of the whole object. What's really tricky is managing the lifetime of things inside the object...

  1  class Foo extends StatefulSnippet {
  2      object x extends SessionVar[Int](5)
  3  }

Now that's tricky. The data held by x has a longer lifetime than the apparently enclosing object Foo. The class name of x is used as a key to lookup the stored value.

It's not a bad solution, until you realize what would happen in some situations:

  1  (1 to 10) map {_ => new Foo}

Each of the created objects refers to the same session variable. Is this right? Well I don't know! It might be but it might not be and you have no control over it, because it's all done by magic.

So what's really going on here? We're trying to decouple object lifetime from object nestedness. The problem is that if you decouple object lifetime from object nestedness you don't have any more nestedness to play with, and you actually want some kind of nestedness because x's lifetime belongs to the session lifetime.

This seems to be a symptom of the general fact that it's nearly impossible to represent two, independent tree-like structures with the same set of syntax -- either the trees will be heavily tied or you will repeat yourself.

So assuming we can't fix the problem directly let's repeat ourselves.

  1  class Foo(x: Var[Int]) {
  2  }

Now Foo takes x in as an argument so the lifetime is decoupled. Unfortunately now the fact that we want x to live in the session has been erased, and it will end up being spread around:

  1  class Session {
  2      val fooX: Var[Int]
  3  }
  4  
  5  ...
  6  new Foo(currentSession.x)

to two different places neither of which are on Foo. If we had a way to declare extention variables we could do

  1  @extention(Session) val x: Var[Int]
  2  
  3  class Foo(implicit session: Session) {
  4  }

the implicit means that even though we have to pass in the particular session in use, it mostly happens automatically. It also provides a static way to tell which objects require sessions and which don't.

Unfortunately this has got us back to requiring that

  1  (1 to 10) map {_ => new Foo}

all refer to the same x. But what that's really saying is that all the constructed Foos are part of the same FooContainer:

  1  class FooContainer {
  2  
  3      @extention(Session) val x: Var[Int]
  4  
  5      class Foo(implicit session: Session) {
  6      }
  7  }

Now the lifetime of x is tied to the product of Session and FooContainer, which is exactly what we wanted, and gives us the flexibility to chose between

  1  val c = new FooContainer
  2  (1 to 10) map {_ => new c.Foo}

and

  1  (1 to 10) map {_ =>
  2      val c = new FooContainer
  3      new c.Foo
  4  }

Making code operate on multiple levels

It would be nice if

  1  <form>
  2      <p>{textField}</p>
  3      <p>{textField}</p>
  4  </form>

could be used to build up both the HTML and the information about the data it represents. The way it is now, this will necesarily produce a plain NodeSeq and the information that two Strings should be produced on submission has been lost.

This would actually be extremely easy to do using delimited continuations, except that Scala's delimited continuations don't have terribly good type inference, so you have to type too many types to make it worth it.

Splitting up the above code isn't such a bad idea, but transforming the language syntax in this way would be generally useful. I implemented something to do this using TemplateHaskell which I will post when I get around to it. It infers types perfectly, for a reason that actually has nothing to do with Haskell having better type inference that Scala. Basically if you do the desugaring before type-checking, you get a lot of the logic of the type-checker for free. The disadvantage is that error messages become unreadable, though I think that could be solved.