Tuesday, July 23, 2013

When building web applications, it is often tempting to look for symmetry in the APIs for the creation and the retrieval of data. We are also nudged in that direction by the popularity of REST and its emphasis on treating data as resources. And by ORMs that encourage a single representation of an entity (the model) for CRUD operations.

Don't be tricked. That symmetry may exist on desktop (or device) applications where APIs are local, but isn't truly there for web applications. Give it up.

We are better off building APIs that are tailored to the specific act being performed. This goes back to the practice of making remote APIs coarse-grained (minimize chattiness, messages are expensive because of latency) and local APIs fine-grained (messages are cheap).

When we create/update entities, we need one representation that contains attributes (strings/numbers/etc) and references to other entities (ids). On the other hand, retrieval usually involves offering different shapes depending on what the needs of the client are (rendering a list, a show page), and responses usually embed a lot more data about related entities to prevent extra queries later (eager-fetching).

Naive ORMs that map tables to models are unsuitable for the task of serving data this way. We need tools that let you map data from the DB to the response in an more ad hoc way.

I see this capability in libraries like Anorm (even it has other issues), Play JSON (json-coast-to-coast), subset (mongodb library), etc. This was one of the guiding principles behind scoop (sql toolkit) and dynasty (dynamodb toolkit). Incidentally, a lot of it is powered by the concept of typeclasses.

Sunday, February 24, 2013

Scala Inferring Nothing

A lot of times i've run an issue with scala inference and type parameters:
abstract class Activity[T] {
  def apply (): T
}

object Count extends Activity[Int] {
  def apply (): Int = ...
}
// a method that needs to know both A and T
def run [T, A <: Activity[T]] (activity: A) = ...

Trying to use the method:
run(Count)
Produces:
error: inferred type arguments [Nothing,Count.type] do not conform to method run's type parameter bounds [T,A <: Activity[T]]

But it works if you pass the type parameters explicitly:
run[Int,Activity[Int]](Count)
A solution:
def run [T, A <: Activity[T]] (activity: A with Activity[T]) = ...

The rule seems to be that the compiler will usually infer only the parameters that appear directly in the parameter list.



Monday, January 16, 2012

Basic Scala HList

I've been trying to follow this excellent article:
http://apocalisp.wordpress.com/2010/07/08/type-level-programming-in-scala-part-6b-hlist%C2%A0folds/

But I couldn't quite make the jump from the part (a) to part (b).

To try to get to an intermediate mental step, I decided to try to implement an HList with just an append method. Here it is:

sealed trait HList {
  type Appended[L] <: HList
  def append[L](l: L) :Appended[L]
}

final case class HCons [H, T<:HList] (head: H, tail: T) extends HList {
  type Appended[L] = HCons[H, tail.Appended[L]]
  
  def ::[C](c: C): HCons[C, HCons[H, T]] = HCons(c, this)
  def append[L](l: L) = HCons(head, tail.append(l))
}

sealed class HNil extends HList {
  type Appended[L] = HCons[L, HNil]
  
  def ::[C](c: C): HCons[C, HNil] = HCons(c, this)
  def append[L](l: L): Appended[L] = l :: HNil
  
  override def toString = "HNil"
}
object HNil extends HNil

object test {
  def main (args:Array[String]) {
    val a = "Test" :: 24 :: HNil
    val b: HCons[String, HCons[Int, HCons[String, HNil]]] = a append "January"
  }
}

Now that I got that working (some good practice, I suppose), I might be able to make sense of the original article.

Saturday, July 30, 2011

Eager Fetching and Mapping Entities with Scala

Note: This post assumes you are doing the sql-results to objects mapping manually. Which I find it is the best approach (we'll leave the reasons for another post).

Lets say we have an Employee entity that is related (in the database) to a few other entities.
case class Employee (firstName: String, lastName :String)
case class Department (name:String)
case class Project (name:String)

To model the relationship, we COULD create a field on the Employee entity that references the department and the project:
case class Employee (
  firstName :String, 
  lastName :String, 
  department :Department,  // <---
  project :Project         // <---
)
But when displaying a page it might turn out that we only need the employee info, not the department or project. When retrieving them from the database, we might choose to leave those null, but that will mean we'll get an NPE (at runtime of course: very bad) if we happen to reference one of the related entities on the page. We might also choose to use Option but that's not entirely accurate, since they are not "Optional" fields, we KNOW that for a particular page we WILL need, say, the Department to be there.

This is where traits and the ability to apply them at instantiation comes in handy.

As an example, let say I would like to display a list of employees with their respective departments. I can create traits that specify that an entity comes with a related entity:
trait HasDepartment {
  val department :Department
}
trait HasProject {
  val project :Project
}
Now, when rendering the table of employees with their departments, I can use the trait to specify the required entity:
def listEmployees (projects:List[Employee with HasDepartment]) = {
  <table>
    <tr>
      <th>Name</th>
      <th>Department</th>
    </tr>
    {for (e <- employees) yield
      <tr>
        <td>{e.firstName} {e.lastName}</td>
        <td>{e.department.name}</td>
      </tr>
    }
  </table>
}
The requirement only specifies that the Employee must have its related Department. If we try to display the related Project using {e.project.name} we will get a compile-time error complaining about the missing "project" property right away (very good).

Now, how do we actually add the related Department to the Employee when querying the database? Like this:
// let's assume we are using some basic db wrapper, like anorm
val employees = queryResults.map(row => {
  new Employee ( row[String]("employee.first_name"), row[String]("employee.last_name") ) 
      with HasDepartment {
    val department = Department( row[String]("department.name") )
  }
})
It is now encoded on the type that this Employee entity has its related Department entity (eagerly fetched). You can now decide on a case by case basis when to include related entities, and have it checked by the compiler that you are only able to use them when they have been included.

As an extra example, here is the same code that actually requires that the Project entity also be there:
def listEmployees (projects:List[Employee with HasDepartment with HasProject]) = {
  <table>
    <tr>
      <th>Name</th>
      <th>Department</th>
      <th>Project</th>
    </tr>
    {for (e <- employees) yield
      <tr>
        <td>{e.firstName} {e.lastName}</td>
        <td>{e.department.name}</td>
        <td>{e.project.name}</td>
      </tr>
    }
  </table>
}
And the code to map the employee from the database:
// let's assume we are using some basic db wrapper, like anorm
val employees = queryResults.map(row => {
  new Employee ( row[String]("employee.first_name"), row[String]("employee.last_name") ) 
      with HasDepartment with HasProject {
    val department = Department( row[String]("department.name") )
    val project = Project( row[String]("project.name") )
  }
})

Tuesday, March 22, 2011

Play! Framework and Templating with Scala: XML Literals

As you know from the previous post, I have been developing using the Play! / Scala / Scalate / Squeryl combination as my platform. Everything has been working well, but I haven't been that happy with Scalate. I am not a fan of the syntax (ssp or any of the others), the fact that the play plugin is outdated (must compile from source), and that eclipse won't compile it and let me know of errors like it does for plain scala files.

This got me thinking, since Scala supports XML literals, there should be some way to take advantage of them for templating. I had considered it before but had dismissed the idea because:
  • It required some sort of compilation unit (class, object) for each template.
  • It was not designer friendly (see above).
  • I wasn't familiar enough with Scala to really try it.
Well, after some time has passed, it turns out I don't care about designer friendliness (not sure why I cared about it before, I've always gotten a PSD file that I end up turning into a templates myself) and I am more comfortable with scala now. So it was time to consider it again.

So I gave it a try and I am REALLY surprised at how well it works. And you get TONs of benefits: Compile time checks, auto-complete in the IDE, full power of scala, familiar syntax, etc.

Layouts

Layouts were the first thing I needed. A way to create a widget that takes parameters (like html attributes) and that wraps around template code. So I started with an object to hold my layout templates. One method for each template. We can also leverage default values, multiple parameter lists, and by-name parameters to make the syntax for the client code extra nice:

object Templates {
  // a page template
  def page (title:String="Default title")(content: => xml.Elem) = {
    <html>
    <head>
      <title>{title}</title>
    </head>
    <body>
      {content}
    </body>
    </html>
  }

  // a panel template, just as an example
  def panel (label:String="Some label")(content: => xml.Elem) = {
    <div class="panel">
      <div class="panel-label">{label}</div>
      <div>{content}</div>
    </div>
  }
}

[Note: As you can see, the syntax highlighter in this blog is unfortunately not as sophisticated as the scala compiler when it comes to XML literals]

Now you can use these directly on your controllers:

import util.Templates._

object Application extends Controller {
  def index = 
    page(title="Welcome to my Page!") {
      <div>
        <h1>Hello</h1>
        
        <p>Some template markup</p>

        {panel(label="Dashboard panel")(
          <div>
            Panel content
          </div>
        )}
      </div>
    }
}

Loops and Ifs

Within your XML, you can use "for" and "if" as you would in plain scala:
...
<div>
{for (user <- users if user.isAdmin) yield
  <div>
    {user.name} 
    {if (user.isAvailable) 
      <img src="available.png"/> 
      else 
      <img src="unavailable.png"/>
    }
  </div>
}
</div>
...

Intra-Application Links

Play! comes with a really cool way to go from a method-call-to-controller to a url (reverse routing), but there's no equivalent way to do that from scala. But it turns out it is really easy to create a method that will do that for you. I'll put it in a Utils object I can import on my pages:
import play.mvc.results._
object Utils {
    def Url (action: => Any) = new ScalaAction(action).actionDefinition.url    
}
We can now use it like this:
import util.Utils._ // import Url method and others

object Application extends Controller {
  def users = 
    page(title="Users page") {
      <div>
        <a href={Url( Application.user(24) )}>Click to see user page</a>
      </div>
    }

  def user (userId:Int) = {
    ...
  }
}

As you can see, XML literals can work great as the templating solution for a web app.

Monday, December 27, 2010

I have settled on a new platform for webapps

Here it is: Play! Framework, Squeryl, and Scalate.

Getting them to work well together was a bit of a challenge. Play! has a collection of modules in different states of development. I got the best results using the most recent releases at the moment: play 1.1, play-scala 0.8, and squeryl 0.9.4RC3. The most recent scalate module 0.7.2 was too old, and I had to use the development version from github and compile it myself.

Here is one issue I run into:

Given an example controller and action:
object Manage extends ScalateController {
  def index = {
    val someVar = getSomeObjectFromSomewhere()
    Template(someVar) 
  }
}

The syntax hints that Template is an object that gets returned by the action. This is actually not correct. Play takes a bit of a strange approach when handling results in the controller. Result objects inherit from RuntimeException and are "thrown" from a controller's action and they are caught by framework. I suppose it makes it easy to break out of the flow in order to redirect, or something like that. So Template() is actually a method that "throws" a Result object.

This made it difficult at first to work with squeryl, which expects all database access to wrapper on a transaction{} block. For a whole day I kept trying to find the right way to do that trying different versions of the modules and different rendering methods. This was one of my attempts:

object Manage extends ScalateController {
  def index = transaction {
    // some squeryl database stuff 
    Template() 
  }
}

Of course it didn't work since the thrown results breaks the transaction. Then I created a base class that would catch and re-throw the result, something like this:

class MyBaseController extends ScalateController {
  def Template (_args:Any*) {
    var e:Result = null
    transaction { 
      try {
        super.Template(_args:_*)
      } catch {
        case r:Result => e = r
      }
    }
    throw e
  }
}

But that broke the "magic" parsing of local variables to send to the templates (maybe play does a little too much magic sometimes).

Finally I realized squeryl has a way to bind a session to the thread without using the transaction method. So, I was able to use that method and an @Before annotation to get the code to work:

class Manage extends ScalateController {
  @Before 
  def initSquerylSession = 
    SessionFactory.newSession.bindToCurrentThread

  def index = { 
    val projects = from(Repo.projects)(p => 
      where(p.public === 1) 
      select(p)
    ).toList

    Template(projects)
  }
}

Hopefully this post will save someone some time.

Friday, December 10, 2010

Gravity: Project Pages

I am currently working on a new Gravity feature called Pages. As you probably guessed from the name, it will allow users to create "Pages" (documents) for their projects. It is the first Gravity feature built using S2JS so I am pretty excited. All of the interactivity (javascript) is actually being written in Scala and it is working great.

I will also use the new feature to document S2JS. So it has been a great feature to test this project with. I'll keep you posted of when it will be available.