Parsing json with over 22 fields with case classes

We recently started to develop a new product using play framework in the back end and angular js for the client side. The back end is rather light and mainly consist of aggregating data from different web services. We do have a local database and for that we chose MongoDB. So what we have is many sources of data and pretty much all of them provide data in json format. Needless to say json parsing has to work flawlessly.

For the most part that worked great. With play you can create a case class with all the same fields as the corresponding json and then you just create a format for it in the companion class. This is easy and elegant and works great… until…

…until you notice that one of the web services you are calling has more than 22 fields and is a problem because Scala does not allow more than 22 fields in a case class. Normally that’s fine and usually that is a red flag about poor design, but sometimes the web services you use have more than 22 fields in the response objects.

case class FooBar(
   field01: String,
   field02: String,
   field03: String,
// ... many fields ...
   field22: String,
   field23: String)

// compilation errors

object FooBar{
   implicit val foobarFormat: Format[FooBar] = Json.format[FooBar]
}

Luckily there is pretty neat way around this and it might even improve your design. You can use nested case classes and all you need to do is implement a wrapper writer and reader for that class, but first you can create case classes that represent subset of the fields of the response.

case class Foo(
   field01: String,
   field02: String,
// ... many fields ...
   field09: String,
   field10: String)

object Foo{
   implicit val fooFormat: Format[Foo] = Json.format[Foo]
}

case class Bar(
   field11: String,
   field12: String,
// ... many fields ...
   field21: String,
   field22: String,
   field23: String)

object Bar{
   implicit val barFormat: Format[Bar] = Json.format[Bar]
}

So now we have smaller classes that contains subset of the original fields and the only thing missing is a “wrapper” class the represents the complete class. This is rather simple and in this case it has two fields, namely the two classes I just defined.

case class FooBar(
   foo: Foo,
   bar: Bar)

object FooBar {
   implicit val foobarReads: Reads[FooBar] = (
      (JsPath).read[Foo] and
      (JsPath).read[Bar])(FooBar.apply _)

   implicit val foobarWrites: Writes[FooBar] = (
      (JsPath).write[Foo] and
      (JsPath).write[Bar])(unlift(FooBar.unapply))
}

Now in scala you can access fields quite neatly using the dot -notion, but the json is serialized back to the original format. This parsing can be easily tested.

val foobarJson: JsValue = Json.parse("""
  { "field1":"value1", 
    "field2":"value2", 
    ...
    "field22":"value22", 
    "field23":"value23"}
  """)

"Parsing json to nested case classes" should {
  "work just fine" in {
    val foobar = foobarJsonAsString.as[FooBar]
    foobar.foo.field1 === "value1"
// ...
    foobar.bar.field23 === "value23"
  }
}
Advertisements

Posted on 09/10/2014, in Scala, Software and tagged , . Bookmark the permalink. 5 Comments.

  1. Good post!
    I don’t know current state of play’s json lib, but in Scala 2.11.x you don’t have casse classes 22 fields limitation any more – awesome : )

    • Hey, thanks for your comment.

      You are right about Scala 2.11 having no issue with case classes with more than 22 fields. Those case classes however lack the unapply -method which is necessary for parsing json this way so you still get a compilation error. I checked this with a new project using 2.11.

      Besides, you probably also benefit from trying to organize the incoming data to nested classes anyway 🙂

  2. Clear post!
    But I wonder if there exist a way without using nested classes in consideration of cost of code-refactoring.
    Thanks.

  1. Pingback: Error: Play JSON Inception | Active Intelligence

  2. Pingback: Scala JSON | Active Intelligence

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: