Squint - Yet dart another JSON processing library

There is a plethora of options if you are looking for a JSON processing library in dart. Two of the most common choices are Dart’s own convert package and the Flutter Favorites library json_serializable. Both are good options but not without their limitations. I decided to write my own JSON processing library which fulfills all my requirements. As they say: “If you want something done right, do it yourself”.

Hello, Squint

Let me introduce squint and let’s talk about its main features:

  • Deserialize JSON properly including (nested) arrays.
  • Deserialize JSON without writing data classes.
  • Code generation.
  • JSON formatting
  • Does not require build_runner or dart:mirrors.
  • Extensible: Write and reuse custom JSON data converters.

Deserialize JSON properly including (nested) arrays.

My project klutter generates dart classes based on data classes written in Kotlin. This code generation process is not very straightforward due to how dart:convert processed (deeply) nested array values.

This does not work for example (it will result in an exception being thrown: type ‘List<dynamic>’ is not a subtype of type ‘List<List<String»’ in type cast):

final json = '''
  {
      "multiples" : [
          [
              "hooray!"
          ]
      ]
  }
''';

List<List<String>> multiples = 
    jsonDecode(json)['multiples'] as List<List<String>>;

Instead, you will have to cast every sub-list to return the proper Dart type:

    List<List<String>> multiples = (jsonDecode(json)['multiples'] as List<dynamic>)
      .map((dynamic e) => (e as List<dynamic>)
        .map((dynamic e) => e as String)
        .toList()
    ).toList();

I don’t like it. I want a strongly typed data class which does not require writing so much repetitive code. So in squint you can just do the following:

    final json = '''
          {
              "multiples" : [
                  [
                      "hooray!"
                  ]
              ]
          }
        ''';

    final multiples = json.jsonDecode.array<List<String>>("multiples");

    assert(multiples is List<List<String>>);

Deserialize JSON without writing data classes.

If you want to store the entire JSON String and use the data in your business logic then it makes sense to write a custom data class for (de)serialization (or generate it, which we will discuss in the next paragraph). But sometimes you just want to store the JSON and retrieve one or more nodes. Squint does the same thing as dart:convert (but without the dynamic typing) and then also stores the JSON metadata. Each node is stored as a JsonNode object containing:

  • key (String)
  • value (String)
  • type (Squint AST format)

Given the following JSON String:

{
  "id": 1,
  "annoyance-rate": [
    { "JarJarBinks" : 9000 }
  ]
}

We have 2 (child) nodes we can access with squint:

  // JSON String stored in json variable
  final object = json.jsonDecode;
  // Get the node with key 'id'
  assert(object.integerNode("id").key == "id");
  assert(object.integerNode("id").data == 1);
  // Get the node with key 'annoyance-rate'
  assert(object.arrayNode<Map<String,int>>("annoyance-rate").key == "annoyance-rate");

Code generation.

Squint uses its own AST (Abstract Syntax Tree) to store the JSON in. This AST makes it possible to:

  • Generate data classes from JSON String.
  • Generate data classes from JSON metadata.
  • Generate extension methods for data classes.

Given the following JSON String:

  {
     "id": 1,
     "isJedi": true,
     "hasPadawan": false,
     "bff": "Leia",
     "jedi": [
       "Obi-Wan", "Anakin", "Luke Skywalker"
     ],
     "coordinates": [22, 4.4, -15],
     "objectives": {
       "in-mission": false,
       "mission-results": [false, true, true, false]
     },
     "annoyance-rate": [
       { "JarJarBinks" : 9000 }
     ],
     "foo": null,
     "listOfObjectives": [
       {
         "in-mission": true,
         "mission-results": [false, true, true, true]
       },
       {
         "in-mission": false,
         "mission-results": [false, true, false, false]
       }
     ],
      "simpleMap": {
        "a": 1,
        "b": 2,
        "c": 4
      }
    }

We can generate a dataclass from Dart code:

    // JSON String stored in json variable.
    final object = json.jsonDecode;
    // Instance of a CustomType which is a squint AST class.
    final customType = object.toCustomType(className: "Example")
    // Use a generator extension method to generate the data class.
    print(customType.generateDataClassFile());

Or run the following from the command-line:

    flutter pub rub squint:generate --type dataclass --input /foo/bar/message.json

It will output the following data class:

// Copyright (c) 2021 - 2022 Buijs Software
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.

import 'package:squint_json/squint_json.dart';

/// Autogenerated data class by Squint.
@squint
class Example {
  const Example({
    required this.id,
    required this.isJedi,
    required this.hasPadawan,
    required this.bff,
    required this.jedi,
    required this.coordinates,
    required this.objectives,
    required this.annoyanceRate,
    required this.foo,
    required this.listOfObjectives,
    required this.simpleMap,
  });

  @JsonValue("id")
  final int id;

  @JsonValue("isJedi")
  final bool isJedi;

  @JsonValue("hasPadawan")
  final bool hasPadawan;

  @JsonValue("bff")
  final String bff;

  @JsonValue("jedi")
  final List<String> jedi;

  @JsonValue("coordinates")
  final List<double> coordinates;

  @JsonEncode(using: encodeObjectives)
  @JsonDecode<Objectives, JsonObject>(using: decodeObjectives)
  @JsonValue("objectives")
  final Objectives objectives;

  @JsonValue("annoyance-rate")
  final List<Map<String, int>> annoyanceRate;

  @JsonValue("foo")
  final dynamic foo;

  @JsonValue("listOfObjectives")
  final List<Objectives> listOfObjectives;

  @JsonValue("simpleMap")
  final Map<String, int> simpleMap;
}

@squint
class Objectives {
  const Objectives({
    required this.inMission,
    required this.missionResults,
  });

  @JsonValue("in-mission")
  final bool inMission;

  @JsonValue("mission-results")
  final List<bool> missionResults;
}

JsonObject encodeObjectives(Objectives objectives) =>
    JsonObject.fromNodes(key: "objectives", nodes: [
      JsonBoolean(key: "in-mission", data: objectives.inMission),
      JsonArray<dynamic>(
          key: "mission-results", data: objectives.missionResults),
    ]);

Objectives decodeObjectives(JsonObject object) => Objectives(
  inMission: object.boolean("in-mission"),
  missionResults: object.array<bool>("mission-results"),
);

There are options to control how code is generated which can be used through both the command-line as directly in Dart. Annotations are still only a code generation tool. They are not a requirement for using squint. So the following code is identical in its functionality:

import 'package:squint_json/squint_json.dart';

class Example {
  const Example({
    required this.id,
    required this.isJedi,
    required this.hasPadawan,
    required this.bff,
    required this.jedi,
    required this.coordinates,
    required this.objectives,
    required this.annoyanceRate,
    required this.foo,
    required this.listOfObjectives,
    required this.simpleMap,
  });
  
  final int id;
  final bool isJedi;
  final bool hasPadawan;
  final String bff;
  final List<String> jedi;
  final List<double> coordinates;
  final Objectives objectives;
  final List<Map<String, int>> annoyanceRate;
  final dynamic foo;
  final List<Objectives> listOfObjectives;
  final Map<String, int> simpleMap;
}

@squint
class Objectives {
  const Objectives({
    required this.inMission,
    required this.missionResults,
  });

  final bool inMission;
  final List<bool> missionResults;
}

JsonObject encodeObjectives(Objectives objectives) =>
    JsonObject.fromNodes(key: "objectives", nodes: [
      JsonBoolean(key: "in-mission", data: objectives.inMission),
      JsonArray<dynamic>(
          key: "mission-results", data: objectives.missionResults),
    ]);

Objectives decodeObjectives(JsonObject object) => Objectives(
  inMission: object.boolean("in-mission"),
  missionResults: object.array<bool>("mission-results"),
);

Does not require build_runner or dart:mirrors

Dart mirrors (a package to use reflection) is pretty neat, but unfortunately it has a negative impact on performance. It also does not support Flutter. I want Squint to be as performant as possible, and it has to support Flutter. Therefore, Squint does not rely on dart:mirrors.

Build_runner is a nice way to automate Dart code generation. It does however require (some) setup and introduces extra dependencies to your package. Squint has its own independent code generation tools which can be easily used from Dart code or command-line.

Conclusion

Thank you for taking the time to have a look at yet another JSON processing library. I hope you’re curious enough to try it out. Be sure to check out the quickstart guide which goes more in-depth and shows how to apply squint properly.

Written on January 20, 2023