AGON: Alternative General Object Notation

Built a binary data interchange format that is 70% faster at encoding and produces 71% smaller files than JSON. Compared against V8's serialisation module, AGON generated files 0.14% smaller for our test payloads.

Approach

JSON prioritises human readability, but this comes at a cost in size and bandwidth. We designed AGON to store data in a custom binary format with a formal grammar specification, then built an encoder and decoder in TypeScript.

The format uses binary markers for primitive data types and start/end markers around non-primitive structures. This eliminates the overhead of string-based encoding while maintaining interoperability with JSON data structures.

The Grammar

Our encoder and decoder adhere to a context-free grammar that covers objects, arrays, numbers, booleans, and strings. The grammar has no left-recursion, and each production rule starts with a unique binary marker, removing the need for look-ahead when parsing.

<S>   ::= <V>
<V>   ::= <O> | <A> | <N> | <B> | <Str>
<N>   ::= <I> | <F>
<I>   ::= t0 <T> | t1 <TT> | t2 <TTTT> | t3 <TTTTTTTT>
<F>   ::= t4 <TTTT> | t5 <TTTTTTTT>
<B>   ::= t6 | t7
<A>   ::= t8 <Ai> t9
<Ai>  ::= <V> | <Ai> <V> | ε
<O>   ::= t10 <Oi> t11
<Oi>  ::= <str> <V> | <Oi> <str> <V> | ε
<str> ::= t12 <P> 0
<P>   ::= <T0> | <P> <T0> | ε
<T0>  ::= 0x1 | 0x2 | ... | 0xFF
<T>   ::= 0x0 | <T0>

Terminal Markers

We use 16 different terminals to represent data types. Each marker is a single byte:

arrayStart:    0xF0    arrayEnd:      0xF1
objectStart:   0xF2    objectEnd:     0xF3
byte:          0xFA    short:         0xFB
int:           0xFC    long:          0xFD
double:        0xFE    float:         0xFF
booleanTrue:   0xF4    booleanFalse:  0xF5
string:        0xF6    buffer:        0xF7
date:          0xEF    null:          0xD0

Encoder

The encoder converts JavaScript objects, arrays, and primitives to binary format. Key design decisions:

Booleans: Single byte (0xF4 or 0xF5) instead of 4-5 bytes for "true"/"false"
Numbers: Type inference determines the smallest C-like type (byte, short, int, long) that can hold the value
Strings: UTF-16 encoded with null termination
Objects/Arrays: Start/end markers with recursive encoding of contents

Decoder

The decoder uses a reverse terminal map to identify markers and a stack to track nested structures. When it encounters an objectStart or arrayStart, it pushes to the stack and creates the appropriate container. Matching end markers pop from the stack.

For numbers, we use a DataView to handle binary conversions according to the platform's endianness automatically.

Benchmarks

We ran 15 benchmark rounds, each generating 15 random objects (roughly 50% arbitrarily nested between 1-20 levels), totalling 225 test objects.

Encoding speed: AGON completed typical serialisation in 16.24 ms versus JSON's 55.08 ms, a 70.5% improvement.

File size: JSON files averaged 2320 bytes; AGON files averaged 667 bytes, a 71.3% reduction. AGON files were also 0.14% smaller than V8's serialisation output.