pro/record format/specification 1.0, tooling 0.9.0

GBLN A typed, bounded record format The type travels with the value

GBLN is a record format in which the type and the size limit travel inside the file, beside the value they describe. A record carries its own type and its own bound: port<u16>(8080) says what it is and how much room it may take, right there, with no schema filed somewhere else to be kept in step. What follows is one argument in seven steps, each proving the one before it, and the places where the honest answer is a shrug say so.

Runs on

  • FreeBSD
  • Linux
  • macOS
  • Windows

Built with

  • Rust
  • C
  • Apache-2.0

Runs with

  • C
  • C++
  • C#
  • Go
  • Java
  • JavaScript
  • Kotlin
  • Perl
  • PHP
  • Python
  • Ruby
  • Rust
  • Swift
  • Tcl
The GBLN mark: the format's comment marker, a colon and a bar, cut out of a green block, which read together are a face keeping a straight one
Specification
1.0, final
Tooling
0.9.0, core and bindings
Kind
Typed record format, text
Core
Rust, with a C FFI beside it
Languages
Fourteen, twelve of them on the C core
Types
Signed, unsigned, float, bounded string, boolean, null
Bounds
In the type: s32 is thirty-two characters
Parsing
Three rules, one character of lookahead
Typing
Optional per field, mixable in one file
Forms
An editable source and a compact wire form
Comment
:| to the end of the line
Source
ProVivianVossNet/gbln
Licence
Apache-2.0

Startbefore the argument

Before the argument, the facts

Two version numbers, and they are not the same claim. The specification is at 1.0 and marked final: the grammar, the type system and the parsing rules are settled, and a file written against them will read the same in five years. The tooling is at 0.9.0, which is the core, the command line and the bindings, and that number is the honest one for software that has not yet had its first public release. The Verbund runs on it: every ticket, every knowledge entry and every roadmap hop behind this site is a GBLN record, which is either a recommendation or a conflict of interest depending on your temperament. So does ZERVO+, whose backup archives are described by a GBLN manifest read before a single byte is unpacked, and which carries its own reader for the format rather than taking on a JSON dependency to describe itself.

The source is at ProVivianVossNet/gbln, Apache-2.0, with the specification, the Rust core, the C FFI and every binding in one tree: one clone brings the lot, which is the arrangement a format wants if it would like to be picked up by somebody who has not met it before.

The argument, in seven steps

Then, for reference

01the type is in the file

A schema kept elsewhere is a schema kept badly

The usual arrangement puts the data in one file and its rules in another, and asks everybody involved to keep the two in step. They do, mostly, until the afternoon somebody adds a field in a hurry. GBLN puts the type where the value is, so the pair cannot drift apart: there is nowhere for them to drift to.

    :| config.gbln, for people and for git
    server{
        host<s64>(api.example.com)
        port<u16>(8080)
        workers<u8>(4)
    }
        

Three things are declared and all three are checked as the file is read. host is a string of at most sixty-four characters. port is an unsigned sixteen-bit number, so nought to sixty-five thousand five hundred and thirty-five, and a port of seventy thousand is refused at the gate rather than four hours later by something further down the pipe. workers fits in a byte.

The gain is not that these checks exist. Every ecosystem has a validator somewhere. It is that they are not a separate step somebody can forget to run, and that the answer to "what shape is this file" is the file.

Which raises the next question: a type says what a value is. What says how much room it will take?

02the bound is in the type

Thirty-two means thirty-two

s32 is a string of at most thirty-two characters. Not a hint, not a convention: the parser knows the ceiling before it has read the first character of the value, and a longer one is refused. The same is true of every numeric type, which carries its width in its name.

What the parser knows in advance
s2 to s256

A string with its maximum length in the name: 2, 4, 8, 16, 32, 64, 128, 256

i8 to i64 · u8 to u64

Signed and unsigned integers, one to eight bytes, range known at parse time

f32 · f64

IEEE 754 single and double, four and eight bytes

Where this earns its keep is the machine that has no spare memory to lose: an allocation that is known before parsing is an allocation that cannot surprise anybody at three in the morning. It is also the reason the format is a reasonable neighbour for Protobuf on a microcontroller, with the difference that a person can still read the file and a text editor can still open it.

And it is the answer to a question that only became urgent recently. A language model writing configuration is a confident author with no idea whether it is right. A field that declares its own bound rejects a hallucinated value on the way in, at the gate, rather than admitting it and discovering the problem in production.

So the file describes itself. What does that cost a parser?

03three rules, one character ahead

The next character decides everything

A record is one of two shapes and the character after the identifier says which. An angle bracket opens a typed terminal value; a brace opens a container. That is the whole of it, and it is why a parser needs to look exactly one character ahead and never has to change its mind.

    identifier<type>(value)   :| terminal: one value, checked against its type
    identifier{ … }            :| container: further records inside
        

The third rule is the comment. :| runs to the end of the line and is stripped before anything else happens, which is why the sequence is reserved and cannot appear anywhere else. It is also, read sideways, a face keeping a straight one, and it is the format's own mark. A comment is after all the one place in a file where a program is entitled to have an opinion and keep it to itself.

What follows from three rules is duller than it sounds and more useful than it looks. There is no significant whitespace to get wrong, no quoting mode to escape out of, no construct that means one thing here and another there. Two parsers written independently agree, which is a lower bar than it ought to be and one that several popular formats do not clear.

Determinism is a fine thing for a machine. What does it ask of a person starting out?

04types where you want them

Nothing insists you decide today

The type is optional. A field may carry one, and a field beside it may not, in the same file, at the same nesting level, with no ceremony about the difference. So a first draft can be written as quickly as JSON and tightened afterwards, field by field, wherever it turns out to matter.

    service{
        name(orders)              :| untyped, and perfectly legal
        port<u16>(8080)            :| typed, because this one has bitten before
        region<s2>(de)             :| two characters, no more
    }
        

This is the part that is genuinely unusual. JSON has no types to add. Protobuf and its relatives want them everywhere before you may write anything at all. GBLN lets the strictness follow the understanding, which is the order those two things actually arrive in.

One file, then, for people. Is it also the file that goes over the wire?

05two forms, one meaning

One to read, one to send

The source form is indented and commented and lives in git, where a person reads it. The wire form is the same records with the whitespace taken out, then compressed, and it is what a cache, a queue or a model context actually receives. Nothing is lost between them, because whitespace outside a value never meant anything in the first place.

    :| source, config.gbln
    server{
        host<s64>(api.example.com)
        port<u16>(8080)
        workers<u8>(4)
    }

    :| wire, config.io.gbln, before compression
    server{host<s64>(api.example.com)port<u16>(8080)workers<u8>(4)}
        

The libraries handle the change of clothes themselves: they use the compressed form if it is there and make it if it is not. There is no build step to remember and no generated artefact to check in, which is the sort of absence one only appreciates after having had the alternative.

One binding sits this out. The JavaScript one runs in WASM with no filesystem to write to, so it works from the source form directly. That is a property of where it runs rather than a gap in the format.

Compact, then. Compact enough to be worth mentioning?

06what it costs in bytes

Level with JSON, and a third of it compressed

One hundred employee records, nested, measured as files rather than argued about. The wire form before compression comes to 49,276 bytes against JSON minified at 49,466: a difference of about a hundred and ninety bytes, which is to say none. Compressed, the same records are around 16,500 bytes, a third of the JSON.

measured49,276Wire form, bytes
measured49,466JSON minified, bytes
measured~16,500Wire form compressed
measured67%Smaller than JSON, compressed

One hundred nested employee records, the same data in every format, sizes taken off the files. The compressed figure is XZ, which JSON would also benefit from; the comparison there is between a format that ships with the compression arranged and one where you arrange it yourself.

Now the part that a marketing page would leave out. Against minified JSON the byte saving is essentially nothing, and the format's own README says so in as many words. What is bought for those same bytes is the type check, the bound and the deterministic parse. The argument is reliability at the same price, not a smaller file.

The token figure belongs to a different question. In a model context, where structure is repeated and every repetition is charged, the compact form runs about eighty-six per cent below JSON. That is a claim about prompts and retrieval systems rather than about disks, and it is the reason the format exists at all.

Fine on one machine. What about the other twelve languages in the building?

07one core, fourteen languages

Fourteen implementations would be fourteen dialects

A format with an implementation per language has a specification in name and a committee in practice: each parser is a little more forgiving than the last, in a slightly different direction, and the file that works everywhere is the intersection nobody documented. GBLN has one core in Rust and a C FFI beside it, and twelve bindings that call into it rather than reimplementing it.

Fourteen, counted honestly
the cores

Rust, and a C FFI layer over it

on the C core

C++, C#, Go, Java, Kotlin, Perl, PHP, Python, Ruby, Swift, Tcl

on its own

JavaScript, compiled to WASM, where there is no filesystem to share

The count is worth a sentence because the project's own README gets it wrong: it says thirteen and then lists fourteen. The directory settles it, and this page follows the directory. It is a small thing, and a page about a format that refuses unvalidated data would look silly repeating a number it had not checked.

Six platform targets, each in two architectures: FreeBSD, Linux, macOS and Windows on x64 and ARM64.

Tableevery type

Every type, and what it is for

The full set, from the specification. Nothing here is inferred: a type either appears in the grammar or it does not exist.

GBLN types with their range and width
TypeRange or boundWidth
i8-128 to 1271 byte
i16-32,768 to 32,7672 bytes
i32±2.1 billion4 bytes
i64±9.2 quintillion8 bytes
u80 to 2551 byte
u160 to 65,5352 bytes
u320 to 4.29 billion4 bytes
u640 to 18.4 quintillion8 bytes
f32IEEE 754 single, about 7 digits4 bytes
f64IEEE 754 double, about 15 digits8 bytes
s2s2562, 4, 8, 16, 32, 64, 128 or 256 charactersbounded
booltrue or false1 byte
nullthe absent value, declared rather than implied

Numbersthe benchmark in full

Four formats, one hundred records

Wire format sizes for one hundred nested employee records
FormatBytesAgainst JSON minified
GBLN, compressed~16,50067% smaller
GBLN, wire form49,2760.4% smaller
JSON, minified49,466the baseline
TOON53,6018.4% larger
YAML57,70116.6% larger

In the source form, which is the one under version control, GBLN comes to 66,273 bytes against JSON at indent two on 79,394: seventeen per cent smaller, with the types included. TOON and YAML are smaller again in that column, and carry neither types nor bounds.

Edgeswhere it stops

Where it stops, and what that leaves you

  • It is not a binary format. The wire form is text with the whitespace removed and then compressed. Where the last byte and the last microsecond both matter, a binary encoding will still beat it, and Protobuf is a fine thing to reach for.
  • The tooling is at 0.9.0. The specification is settled at 1.0 and files written against it will keep; the core, the command line and the bindings have not had a public release yet, and the version says so.
  • Bounded strings are a discipline, not a free lunch. Choosing s32 is choosing, and a value that outgrows its bound is rejected rather than quietly truncated. That is the point, and it is occasionally inconvenient at exactly the moment one had hoped it would not be.
  • The byte saving against minified JSON is nothing worth mentioning. It is mentioned here anyway, because a page that only quoted the compressed figure would be describing a different comparison than the one a reader is holding.
  • Ecosystem is JSON's, and will remain so. Every language, every tool and every service speaks it. GBLN is worth choosing where a file has to be right, not where it merely has to be read by something.