pro/record format/specification 1.0, tooling 0.9.0
GBLN A typed, bounded record format The type travels with the value
GBLN is a record format in which the type and the size limit travel inside the
file, beside the value they describe. A record carries its own type and its own bound:
port<u16>(8080) says what it is and how much room it may take, right there, with no
schema filed somewhere else to be kept in step. What follows is one argument in seven steps, each proving the one before it, and the
places where the honest answer is a shrug say so.
Runs on
- FreeBSD
- Linux
- macOS
- Windows
Built with
- Rust
- C
- Apache-2.0
Runs with
- C
- C++
- C#
- Go
- Java
- JavaScript
- Kotlin
- Perl
- PHP
- Python
- Ruby
- Rust
- Swift
- Tcl
- Specification
- 1.0, final
- Tooling
- 0.9.0, core and bindings
- Kind
- Typed record format, text
- Core
- Rust, with a C FFI beside it
- Languages
- Fourteen, twelve of them on the C core
- Types
- Signed, unsigned, float, bounded string, boolean, null
- Bounds
- In the type:
s32is thirty-two characters - Parsing
- Three rules, one character of lookahead
- Typing
- Optional per field, mixable in one file
- Forms
- An editable source and a compact wire form
- Comment
:|to the end of the line- Source
- ProVivianVossNet/gbln
- Licence
- Apache-2.0
Startbefore the argument
Before the argument, the facts
Two version numbers, and they are not the same claim. The specification is at 1.0 and marked final: the grammar, the type system and the parsing rules are settled, and a file written against them will read the same in five years. The tooling is at 0.9.0, which is the core, the command line and the bindings, and that number is the honest one for software that has not yet had its first public release. The Verbund runs on it: every ticket, every knowledge entry and every roadmap hop behind this site is a GBLN record, which is either a recommendation or a conflict of interest depending on your temperament. So does ZERVO+, whose backup archives are described by a GBLN manifest read before a single byte is unpacked, and which carries its own reader for the format rather than taking on a JSON dependency to describe itself.
The source is at ProVivianVossNet/gbln, Apache-2.0, with the specification, the Rust core, the C FFI and every binding in one tree: one clone brings the lot, which is the arrangement a format wants if it would like to be picked up by somebody who has not met it before.
The argument, in seven steps
- OneThe type is in the fileBeside the value, not in a schema kept somewhere else.
- TwoThe bound is in the types32 is thirty-two characters, and the parser knows it before it reads one.
- ThreeThree rules, one character aheadThe next character decides the structure. No backtracking, no ambiguity.
- FourTypes where you want themStart untyped, add types field by field, mix both in one file.
- FiveTwo forms, one meaningAn editable source and a compact wire form, with nothing lost between them.
- SixWhat it costs in bytesLevel with JSON minified, and a third of it once compressed.
- SevenOne core, fourteen languagesRust and a C FFI, with twelve bindings that all behave the same.
Then, for reference
01the type is in the file
A schema kept elsewhere is a schema kept badly
The usual arrangement puts the data in one file and its rules in another, and asks everybody involved to keep the two in step. They do, mostly, until the afternoon somebody adds a field in a hurry. GBLN puts the type where the value is, so the pair cannot drift apart: there is nowhere for them to drift to.
:| config.gbln, for people and for git
server{
host<s64>(api.example.com)
port<u16>(8080)
workers<u8>(4)
}
Three things are declared and all three are checked as the file is read.
host is a string of at most sixty-four characters. port is an
unsigned sixteen-bit number, so nought to sixty-five thousand five hundred and
thirty-five, and a port of seventy thousand is refused at the gate rather than four hours
later by something further down the pipe. workers fits in a byte.
The gain is not that these checks exist. Every ecosystem has a validator somewhere. It is that they are not a separate step somebody can forget to run, and that the answer to "what shape is this file" is the file.
Which raises the next question: a type says what a value is. What says how much room it will take?
02the bound is in the type
Thirty-two means thirty-two
s32 is a string of at most thirty-two characters. Not a hint, not a
convention: the parser knows the ceiling before it has read the first character of the
value, and a longer one is refused. The same is true of every numeric type, which carries
its width in its name.
A string with its maximum length in the name: 2, 4, 8, 16, 32, 64, 128, 256
Signed and unsigned integers, one to eight bytes, range known at parse time
IEEE 754 single and double, four and eight bytes
Where this earns its keep is the machine that has no spare memory to lose: an allocation that is known before parsing is an allocation that cannot surprise anybody at three in the morning. It is also the reason the format is a reasonable neighbour for Protobuf on a microcontroller, with the difference that a person can still read the file and a text editor can still open it.
And it is the answer to a question that only became urgent recently. A language model writing configuration is a confident author with no idea whether it is right. A field that declares its own bound rejects a hallucinated value on the way in, at the gate, rather than admitting it and discovering the problem in production.
So the file describes itself. What does that cost a parser?
03three rules, one character ahead
The next character decides everything
A record is one of two shapes and the character after the identifier says which. An angle bracket opens a typed terminal value; a brace opens a container. That is the whole of it, and it is why a parser needs to look exactly one character ahead and never has to change its mind.
identifier<type>(value) :| terminal: one value, checked against its type
identifier{ … } :| container: further records inside
The third rule is the comment. :| runs to the end of the line and is
stripped before anything else happens, which is why the sequence is reserved and cannot
appear anywhere else. It is also, read sideways, a face keeping a straight one, and it is
the format's own mark. A comment is after all the one place in a file where a program is
entitled to have an opinion and keep it to itself.
What follows from three rules is duller than it sounds and more useful than it looks. There is no significant whitespace to get wrong, no quoting mode to escape out of, no construct that means one thing here and another there. Two parsers written independently agree, which is a lower bar than it ought to be and one that several popular formats do not clear.
Determinism is a fine thing for a machine. What does it ask of a person starting out?
04types where you want them
Nothing insists you decide today
The type is optional. A field may carry one, and a field beside it may not, in the same file, at the same nesting level, with no ceremony about the difference. So a first draft can be written as quickly as JSON and tightened afterwards, field by field, wherever it turns out to matter.
service{
name(orders) :| untyped, and perfectly legal
port<u16>(8080) :| typed, because this one has bitten before
region<s2>(de) :| two characters, no more
}
This is the part that is genuinely unusual. JSON has no types to add. Protobuf and its relatives want them everywhere before you may write anything at all. GBLN lets the strictness follow the understanding, which is the order those two things actually arrive in.
One file, then, for people. Is it also the file that goes over the wire?
05two forms, one meaning
One to read, one to send
The source form is indented and commented and lives in git, where a person reads it. The wire form is the same records with the whitespace taken out, then compressed, and it is what a cache, a queue or a model context actually receives. Nothing is lost between them, because whitespace outside a value never meant anything in the first place.
:| source, config.gbln
server{
host<s64>(api.example.com)
port<u16>(8080)
workers<u8>(4)
}
:| wire, config.io.gbln, before compression
server{host<s64>(api.example.com)port<u16>(8080)workers<u8>(4)}
The libraries handle the change of clothes themselves: they use the compressed form if it is there and make it if it is not. There is no build step to remember and no generated artefact to check in, which is the sort of absence one only appreciates after having had the alternative.
One binding sits this out. The JavaScript one runs in WASM with no filesystem to write to, so it works from the source form directly. That is a property of where it runs rather than a gap in the format.
Compact, then. Compact enough to be worth mentioning?
06what it costs in bytes
Level with JSON, and a third of it compressed
One hundred employee records, nested, measured as files rather than argued about. The wire form before compression comes to 49,276 bytes against JSON minified at 49,466: a difference of about a hundred and ninety bytes, which is to say none. Compressed, the same records are around 16,500 bytes, a third of the JSON.
One hundred nested employee records, the same data in every format, sizes taken off the files. The compressed figure is XZ, which JSON would also benefit from; the comparison there is between a format that ships with the compression arranged and one where you arrange it yourself.
Now the part that a marketing page would leave out. Against minified JSON the byte saving is essentially nothing, and the format's own README says so in as many words. What is bought for those same bytes is the type check, the bound and the deterministic parse. The argument is reliability at the same price, not a smaller file.
The token figure belongs to a different question. In a model context, where structure is repeated and every repetition is charged, the compact form runs about eighty-six per cent below JSON. That is a claim about prompts and retrieval systems rather than about disks, and it is the reason the format exists at all.
Fine on one machine. What about the other twelve languages in the building?
07one core, fourteen languages
Fourteen implementations would be fourteen dialects
A format with an implementation per language has a specification in name and a committee in practice: each parser is a little more forgiving than the last, in a slightly different direction, and the file that works everywhere is the intersection nobody documented. GBLN has one core in Rust and a C FFI beside it, and twelve bindings that call into it rather than reimplementing it.
Rust, and a C FFI layer over it
C++, C#, Go, Java, Kotlin, Perl, PHP, Python, Ruby, Swift, Tcl
JavaScript, compiled to WASM, where there is no filesystem to share
The count is worth a sentence because the project's own README gets it wrong: it says thirteen and then lists fourteen. The directory settles it, and this page follows the directory. It is a small thing, and a page about a format that refuses unvalidated data would look silly repeating a number it had not checked.
Six platform targets, each in two architectures: FreeBSD, Linux, macOS and Windows on x64 and ARM64.
Tableevery type
Every type, and what it is for
The full set, from the specification. Nothing here is inferred: a type either appears in the grammar or it does not exist.
| Type | Range or bound | Width |
|---|---|---|
i8 | -128 to 127 | 1 byte |
i16 | -32,768 to 32,767 | 2 bytes |
i32 | ±2.1 billion | 4 bytes |
i64 | ±9.2 quintillion | 8 bytes |
u8 | 0 to 255 | 1 byte |
u16 | 0 to 65,535 | 2 bytes |
u32 | 0 to 4.29 billion | 4 bytes |
u64 | 0 to 18.4 quintillion | 8 bytes |
f32 | IEEE 754 single, about 7 digits | 4 bytes |
f64 | IEEE 754 double, about 15 digits | 8 bytes |
s2 … s256 | 2, 4, 8, 16, 32, 64, 128 or 256 characters | bounded |
bool | true or false | 1 byte |
null | the absent value, declared rather than implied | — |
Numbersthe benchmark in full
Four formats, one hundred records
| Format | Bytes | Against JSON minified |
|---|---|---|
| GBLN, compressed | ~16,500 | 67% smaller |
| GBLN, wire form | 49,276 | 0.4% smaller |
| JSON, minified | 49,466 | the baseline |
| TOON | 53,601 | 8.4% larger |
| YAML | 57,701 | 16.6% larger |
In the source form, which is the one under version control, GBLN comes to 66,273 bytes against JSON at indent two on 79,394: seventeen per cent smaller, with the types included. TOON and YAML are smaller again in that column, and carry neither types nor bounds.
Edgeswhere it stops
Where it stops, and what that leaves you
- It is not a binary format. The wire form is text with the whitespace removed and then compressed. Where the last byte and the last microsecond both matter, a binary encoding will still beat it, and Protobuf is a fine thing to reach for.
- The tooling is at 0.9.0. The specification is settled at 1.0 and files written against it will keep; the core, the command line and the bindings have not had a public release yet, and the version says so.
- Bounded strings are a discipline, not a free lunch. Choosing
s32is choosing, and a value that outgrows its bound is rejected rather than quietly truncated. That is the point, and it is occasionally inconvenient at exactly the moment one had hoped it would not be. - The byte saving against minified JSON is nothing worth mentioning. It is mentioned here anyway, because a page that only quoted the compressed figure would be describing a different comparison than the one a reader is holding.
- Ecosystem is JSON's, and will remain so. Every language, every tool and every service speaks it. GBLN is worth choosing where a file has to be right, not where it merely has to be read by something.