JSON to NDJSON

The format bulk loaders insist on

You have a JSON array and something refuses it. BigQuery calls what it wants newline-delimited JSON and rejects an array outright. The Elasticsearch bulk API reads a stream of lines and will not accept a document. Batch endpoints, log shippers and most streaming consumers are the same. All of them want NDJSON: one complete JSON value per line, no enclosing brackets, no commas between records.

The reason is not fussiness. A line-delimited file can be appended to without rewriting it, split across workers at any newline, and read a record at a time regardless of how large it gets. An array has to be parsed as one document, so a loader would have to hold the entire thing in memory before it could begin.

So this takes your array and writes each element on its own line, compact. Compact is not a preference — a record spread over several lines is no longer one line, and the file stops being NDJSON at all. That is worth stating because "format the output nicely" is the one instinct that breaks this particular conversion.

Long integers keep every digit, which matters more here than almost anywhere: the arrays people load in bulk are full of primary keys and event IDs, and a naive round trip rounds anything past fifteen digits without a word. Nothing is uploaded.

Building the file

1. Paste the JSON array – An array of objects is the usual case. A single object works too and becomes one line, which is what you want when you are testing a loader with one record.
2. Read the record count – The chip beside the output is the number of lines written, which should match the number of records you expected to load. It is the cheapest check there is against a truncated paste.
3. Copy or download it – Download writes a .jsonl file ready to upload. Copy takes the whole thing regardless of what the editor is showing.
4. Check it if the load matters – NDJSON to Table reads the result back and will name any line that does not parse — a useful last look before a load you cannot easily undo.

The Elasticsearch bulk API is the exception worth knowing about: it wants alternating lines — an action line, then the document, then the next action line. This page writes one line per array element, so it produces the document lines and you still have to interleave the actions. Every other consumer listed here takes the output as it stands.

An array of three, as a load file

Nothing subtle happens here, and that is rather the point — the value is in what does not change. The 19-digit event IDs come through with every digit, where a round trip through JSON.parse would have rounded all three at the sixteenth.

[
  { "eventId": 9007199254740993,
    "type": "attach" },
  { "eventId": 9007199254740994,
    "type": "detach" }
]

# BigQuery rejects this shape

{"eventId":9007199254740993,"type":"attach"}
{"eventId":9007199254740994,"type":"detach"}

# compact is not a style choice -
# a record split over lines is no
# longer one line, and the file
# stops being NDJSON.

# ids past 2^53, digit for digit.

What people use it for

Preparing a BigQuery load file

BigQuery will not load a JSON array — the documentation is explicit that the format is newline-delimited JSON. Converting an export or a query result before uploading it to Cloud Storage is the most common reason people end up here.

Building a batch request

Batch endpoints that process many items in one submission generally take a .jsonl file, one request object per line. Assembling the array first and converting last is easier than writing lines by hand and getting one comma wrong.

Seeding a log or event pipeline

Test fixtures for anything that consumes a stream want the stream format. One record per line means you can feed it a few lines, watch what happens, and append more without rewriting the file.

Making a large file appendable

Converting an array to NDJSON is what makes it possible to add records later without reading and rewriting the whole document. For anything that grows, that is the difference between an append and a rewrite.

What it handles

• One compact line per record, which is what the format requires — a pretty-printed record would span lines and stop being NDJSON.
• Long integers stay exact. Primary keys and event IDs past 15 digits keep every one of them.
• A single object becomes one line rather than an error, for testing a loader with one record.
• Parse errors name the position so a bad paste is quick to find.
• Downloads as .jsonl, the extension most loaders expect.
• Nothing is uploaded.

Questions that come up

Why is the output not indented?

Because it cannot be. Every line in an NDJSON file must be a complete JSON value, so a record spread over several lines breaks the format — the second line is not valid JSON on its own. Compact output is not a style choice here, it is the specification. If you want to read the data rather than load it, NDJSON to Table is the page for that.

What is the difference between .jsonl and .ndjson?

The extension, and nothing else. JSON Lines and NDJSON describe the same convention. .jsonl is the more common extension; some tools also accept plain .json, which is how people end up handing an array to something that wanted lines.

My input is a nested array of arrays.

Each top-level element becomes one line, so an array of arrays produces a line per inner array — valid NDJSON, since an array is a JSON value. Whether your loader accepts it is a separate question; most bulk endpoints expect an object per line.

Will this round my ID numbers?

No, and it is the failure that costs the most when it happens, because a rounded primary key loads successfully and is simply wrong. A JavaScript number holds about 15 exact digits, so JSON.parse silently turns 8901240544102066246 into 8901240544102066000. The digits are carried through as text here and written back as bare numbers.

Can I do this at a terminal instead?

Yes, and for a large file you should: jq -c '.[]' events.json > events.jsonl. The -c is the important part — without it jq pretty-prints and the result is not NDJSON. The jq manual covers the rest. This page is for when you have the data in front of you and would rather not.

Related tools

Worth reading

• JSON Lines – The rules, including why every line has to stand on its own
• RFC 8259 — JSON – What each individual line has to be, and its silence on numeric precision