Introducing zq: an easier (and faster) alternative to jq

With zq, this SQL pattern would appear as Zed record values that are operated upon via the field x instead of top-level values:

$ echo '{"x":1}{"x":2}{"x":3}' | zq -j 'sum(x)' -
{"sum":6}

To do this with jq, you have to slurp the input objects and emulate stateful dataflow:

$ echo '{"x":1}{"x":2}{"x":3}' | jq -s '[.[].x] | add | {sum:.}'
{
  "sum": 6
}

Would you rather say sum(x) or [.[].x] | add | {sum:.}? Are you starting to get why the stateless model of jq is difficult?

Perhaps I am just more familiar with it, but I find the jq example to be much easier to understand. Yes, it takes more typing, but it's clear (to me) what each step does, whereas the sum(x) example feels like magic and I don't know how to break it down to constituent parts. - Why is my json parsing tool accepting non-JSON input (multiple documents is outside of the JSON spec)? - How can I tell that x refers to a field name? - What is up with {"sum": ...}? I just asked for the sum, not an object with the sum contained within a field named "sum".

Most of these examples deal with inputs that are NOT JSON.

$ echo '1 2 3' | zq -j 'sum(this)' -
{"sum":6}

Uhm, 1 2 3 is not valid input! jq . would reject that input, which is a key feature in my book.

Similarly, for the over command, the goal seems to be to get values out of json into zq's model and working on streams of numbers, rather than getting values in to JSON data model and working on that model directly.

In other words, any dataflow computation whatsoever can be performed upon the sequence of iterated values, not just inputs that fit into a memory-resident array. (emphasis mine)

For some generic calculator tool, that would make sense. However, jq is not a generic calculator, it's a tool for working with JSON data. The JSON model does not have a concept of a "sequence of iterated values", only a concept of arrays. Having to put everything into a JSON model makes perfect sense to me; the syntax can be slightly cumbersome, but the mental model is easy because it is mostly just JSON concepts that I already know.

For me, the primary purpose of jq is parsing JSON and extracting data from it. Performing calculations on the results is a secondary or even tertiary purpose, so jq's design makes more sense than zq. If you're primarily focused on the calculation and just want support for JSON (and streams of JSON documents) as an I/O format, then perhaps zq is better suited.

/r/programming Thread Link - brimdata.io