Those methods will wait until the entire response has been downloaded, and then parse it. That's because JSON is not a streaming format
To consume JSON in a streaming way, use jq
Multiplying two objects will merge them recursively: this works like addition but if both objects contain a value for the same key, and the values are objects, the two are merged with the same strategy.
Unfortunately, it doesn't merge/concatenate arrays. Sometimes that's what you want (you want the 2nd value to override the 1st but sometimes not.
If you want it to concatenate instead, here are some workarounds:
https://stackoverflow.com/questions/53661930/jq-recursively-merge-objects-and-concatenate-arrays
If you only need/want to concatenate for some fixed list of keys, you could do it more simply like this (but could get repetitive to repeat for each key you want it for):
⟫ jq -n '[{hosts: ["a"]}, {hosts: ["b"]}] | .[]' | jq -s '.[0] * .[1] * {hosts: (.[0].hosts + .[1].hosts)}'
{
"hosts": [
"a",
"b"
]
}
jq -s '.[0] * .[1]' $config_file $local_config_file
jq '.profile.exit_type= "Normal" | .profile.exited_cleanly = true' ~pi/.config/chromium/Default/Preferences
Colin D asks how to preserve the JSON structure of the array, so that the final output is a single JSON array rather than a stream of JSON objects. The simplest way is to wrap the whole expression in an array constructor:
jq uses the Oniguruma regular expression library, as do php, ruby, TextMate, Sublime Text, etc, so the description here will focus on jq specifics.
systemctl show --no-page iptables \ | jq --slurp --raw-input \ 'split("\n") | map(select(. != "") | split("=") | {"key": .[0], "value": (.[1:] | join("="))}) | from_entries'
Decode of jq command:
Gitlab API can be used