Jq

JC - JSON from CLI

OpenBSD

how to build json from cli

we all like json, do we ? https://kellyjonbrazil.github.io/jc/docs/parsers/ping

add package

doas pkg_add jc

try ping

openbsd-box # ping -c 3 1.1.1.1 |jc --ping -p 2>/dev/null
{
  "destination_ip": "1.1.1.1",
  "data_bytes": 56,
  "pattern": null,
  "destination": "1.1.1.1",
  "packets_transmitted": 3,
  "packets_received": 3,
  "packet_loss_percent": 0.0,
  "duplicates": 0,
  "round_trip_ms_min": 9.219,
  "round_trip_ms_avg": 9.826,
  "round_trip_ms_max": 10.158,
  "round_trip_ms_stddev": 0.43,
  "responses": [
    {
      "type": "reply",
      "bytes": 64,
      "response_ip": "1.1.1.1",
      "icmp_seq": 0,
      "ttl": 59,
      "time_ms": 10.158,
      "duplicate": false
    },
    {
      "type": "reply",
      "bytes": 64,
      "response_ip": "1.1.1.1",
      "icmp_seq": 1,
      "ttl": 59,
      "time_ms": 9.219,
      "duplicate": false
    },
    {
      "type": "reply",
      "bytes": 64,
      "response_ip": "1.1.1.1",
      "icmp_seq": 2,
      "ttl": 59,
      "time_ms": 10.101,
      "duplicate": false
    }
  ]
}

Compatible platforms: linux, darwin, freebsd -> had to redirect the stderr to /dev/null because OpenBSD is not (yet) supported officially…

HTMLQ

OpenBSD

stumpled upon some thing cool, htmlq! It’s like jq, but for HTML.

Installation Rust

htmlq need rust. so, let’s install rust first.

doas pkg_add rust

cat << 'EOF' |doas tee -a /etc/profile
# Rust/Cargo
export PATH=$PATH:/root/.cargo/bin

EOF
. /etc/profile

Install HTMLQ

doas cargo install htmlq

some Examples

curl -s https://www.openbsd.org | htmlq --attribute href a |head

Example

user@nixbox$ curl -s https://www.openbsd.org | htmlq --attribute href a |head
goals.html
plat.html
security.html
crypto.html
events.html
innovations.html
faq/faq4.html#Download
anoncvs.html
https://cvsweb.openbsd.org/
https://github.com/openbsd

curl --silent https://www.nytimes.com | htmlq a --attribute href -b https://www.nytimes.com

Example

Jq

OpenBSD

Json Query

some basics about JQ

RAW Data

cat history.shelly.0.SHEM-3#40F52000B661#1.Total.Current.json | jq '.[0:3]'
[
  {
    "val": 2.64,
    "ack": 1,
    "ts": 1607900404883,
    "q": 0,
    "user": "system.user.admin"
  },
  {
    "val": 2.61,
    "ack": 1,
    "ts": 1607900410483,
    "q": 0,
    "user": "system.user.admin"
  },
  {
    "val": 2.58,
    "ack": 1,
    "ts": 1607900416083,
    "q": 0,
    "user": "system.user.admin"
  }
]

Query First Record

cat history.shelly.0.SHEM-3#40F52000B661#1.Total.Current.json | jq '.[0]'
{
  "val": 2.64,
  "ack": 1,
  "ts": 1607900404883,
  "q": 0,
  "user": "system.user.admin"
}

Filter val and ts

cat history.shelly.0.SHEM-3#40F52000B661#1.Total.Current.json | jq '.[0] |.ts,.val'
1607900404883
2.64

Filter val and ts on one line

cat history.shelly.0.SHEM-3#40F52000B661#1.Total.Current.json | jq '.[0] | (.ts |tostring) + ";" + (.val |tostring)'
"1607900404883;2.64"

Filter first 10 val and ts on one line

cat history.shelly.0.SHEM-3#40F52000B661#1.Total.Current.json | jq '.[] | (.ts |tostring) + ";" + (.val |tostring)' |head -10
"1607900404883;2.64"
"1607900410483;2.61"
"1607900416083;2.58"
"1607900421739;2.62"
"1607900427335;2.62"
"1607900433003;2.57"
"1607900438543;2.72"
"1607900444131;2.67"
"1607900449791;2.6"
"1607900455383;2.55"

Filter first 10 val and ts on one line

cat history.shelly.0.SHEM-3#40F52000B661#1.Total.Current.json | jq '.[] |.ts,.val' |paste - - |head -10
1607900404883	2.64
1607900410483	2.61
1607900416083	2.58
1607900421739	2.62
1607900427335	2.62
1607900433003	2.57
1607900438543	2.72
1607900444131	2.67
1607900449791	2.6
1607900455383	2.55

Current over 50A

cat history.shelly.0.SHEM-3#40F52000B661#1.Total.Current.json | jq -c '.[] | select (.val >= '50')'
{"val":52.55,"ack":1,"ts":1607907152399,"q":0,"user":"system.user.admin"}
{"val":52.54,"ack":1,"ts":1607907157975,"q":0,"user":"system.user.admin"}
{"val":52.53,"ack":1,"ts":1607907163639,"q":0,"user":"system.user.admin"}
{"val":52.5,"ack":1,"ts":1607907169220,"q":0,"user":"system.user.admin"}
{"val":52.49,"ack":1,"ts":1607907174863,"q":0,"user":"system.user.admin"}
{"val":52.53,"ack":1,"ts":1607907180639,"q":0,"user":"system.user.admin"}
...

BGP Stuff

dump networks from AS 3303