notes

Log | Files | Refs

gawk.txt (3163B)

      1 Awk
      2 
      3 Remove all ollama models locally:
      4 
      5 ollama list | awk 'NR>1 {print $1}' | xargs -r ollama rm
      6 
      7 ================================================================================
      8 GAWK — quick notes (Arch Linux)
      9 
     10 GNU Awk. Handy for shaping psql or other CLI output in a pipe.
     11 
     12 INSTALL
     13 sudo pacman -S gawk postgresql
     14 
     15 psql is in the postgresql package.
     16 
     17 PSQL: OUTPUT THAT PIPES WELL
     18 Use unaligned, one-field-per-line rows:
     19 
     20 psql -h localhost -U myuser -d mydb -t -A -F','\
     21 -c "SELECT id, name, active FROM users LIMIT 5"
     22 
     23 Flags: -t rows only (no header/footer) -A unaligned -F',' field separator (comma
     24 or tab)
     25 
     26 SIMPLE EXAMPLES
     27 Print one column
     28 psql -h localhost -U myuser -d mydb -t -A -c "SELECT email FROM users" \
     29   | gawk '{ print $1 }'
     30 
     31 With comma-separated psql output:
     32 psql ... -t -A -F',' -c "SELECT id, email FROM users" \
     33   | gawk -F',' '{ print $2 }'
     34 
     35 Filter rows
     36 Keep rows where active (3rd field) is true:
     37 psql ... -t -A -F',' -c "SELECT id, name, active FROM users" \
     38   | gawk -F',' '$3 == "t" { print $1, $2 }'
     39 
     40 Add a CSV header
     41 psql ... -t -A -F',' -c "SELECT id, name FROM users"\
     42 | gawk -F',' 'BEGIN { print "id,name" } { print }'
     43 
     44 Sum a numeric column
     45 psql ... -t -A -c "SELECT amount FROM orders" \
     46   | gawk '{ sum += $1 } END { print sum }'
     47 
     48 key=value lines
     49 psql ... -t -A -F',' -c "SELECT id, name FROM users WHERE id = 42" \
     50   | gawk -F',' '{ printf "id=%s name=%s\n", $1, $2 }'
     51 
     52 COMPLEX EXAMPLE — MULTI-RECORD CLI OUTPUT
     53 -----------------------------------------
     54 When input is not one row per line (nested structs, repeated fields):
     55 - set RS (record separator) to split on blocks, not newlines
     56 - use gawk match() with a 3rd arg for capture groups (needs gawk, not awk)
     57 - loop with match() to collect repeated fields (e.g. roles)
     58 
     59 Turn verbose get-all-profiles output into a table:
     60 kioko cad "$ENV" db user get-all-profiles | gawk '
     61 BEGIN {
     62   RS = "UserProfile"
     63   printf "%-3s %-22s %-20s %-8s %-5s %s\n", "ID", "USERNAME", "NAME", "BLOCKED", "LINK", "ROLES"
     64 }
     65 /TrimmedUser/ {
     66   match($0, /id: ([0-9]+)/, a)
     67   match($0, /username: "([^"]*)"/, u)
     68   match($0, /first_name: "([^"]*)"/, f)
     69   match($0, /last_name: "([^"]*)"/, l)
     70   match($0, /blocked: ([a-z]+)/, b)
     71   match($0, /email_link: ([0-9]+)/, e)
     72   roles = ""
     73   s = $0
     74   while (match(s, /role: ([A-Za-z]+)/, r)) {
     75     roles = roles (roles ? "," : "") r[1]
     76     s = substr(s, RSTART + RLENGTH)
     77   }
     78   printf "%-3s %-22s %-20s %-8s %-5s %s\n", a[1], u[1], f[1] " " l[1], b[1], e[1], roles
     79 }'
     80 
     81 What each piece does:
     82 RS = "UserProfile"     split stream on each profile block
     83 /TrimmedUser/          only process records with that marker
     84 match($0, /.../, arr)  put capture group 1 into arr[1]
     85 while (match(s, ...))  walk the string; collect every role:
     86 printf "%-3s ..."      fixed-width columns
     87 
     88 Same idea works on any verbose tool output, or psql nested text
     89 (e.g. json_agg) when flat -F splitting is not enough.
     90 
     91 TIPS
     92 ----
     93 * Match gawk -F to the separator you pass to psql -F.
     94 * Use gawk (not plain awk) for match(str, regex, array) captures.
     95 * Quote SQL in -c "..." so the shell does not eat $ in awk patterns.
     96 * In psql, \copy and \o are fine interactively; gawk fits scripts and pipes.