LLI Wiki — My Digital Knowledge Base

Linux: awk patterns, variables and fields

awk examples: built-in variables, profiling, patterns, counting and length filtering, and extracting fields from a MySQL log.

awk reads its input line by line, tests each line against patterns and runs actions on the matching lines. The examples use GNU awk (gawk); the --dump-variables and --profile options are gawk-specific. Reference: awk built-in variables.

Built-in variables

$0 holds the entire current input line, and $1, $2, ... hold its fields.

Write the global variables and their values to a file, then display it:

awk --dump-variables ''
cat awkvars.out

Profile a program

Run an awk program with profiling enabled. The profile is written to awkprof.out:

awk --profile 'BEGIN{printf"---|Header|--\n"} {print} END{printf"---|Footer|---\n"}' marks.txt > /dev/null
cat awkprof.out

Pattern examples

Print the lines that match a pattern (the two forms are equivalent):

awk '/a/' marks.txt
awk '/a/{print}' marks.txt

Count the lines that match a pattern:

awk '/a/{++cnt} END {print "Count = ", cnt}' marks.txt

Print the lines longer than 18 characters:

awk 'length($0) > 18' marks.txt

Parse the MySQL general log

Extract the third field of the Connect entries, excluding Query entries, and list each value once:

cat /var/log/mysqld_gen.log | awk '/Connect/  && ! /Query/ &&  gsub(/\t\t/, "") {print $3}'  | sort |  uniq

Notes

Mastodon LLIMastodon SocialFosstodon