Linux: awk patterns, variables and fields
awk examples: built-in variables, profiling, patterns, counting and length filtering, and extracting fields from a MySQL log.
awk reads its input line by line, tests each line against patterns and runs actions on the matching lines. The examples use GNU awk (gawk); the --dump-variables and --profile options are gawk-specific. Reference: awk built-in variables.
Built-in variables
$0 holds the entire current input line, and $1, $2, ... hold its fields.
Write the global variables and their values to a file, then display it:
awk --dump-variables ''
cat awkvars.out
Profile a program
Run an awk program with profiling enabled. The profile is written to awkprof.out:
awk --profile 'BEGIN{printf"---|Header|--\n"} {print} END{printf"---|Footer|---\n"}' marks.txt > /dev/null
cat awkprof.out
Pattern examples
Print the lines that match a pattern (the two forms are equivalent):
awk '/a/' marks.txt
awk '/a/{print}' marks.txt
Count the lines that match a pattern:
awk '/a/{++cnt} END {print "Count = ", cnt}' marks.txt
Print the lines longer than 18 characters:
awk 'length($0) > 18' marks.txt
Parse the MySQL general log
Extract the third field of the Connect entries, excluding Query entries, and list each value once:
cat /var/log/mysqld_gen.log | awk '/Connect/ && ! /Query/ && gsub(/\t\t/, "") {print $3}' | sort | uniq
Notes
- The general log can contain account names and host names. Review the output before sharing it.
- The longer one-liners from the source note that mixed date handling and masking were removed because they were hard to read and depended on an exact log layout.