awk: process text by columns

Updated November 2026

awk reads text line by line, splits each line into fields, and runs a small program on them. It is perfect for pulling columns out of logs and command output, adding up numbers and producing quick reports.

awk [-F separator] 'pattern { action }' [file ...]

Fields

awk splits each line at spaces and tabs by default. $1 is the first field, $2 the second, $NF the last, and $0 the whole line. NR is the line number and NF the number of fields.

awk '{print $1}' access.log                 # first column: client IP
ps aux | awk '{print $2, $11}'              # PID and command
awk '{print $NF}' file.txt                  # last field of each line
awk -F: '{print $1, $7}' /etc/passwd        # user and shell (":" separator)
awk -F, '{print $3}' data.csv               # third column of a simple CSV
awk 'NR==1 || NR==10' file.txt              # lines 1 and 10
awk 'NF' file.txt                           # skip empty lines

Filter rows

awk '$9 == 404' access.log                  # rows whose 9th field is 404
awk '$3 > 1000' sizes.txt
awk -F: '$3 >= 1000 {print $1}' /etc/passwd # human users
awk '/ERROR/ {print $1, $2}' app.log        # lines matching a regex
awk 'length($0) > 120' code.c               # long lines

Sum, count and average

awk '{sum += $10} END {print sum}' access.log                     # total bytes sent
awk '{sum += $1; n++} END {if (n) print sum/n}' numbers.txt        # average
df -k | awk 'NR>1 {used += $3} END {printf "%.1f GB\n", used/1024/1024}'

The END block runs after the last line; BEGIN runs before the first.

Group with arrays

# requests per client IP, busiest first
awk '{n[$1]++} END {for (ip in n) print n[ip], ip}' access.log | sort -rn | head

# bytes per HTTP status code
awk '{bytes[$9] += $10} END {for (s in bytes) print s, bytes[s]}' access.log

# remove duplicate lines but keep the original order
awk '!seen[$0]++' list.txt

Formatted output

awk -F: '{printf "%-15s %5d %s\n", $1, $3, $7}' /etc/passwd
awk 'BEGIN {OFS=","} {print $1, $4}' data.txt          # comma-separated output

A small awk program

#!/usr/bin/awk -f
# slow.awk - report slow requests from an nginx log with $request_time as the last field
$NF > 1.0 { slow++; print $7, $NF }
END       { printf "%d slow requests out of %d\n", slow, NR }

Run it with awk -f slow.awk access.log, or make it executable.

Notes