uniq: remove and count duplicate lines

Updated November 2026

uniq removes repeated lines, or counts them. It only compares each line with the one before it, so you almost always sort the input first.

Common options

Examples

sort list.txt | uniq                         # unique lines (same as sort -u)
sort list.txt | uniq -c | sort -rn           # how often each line occurs, most common first
sort emails.txt | uniq -d                    # e-mail addresses that appear more than once
sort a.txt b.txt | uniq -u                   # lines in only one of the two files (if neither file repeats lines itself)
awk '{print $9}' access.log | sort | uniq -c # count HTTP status codes
grep -o "user=[a-z]*" app.log | sort | uniq -c | sort -rn | head

Without sorting

Unsorted input only loses adjacent duplicates:

$ printf 'a\na\nb\na\n' | uniq
a
b
a

To remove duplicates but keep the original order, use awk instead: awk '!seen[$0]++' file. See awk.