Bash uniq Command - Remove Duplicate Lines

Using the uniq Command

The uniq command removes repeated lines, so each line is shown only once.

The examples below use this file, fruits.txt:

apple
banana
apple
cherry
banana
apple

uniq Needs Sorted Input

uniq only removes duplicates that are next to each other. In fruits.txt, no line is followed by the same line, so nothing is removed:

uniq fruits.txt
apple
banana
apple
cherry
banana
apple

Sort the lines first, so the same lines end up next to each other:

sort fruits.txt | uniq
apple
banana
cherry

sort -u gives the same result in one command. Use uniq when you need its options, like counting.

Count Each Line (-c)

Use -c to show how many times each line appears:

sort fruits.txt | uniq -c
      3 apple
      2 banana
      1 cherry

Show Only Duplicates (-d)

Use -d to show only lines that appear more than once:

sort fruits.txt | uniq -d
apple
banana

Show Only Unique Lines (-u)

Use -u to show only lines that appear exactly once:

sort fruits.txt | uniq -u
cherry

Ignore Case (-i)

Use -i to treat upper and lower case as the same, so "Apple" and "apple" count as duplicates:

printf "Apple\napple\nAPPLE\n" | uniq -i
Apple

Options Overview

• -c: Count how many times each line appears
• -d: Show only repeated lines
• -u: Show only lines that are not repeated
• -i: Ignore upper and lower case