How to Compare Two Lists in Bash

Q: What is the comm command and what do its columns mean?

comm takes two sorted files and prints three tab-separated columns: lines only in the first file, lines only in the second, and lines common to both. comm -23 a.txt b.txt suppresses columns 2 and 3, leaving only lines unique to a.txt .

Q: Why does comm require sorted input?

comm compares both files line by line in a single pass and relies on matching lines appearing in the same relative order in each file - it doesn't build any kind of index. Unsorted input breaks this assumption and produces wrong or incomplete results without any error message, which is why both inputs must go through sort first, typically via process substitution: comm -23 <(sort a.txt) <(sort b.txt) .

Q: What's the simplest way to compare two lists in Bash without sorting them first?

grep -Fxvf b.txt a.txt - this prints every line of a.txt that does NOT appear anywhere in b.txt , using fixed-string (-F) exact whole-line (-x) matching, and doesn't require either file to be sorted.

Q: What do the grep flags -Fxvf actually mean?

-F treats patterns as literal fixed strings rather than regex, -x requires the whole line to match exactly rather than matching a substring, -v inverts the match to print non-matching lines, and -f b.txt reads the list of patterns to match from the file b.txt instead of the command line.

Q: How do I find lines common to both files (intersection) in Bash?

With sorted input: comm -12 <(sort a.txt) <(sort b.txt) suppresses the unique-to-each columns and leaves only the common lines. Without sorting: grep -Fxf b.txt a.txt (same flags as the difference version, minus -v ).

Q: Why do two lines that look identical not match in comm or grep?

Trailing whitespace, trailing carriage returns from Windows-created files (CRLF vs LF line endings), or non-breaking spaces are the usual causes - the lines look the same when printed but differ byte-for-byte. Run cat -A file.txt to reveal hidden characters, or use the Invisible Character Detector for a visual view of exactly which characters are present.

Q: How do I compare two files by a specific column, not the whole line?

join is built for this: join -v 1 <(sort -k1,1 a.txt) <(sort -k1,1 b.txt) finds lines from a.txt whose first field has no match in b.txt . Both files need to be sorted on the join key, and the field delimiter defaults to whitespace unless you pass -t .

Bash doesn't need a scripting language for this - a handful of standard Unix tools already do exactly what you want, as long as you know their sorting requirements and matching rules. Here are the four commands worth knowing, ordered by how often you'll reach for each.

Quick answer

comm -23 <(sort a.txt) <(sort b.txt) prints lines only in a.txt, but both files must be sorted first - comm compares line by line and produces wrong results on unsorted input without any warning. grep -Fxvf b.txt a.txt does the same job without requiring sorted input, using exact fixed-string whole-line matching. Use join instead of comm/grep when you need to match on one column rather than the whole line.

Need the same thing in another language?

See the full reference for Python, JavaScript, SQL, Java, PowerShell, and more.

See every language →

comm - the standard tool, if you sort first

comm -23 <(sort a.txt) <(sort b.txt)

comm prints three tab-indented columns by default: lines only in the first file, lines only in the second, and lines in both. The numbers after - suppress columns - -23 hides columns 2 and 3, leaving only lines unique to a.txt. The <(sort ...) process substitution sorts each file on the fly without creating a temp file - this step is mandatory, since comm assumes sorted input and gives silently wrong results otherwise.

grep -Fxvf - no sorting required

grep -Fxvf b.txt a.txt

This reads patterns from b.txt (-f b.txt) as literal strings rather than regex (-F), requires a full-line exact match (-x), and inverts the result to show non-matches (-v). The result: every line of a.txt that doesn't appear anywhere in b.txt, in a.txt's original order, with no sorting step needed.

diff - when you need context, not just membership

diff a.txt b.txt

diff answers a different question than comm and grep: it shows how to transform one file into the other, matching by position and context, not just by which lines exist in each file. It's the right tool when line order and grouping matter (like comparing two versions of a config file); it's the wrong tool for "which items exist in A but not B" on lists where order doesn't matter, since a single reordering will show up as a wall of unrelated-looking changes.

join - comparing by a column, not the whole line

join -v 1 <(sort -k1,1 a.txt) <(sort -k1,1 b.txt)

When each line has multiple fields and you want to match on just one of them (like an ID column) rather than the entire line, join does what comm can't. Both inputs need to be sorted on the join key first (-k1,1 sorts on the first field), and -v 1 prints unmatched lines from the first file only.

The usual culprit when lines "should" match but don't

Trailing whitespace, a stray carriage return left over from a file created on Windows (CRLF line endings instead of Unix LF), or a non-breaking space pasted from a web page are the most common reasons two visually identical lines fail to match in any of these tools. cat -A file.txt reveals line endings and tabs directly in the terminal; the Invisible Character Detector gives a clearer visual breakdown if you'd rather not parse terminal escape codes by eye.

Just need the answer, not the code? Paste both lists.

Open Compare 2 Lists →

Frequently Asked Questions

What is the comm command and what do its columns mean?

comm takes two sorted files and prints three tab-separated columns: lines only in the first file, lines only in the second, and lines common to both. comm -23 a.txt b.txt suppresses columns 2 and 3, leaving only lines unique to a.txt.

Why does comm require sorted input?

comm compares both files line by line in a single pass and relies on matching lines appearing in the same relative order in each file - it doesn't build any kind of index. Unsorted input breaks this assumption and produces wrong or incomplete results without any error message, which is why both inputs must go through sort first, typically via process substitution: comm -23 <(sort a.txt) <(sort b.txt).

What's the simplest way to compare two lists in Bash without sorting them first?

grep -Fxvf b.txt a.txt - this prints every line of a.txt that does NOT appear anywhere in b.txt, using fixed-string (-F) exact whole-line (-x) matching, and doesn't require either file to be sorted.

What do the grep flags -Fxvf actually mean?

-F treats patterns as literal fixed strings rather than regex, -x requires the whole line to match exactly rather than matching a substring, -v inverts the match to print non-matching lines, and -f b.txt reads the list of patterns to match from the file b.txt instead of the command line.

How do I find lines common to both files (intersection) in Bash?

With sorted input: comm -12 <(sort a.txt) <(sort b.txt) suppresses the unique-to-each columns and leaves only the common lines. Without sorting: grep -Fxf b.txt a.txt (same flags as the difference version, minus -v).

Why do two lines that look identical not match in comm or grep?

Trailing whitespace, trailing carriage returns from Windows-created files (CRLF vs LF line endings), or non-breaking spaces are the usual causes - the lines look the same when printed but differ byte-for-byte. Run cat -A file.txt to reveal hidden characters, or use the Invisible Character Detector for a visual view of exactly which characters are present.

How do I compare two files by a specific column, not the whole line?

join is built for this: join -v 1 <(sort -k1,1 a.txt) <(sort -k1,1 b.txt) finds lines from a.txt whose first field has no match in b.txt. Both files need to be sorted on the join key, and the field delimiter defaults to whitespace unless you pass -t.