How to Remove Duplicates from an Array in PowerShell

Q: What is the standard way to remove duplicates from an array in PowerShell?

$items | Select-Object -Unique is the most common approach. It preserves the original order of first occurrence and is case-insensitive by default.

Q: How do I do a case-sensitive dedup in PowerShell?

Use a HashSet with an explicit case-sensitive comparer: [System.Collections.Generic.HashSet[string]]::new([StringComparer]::Ordinal) , then add each item and keep only the ones that were newly added (Add() returns $false for items already present).

PowerShell's most common deduplication command has a default behavior that catches people off guard: it's case-insensitive out of the box. Here's the standard approach, the sorted variant, and how to force case-sensitive comparison when it matters.

Quick answer

$items | Select-Object -Unique preserves the original order of first occurrence and is case-insensitive by default, so 'Apple' and 'apple' are treated as duplicates. Use Sort-Object -Unique instead if you want the output sorted rather than in original order. For case-sensitive dedup, use a [System.Collections.Generic.HashSet[string]] with an Ordinal comparer.

Need the same thing in another language?

See the full reference for Python, JavaScript, SQL, Bash, and more.

See every language →

The standard way: Select-Object -Unique

$items = @('apple', 'banana', 'apple', 'cherry', 'banana')

$unique = $items | Select-Object -Unique
$unique # apple, banana, cherry

This preserves the order of first occurrence, which makes it the go-to choice whenever the original ordering of your data matters.

The case-insensitivity gotcha

$items = @('Apple', 'apple', 'Banana', 'APPLE')

$unique = $items | Select-Object -Unique
$unique # Apple, Banana - only 2 items, not 3 or 4!

Select-Object -Unique compares strings case-insensitively by default, matching PowerShell's overall default string comparison behavior. If your data includes values that differ only by case and should be treated as genuinely distinct - case-sensitive codes or identifiers, for example - this silently merges them, which is a common source of confusing bugs.

Case-sensitive deduplication

$set = [System.Collections.Generic.HashSet[string]]::new([StringComparer]::Ordinal)
$unique = $items | Where-Object { $set.Add($_) }

HashSet[string].Add() returns $true when the item was newly added and $false if it was already present, which makes it a natural fit for Where-Object as an order-preserving filter. Passing [StringComparer]::Ordinal makes the comparison strictly case-sensitive, unlike the default behavior of Select-Object -Unique.

Sort-Object -Unique

$sorted_unique = $items | Sort-Object -Unique

This sorts the array and removes duplicates in the same step, similar to Bash's sort -u. Use it when you want sorted, deduplicated output in one command rather than caring about the original order.

Why Get-Unique gives wrong results

# Wrong - Get-Unique only catches ADJACENT duplicates
$items | Get-Unique

# Correct - sort first so duplicates become adjacent
$items | Sort-Object | Get-Unique

Like Unix's uniq, Get-Unique only removes duplicate items that are immediately next to each other in the array - it has no memory of items further back. Without sorting first, non-adjacent duplicates pass through untouched.

Just need the answer, not the script? Paste your list.

Open Remove Duplicate Lines →

Doing the same thing on the command line instead?

See the Bash equivalent - and the classic uniq-without-sort mistake.

Read the Bash guide →

Frequently Asked Questions

What is the standard way to remove duplicates from an array in PowerShell?

$items | Select-Object -Unique is the most common approach. It preserves the original order of first occurrence and is case-insensitive by default.

Is Select-Object -Unique case-sensitive or case-insensitive?

Case-insensitive by default - "Apple" and "apple" are treated as the same value, and only the first one encountered is kept. This surprises people coming from languages where string comparisons default to case-sensitive.

How do I do a case-sensitive dedup in PowerShell?

Use a HashSet with an explicit case-sensitive comparer: [System.Collections.Generic.HashSet[string]]::new([StringComparer]::Ordinal), then add each item and keep only the ones that were newly added (Add() returns $false for items already present).

What's the difference between Select-Object -Unique and Sort-Object -Unique?

Select-Object -Unique preserves the original order of first occurrence. Sort-Object -Unique sorts the array and removes duplicates in the same step, so the output ends up alphabetically (or numerically) sorted instead of in original order.

Why does Get-Unique give wrong results?

Get-Unique, like Unix's uniq, only removes consecutive duplicate items - it doesn't scan the whole array. It needs sorted input first ($items | Sort-Object | Get-Unique) to catch every duplicate, which also means it doesn't preserve original order either.

What is the fastest approach for a very large array?

A [System.Collections.Generic.HashSet[string]] gives O(n) deduplication with full control over the comparer (case-sensitive or not), and is generally faster than piping through Select-Object -Unique for arrays with tens of thousands of items or more.

What if I don't want to write a PowerShell script at all?

Paste your list into the Remove Duplicate Lines tool and it dedupes instantly in the browser, with an explicit case-sensitivity toggle so you don't need to remember PowerShell's default behavior.