Remove Duplicate Lines from a List
Paste a list and get it back with duplicates gone — order kept, counts shown, case and spacing handled sensibly.
- Lines in
- —
- Unique out
- —
Fast, easy and free — your answer appears as you type, and whatever you enter stays on your device.
Paste a list, get it back with every repeat removed — first occurrence kept, original order untouched, and the number of duplicates found shown as the headline. Case and stray spaces are treated as the same line unless you say otherwise.
How to use it
- Paste the listEmails, order numbers, keywords, names — one per line.
- Check the togglesIgnore case and Ignore spaces are on, so 'Anna ' and 'anna' count as one. Turn either off for strict matching.
- Copy the clean listThe unique lines, in the order they first appeared.
Where duplicates come from
Nobody types a list with duplicates; they merge their way in. Two exports of overlapping date ranges. A mailing list combined with last year's. Keyword research from three tools that mostly agree. The overlap is invisible in a thousand-line paste, and the cost is real — double emails sent, inflated counts reported, the same work done twice.
The headline number here is the duplicates found, not the survivors, because that is the diagnostic: paste a "clean" list and a big number tells you the upstream process is merging something it should not.
Forgiving matching, faithful output
Strict equality is the wrong default for pasted data. The same email arrives as Name@Example.com from one export and name@example.com from another; spreadsheet copies carry trailing spaces that no eye will ever see. The default toggles treat those as the duplicates they really are — while the line that survives is your original, byte for byte. Matching is normalised; your data is not rewritten.
When the difference is the data — case-sensitive codes, IDs where trailing space is meaningful — both toggles switch off and matching becomes exact.
Order is information
Plenty of dedupe methods sort as a side effect, which quietly destroys whatever the order meant: priority, chronology, the sequence a customer signed up in. This tool never reorders — first appearance wins its place and keeps it. Deduplication and sorting are different decisions, and you can take the result next door to sort-lines when you want both.
Lists like these are usually personal data. Everything runs on your device, nothing is uploaded, and closing the tab is the delete button.
Questions people ask
Which copy of a duplicate is kept?
The first one, exactly as it appeared, and the list order is otherwise unchanged. Nothing is sorted behind your back — if you want sorting too, the sort-lines tool has a dedupe option built in.
Are 'Anna' and 'anna ' really duplicates?
With the default toggles, yes: case and surrounding spaces are ignored for matching, because in pasted lists those differences are almost always noise from different sources. The kept line is still your original text — matching is forgiving, output is untouched.
Why did my count of removed lines seem high?
Usually invisible whitespace: lines that differ only by a trailing space or a tab collapse into one when Ignore spaces is on. That is normally the correct outcome — turn the toggle off to see the strict count.
What happens to blank lines?
Dropped. A deduplicated list with holes in it is rarely what anyone wants, and blank lines in pasted data are almost always copying artefacts.
Can I dedupe a comma-separated list?
Put each item on its own line first — paste into the line-breaks tool and split there, or use your editor's find-and-replace to turn commas into newlines. This tool deliberately works line-wise so that what counts as an item is never ambiguous.
How big a list can it handle?
Tens of thousands of lines without noticing — matching uses a hash set, so size barely matters. And the list never leaves your device, which is the point when it is full of email addresses.