Remove Duplicate Lines
Why use this tool
Duplicate lines creep into all kinds of lists — email addresses exported twice from a CRM, log lines repeated by a noisy script, a spreadsheet column pasted from two overlapping sources. Sorting them out by eye is slow and error-prone once a list runs past a few dozen lines. This tool removes duplicates in one pass, entirely in your browser, with three options most competing tools only offer one or two of at once: a case-sensitivity toggle, alphabetical sorting, and a view of exactly what was removed.
How deduplication works here
The tool reads your text one line at a time and keeps only the first time each line appears, dropping every repeat after it. Line order is preserved unless you turn on "Sort alphabetically." Comparison is exact by default — a line is only a duplicate of another if every character matches — which you can loosen with the case-sensitivity toggle below.
Worked example
Paste this list:
apple
banana
Apple
banana
cherry
With case-sensitive comparison on (the default), Apple (capital A) is kept as a separate line from apple, because the comparison is exact. The result is:
apple
banana
Apple
cherry
Turn case-sensitive off, and Apple is now treated as a duplicate of apple (whichever came first is kept), producing:
apple
banana
cherry
Either way, banana — an exact repeat — is removed both times.
Edge cases this tool handles correctly
- Empty input. Pasting nothing (or clearing the box) produces an empty result with no error, rather than crashing or showing "0 duplicates."
- A list with no duplicates. The output matches your input exactly, line for line, and the tool reports "No duplicate lines found" instead of silently doing nothing and leaving you unsure if it ran.
- A list that's entirely duplicates. If every line is the same, the output is a single line — this is correct behavior, not a bug, and the removed-lines view will show every repeat that got cut.
- Blank lines. Repeated empty lines are deduplicated exactly like any other line, collapsing to a single blank line if they appear more than once.
- Mixed line endings. Lists pasted from Windows (
\r\n), classic Mac (\r), or Unix (\n) sources are all normalized correctly before comparison, so a list built from copy-pasting across operating systems still dedupes correctly. - Single-line input. A one-line paste returns that line unchanged, with zero removed.
When to use case-sensitive vs. case-insensitive
Keep case-sensitivity on (the default) for anything where casing is meaningful — code identifiers, tags, or data where ID123 and id123 are genuinely different values. Turn it off for human-entered lists like names or email addresses, where John@Email.com and john@email.com almost always represent the same real-world duplicate that should be merged.
Frequently asked questions
- Does this tool keep the first or last copy of a duplicate line?
- It keeps the first occurrence and discards every line after it that repeats. If you need the reverse — keeping the last copy — reverse your list before pasting it in, run the tool, then reverse the result back.
- Are blank lines treated as duplicates too?
- Yes. An empty line is a line like any other, so five blank lines in a row collapse down to one. If you want to keep your original spacing, this tool isn't the right step for that — dedupe it first, then add blank lines back manually.
- What happens with lines that only differ by capitalization, like "Apple" and "apple"?
- By default the comparison is case-sensitive, so "Apple" and "apple" are treated as different lines and both are kept. Turn off "Case-sensitive" to treat them as duplicates — the tool keeps whichever casing appeared first and removes the rest.
- Does this trim extra spaces before comparing lines?
- No — comparison is exact, so "item" and "item " (with a trailing space) are currently treated as different lines. If your list has inconsistent spacing, clean it up first (e.g. with a find-and-replace) for reliable deduplication.
- Can I see which lines got removed?
- Yes — check "Show removed lines" and a second box appears listing every line that was dropped as a duplicate, in the order it was encountered, so you can double-check nothing important was cut.
- Does sorting change which lines count as duplicates?
- No. Deduplication always happens first, based on original order; "Sort alphabetically" only changes the order of the lines you keep in the output. It doesn't affect which lines are considered duplicates.
- Will this work on a huge pasted list, like thousands of email addresses?
- Yes — it runs a single pass over your input entirely in your browser, so lists with thousands of lines process instantly with no upload or size limit imposed by the tool itself.
- Is my data uploaded anywhere?
- No. Deduplication happens client-side in JavaScript — your list never leaves your browser or touches a server, which matters if you're pasting anything sensitive like customer emails or internal data.