Duplicate Line Remover

The Duplicate Line Remover scans a list and deletes repeated lines, keeping either the first or the last occurrence of each unique value while preserving the surviving lines in their original order. It's the fast way to deduplicate a list without opening a spreadsheet — paste in a list of emails, URLs, tags or log lines, and get back only the unique ones.

Typical uses: merging two contact lists and removing the overlap before a mail merge, cleaning a list of URLs scraped from multiple pages that contains the same link several times, deduplicating tags or keywords before a content brief, and tidying up a log file where the same error line repeated during a retry loop. Because comparison options are configurable, it handles messier real-world data too — a case-insensitive toggle catches '[email protected]' and '[email protected]' as the same entry, and a trim-before-compare option ignores accidental leading or trailing spaces that would otherwise make two visually identical lines register as different.

The choice between keeping the first or last occurrence matters when duplicate lines aren't perfectly identical in formatting — if a later entry in the list has more complete information (a fuller name, an updated URL), keeping the last occurrence preserves the presumably more current version instead of an earlier, possibly stale one. In both modes, every surviving line stays exactly where it appeared in the original list; only the position of the specific duplicate that's kept differs between the two modes.

This is a line-based, exact-match deduplication tool, not a fuzzy one — 'apple' and 'apples' are treated as entirely different entries, and there's no similarity threshold or typo tolerance. For genuinely fuzzy duplicate detection you'd need a different kind of tool, but for the vast majority of list-cleanup tasks — exact repeats from copy-paste, exports, or merges — exact matching is exactly what's needed. Everything runs locally in your browser.