Remove Duplicate Lines
Paste a list and get back the same list with every repeat removed. The first copy of each line stays, in its original place. Nothing is sent to a server.
- Free, no sign-up
- First occurrence kept
- Order preserved
- Text stays on your device
- Runs on your device. Files are never uploaded.
- Free and unlimited. No sign-up, no watermark.
Remove duplicate lines in three steps
No account, no email, no waiting for an upload.
- STEP 01
Paste your list
Type or paste lines into the box, or open a .txt, .csv or other text file.
- STEP 02
Set the matching rules
Choose whether upper and lower case count as the same, whether spaces at the ends are ignored, and whether to delete empty lines.
- STEP 03
Copy or download the result
The note says how many lines were removed and how many were kept.
What Remove duplicate lines does, and what it doesn't
We'd rather tell you up front than surprise you after the download.
What you get
- First copy winsThe first time a line appears it stays, and later repeats are dropped.
- Order is untouchedRemaining lines stay in their original sequence, so no sorting is applied.
- Ignore caseTreat Apple and apple as the same line.
- Ignore stray spacesSpaces at the ends of a line, and repeated spaces inside it, do not make two lines different. Kept lines have their end spaces trimmed.
- Optional empty-line removalRemove blank lines at the same time.
Good to know
- Whole lines onlyLines are compared in full. 'red apple' and 'apple' are different. To work on columns of a table use CSV deduplicator.
- Blank lines are not deduplicatedUnless you tick remove empty lines, blank lines are all kept so paragraph gaps do not disappear.
- Not for huge filesMillions of lines depend on your device memory.
Your documents never leave your device
The tool runs inside your browser instead of on our servers, so there is nothing to upload, store or delete afterwards.
Typical online tool upload
Your file travels over the internet, sits on someone else's computer for a while, and you wait for the upload and the download.
Mediaextractr no upload
The tool loads into your browser once, then works on your file locally. Nothing is sent, so nothing can leak.
When people use Remove duplicate lines
Clean an email list
Remove repeated addresses before importing contacts.
Tidy keyword lists
Get one copy of every search term, URL or tag.
Dedupe logs and IDs
Reduce a log or export to its unique lines.
Make a unique word list
Combine notes from several sources without repeats.
Mediaextractr vs typical upload tools
| MediaextractrPrivate | Typical upload tools | |
|---|---|---|
| Your file is uploaded | Never | Yes |
| Sign-up needed | No | Often, after a few files |
| Daily file limit | None | Often 1–2 free files |
| Watermark | Never | Sometimes |
| Wait for upload & download | No | Yes |
| Speed on very large files | Depends on your device | Depends on their servers |
Choosing the matching rules
The defaults ignore spaces at the ends of lines, because trailing spaces are a very common reason two identical-looking lines are not equal. Turning on Ignore upper/lower case helps with names and email addresses, which people type in different ways.
Leave case sensitivity on when capitals matter, for example with product codes or file names.
A typical workflow
Many list tasks take three quick steps:
- Remove repeats here.
- Put the result in order with the text sorter, which can also remove duplicates and empty lines.
- Check what changed with Text compare.
Spreadsheet duplicates
For tables where whole rows repeat, Remove duplicate CSV rows and Remove duplicate Excel rows understand columns.
Lists pasted from different places often hide small differences, such as a trailing space or a capital letter. If the count of removed lines looks lower than you expected, turn on the two ignore options and run it again.
Questions, answered
How do I remove duplicate lines from a text?
Paste it into the box, choose your matching rules, and copy the result.
Which duplicate is kept?
The first one. Later copies are removed.
Is my list sent to a server?
No. The work is done in your browser and the list stays on your device.
Will the order of my lines change?
No. Only the repeats are removed. Use the text sorter if you want sorting.
Can it treat uppercase and lowercase as the same?
Yes. Tick Ignore upper/lower case.
Does this work on Android or iPhone?
Yes, in the mobile browser as well as on computers.