PhantomBuster Phantoms automatically remove duplicates within a single launch, no setup needed. To remove duplicates across multiple launches or search inputs, enable the "Remove duplicates" checkbox in the Behavior step during setup (available on supported Phantoms). Phantoms don't compare results with each other, so if you run similar searches across different Phantoms, you'll need to deduplicate manually after exporting, or use the LinkedIn Leads page which deduplicates automatically.
What counts as a duplicate
A duplicate is any repeated entry, usually a profile, post, business, or page, that appears more than once in your results. Phantoms usually identify duplicates based on a unique value like a link or ID (e.g., a LinkedIn profile URL).
Duplicates within a single launch
Most Phantoms automatically remove duplicates during a single launch.
- If the same profile, page, or post appears more than once in your search or file, the Phantom will only save it once.
- No setup is required. This works by default to keep your results clean during each individual launch.
Duplicates across multiple launches or input files
Duplicates can appear across separate launches if you:
- Run the same Phantom more than once. For example: with different search URLs or spreadsheets.
- Upload overlapping input files across campaigns or team members.
- Use the same list in multiple Phantoms, even if it's for a different goal.
Phantoms don't remove repeated entries across different runs or lists, unless they include a setting made for that.
How to automatically remove duplicates between launches
Some Phantoms include a Remove duplicates checkbox in their setup.
This feature filters out repeated entries across different inputs (also known as cross-query duplicate removal).
To use it:
- Log in to your PhantomBuster workspace and go to your Dashboard if you want to update one of your existing Phantoms.
→ If you're setting up a new Phantom, this option will appear in the Behavior step during setup. - Click on the three dots menu on the Phantom's card you want to update, select Setup, and scroll to the Behavior step in the left-hand side menu.
- Look for the Remove duplicates checkbox and make sure it's checked.
→ Wording may vary depending on the Phantom, for example:
• Remove duplicate profiles between different groups
• Remove duplicate profiles between different posts
This setting is off by default, you must turn it on manually. Some users prefer to keep duplicates for analysis.
Phantoms that support duplicate removal between searches
These Phantoms support the Remove duplicates setting:
The Google Maps Search Export Phantom is a special case: it skips duplicates by default when comparing new results to those already saved. No checkbox needed.
LinkedIn automations
- LinkedIn Event Guests Export
- LinkedIn Group Members Export
- LinkedIn Poll Voters Export
- LinkedIn Post Likers Export
- LinkedIn Search Export
Sales Navigator automations
- Sales Navigator List Export
- Sales Navigator Search Export
Instagram automations
- Instagram Hashtag Search Export
- Instagram Photo Likers
Other platforms
- Facebook Group Members Export
- GitHub Stargazers Export
- GitHub User Search Export
- Twitter Following Collector
Duplicates between different Phantoms
Phantoms don't compare results with one another.
That means if you run LinkedIn Search Export and Sales Navigator Search Export on similar keywords, the same profile may appear in both result files.
To remove these overlaps, you'll need to deduplicate manually after exporting.
Best practices to keep your data clean
Here are tips to avoid or remove duplicates:
-
Use one spreadsheet when possible:
Combine search URLs or profile links in a single spreadsheet when you can. This allows you to use one Phantom to process all data and helps avoid repeated entries across separate launches. -
Turn on the "Remove duplicates" option (if available):
Some Phantoms include a checkbox to automatically skip duplicate profiles between multiple search inputs. Make sure this is enabled in the Behavior step during setup. -
Use the LinkedIn Leads page to manage lead lists:
If you're working with LinkedIn or Sales Navigator Phantoms, use the LinkedIn Leads page in your workspace to create dynamic lists. Leads added to a list are automatically deduplicated, so even if the same profile is collected by multiple Phantoms, it won't be added twice. -
Clean results manually in Google Sheets or Excel:
Export your results and remove duplicates using a unique column like a LinkedIn profile URL. See Google's guide to removing duplicates in Sheets for step-by-step instructions.
Frequently asked questions
Does PhantomBuster automatically remove duplicates?
Yes, within a single launch. If the same profile or page appears more than once in your search or input file, the Phantom saves it only once. Across multiple launches, you need to enable the Remove duplicates checkbox in the Behavior step.
How do I remove duplicates between different search URLs or input files?
Enable the Remove duplicates checkbox in the Behavior step of your Phantom's setup. This filters out repeated entries across different inputs. Not all Phantoms support this, check the supported list in this article.
Can PhantomBuster remove duplicates between different Phantoms?
No. Phantoms don't compare results with each other. If you run similar searches across different Phantoms, deduplicate manually in Google Sheets or Excel using a unique column like the LinkedIn profile URL, or use the LinkedIn Leads page which deduplicates automatically.
Is the "Remove duplicates" setting on by default?
No, it's off by default. You need to enable it manually in the Behavior step during setup.