Avoid and Remove Duplicates from your Phantom Results

This guide explains how PhantomBuster handles duplicates and how to keep your data clean across different automations. You’ll learn what happens during launches, how duplicates can appear across multiple runs, and what settings or steps can help you avoid or remove them.

What counts as a duplicate

A duplicate is any repeated entry, usually a profile, post, business, or page, that appears more than once in your results. Phantoms usually identify duplicates based on a unique value like a link or ID (e.g., a LinkedIn profile URL).

Duplicates within a single launch

Most Phantoms automatically remove duplicates during a single launch.

  • If the same profile, page, or post appears more than once in your search or file, the Phantom will only save it once.
  • No setup is required. This works by default to keep your results clean during each individual launch.

Duplicates across multiple launches or input files

Duplicates can appear across separate launches if you:

  • Run the same Phantom more than once. For example: with different search URLs or spreadsheets.
  • Upload overlapping input files across campaigns or team members.
  • Use the same list in multiple Phantoms, even if it’s for a different goal.

Phantoms don’t remove repeated entries across different runs or lists, unless they include a setting made for that.

How to automatically remove duplicates between launches

Some Phantoms include a Remove duplicates checkbox in their setup
This feature filters out repeated entries across different inputs (also known as cross-query duplicate removal).

To use it:

  1. Log in to your PhantomBuster workspace and go to your Dashboard if you want to update one of your existing Phantoms.
    → If you’re setting up a new Phantom, this option will appear in the Behavior step during setup.
  2. Click on the three dots menu on the Phantom’s card you want to update, select Setup, and scroll to the Behavior step in the left-hand side menu.
  3. Look for the Remove duplicates checkbox and make sure it’s checked.
    → Wording may vary depending on the Phantom, for example:
     • Remove duplicate profiles between different groups
     • Remove duplicate profiles between different posts

This setting is off by default, you must turn it on manually. Some users prefer to keep duplicates for analysis.

Phantoms that support duplicate removal between searches

These Phantoms support the Remove duplicates setting:

LinkedIn automations

  • LinkedIn Event Guests Export
  • LinkedIn Group Members Export
  • LinkedIn Poll Voters Export
  • LinkedIn Post Likers Export
  • LinkedIn Search Export

Sales Navigator automations

  • Sales Navigator List Export
  • Sales Navigator Search Export

Instagram automations

  • Instagram Hashtag Search Export
  • Instagram Photo Likers

Other platforms

  • Facebook Group Members Export
  • GitHub Stargazers Export
  • GitHub User Search Export
  • Twitter Following Collector

The Google Maps Search Export Phantom skips duplicates by default when comparing new results to those already saved, no checkbox needed.

Duplicates between different Phantoms

Phantoms do not compare results with one another.
That means if you run LinkedIn Search Export and Sales Navigator Search Export on similar keywords, the same profile may appear in both result files.

To remove these overlaps, you’ll need to deduplicate manually after exporting.

Best practices to keep your data clean

Here are tips to avoid or remove duplicates:

  • Use one spreadsheet when possible:
    Combine search URLs or profile links in a single spreadsheet when you can. This allows you to use one Phantom to process all data and helps avoid repeated entries across separate launches.
  • Turn on the “Remove duplicates” option (if available):
    Some Phantoms include a checkbox to automatically skip duplicate profiles between multiple search inputs. Make sure this is enabled in the Behavior step during setup.
  • Use the LinkedIn Leads page to manage lead lists:
    If you're working with LinkedIn or Sales Navigator Phantoms, use the LinkedIn Leads page in your workspace to create dynamic lists. Leads added to a list are automatically deduplicated, so even if the same profile is collected by multiple Phantoms, it won’t be added twice.
  • Clean results manually in Google Sheets or Excel:
    Export your results and use filters or remove duplicates using a unique column like a LinkedIn profile URL or website link. This is especially useful if you're combining results from multiple Phantoms.

 

Was this article helpful?

0 out of 0 found this helpful