NEW PROBLEM: Scraping LinkedIn MutualConnections CHANGED
Mutual connections on LinkedIn used to be easy to scrape. We could create the mutual connections search URL just by stitching together a few values.
Now, it doesn't even look like you can directly go to someone's LinkedIn profile, copy the URL of the mutual connections link, and paste it into a browser. That will just redirect to your list of linkedin connections.
Does anyone know if there are ways around this? We had set up a process that created a bunch of mutual connections URLs and had a Phantom go one-by-one scraping all the results from those search URLs. This doesn't look like it works anymore.
thanks for any help. it's key to one of the workstreams we run.
Comments
8 comments
Hey Jason,
Can you tell me the workflow you're using here?
I'd recommend you taking a look at the LinkedIn Profile Scraper where you'll be able to get that Mutual Connections URL.
We had been constructing the mutualconnectiionsURL by stitching together the search URL format with a VMID etc. That had no problems forever. Now it's having issues.
We just ran a test of the linkedin profile scraper and the URL it gets for mutualconnectionsURL. Those links don't work either. You can see it in our account here: https://phantombuster.com/8514593289174712/phantoms/5617261029931232/console
run the test for yourself. We believe there's some sort of block for people whose accounts are above a certain size so test it with someone who has at least 1000 connections. When we tested it with my teammate who has 450 connections, they worked.
Would love to hear what you find…
btw an easy way to test it is for you to go into 2nd degree searches on LInkedIn yourself and see what happens.
I walk through exactly the behavior in this 2 minute loom video [Redacted content].
Hey Jason,
I went on your Phantom and it seems that on your example I can still see my mutual connections with the second contact on the list (Mike Lu). I can't access the ones of the two other contacts though.
The reason for this is because they're hiding their connections to the public. You'll see that you can do the same in your settings :
I've also tried with people having over 1000 connections (for instance that guy) and haven't had any issue in getting an access to his connections. So, for me, and as I said, it's rather a setting selected by some LinkedIn users that will prevent you seeing their connections.
Regarding your Loom video, thanks a lot for sharing. I can indeed not copy paste that mutual connections link. The reason is that the link is dynamic and changing once you click on it. If you click on the link, please take a look at the URL : you'll see that you're getting something like this at the end : connectionOf=%5B"ACoAAB_y48UBFMfqEzUj-JF3JmUSOvT_BG3KZLw"%5D
And I guess that the last part is the VMID. So if you are adding &connectionOf=VMID to your pre-built URL, you should be able to have access to the mutual connections page of your leads, but only if they decided to show publicly their connections.
Let me know if you have further questions!
Hi Nathan,
Thanks for spending so much time on this. I can't quite reconcile what you said in the beginning with clicking on Mike Lu's mutual connection and being able to see those mutuals with your reaction to my Loom video.
My concern is even if we use the profile scraper phantom for a list of LinkedIn profiles, and that profile scraper gets us a spreadsheet of mutual connections URLs, because of the inability to copy and paste the mutual connections link, we wouldn't be able to run a search export phantom on that profile scraper output.
Said another way, what we're trying to do is at scale scrape all of the mutual connections from a list of profile URLs. Do you think that's possible now given the dynamic changing of the mutual connections link from my Loom video?
p.s. I just tried viewing the mutual connection links from my example phantom with Mike Lu and it didn't work. can you record a loom of it working for you? i don't know if we're talking about the same thing → https://www.loom.com/share/68f7ba91c9f2420bacb52920d7f9c081?sid=dc1fc59a-970e-4daa-a62f-3d72537d9398
Hey Jason,
Yes, I was able to click on Mike Lu's mutual connections link and have access to it. Please let me know if you can also see your mutual connections with him?
Regarding the copy-pasting you want to do with the mutual connections URLs, I'm not quite sure what you mean and need here?
With the LinkedIn Profile Scraper, you'll get the mutual connections URLs with your leads. You can sometimes access those lists of shared relations when they accept to display publicly their network. If they don't, you won't be able to see anything.
And, if you just need to build yourself those mutual connections link, you can will need to use the LinkedIn Profile Urn and create those links yourself like followed :
https://www.linkedin.com/search/results/people/?origin=MEMBER_PROFILE_CANNED_SEARCH&facetNetwork=%5B%22F%22%5D&facetConnectionOf=%5B%22LinkedInProfileUrn%22%5D
Replacing LinkedInProfileUrn by the right profiles urn :)
Then, after getting the right URLs, I'd recommend you using LinkedIn Search Export to extract information about these mutual connections.
Let me know if that solution works for you!
ha yeah we're definitely speaking past each other.
Maybe this will help: I would like to create a workflow like this:
i don't believe that will work now because of how linkedin is handling mutual connections search links.
And here is one other display of the Mike Lu mutual connections link.. visiting it from the page versus clicking on the link that you pasted in your comment above: https://www.loom.com/share/6c19ddc896da44cbab631d228438f260?sid=bd38b043-54be-4c30-a29a-5998d858e4c4
Hey Jason,
Thanks for the details.
You don't need to copy paste your URLs so you won't encounter this issue.
When you'll run the LinkedIn Search Export on the step 4 of your workflow, you just need to select the correct field on the input source of your Phantom.
Please sign in to leave a comment.