Learn about the Rate limits to safely scrape and automate. Check out below to establish how many actions your Phantoms can process daily.
1. Basic rate limits to keep in mind
2. Disconnected again? Your cookies have expired
3. LinkedIn
4. Instagram
5. Facebook
6. Twitter
Have you ever noticed that after a few months (or days) of extensive scrolling, posting, and liking, your social network account disconnects you and makes you login again? While Silicon Valley probably loves the time you spend on social media, there are still limits to the amount of activity that they believe to be humanly possible. Social networks disconnect your account to make you re-identify yourself and confirm the security of your account.
How does this affect your Phantom’s automations? As a Phantom is a simple bot that acts on your behalf, the network on which you’re automating will believe you’re the one manually performing each action. Phantoms gain access to your account through your browser’s session cookie, which expires every time you’re logged out of the account. (For more on that, check out How to get your cookies with PhantomBuster). This means you have to reconnect the session cookie to your Phantom to allow the Phantom to continue its work.
1. Basic rate limits to keep in mind
Regardless of the network you’re using, remember that these limits apply to the total number of actions done on your account in a single day. This includes actions you might be doing manually, as well as the sum of all of your Phantoms’ actions.
2. Disconnected again? Your cookies have expired
Session cookies naturally expire every couple of months, so occasionally reconnecting your cookie to your Phantom is inevitable. If you find yours expiring too often and you're not logging out yourself, it could be that the network is logging you out as your activity rate is too high.
To prevent this, we recommend reducing your activity, keeping in mind the rate limits for different websites.
You should always make sure to spread your actions over several launches throughout the day, within business hours where possible. Do this by setting your Phantom to automatic, repeated launches.
It's also important to note that if you're just starting to automate your account, you should start with a similar amount of actions as you would normally do manually and increase gradually.
3. LinkedIn
LinkedIn depends on whether or not you’re paying for your LinkedIn account. If you have a regular, free account, we recommend processing a maximum of around 80 profiles per day with profile-scraping Phantoms. With paid, premium, or Sales Navigator accounts, it’s possible to do ~150.
You can check out our blog article on the subject for comprehensive recommendations on LinkedIn rate limits.
4. Instagram
Instagram limits depend more on the actions you’re performing.
If you've received the message: "This action was blocked. Please try again later. We restrict certain content and actions to protect our community. Tell us if you think we made a mistake," you have definitely hit Instagram's limit.
For example, if you’re looking at extracting a large number of followers with the Instagram Follower Collector, accounts a profile is following with the Instagram Following Collector, or posts associated with hashtags from the Instagram Hashtag Search Export -- all of these Phantoms can take around 5,000 - 9,000 in one launch, with up to 20 launches per day. Direct actions, however, such as liking, commenting, or auto-following accounts, are much more limited. For the entire breakdown, see Every social network limitation any digital-marketer should know.
5. Facebook
Facebook is a tricky one, as it absolutely requires proxies to ensure your Phantom is able to connect to your account in the first place. Luckily, we have a couple of articles explaining that already: Facebook Phantoms and How and why to use a proxy. Be sure to check out Troubleshooting with proxies to avoid common user errors for proxy configurations.
Once you’ve set up your proxy, you’ll still need to follow the recommended rate limits for automation. Similar to Instagram, extracting profiles from a large list is fairly easy -- the Facebook Group Members Export can take up to 4,000-5,000 members from a group. Profile scraping is much more restricted. We recommend scraping 5 per hour with the Facebook Profile Scraper.
6. Twitter
Twitter’s limits vary yet again, but we still have our best practice suggestions! Stick to 10 accounts per launch, 5-8 launches per day with the Twitter Auto Follow, Twitter Auto Retweeter, and Twitter Message Sender. Like up to 1,000 tweets per day with the Twitter Auto Liker, and scrape 5,000 followers every 2 hours with the Twitter Follower Collector.
Did we miss a limit? For any limits unaddressed here, be sure to check out our Blog for the complete list: Every social network limitation any digital-marketer should know.
If you think this article does not address your issue, please contact Support directly. We are continuously improving, so your feedback means the world to us!