Puppeteer-extra: Tiktok is detecting puppeteer-extra-plugin-stealth

[MerwinV1]

Hi everyone,

I'm curently working on a tiktok scraper, using the stealth plugin from the begining of it, but today when i'm trying to go on a tiktok profile i'm getting this:

Any Idea how to fix it ?

For the code i'm using (code for testing the connection):

const puppeteer = require('puppeteer-extra');
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
var fs = require("fs");

(async () => {
    try {
      puppeteer.use(StealthPlugin())
      var browser = await puppeteer.launch({
          headless: false,
        });
      var page = await browser.newPage();
      await page.goto(`https://www.tiktok.com/@miss_extra2.0`);
      await page.waitFor(50000)
      await browser.close();
    } catch (err) {
      await browser.close();
    }
  })();

[danielgwilson]

Also experiencing this issue

Edit:

Previously, puppeteer-extra-plugin-stealth functioned correctly and interacted with TikTok without issue. As of approximately ~3 days ago, TikTok no longer loads the following:

• User profiles (including the logged in user)
• Hashtag pages

[BrightLad990]

Google Chrome seems to be detected by sites lately. Using puppeteer-firefox is a good way to scrape Tiktok. Hoping someone could find out why chrome lets sites know about automation as this is only a temporary fix.

[danielgwilson]

@Torpedo99 thanks for that—trying now

[delmotte]

up

It works with puppeteer-firefox but it would be better to see it working with puppeteer/puppeteer-extra and stealth plugin

[Daniel-Griffiths]

Same here, had a lot of issues with puppeteer being detected recently. The firefox work around works for now :slightly_smiling_face:

[rezemble]

it's already the --remote-debugging-port flag that trips the detection.

running chromium (bare) without loads fine,
once the debugging port is set, the profile request doesn't return anything.

[Daniel-Griffiths]

@rezemble How did you disable the debugging port?

I am guessing it’s just another launch option you set to false?

[rezemble]

@Daniel-Griffiths well I took the chromium binary provided with puppeteer and ran it without any flags, opened a page, loaded fine
then started the same binary with --remote-debugging-port=0, opened a page, doesn't load.
without puppeteer.

So it would seem they're somehow detecting that remote debugging is enabled

[rezemble]

you can't use puppeteer without the remote debugging port, it's what allows it to interact with the browser.

[Daniel-Griffiths]

Thanks for the explanation @rezemble I guess that's going to be a tricky one to work around for now.

Sites keep geting better at detecting puppeteer :sweat_smile:

[vityas-off]

Somehow i ended with this

const puppeteer = require("puppeteer-extra");
const stealthPlugin = require("puppeteer-extra-plugin-stealth")();

["chrome.runtime", "navigator.languages"].forEach(a =>
  stealthPlugin.enabledEvasions.delete(a)
);

puppeteer.use(stealthPlugin);

main();
async function main() {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();

  await page.evaluateOnNewDocument(() => {
    delete navigator.__proto__.webdriver;
  });

  await page.goto(`https://www.tiktok.com/@miss_extra2.0`);
  await page.screenshot({ path: "./11.png" });
  browser.close();
}

[momala454]

https://blog.chromium.org/2018/06/improving-extension-transparency-for.html

window.chrome.webstore is undefined now, it should be removed from here https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-stealth/evasions/shared.js

but it should be also improved further, you can detect the fake functions with
window.chrome.csi.toString()

chrome.runtime.hasOwnProperty('id')
returns true, but puppeteer extra return false

chrome.app is also missing a lot of attributes/methods

Object.getOwnPropertyDescriptors(navigator.languages)

it's writable using the languages evasion :
[value] => en-US
[writable] => 1
[enumerable] => 1
[configurable] => 1

while it should be
configurable: false
enumerable: true
value: "en-US"
writable: false

[olitomas]

Related to this? 🤔
https://github.com/digitalhurricane-io/puppeteer-detection-100-percent

[momala454]

tiktok calls
chrome.runtime.connect

removing this function from the evasion ( https://github.com/berstend/puppeteer-extra/blob/master/packages/puppeteer-extra-plugin-stealth/evasions/shared.js#L66 ) make tiktok works, but one should probably simulate its functionnality to prevent being detected using this.

As i mentioned earlier, the whole window.chrome object must be properly implemented to prevent being detected

[berstend]

This should be fixed as soon as #292 is merged and published. :-)

[berstend]

Published in [email protected]