Puppeteer-extra: What are all you guys using pupeteer for?

[AdityaAnand1]

I'm just curious to know what all everyone is scraping 😉 There isn't much of a community around it so thought this is the best place to discuss/

[dantgal]

Google search results - but it's impossible now using this plugin - Google changed reCaptcha and no chance to solve it

[AdityaAnand1]

Interesting, would love to know your use case. How exactly are you using Google search results? In a derivative product or some internal tool or something else?

Assuming you're talking about this issue, they seem to have a workaround.

[dench1k]

I make music mixes every month and post it on a couple of platforms including youtube. There is a thing called music policies, where you can search for a track and get the info about it's license (i.e. can you actually use it without restrictions or it can cause ban). So I made a bot, which logs in the gmail account and searches via this service through my tracklist, tells me the info about a track and makes a screenshot to check this info.

[seuaCoder]

Google search results - but it's impossible now using this plugin - Google changed reCaptcha and no chance to solve it

Did you find any alternative ?

[dantgal]

nope and I will be very happy when recaptcha plugin for puppeteer is upgraded cause alternatives are very expensive or maybe better statement: too expensive for me :)

[Ross1337]

https://github.com/danielgatis/puppeteer-recaptcha-solver , here is a free solver working (for me lol)

[dantgal]

But it's for old style reCaptcha (embedded) for such things puppeteer works like a charm too

[Ross1337]

U need v3 ?

[dantgal]

yuup! Tried https://github.com/danielgatis/puppeteer-recaptcha-solver but it also failed: https://postimg.cc/kD24p1Hb - tough times for google serp scrapers :(

[lmourabraga]

I have been using to scrap all images of the e-commerce of the company I work for and save each file naming with SKU of their respective products in order to centralize all these images in an API we created for our products.

[lmourabraga]

yuup! Tried https://github.com/danielgatis/puppeteer-recaptcha-solver but it also failed: https://postimg.cc/kD24p1Hb - tough times for google serp scrapers :(

Dantgal, regarding your situation there are not solutions known yet for reCaptcha v3 once it was developed to not be beaten by an automatization. Actually you will always need a human solver for it. As you can see, the solutions we have over the internet depends on the human hands to deliver their solution. For instance: https://2captcha.com/?from=9148648 see the content for: Work for 2Captcha.

[dantgal]

Yes, you are absolutely right! I have used 2captcha with puppeteer until it was possibe - now it's not :(

[RadoslavK]

Browser game bot. More reliable than simple http requests :D Just fun personal project.

[cnscorpions]

I scrape google serps with Puppeteer, but now google ban my bot. :(

[cnscorpions]

@dantgal Hi, just wonder how to identify recap version? I'm totally confused.

[itsdarrylnorris]

@AdityaAnand1

I'm just curious to know what all everyone is scraping 😉 There isn't much of a community around it so thought this is the best place to discuss

We have a discord server with a community around scraping/puppeteer/playwright. Here is the discord invite -> https://discord.gg/vz7PeKk

[cnscorpions]

@itsdarrylnorris thanks

[dantgal]

@dantgal Hi, just wonder how to identify recap version? I'm totally confused.

Hello Gavin! Sorry but I gave up with puppeteer, too many sleepless nights to fight with it :(

[cnscorpions]

@dantgal okay, good luck. it's hard work for me too.
I am working on a google scraping project overnight. :(
oh shit, Google bans all my proxy IPs.

[Dwight-Schrute]

@AdityaAnand1 - onlyFans automation for a growing 'influencer'. Those girls make serious cash

[berstend]

Closing this for now 😄

There isn't much of a community

Not anymore! Our discord has an active and cool community (which is better suited for these types of discussion): https://github.com/berstend/puppeteer-extra/wiki/Scraping-Chat