Streetmerchant: Proxy - Cannot read property 'url' of null

[MidnightsFX]

Expected Behavior

No errors :)

Current Behavior

[4:54:47 PM] info :: ℹ [amd] Next proxy index: 0 / Count: 2
[4:54:56 PM] error :: ✖ [bestbuy] evga 3080 ftw3 ultra - Navigation timeout of 30000 ms exceeded
[4:54:56 PM] info :: ℹ [bestbuy] Next proxy index: 1 / Count: 2
[4:54:57 PM] error :: ✖ [asus] asus 3080 strix oc - Navigation timeout of 30000 ms exceeded
[4:54:59 PM] error :: Cannot read property 'url' of null
[4:55:00 PM] error :: Cannot read property 'url' of null
[4:55:00 PM] error :: Cannot read property 'url' of null
[4:55:01 PM] info :: ✖ [bestbuy] [evga (3080)] ftw3 :: OUT OF STOCK

Reproduction

1. Setup global.proxies
2. Run

Environment

Windows 10 WSL2 - Ubuntu

# ** All configuration variables are optional **
######
USER_AGENT=
MAX_PRICE_SERIES_3080=850
MAX_PRICE_SERIES_RX6800XT=850
MAX_PRICE_SERIES_RX6900XT=1100
PAGE_BACKOFF_MIN=900000
PAGE_BACKOFF_MAX=1800000
PAGE_TIMEOUT=30000
SHOW_ONLY_SERIES=ryzen5600,rx6800xt,rx6900xt,3080
STORES=amazon:60000:90000,amd,asus,bandh:300000:600000,bestbuy,newegg:60000:90000,target,walmart,zotac

global.proxies

http://test-ip:8888
http://nsx-test-2:8888

Those DNS names are fake. I do have working ones in there.

I believe the issue here is that the proxies list is being accessed as zero indexed, so it is accessing proxies 0, 1, 2- however I only defined 2 proxies.

[jef]

@Doridian, do you mind taking a look at this when you get the chance? Thanks!

[Doridian]

The proxy list is being accessed correctly. As you can see it only says Next proxy index: 0 and Next proxy index: 1.
However this Cannot read property 'url' of null error has happened to at least two different people now, and I want to figure out what it actually is. The only objects I know of that have a url property are stores and puppeteer request, and somehow the stack trace is missing, too...

[Doridian]

@MidnightsFX Could you provide the output of node --version?

[MidnightsFX]

node --version
v14.15.1

Well excited its not the proxy settings then! Interestingly I only started seeing this when I setup the global proxy list.

[MidnightsFX]

I am also seeing a much higher rate of request interception not enabled complaints:

[9:31:05 AM] info :: ℹ [bestbuy] Next proxy index: 1 / Count: 2
[9:31:05 AM] error :: Request Interception is not enabled!
[9:31:05 AM] error :: Request Interception is not enabled!
[9:31:06 AM] error :: Request Interception is not enabled!
[9:31:06 AM] error :: Request Interception is not enabled!
[9:31:07 AM] error :: Request Interception is not enabled!
[9:31:07 AM] error :: Request Interception is not enabled!
[9:31:07 AM] error :: Request Interception is not enabled!
[9:31:08 AM] info :: ℹ [asus] Next proxy index: 0 / Count: 2
[9:31:08 AM] error :: Cannot read property 'url' of null
[9:31:09 AM] info :: ✖ [bestbuy] [evga (3080)] xc3 black :: OUT OF STOCK
[9:31:09 AM] info :: ℹ [bestbuy] Next proxy index: 0 / Count: 2
[9:31:14 AM] error :: Cannot read property 'url' of null

Which does not happen when just using a singular proxy from the dotenv. Happy to provide any more details that might be useful too.

[Doridian]

@MidnightsFX I figured out why those happen. That's being fixed in my next PR. Those are correlated to the ZOTAC store, usually when it CAPTCHAs you.

[Doridian]

Turns out you hit two seperate bugs! Fixed by https://github.com/jef/streetmerchant/pull/1408