Testcafe: Parent domain cookies aren't being sent from an incoming request on a subdomain

[JasonRammoray]

What is your Test Scenario?

Suppose we have two domains:

• parent.com that hosts a static font and sends two cookies (geo and ccl) if a client didn't provide those cookies
• subdomain.parent.com where our app is hosted. The app loads a font from `parent.com

Steps to Reproduce:

1. Create 300+ simple scenarios that make TestCafe opening at sudomain.parent.com
2. Run these tests in Safari

What is the Current behavior?

At some point TestCafe just freezes up because of hitting 8kb headers payload limit on an incoming request. The reason is that TestCafe doesn't include cookies from parent.com into a request headers when the app loads a static resource from parent.com. Consequently, parent.com sets cookies for .parent.com that are gradually accumulating and are being transferred for any request on subdomain.parent.com.

What is the Expected behavior?

TestCafe should include cookies from parent.com when the app from subdomain.parent.com requests a resource from parent.com.

Environment

• testcafe version: 1.8.5
• node.js version: 10.15.0
• browser name and version: 13.1 (15609.1.20.111.8)
• platform and version: macOS 10.15.4 (19E287)

[miherlosev]

Hi @JasonRammoray

8 kb is the limit for header size in Node.js application. Try to increase this limit using the --max-http-header-size flag.

[JasonRammoray]

Thanks, @miherlosev .
I've tried that before and it worked, however, I don't think this is a step in the right direction moving forward because a new limit could also be easily hit when repeating tests a bunch of times.
The way how I worked around the issue, for now, is that I added an instance of a custom class that extends RequestHook to all the fixtures.
In that class, in onRequest method I am adding certain cookies upon intercepting a request for a static resource on parent.com such that I don't get a new portion of cookies in a response.

Although it works, my gut feeling tells me this is rather a hacky approach and there should be a better solution which is why I raised this issue.

The way I see it is that the problem could be solved in two ways:

• send all the cookies for parent.com when an app from subdomain.parent.com makes a request to a resource from parent.com
• wipe all the cookies not only for the current domain but for all higher-level domains after each test

What do you think?

[Dmitry-Ostashev]

@JasonRammoray We need some additional time to research this scenario. We'll notify you as soon as we have any news.

[JasonRammoray]

@Dmitry-Ostashev , sure.
If you guys experience shortage of resources, I'll be happy to contribute.

[LavrovArtem]

Hello @JasonRammoray,
Unfortunately, the information you shared doesn't give us a clue. We need a simple sample that demonstrates the incorrect behavior you described.

[JasonRammoray]

@LavrovArtem, in a pictorial form the issue looks as follows:

As we can imagine, the accumulated cookies will eventually exceed a default threshold of 8kb.
Increasing that threshold doesn't really solve the issue for the long-term perspective because the new value could be easily exceeded with enough repetitions.

I'll try to prepare a demo this weekend (May 30 - May 31, 2020).

[Farfurix]

@JasonRammoray

Hello,

Thank you. We'll wait for your update.

[JasonRammoray]

@Farfurix, @LavrovArtem, @Dmitry-Ostashev, @miherlosev, gentlemen, please, find a demo illustrating the issue attached (bloating-cookie-1.0.0.zip)

For testing purposes, I took two subdomains on apple.com:

• developer.apple.com
• register.apple.com

Both domains load static data from apple.com and the latter sets cookies for .apple.com.

As we can see on a video, that process goes over and over again leading to linear growth in headers data for each request to an html document eventually hitting 8KB limit for the headers payload which renders TestCafe hanging.

By looking at cookies sample that TestCafe sends during these tests, one might think that TestCafe associates cookies from .apple.com with a session that is different for each test.
Turns out these sessions cookies aren't being cleared out upon completion of each test.

Below we can see traces of 14 sessions:

• TioWPdzt5
• wOMO2GsU0
• BbquHeNEi
• V_twHgEnh
• DH04jH3R1
• NSTYEzOEx
• OreqkIX1n
• BU4lS14ez
• fPmu0sE3D
• tAGlfr0o4
• jtwVPgZA6
• nTXKj3kWW
• wyep7JsP8
• wuCcD8Zd8

c|4h6_waAJs|s_fid|apple.com|%2F|mbd5c6wo|kavzaq60=480FD726007CE260-27194D48614C796A; c|4h6_waAJs|s_sq|apple.com|%2F||kavzaq65=awdappledeveloper%3D%2526pid%253Dios%252520-%252520%252528english%252529%2526pidt%253D1%2526oid%253Dhttps%25253A%25252F%25252Fdeveloper.apple.com%25252F%2526ot%253DA; s|TioWPdzt5|s_vi|apple.com|%2F|l3v2qsq8|kavzaoak=[CS]v1|2F6A40438515973B-60000A5C1F2368A3[CE]; s|TioWPdzt5||register.apple.com|%2Fassets%2Ffonts||kavzaods=; s|TioWPdzt5||register.apple.com|%2Fassets%2Fimages%2Flogin%2Fstart%2FDesktop||kavzaoat=; s|TioWPdzt5||register.apple.com|%2Fassets%2Fimages%2Flogo||kavzaonl=; s|wOMO2GsU0|s_vi|apple.com|%2F|l3v2phns|kavzamg9=[CS]v1|2F6A40428515EC13-40000985AFCB08EC[CE]; s|BbquHeNEi|s_vi|apple.com|%2F|l3v2phns|kavzakks=[CS]v1|2F6A40410515D5E9-600007A5DF130913[CE]; s|BbquHeNEi||register.apple.com|%2Fassets%2Ffonts||kavzakpf=; s|BbquHeNEi||register.apple.com|%2Fassets%2Fimages%2Flogin%2Fstart%2FDesktop||kavzakl2=; s|BbquHeNEi||register.apple.com|%2Fassets%2Fimages%2Flogo||kavzakt2=; s|V_twHgEnh|s_vi|apple.com|%2F|l3v2phns|kavzaig5=[CS]v1|2F6A40400515EE60-600008661F270650[CE]; s|DH04jH3R1|s_vi|apple.com|%2F|l3v2phns|kavzagov=[CS]v1|2F6A403F05159E4F-600007375F185254[CE]; s|DH04jH3R1||register.apple.com|%2Fassets%2Ffonts||kavzagr3=; s|DH04jH3R1||register.apple.com|%2Fassets%2Fimages%2Flogin%2Fstart%2FDesktop||kavzagp2=; s|DH04jH3R1||register.apple.com|%2Fassets%2Fimages%2Flogo||kavzagt2=; s|NSTYEzOEx|s_vi|apple.com|%2F|l3v2phns|kavzaf2j=[CS]v1|2F6A403D851597EB-6000074FAA8DDDF8[CE]; s|OreqkIX1n||register.apple.com|%2Fassets%2Fimages%2Flogo||kavzad96=; s|OreqkIX1n|s_vi|apple.com|%2F|l3v2phns|kavzad4s=[CS]v1|2F6A403C8515E804-400006E45F07E390[CE]; s|OreqkIX1n||register.apple.com|%2Fassets%2Ffonts||kavzad6y=; s|OreqkIX1n||register.apple.com|%2Fassets%2Fimages%2Flogin%2Fstart%2FDesktop||kavzad51=; s|BU4lS14ez|s_vi|apple.com|%2F|l3v2phns|kavzabp6=[CS]v1|2F6A403B85159474-600009574FE81675[CE]; s|fPmu0sE3D|s_vi|apple.com|%2F|l3v2phns|kavzaabf=[CS]v1|2F6A403B0515EA77-4000082D135F8FE6[CE]; s|fPmu0sE3D||register.apple.com|%2Fassets%2Ffonts||kavzaadn=; s|fPmu0sE3D||register.apple.com|%2Fassets%2Fimages%2Flogin%2Fstart%2FDesktop||kavzaabr=; s|fPmu0sE3D||register.apple.com|%2Fassets%2Fimages%2Flogo||kavzaah9=; s|tAGlfr0o4|s_vi|apple.com|%2F|l3v2phns|kavza90m=[CS]v1|2F6A403A0515832D-60000BCB3F26FEE4[CE]; s|jtwVPgZA6|s_vi|apple.com|%2F|l3v2phns|kavza7k5=[CS]v1|2F6A40390515B9D9-4000082D135F8521[CE]; s|jtwVPgZA6||register.apple.com|%2Fassets%2Ffonts||kavza7m5=; s|jtwVPgZA6||register.apple.com|%2Fassets%2Fimages%2Flogin%2Fstart%2FDesktop||kavza7kq=; s|jtwVPgZA6||register.apple.com|%2Fassets%2Fimages%2Flogo||kavza7ov=; s|nTXKj3kWW|s_vi|apple.com|%2F|l3v2phns|kavza68y=[CS]v1|2F6A40380515CC2C-40000ACADF1228BC[CE]; s|wyep7JsP8|s_vi|apple.com|%2F|l3v2phns|kavza4r4=[CS]v1|2F6A403705159D4D-60000AC67F29B19C[CE]; s|wyep7JsP8||register.apple.com|%2Fassets%2Ffonts||kavza4uj=; s|wyep7JsP8||register.apple.com|%2Fassets%2Fimages%2Flogin%2Fstart%2FDesktop||kavza4s4=; s|wyep7JsP8||register.apple.com|%2Fassets%2Fimages%2Flogo||kavza4yb=; s|wuCcD8Zd8|s_vi|apple.com|%2F|l3v2phns|kavza379=[CS]v1|2F6A40360515E621-60000ACADF121C53[CE]

[AlexanderMoiseev]

Hello,

Thank you for your example project. We’ve reproduced the issue and need some time to investigate it. Follow this issue to stay tuned.

Note for the team - ask @AndreyBelym if you need help with reproducing the problem

[JasonRammoray]

Got it.
Please, let me know if I can contribute.

[JasonRammoray]

Gentlemen, could you, please, give an update on the issue?

[Ogurecher]

Hello,

Thank you for your patience. We need more time to investigate this issue. Once we get any results, we will share them here.

[JasonRammoray]

Just checking in to see if there are any updates on the issue.
Gentlemen, could you, please, shed some light on the current status?
Please, let me know if you need some help.

[miherlosev]

Hi @JasonRammoray

We cannot say something definitely because we don't know the cause of the problem. We will update this thread once we have any results.