Archisteamfarm: Suggestions to optimize HTTP requests and performance

[guihkx]

I realized ASF makes GET requests when a HEAD request would consume much less bandwith and at the same time improve performance. A example of this is the method IsLoggedIn():

1. Makes a GET request to /my/profile;
2. Parses the response header to find the URL it will redirect to;
3. Redirects to the actual user profile;
4. Parses the entire HTML response and transforms it to an object;
5. Searches for a XPath pattern: //span[@id='account_pulldown'];
6. If found, the user is logged in.

It would be much more efficient if you do this:

1. Make a HEAD request to /my/profile;
2. Check if the Location: header contains steamcommunity.com/login/;
3. If the above condition is true, the user is not logged in.

The way you can do step 2 is up to you (I'm not familiar with C# libraries), but I would simply do something like:

return (wholeResponseHeaderStr.Contains("steamcommunity.com/login/")); // true: not logged in

This can be applied to other methods too. I didn't look through the entire project searching for all the requests, but there's another method I noticed that can be changed: MarkInventory(). I've already tested it, if you make a HEAD request to /id/blahblah/inventory, it will dismiss the notifications too (but you're free to check it yourself). However, as I don't know the behavior of C#, I wonder if you first make a HEAD request to /my/inventory, will the C# library make the following request using GET or HEAD request to the URL found in the Location: header? By the logic, it should use HEAD...

That's about it. Sorry for the wall of text! Thanks.

FUN DATA!

[JustArchi]

This won't optimize in any significant way network usage of ASF, as network usage of ASF is mostly based on badge pages and games pages, those two requests are probably the only ones that could be adapted to use HEAD, and it complicates things further because of need for new methods in my WebBrowser

Wishlist.

[JustArchi]

As I initially suspected, MarkInventory() and IsLoggedIn() are the only 2 methods where such change is even possible, as everything else depends highly either on content of GET, or is using POST messages.

It won't result in any significant improvements as those requests are not heavy, but I followed your suggestion anyway as better performance is always nice. I'll need to test it carefully though.

[JustArchi]

Also released new pre-release dedicated to testing if this commit didn't break anything: https://github.com/JustArchi/ArchiSteamFarm/releases/tag/2.0.4.1

[guihkx]

I'll update it right now and report to you if I find anything unusual.

[JustArchi]

BTW, if you want to reduce your network usage then FarmingDelay is the most important variable as it directly affects the frequency of one of the most heavily used request - the one which checks if all cards are dropped already. Increasing it from 5 to e.g. 15 should result in much less bandwidth usage, for the cost of being less-precise and wasting up to 15 minutes after game is fully farmed, of course.

[guihkx]

Duly noted. To be honest, probably the most traffic was from accepting things to trade or sell them in the market. Last month I sold and trade a lot of things.

By the way, I noticed that isn't necessary to make a request to /my/profile, because Steam will redirect you to /id/blahblah/profile, simply requesting /my/ will redirect "correctly". However, It's kinda useless since both requests will work anyway. lol

[JustArchi]

/my also works with several other things - ASF uses it in various places, and when we expect to land on profile it should be stated, just in case Valve decides to change default link to something like home or whatever.

[guihkx]

Smart decision.

[JustArchi]

I can't notice anything abnormal, everything seems to work properly, both session refreshes and inventory marking, which is good :+1: