Crystal: HTTP::Client allow sessions

[Rinkana]

It would be nice to do multiple HTTP requests while not having to manually parse the Set-Cookie header for the next request. Ruby already does this when using the .start function.

I was thinking of implementing something like the .start. It would re use an HTTP::Cookies object for each request.

An idea for the next iteration might be to allow a cookie file so cookies can be saved for later use.

I'd like to hear your thoughts on this before i start writing on a PR. Especially on _how_ you'd want this to be implemented.

[jhass]

I can see a HTTP::Cookies.from_json/#to_json for easy de-/serialization, but I wouldn't want any automatic file writing/reading in the standard library.

I think I'd also like an explicit interface for this, maybe through a client middleware, as discussed in #2721 already.

[ysbaddaden]

Yes, we should be capable to pass an abstract CookieJar class to store cookies between requests, so depending on usages we could choose to store cookies in memory, file, database or whatever.

[jhass]

With the middleware approach we could have a basic in memory implementation in stdlib and more complex stuff as shards.

[Rinkana]

I quite like the middleware idea and i started on a basic implementation.
While working on that is saw that the HTTP::Server already supports middleware trough the HTTP::Handler classes. The files for these classes are now located in the http/server folder but their namespace is HTTP (missing the server).

I was thinking of using i similar approach to the server handlers for the client and renaming the sever one.
This would give you two abstract classes:

• HTTP::Client::Handler
• HTTP::Server::Handler

And they would be in the correct dirs (http/client and http/server) for their used namespace.
Is there anything against this? As it would mean that there needs to be some simple refactoring to make old server handlers work.

[Rinkana]

Alright so i have something working however i'm not sold on it.
The way i've currently implemented it is that you pass an (optional) middleware array with the client creation:

client = HTTP::Client.new("http://www.example.com", handlers: [HTTP::SessionHandler.new])

It will build the middleware the same way as the HTTP::Server middleware (chaining) and use that middleware before requesting and after the response. However it currently means that the middleware has two optional functions. One handles the request and the other handles the response.

So instead of:
Request <--> Middleware <--> Response
You get:
Request --> Middleware --> Response --> Middleware

I tried making it as the first but that would mean a big overhaul of the current HTTP::Client class to support this method as the request and response handling are intertwined with each other and need to be split to make this work. The client class needs to just be the interface between an request and the connection response.

I'd like to have some input on this matter and please view my current implementation at my repo as i might be missing something that would help out.

[TheLonelyGhost]

If it helps, I contributed something similar within Cossack which handles sessions via a cookie jar middleware.

Similar inspiration, but focusing on serialization to line-delimited plaintext format, in a way that would allow us to read each line and put them into HTTP::Cookies via some Set-Cookie headers and HTTP::Cookies#fill_from_headers. (source) My reasoning was ease of implementation, reasonable legibility in the text file, and code reuse for importing cookies via headers. No reason to duck established standards...

[straight-shoota]

I don't know why #3111 was closed, but I think it was not a bad start.

Injecting middleware into HTTP::Client is a great thing and offers flexibility to implement more specific features.
I'd suggest not to name the middleware objects Handler this just creates confusion with the server handlers. The term Middleware like in cossack is probably a good idea, maybe there is something even better. Even with the different name, it makes sense to put the server handlers in the HTTP::Server namespace to keep things organized. I never understood why these obviously server specific classes are directly in the HTTP module (the source code is even in http/server/handlers/*).
I also like the method of operation much better in cossack, that middleware is invoked as a single method (or proc) which takes the request as argument, calls the next middleware and returns the response. So the first middleware will send the actual request to the connection and then parse the response (what exec_internal currently does).

Or, a more radical idea would be to make no difference between server and client handlers and invoke client handlers with a context of request and response. Of course, many handlers only make sense in one context but some like LogHandler could be perfectly used as a client middleware! Similar are TimeoutHandler or even CacheHandler - there should not be much difference between server and client context. Want to mock an HTTP request? Just put a StaticFileHandler in the client middleware! Why not use the same HTTP pipeline on both ends?

[sdogruyol]

This would be a good addition to HTTP::Client