Urllib3: Add functionality to get the final request URL after redirects, like the function urllib2.urlopen().geturl().

[bosihu]

[Lukasa]

You cannot: this functionality is not supported at this time. Patches to add it would be welcome. :smile:

[crwilcox]

I was investigating this and have a possible solution, but it may make more sense to have this as a utility than to put it off of urlopen. HttpResponse, which is returned by urllib3.PoolManager().request and urllib3.PoolManager().urlopen, doesn't seem to have the url that the data came from. However, HttpResponse does have retry history which, in the case there was a redirect, the url can be discovered. A field containing the location could be added but I am not sure it makes sense to add an additional field to each HttpResponse object. This field would be needed to handle the case that the response was not from a redirect.

If you try

r = http.request('GET', 'http://github.com/shazow/urllib3')
r.retries.history

You will get the following:

RequestHistory(method='GET', url='http://github.com/shazow/urllib3', error=None, status=301, redirect_location='https://github.com/shazow/urllib3')
RequestHistory(method='GET', url='https://github.com/shazow/urllib3', error=None, status=301, redirect_location='https://github.com/urllib3/urllib3')

This will be empty in the case of no redirects.

The following code seems to do what you want:

def geturl(request_type, url):
    http = urllib3.PoolManager()
    response = http.request(request_type, url)
    if len(response.retries.history):
        return response.retries.history[-1].redirect_location
    else:
        return url

[sethmlarson]

Hey @crwilcox could you turn that into a method of Response? I'd gladly accept that change. :)

[crwilcox]

@SethMichaelLarson sure. feel free to assign this to me and I can probably do it Friday :)

[sethmlarson]

Consider yourself officially assigned! (GitHub isn't letting me assign non-collaborators?)

[crwilcox]

@SethMichaelLarson,
I looked at this again and recall why I didn't just make a method and PR. From what I could tell earlier the request url isn't recorded in the response object. So adding response.geturl() would require additional data to be stored on response in the case there wasn't a redirect. Holding this extra data for, essentially, a helper method seemed heavy-handed. Did I perhaps miss a field on response that would allow me to retrieve the requested url?

[sethmlarson]

If you've got time I'd still like to see a PR for this functionality even if it adds a field to response objects. It's genuinely useful functionality we're missing compared to httplib.

[sethmlarson]

Closed by #1382