Keepassxc: Remove qhttp and KeePassHTTP functionality

[applesea2]

When I create any entry for a website for a known favicon, clicking "download favicon" reports the error " unable to fetch favicon" in all cases

Expected Behavior

I expect clicking "download favicon" to correctly pull the icon from the database

Current Behavior

ERROR: unable to fetch favicon

Possible Solution

Steps to Reproduce (for bugs)

1. Create or edit an existing entry for a website with a known favicon
2. Click icon on the left side
3. Click download favicon on the bottom

Context

I would have to manually import every favicon with a file instead of having them being pulled from the database

Debug Info

KeePassXC - 2.20
Revision: caa49a8ef3ee28ed478192389b21d61107b3b8e0

Libraries:

• Qt 5.9.0
• libgcrypt 1.7.7

Operating system: Windows 10
CPU architecture: 64 bit
Kernel: Windows NT

Enabled extensions:

• KeePassHTTP
• Auto-Type
• YubiKey
  2017-08-30 14-50-17.zip

[droidmonkey]

I cannot reproduce this issue. Have you enabled the use of the google favicon service in settings? Sometimes that clesrs up "hard to find" favicons.

[applesea2]

Yes it is enabled under settings. I did, however get one of my sites to find a favicon (battle.net), but the 70 other would not fetch. I tested out google using the url http://www.google.com and it wouldn't work, for example. After I noticed when I tried to save it, it said access is denied. That could possibly be because I migrated from Keepass 2 using the new format. I also don't like the fact that I have to manually click the download button on each entry so maybe after you fix this you can make it easier to download multiple icons at a time. 
On 9/4/2017 8:30:32 AM, Jonathan White [e-mail] wrote:
I cannot reproduce this issue. Have you enabled the use of the google favicon service in settings? Sometimes that clesrs up "hard to find" favicons.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub [https://github.com/keepassxreboot/keepassxc/issues/913#issuecomment-326951332], or mute the thread [https://github.com/notifications/unsubscribe-auth/ATNFzamKUV7d_gSKSDwudaU02KmTpAMYks5se-1mgaJpZM4PH0i6].

[droidmonkey]

I can't fix what I can't reproduce. I just tried downloading a favorite icon from multiple sites and it worked fine. Try creating a new database, a new entry, and see if it still fails for you.

KeePassXC - Version 2.2.0
Revision: caa49a8ef3ee28ed478192389b21d61107b3b8e0

Libraries:
- Qt 5.9.0
- libgcrypt 1.7.7

Operating system: Windows 10 (10.0)
CPU architecture: x86_64
Kernel: winnt 10.0.15063

Enabled extensions:
- KeePassHTTP
- Auto-Type
- YubiKey

[ArchangeGabriel]

I’m also having an issue downloading favicons, but not for all websites. It doesn’t work for https://www.openstreetmap.org for instance.

Also, I’ve found no reference to Google Favicon Service in the settings, is this a Windows thing?

[droidmonkey]

My apologies i thought we added that option. All requests for fav icons go to the original entered site first, then to the google lookup service.

[applesea2]

I didn't see that option too and its not a Windows thing. I am not exactly sure why it only works on some urls, but using the official keepass and the favicon downloader works perfectly and I don't have to download each entry individually.
On 9/5/2017 11:45:54 AM, Jonathan White [e-mail] wrote:
My apologies i thought we added that option. All requests for fav icons go to the original entered site first, then to the google lookup service.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub [https://github.com/keepassxreboot/keepassxc/issues/913#issuecomment-327217599], or mute the thread [https://github.com/notifications/unsubscribe-auth/ATNFzXnzGEXVpTsMgzrh3ac8vkEmHRh9ks5sfWyxgaJpZM4PH0i6].

[applesea2]

I did a test on my Linux machine and the issue is nonexistent. I believe
it's just a Windows bug where it will not download any favicons. Just push
out an update when you can that will fix it along with possibly making it
easier to download multiple favicons at once.

On Sep 4, 2017 8:30 AM, "Jonathan White" [e-mail] wrote:

I cannot reproduce this issue. Have you enabled the use of the google
favicon service in settings? Sometimes that clesrs up "hard to find"
favicons.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/keepassxreboot/keepassxc/issues/913#issuecomment-326951332,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ATNFzamKUV7d_gSKSDwudaU02KmTpAMYks5se-1mgaJpZM4PH0i6
.

[droidmonkey]

There is nothing to fix from my perspective because it works perfectly fine for me (and many more people) on Windows.

[phoerious]

I've also had some more issues with HTTPS. Need to check if they are fixed with develop and if not fix them myself once I find the time (moving into a new flat right now, so very tight schedule atm).

[ArchangeGabriel]

Please tell me if I can provide any information for the non-working cases (and if you could reproduce them on your side by creating dummy entries, that could likely help I guess).

[applesea2]

I got some non working cases in the attachment. Virtually no site that I have tried will download a favicon even though it should be. I swear these sites worked on my Linux machine, but not on Windows, yet the original keepass with favicon downloader plugin downloads the sites' favicon perfectly for some reason. I even created a completely brand new virtual machine and the same bug exists on my end
Videos.zip

[jamiecon]

I am seeing this issue with KeepassXC on Ubuntu, installed via Snap. Occurs consistently for any domain.

My database was imported from a Keepass2 database but it also occurs on a brand new test database I created.

KeePassXC - Version 2.2.0
Revision: 9a7e6850d6b79c47c2d46d99fdaf433c2ff13716

Libraries:

• Qt 5.7.1
• libgcrypt 1.7.6-beta

Operating system: Ubuntu Core 16
CPU architecture: x86_64
Kernel: linux 4.10.0-33-generic

Enabled extensions:

• KeePassHTTP
• Auto-Type
• YubiKey

If there's any further diagnostic info I can provide I'm happy to help.

EDIT: Added diagnostic info

[droidmonkey]

OK watched your videos and repeated your keystrokes EXACTLY and I downloaded the favicon for reddit and yahoo without any issue. I can only conclude that the failure to download the icons is due to a configuration error on your windows computer. Whether that is a proxy server, firewall, router, DNS, I have no idea. I am almost 99% certain it is not KeePassXC's code though.

[applesea2]

OK I verified that I didn't have an corrupted system files and no abnormal internet configuration. I tested every single option through troubleshoot compatibility and same issue. Replicating the issue in a virtual machine and same issue.

[MichJB93]

I'm having the same issue downloading favicons I just downloaded keepasXC, created a new database and a new entry and still having the same issue.

KeePassXC - Version 2.2.0
Revision: caa49a8ef3ee28ed478192389b21d61107b3b8e0

Libraries:

• Qt 5.9.0
• libgcrypt 1.7.7

Operating system: Windows 10 (10.0)
CPU architecture: x86_64
Kernel: winnt 10.0.15063

Enabled extensions:

• KeePassHTTP
• Auto-Type
• YubiKey

[droidmonkey]

I forgot this feature was worked on by @TheZ3ro in PR #719. This is slated for release in 2.2.1 so tune back once its released to let us know if its still broken.

[peternm22]

I upgraded to 2.2.1 and I still can't download any favicons on Windows. Every time I try, it waits for a few seconds, and I get this error:

Unable to fetch favicon.
Hint: You can enable Google as a fallback under Tools>Settings>Security

I enabled that setting, and I still get the same error message. I tried it with a bunch of different entries in my database, and not a single one works. This is on Windows 10 64 bit.

I decided to try it on a computer at work (running Windows 7 64 bit). I downloaded KeepassXC portable 64 bit, and created a new dummy database to test with. Every site I tested (Google, Yahoo, Reddit, etc), resulted in the same error message as my home computer. No fancy firewall or antivirus running on either system, just the built in Windows stuff.

Is there anything I can do to help debug this?

[phoerious]

Can anyone else confirm this behavior?

[jamiecon]

I see the same message on Linux, however when I check 'Use Google as a fallback' i am able to retrieve the icons.

For some sites I see the message "Unable to retrieve favicon" without the hint to use Google.

I did a quick tcpdump without the Google checkbox selected and I can see the request going out and a response arriving. I didn't attempt to decode the response as it's HTTPS.

However a Windows 10 machine on the same network with a fresh KeepassXC install had no problems grabbing the icons.

Happy to help with further info.

KeePassXC - Version 2.2.1
Revision: 2bce9c8add07226e9a05e9e0fd0e5e66b236d5b6

Libraries:

• Qt 5.7.1
• libgcrypt 1.7.6-beta

Operating system: Ubuntu Core 16
CPU architecture: x86_64
Kernel: linux 4.10.0-37-generic

Enabled extensions:

• KeePassHTTP
• Auto-Type
• YubiKey

[droidmonkey]

That is interesting. These anomalies may go away one i replace qhttp with qhttpengine.

[phoerious]

Is QHttpEngine less buggy than QHttp? I'm getting increasingly fed up with the lack of minimalistic quality C++ HTTP libraries.

[droidmonkey]

It is at least supported by Debian and under active development.

[simgunz]

I've just installed keepassxc on Linux and I also have some trouble downloading some favicons. It seems that the URL needs to be specified precisely. As an example if the URL is "live.com" it doesn't work if it is "https://live.com" it manages to download the favicon.

Maybe the downloader can just try to append the "https://" prefix if the download fails on the first trial (or maybe there is a more elegant solution).

Also I can't find the option "Use Google as a fallback". Should it appear in the error dialog?

[phoerious]

I recently found this workaround for the old bearer management issue in QNetworkAccessManager: https://lostdomain.org/2017/06/17/qt-qnetworkaccessmanager-causing-latency-spikes-on-wifi/
I suppose with that fix we could dump this horrific QHttp library and go back the otherwise stable native Qt tools.

[droidmonkey]

I am very close to finalizing my removal of qhttp. I had to use curl for the favicon downloading, but that let me cut off many lines of code and recursion.

[droidmonkey]

This will be corrected by removing KeePassHTTP functionality in v2.4.0. KeePassXC-Browser is the replacement.