Wappalyzer: Bug

[fildenis]

After I was in the store on the opencart, I went back to Google and saw opencart on the wappalyzer. After updating the page of Google and even after updating the page with cache cleaning, there is still an opencart. Sorry for my bad english.

[gadcam]

You can trigger this bug by searching inurl:index.php?route= in google.
As usual the html pattern is the culprit, like in #2060 ...
Maybe we should try to use Xpath instead of matching plain html.
What do you think @AliasIO, @jvoisin, @q--, @AlexByte, @kkadosh ?

[AlexByte]

@gadcam How will XPath help?

[fildenis]

@gadcam How will XPath help?
how to do it?

[AlexByte]

Why in version 5.1.6 OpenCart isn't detected in Google?

[gadcam]

@AlexByte It wasn't not detected in previous releases because the HTML cap was not the same before 4a711891fc4022ba8138c55c3ed44bbba32b59bb.

[gadcam]

@fildenis We can not use XPath currently so we need to implement a new way to detect technologies.

@gadcam How will XPath help?

@AlexByte HTML detection is really the weak point of W. :

• the patterns are too loose cf this issue but also #2060
• we can not analyze a lot of HTML with regexes so we have to cap the amount of HTML to process

With a lot of work we can address the first concern.
The second point however is really bad because : we use a lot resources (sometimes it does things like #2061 or #2073) AND we surely miss a lot of things in the body of the page...

So XPath are selectors and so if a framework is using a given structure we could spot it and it will be different than in Google.
You could tell me that you can do that with regexes but I think XPath is really the tool designed to do this sort of things and you also address the performance problem.

[AlexByte]

@gadcam how specifically could you solve this problem of false detection with XPath?

[gadcam]

To be fair I have no knowledge on XPath, I only used them a few times and neither I know how is built a OpenCart website.

But the main idea would be for example to check all div elements with a given class which is specific or all elements inside a p inside a ....

[AliasIO]

We can just use native DOM methods instead of XPath, I'd be in favour of that.

The problem here though is that the pattern is matching plain text. The tests didn't pick it up because there is a < elsewhere in the pattern. I think we need to remove the first half as it targets any link to an Opercart powered website.

[AlexByte]

@gadcam The name of a class is unique for each website.
My idea is to add the ability to do so
"html": "<a href=\"[^>]+{{locationOrigin}}/index\\.php\\?route=[a-z]+/",

[kyletaylored]

Fixed in #2650