Web-bugs: www.fcc.gov - ECFS search results not paginated

[miketaylr]

URL: https://www.fcc.gov/ecfs/search/filings?date_received=%5Bgte%5D2017-5-7%5Blte%5D2017-5-10&express_comment=0&sort=date_disseminated,DESC
Browser / Version: Firefox 55.0
Operating System: Mac OS X 10.11
Problem type: Something else - I'll add details below

Steps to Reproduce

1. Navigate to: https://www.fcc.gov/ecfs/search/filings?date_received=%5Bgte%5D2017-5-7%5Blte%5D2017-5-10&express_comment=0&sort=date_disseminated,DESC

Expected Behavior: There should be 3,453 results, with pagination

Actual Behavior: It says 0 results (but is showing more than 0), and no pagination is available.

_From webcompat.com with ❤️_

[denschub]

AngularJS. Pagination is defined in https://www.fcc.gov/ecfs/scripts/scripts.min.js, searchable via searchPagination. HTML markup is there in Firefox, but hidden and the list of pages only contains one element.

[wisniewskit]

The template filings.html gets this for its headers value in Chrome (I just had it dump the json value):

{ "params": "[object Object]", "total": "3453", "offset": "0", "content-type": "application/json; charset=utf-8", "access-control-allow-origin": "https://www.fcc.gov", "x-response-time": "368.923ms", "cache-control": "no-cache", "limit": "25" }

In Firefox, this is what it dumps:
{ "content-type": "application/json; charset=utf-8", "cache-control": "no-cache" }

[denschub]

That's the module, basically:

}), angular.module("fcc.ecfs").directive("searchPagination", [
  "$stateParams",
  "$state",
  function(a, b) {
    return {
      template: '<pagination direction-links="false" max-size="8" total-items="meta.total" items-per-page="meta.limit" ng-model="page" ng-change="changePage()" class="pagination-sm pagination" boundary-links="false" rotate="false" num-pages="numPages" ng-show="{{pageTotal > pageLimit}}"></pagination>',
      restrict: "EA",
      scope: { meta: "=" },
      link: function(c, d, e) {
        c.changePage = function() {
          var d = _.clone(a);
          d.limit = parseInt(d.limit || 25, 10), d.offset = (c.page - 1) *
            d.limit, window.scrollTo(0, 0), b.go(".", d);
        };
        var f = c.meta || {};
        c.page = Number(f.offset) / Number(f.limit) + 1, c.pageLimit = Number(
          f.limit
        ) ||
          25, c.pageTotal = Number(f.total) || 0;
      }
    };
  }
]), angular.module("fcc.ecfs").directive("facetSearchFilter", function() {

Interesting is the line:

c.page = Number(f.offset) / Number(f.limit) + 1

f seems to be the header object, as @wisniewskit found. But yeah, that's different in Chrome and Firefox.

[denschub]

Both browsers receive the same response from the server, so nothing fishy here.

(For the XHR to https://ecfsapi.fcc.gov/filings?date_received=%5Bgte%5D2017-5-7%5Blte%5D2017-5-10&express_comment=0&limit=25&sort=date_disseminated,DESC, that is.)

[miketaylr]

Both browsers receive the same response from the server, so nothing fishy here.

I'm seeing the same thing, for both the https://ecfsapi.fcc.gov/filings?date_received=%5Bgte%5D2017-5-7%5Blte%5D2017-5-10&express_comment=0&limit=25&sort=date_disseminated,DESC and https://ecfsapi.fcc.gov/documents?id_submission=105100914620866,10510204081248,10510251716417,1051009230015,105102457227022,10510666509421,1051027075503,105102927106512,10510215301771,1051086390924,10510273721859,10510724612672,1051090890327,1051098446493,105102127108767,1051033958850,10510295366600,10510473006830,10510659520640,105102239124169,10510145239750,10510246821461,10510883015246,1051021019305,105102432410051 xhr requests.

[miketaylr]

FWW, they're using AngularJS 1.3.20

{
  "full": "1.3.20",
  "major": 1,
  "minor": 3,
  "dot": 20,
  "codeName": "shallow-translucence"
}

[denschub]

In angular.min.js, there is a call to C.getAllResponseHeaders() in the XHRs onload. Interestingly, there are some differences in their return values.

In Chrome:

"params: [object Object]
total: 3453
offset: 0
Content-Type: application/json; charset=utf-8
Access-Control-Allow-Origin: https://www.fcc.gov
X-Response-Time: 370.409ms
limit: 25"

In Firefox:

"Content-Type: application/json; charset=utf-8

"

---

Just double checked if this is the same request.

Chrome:

Firefox:

One idea is that the squared brackets are throwing our parser off. @wisniewskit is having a look, I'll do so as well.

[denschub]

Isolated test-case with that URL: http://jsbin.com/wejehuqaxo/1/edit?js,console

Seems like there are indeed issues with the header parsing.

[denschub]

This seems to be both header format abuse (technically, [ and ] are used as field separators as per RFC2616) and we're not recovering as nice as Chrome does.

@wisniewskit is filing a Necko/Gecko issue as well as an issue to track a possible change to the XHR spec to add a note that headers have to be parsed differently. Edge, Safari, and Chrome seem to recover just fine, so I guess it's fine to make that a default behavior for everyone.

As for a fix the site could deploy, they probably should wrap their header into double quotes, which would make this issue disappear. What do you think, @miketaylr?

[miketaylr]

@denschub yeah, that sounds cool -- let's move this to needscontact to explain the double quote workaround while @wisniewskit works to get the right bug filed.

[miketaylr]

I see the @FCC is on GitHub. I wonder if @bizui @brianfunk @antoinegreen could help us find the right contact to fix this issue?

To summarize, ECFS search results aren't paginated properly in Firefox, due to an invalid HTTP response header format (which other browsers are able to recover from). To fix this, the params response should be double quoted (possibly returning "[object Object]" isn't actually useful anyways?).

[adamopenweb]

Reaching out via Linkedin as well.

[wisniewskit]

Update: actually the problem is not the brackets, it's that their CORS "expose headers" line has a stray space in the list of allowed headers:

Access-Control-Expose-Headers:server,limit,offset,params,response-time,total,authorization,access-control-allow-origin,x-response- time

Chrome is fine with that. Firefox just ignores the entire header, which is why none of the headers end up being accessible.

[miketaylr]

Ah, nice find!

[wisniewskit]

Alright, I've filed bug 1364598. Turns out that Firefox, Chrome/Safari, and Edge all treat this case differently, so it's a subtle webcompat issue.

Regardless, it's something the FCC should fix on their end as well, as it's an error in their CORS header even if some browsers try to work around it for them.

[brianfunk]

Thanks for investigating and thanks for tag @miketaylr . On it.

[wisniewskit]

Just a quick update: it was decided that Firefox's behavior is correct, given that it is what the spec says and there is no evidence of that this is a widespread webcompat issue. As such the web platform tests have been updated and Chrome, Edge, and Safari have been contacted to adhere to the spec, with signs that Chrome and Edge are already working on patches.

[brianfunk]

Update: the issue has been resolved on FCC.gov with the extra space in header removed.

[adamopenweb]

Thanks for the help @brianfunk! I can confirm the issue is resolved. Getting 3508 results from the original query, tested in Firefox 55 for OSX.

Let's close this report.