Zeronet: Big json.gz , LimitedGzipFile ,then cause Unterminated string starting at: line 1 column 6291436 (char 6291435)

[blurHY]

rev 3571

                if file_path.endswith("json.gz"):
                    data = json.load(helper.limitedGzipFile(fileobj=file))
                else:
                    data = json.load(file)
        except Exception, err:
            self.log.debug("Json file %s load error: %s" % (file_path, err))
            data = {}

Unterminated string starting at: line 1 column 6291436 (char 6291435)

6291456=1024*1024*6

def limitedGzipFile(*args, **kwargs):
    import gzip
    class LimitedGzipFile(gzip.GzipFile):
        def read(self, size=-1):
>>>          return super(LimitedGzipFile, self).read(1024*1024*6)
    return LimitedGzipFile(*args, **kwargs)

Is it intentionally

I checked my big json file,
at col 6291456 ,it/?Post:9115",

[HelloZeroNet]

So you have a anything.json.gz file that contains a file that is larger, than 6MB, right?

[blurHY]

17MB and growing.That's the data of horizon

[HelloZeroNet]

I have increased the limit to 25MB in latest Rev, but I recommend to split your data to multiple file, because in this way the users don't have to re-download the whole data when it's got changed and due other performance reasons.

[blurHY]

I'm going to do that,and make some data files optional

[DaniellMesquita]

@blurHY
Do you have Tox or ZeroMail?

[blurHY]

tox 867226A9F397786A82F9E8E547A0CFDA080E180B8500D58B7830100D68B0AA3F2B5AFF62F744