Tfjs: tf.loadModel Fails to load keras model

[YajJackson]

To get help from the community, check out our Google group.

TensorFlow.js version

0.13.5

Browser version

Version 70.0.3538.110 (Official Build) (64-bit)

Describe the problem or feature request

I can't load a keras model created by tfjs.converters.save_keras_model

Code to reproduce the bug / link to feature request

Used this to create model.json

import keras
import tensorflowjs as tfjs
mobilenet = keras.applications.mobilenet.MobileNet()
tfjs.converters.save_keras_model(mobilenet, './tfjs-models/MOBILENET')

Attempt to load model here

const model = await tf.loadModel(`./tfjs-models/MOBILENET/model.json`);

result:

Uncaught (in promise) RangeError: byte length of Float32Array should be a multiple of 4
    at new Float32Array (<anonymous>)
    at o (io_utils.ts:116)
    at Object.decodeWeights (io_utils.ts:79)
    at models.ts:287
    at index.ts:68
    at Object.next (index.ts:68)
    at o (index.ts:68)

Any help would be greatly appreciated.

[caisq]

Can you open the browser's devtools and check the status of the responses to the weight requests. Those should show up as names such as "group1-shard1of2", etc. Make sure that the response status is 200 OK and that the content of the response is correct. We've seen various kinds of HTTP servers having trouble serving binary weight files like those correctly.

[YajJackson]

@caisq I'm serving these files locally, and see that rather than 200, each file has a 304 'Not Modified' status.

[caisq]

What kind of HTTP server are you using, @YajJackson ?

[YajJackson]

I've been using parceljs.

[YajJackson]

Is there anyway for me to test that the binaries are served properly besides tf.loadModel() failing?

[dsmilkov]

Couple of things:

• Can you share with us the model.json? In particular search for "group1-shard1ofX" and see if you can access that file directly from the browser.

• On our side, we will improve the error message shown to the user when calling loadModel():

  • inspect the http response code and if it's not 200 OK, let the user know the response code
  • inspect the http encoding and make sure it's octet-stream / binary
  • print out the actual url that we tried to access (the url obtained by merging the base path of model.json and the "paths" field)
  • print the relevant sub-section of model.json that was used to make this request.

[YajJackson]

@dsmilkov I can access the files directly from the browser. After clearing my cache, the 304s returned to 200 OK, so looks like we're good there. Turns out that the group1-shardxofy responses had Content-Type: text/html; charset=UTF-8.

[caisq]

+1 to Daniel's point that we should throw a more informative error message when the response code isn't 200 OK.

[YajJackson]

@caisq I think 304 would also be acceptable.

[YajJackson]

Good news! after switching from serving my files with parcel to webpack-dev-server, the content-type header was updated appropriately to application/octet-stream. I'm no longer receiving the previous error, however, it now appears to be silently failing.

Consider this snippet:

let model;
const loadModel = async m => {
  $("#loading-spinner").show();
  model = await tf.loadModel(`/assets/tfjs-models/${m}/model.json`);
  console.log(model);
  $("#loading-spinner").hide();
};

*where m is 'MOBILENET'
Previously, tf.loadModel() would fail with the error I provided initially. Now, it seems that it's endlessly loading, without ever timing out. What could be the cause of this?

[YajJackson]

I can't say exactly what solved the issue I just mentioned, but after the following steps I was able to get that snippet to work properly:

1. I changed tf.loadModel() to target 'https://storage.googleapis.com/tfjs-models/tfjs/iris_v1/model.json' (from example 4 in the loadModel docs)
2. Cleared browser cache
3. Loaded site, and observed that the example model loaded successfully.
4. Switched back to tf.loadModel(`/assets/tfjs-models/${m}/model.json`);
5. Loaded site, observed successful MOBILENET model load.

¯_(ツ)_/¯

[JWSunny]

@YajJackson hello, when i use the tfjs loadfrozenmodel，i have the same problem: RangeError: byte length of Float32Array should be a multiple of 4, the statue is 200 and i alse use the parceljs, but also have the mistake, can you have any other suggestions? thanks

[JWSunny]

@YajJackson

[caisq]

I find it surprising that the weight shard files (group1*) have a size of
1.4 KB. They should be about 4 MB (except the last one), because that's the
default shard size. So it appears that they somehow got the wrong content.

On Mon, Jan 14, 2019 at 3:39 AM sunny [e-mail] wrote:

@YajJackson https://github.com/YajJackson
[image: image]
https://user-images.githubusercontent.com/29625315/51102702-f0d1ac00-181a-11e9-9833-3a05642ee1c1.png

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tfjs/issues/924#issuecomment-453931352,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQC5fk978I-jeTD6ra9ok-cfLcEaRRT7ks5vDEJDgaJpZM4YqSzn
.

--

Shanqing Cai
Software Engineer
Google
[email protected]

[JWSunny]

@YajJackson @caisq hello，i have similar problem，but the group* file is 1.4k，because of the parcel Can't serve static resources with no filename extension( https://github.com/parcel-bundler/parcel/issues/1098 )，how can you success load the group* files? can you provide some suggestions, thanks.

[caisq]

Can you the the following workaround:

• Rename the weight shard files so that they can have extension names,
  e.g., group*.bin
• Modify the weightsManifest part of the JSON in the model.json file to
  reflect the name changes in the weight shard files.

Feel free to file an issue at https://github.com/tensorflow/tfjs/issues for
us to add an extension name to the weight files.

On Tue, Jan 15, 2019 at 8:27 PM sunny [e-mail] wrote:

@YajJackson https://github.com/YajJackson @caisq
https://github.com/caisq hello，i have similar problem，but the group*
file is 1.4k，because of the parcel Can't serve static resources with no
filename extension( parcel-bundler/parcel#1098
https://github.com/parcel-bundler/parcel/issues/1098 )，how can you
success load the group* files? can you provide some suggestions, thanks.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tfjs/issues/924#issuecomment-454616094,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQC5fjZagMrxrqqXuFMko7rCZQWszDXoks5vDoATgaJpZM4YqSzn
.

--

Shanqing Cai
Software Engineer
Google
[email protected]

[JWSunny]

@caisq i rename the group* file as group.bin and modify the key of "path" in the json, but failed, because the parcel use the require("weightsManifest.json") and return the file path is weightsManifest.(hash value).json, so the new path of group.bin is group*.(hash value).bin。

[caisq]

Have you tried this parcel plugin?
https://www.npmjs.com/package/parcel-plugin-static-files-copy

It can copy files into the dist/ folder without adding the hash string to
the file name.

On Tue, Jan 15, 2019 at 9:44 PM sunny [e-mail] wrote:

@caisq https://github.com/caisq i rename the group* file as group.bin
and modify the key of "path" in the json, but failed, because the parcel
use the require("weightsManifest.json") and return the file path is
weightsManifest.(hash value).json, so the new path of group.bin is
group*.(hash value).bin。
[image: image]
https://user-images.githubusercontent.com/29625315/51223155-b59abe80-197b-11e9-928a-ea1f7cab50e3.png

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tfjs/issues/924#issuecomment-454631015,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQC5fvoUpRKCcr4M-WQARNuZPUBOAjyDks5vDpIdgaJpZM4YqSzn
.

--

Shanqing Cai
Software Engineer
Google
[email protected]

[JWSunny]

@caisq thanks，i'll try it.

[JWSunny]

@caisq thanks for your suggestions, i solved the problem successfully，yesterday.

[vpanichkin]

Try to collect the solution in step-by-step tutorial:

1) Install parcel-plugin-static-files
2) Create static folder in your project folder
3) Locate you model, weights and chunks there
4) Add to your chunks (beginning with _group1-shard_) extension .bin
5) Change file weights_manifest.json : find key "paths" and add extension .bin to all items in array
6) Alter MODEL_URL and WEIGHTS_URL in your script respectively and parcel will serve files for you: http://localhost:1234/tensorflowjs_model.pb
http://localhost:1234/weights_manifest.json
7) Restart parcel

After that, the issue should disappear!

[mrmartin]

Same problem. What worked for me was

1. adding .bin extension to group1-shard*of*
2. changing the .json file to reflect this

[dsmilkov]

If you've experienced problems loading models in the latest release, we are working on a fix: https://github.com/tensorflow/tfjs/issues/1188.

It's related to the problem that we are strict about the Content-Type header produced by the http server - we expect application/json for model.json and octet-stream for weights.

We are going to revert the content type check.

[mrmartin]

1188 will be helpful, but I don't think it's the root of the problem.

At least for me, the weights are an octet-stream even when it was failing.
The error was:
Expected content type application/octet-stream but got null.

and here's the file, with Content-Type:

Could it be that the issue is with misreading file type instead?

[dsmilkov]

Hi all,

The fix just went in: https://github.com/tensorflow/tfjs-core/pull/1532 and will be available in 0.15.1, which we'll release in 2 days.

[adrienZ]

Same problem. What worked for me was

1. adding .bin extension to group1-shardof
2. changing the .json file to reflect this

Worked like a charm for me thanks @mrmartin

[yingshaoxo]

Try to collect the solution in step-by-step tutorial:

1. Install parcel-plugin-static-files
2. Create static folder in your project folder
3. Locate you model, weights and chunks there
4. Add to your chunks (beginning with _group1-shard_) extension .bin
5. Change file weights_manifest.json : find key "paths" and add extension .bin to all items in array
6. Alter MODEL_URL and WEIGHTS_URL in your script respectively and parcel will serve files for you: http://localhost:1234/tensorflowjs_model.pb
   http://localhost:1234/weights_manifest.json
7. Restart parcel

After that, the issue should disappear!

It just too complex for using!

https://github.com/elwin013/parcel-plugin-static-files-copy/issues/37

https://github.com/parcel-bundler/parcel/issues/1098

[theairbend3r]

I have a similar error - RangeError: attempting to construct out-of-bounds TypedArray on ArrayBuffer

I'm using express.js to send the json+bin files to react so that I can run inference in the browser itself.

Here's the relevant Express.js code. The json+bin files are all in the same folder.

app.use(
  "/api/pokeml/classify",
  express.static(path.join(__dirname, "classifier_models/original/model.json"))
)

Here's how I'm loading this in React -

  useEffect(() => {
    async function fetchModel() {
      // const mobileNetModel = await mobilenet.load()
      // const classifierModel = await axios.get("api/pokeml/classify")
      const classifierModel = await tf.loadLayersModel(
        "http://localhost:3001/api/pokeml/classify"
      )
      setModel(classifierModel)
    }
    fetchModel()
  }, [])

The model.json file loads correctly but the shards do not -