Isomorphic-git: Sometimes it is not possible to read the tree property with readObject after git clone.

Created on 6 Sep 2018 · 7Comments · Source: isomorphic-git/isomorphic-git

My mention:

Browser(Google Chrome v68)
[email protected]
IndexedDB

Because I can not use English much, I am using a translation machine.

Sometimes it is not possible to read the tree property with readObject after git clone.
I tried it with the following sample.

I opened the chrome developer tool and initialized with Application from Clear Storage.
After that, I ran init, created a directory and git cloned.
Finally, an error occurred when 10 tree objects were git logged and readObjected.

However, reloading the browser after git clone did not cause an error.

Is there any way to solve this problem?

sample html file

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta http-equiv="X-UA-Compatible" content="ie=edge">
  <title>Document</title>
  <script src="https://isomorphic-git.org/js/browserfs.js"></script>
  <script src="https://isomorphic-git.org/js/pify.js"></script>
  <script src="https://unpkg.com/isomorphic-git"></script>
</head>
<body>
<p>case1: clear site data → init → mkdir → clone1 → log (ng)</p>
<p>case2: case1 → reload browser → init → log (ok)</p>
<p>case3: clear site data → init → mkdir → clone1 → init → log (ng)</p>
<p>case4: case2 → fsGetRootFsEmpty → mkdir → clone2 → log (ng)</p>
<script>
function init (params) {
  let fsOptions = {
    fs: 'IndexedDB',
    options: {}
  }

  BrowserFS.configure(fsOptions, function (err) {
    if (err) return console.log(err)

    window.fs = BrowserFS.BFSRequire('fs')

    // Initialize isomorphic-git with our new file system
    git.plugins.set('fs', fs)

    // make a Promisified version for convenience
    window.pfs = pify(fs)
    console.log('finish initializing')
  })
}

async function clone (dir, url) {
  await git.clone({
    dir,
    corsProxy: 'https://cors.isomorphic-git.org',
    url: url
  })
  console.log('done')
}

async function log (dir) {
  let log = await git.log({
    dir,
    depth: 10
  })
  await readObject(dir, log)
}

async function readObject(dir, log) {
    for (const { oid, tree } of log) {
    let blobs = await git.readObject({
      dir,
      oid: tree,
    })
    console.log({ oid }, { tree }, blobs.object)
  }
}

async function mkdir (dir) {
  let res = await pfs.mkdir(dir)
  console.log('done')
}

async function readdir (dir) {
  let res = await pfs.readdir(dir)
  console.log(res)
}

async function fsGetRootFsEmpty() {
  await fs.getRootFS().empty()
  console.log('done')
}
</script>

<button onclick="init()">init</button>
<button onclick="clone('tutorial', 'https://github.com/tomatoaiu/vue-parcel-pack.git')">clone 1</button>
<button onclick="clone('tutorial', 'https://github.com/tomatoaiu/vue-vuex-counter.git')">clone 2</button>
<button onclick="log('tutorial')">log</button>
<button onclick="mkdir('tutorial')">mkdir</button>
<button onclick="readdir('tutorial')">readdir</button>
<button onclick="fsGetRootFsEmpty()">fsGetRootFsEmpty</button>
</body>
</html>

error

Uncaught (in promise) u: An internal error caused this command to fail. Please file a bug report at https://github.com/isomorphic-git/isomorphic-git/issues with this error message: GitTree: Error parsing buffer at byte location 0: Could not find the next space character.
    at https://unpkg.com/[email protected]/dist/bundle.umd.min.js:8:28437
    at new a (https://unpkg.com/[email protected]/dist/bundle.umd.min.js:8:28955)
    at Function.from (https://unpkg.com/[email protected]/dist/bundle.umd.min.js:8:29142)
    at Module.<anonymous> (https://unpkg.com/[email protected]/dist/bundle.umd.min.js:8:286250)
    at Generator.next (<anonymous>)
    at n (https://unpkg.com/[email protected]/dist/bundle.umd.min.js:8:285413)
    at https://unpkg.com/[email protected]/dist/bundle.umd.min.js:8:285507

bug

Source

tomatoaiu

All 7 comments

Thank you! I have heard about this problem before, but have never been able to reproduce this! This is very helpful.

I understand the problem now! I do not know the solution yet, but I will solve this problem.

wmhilton on 6 Sep 2018

😄1

Note to self: this problem occurs with both [email protected] and [email protected]

wmhilton on 6 Sep 2018

Note: I've been able to reproduce the problem in Node! So it is NOT a browserfs bug as I previously suspected.

const fs = require('fs')
const git = require('..')
const pfs = require('fs').promises
git.plugins.set('fs', fs)

async function mkdir (dir) {
  let res = await pfs.mkdir(dir)
  console.log('done')
}

async function clone (dir, url) {
  await git.clone({ dir, url })
  console.log('done')
}

async function log (dir) {
  let log = await git.log({ dir, depth: 10 })
  await readObject(dir, log)
}

async function readObject(dir, log) {
    for (const { oid, tree } of log) {
    let blobs = await git.readObject({ dir, oid: tree })
    console.log({ oid }, { tree }, blobs.object)
  }
}

async function readdir (dir) {
  let res = await pfs.readdir(dir)
  console.log(res)
}

async function fsGetRootFsEmpty() {
  await fs.getRootFS().empty()
  console.log('done')
}

async function case1 () {
  await mkdir('tutorial')
  await clone('tutorial', 'https://github.com/tomatoaiu/vue-parcel-pack.git')
  await log('tutorial')
}
async function case2 () {
  await log('tutorial')
}

// Instructions: uncomment one of these to run one case at a time
// case1();
case2();

wmhilton on 8 Sep 2018

and it does not depend on the url 'https://github.com/tomatoaiu/vue-parcel-pack.git'. Also happens if you use 'https://github.com/isomorphic-git/isomorphic-git.git'

it's got to be a race condition. somehow it's trying to read parts of the packfile that aren't ready yet.

wmhilton on 8 Sep 2018

I think I solved it! It was a subtle case of mutability, where readObject was modifying the raw object in the internal offsetCache and replacing it with its parsed version.

wmhilton on 8 Sep 2018

:tada: This issue has been resolved in version 0.35.2 :tada:

The release is available on:

Your semantic-release bot :package::rocket: