Turicreate: What to put in the annotations csv file

[bobdillon48]

I watched the Turi Create WWDC 2018 tutorial video where Zach runs the following line of code:
annotations = tc.SFrame('annotations.csv')

What exactly is in this annotations.csv file/ how is it formatted?

[znation]

Hi @bobdillon48, please see the User guide chapter on Object Detection, which gives examples of the JSON format used for describing the bounding boxes. The bounding box description for a single row (where each image's metadata is a row of your CSV), would look something like this:

[{"coordinates": {"height": 104, "width": 110, "x": 115, "y": 216}, "label": "ball"}]

Note that there could be more than one object annotation per image, which is why this is a JSON array. I think a single object is also valid.

To construct a CSV file that would describe each image this way, you'd want two columns, one for the path to the image (relative paths from within a directory where you will execute the script tends to work best), and one for the annotations. The resulting CSV file would look something like:

path,annotations
images/image1.jpg,[{"coordinates": {"height": 104, "width": 110, "x": 115, "y": 216}, "label": "ball"}]
images/image2.jpg,[{"coordinates": {"height": 208, "width": 40, "x": 48, "y": 29}, "label": "tree"}]

Then, you can read the resulting file into SFrame format with turicreate.SFrame.read_csv and join it to your images with join (assuming the path column name is the same, and contains the same image paths, in both SFrames).

I'm also going to leave this issue open and tag as docs, so we can use it to track updating the documentation to make it clearer. Now that I'm reading that user guide section, it's not clear how to get these JSON annotations into a CSV file (i.e. what's the right CSV syntax), how to load that into SFrame format, and how to combine it with the images SFrame.

[yousifKashef]

Hi,

follow up question: is there a way to use a JSON file instead of a csv?

my JSON looks like this:

{
  "1.jpg" : [
    {
      "coordinates" : {
        "height" : 94,
        "width" : 106,
        "x" : 155,
        "y" : 99
      },
      "label" : "2"
    }
  ],
  "2.jpg" : [
    {
      "coordinates" : {
        "height" : 228,
        "width" : 298,
        "x" : 531,
        "y" : 140
      },
      "label" : "1"
    }
  ],
  "3.jpg" : [
    {
      "coordinates" : {
        "height" : 291,
        "width" : 274,
        "x" : 136,
        "y" : 202
      },
      "label" : "1"
    }
  ]

if not, what's the easiest way to convert it to the right kind of csv?

[znation]

Hi @yousifKashef,

To read this kind of JSON format into SFrame, it would be easiest to use the built-in json package, since that type of structure (a single object with key/value pairs) doesn't map directly to the SFrame table structure and thus isn't supported by turicreate.SFrame.read_json. Instead, we can provide that mapping through code:

import json
import turicreate as tc

# read the JSON file into an in-memory object
with open('file.json', 'rb') as f:
    obj = json.load(f)

# get that JSON object into SFrame format (gives a single column of [key, value])
# note that `items` is what allows us to map this key/value structure into a table structure
sf = tc.SFrame(obj.items())

# transform the SFrame into two columns, and rename them appropriately
sf = sf.unpack('X1')
sf = sf.rename({'X1.0': 'path', 'X1.1': 'annotations'})

[srikris]

This seems more like a question than a doc change. Closing.