Turicreate: Style transfer should work with 1 style image

[enzyme69]

I tried once with no max iterations, default to 5000, it was too long with CPU, at around 75, I stopped it.

So then I tried max iteration option at 100, it seems to hang at around 17...
model = tc.style_transfer.create(style, content, max_iterations = 100)

Is this iteration something that can be stopped at any time and then saved? Because currently it seems to freeze at random process iteration and I started again from 0...

[srikris]

That is most unusual. Can you share your content and style images with us so we can give it a spin and see what's wrong?

[enzyme69]

Content image:

Style images:

They seem pretty hires and I put the all inside one folder. And over night training for 1000 iterations, for example, hang at around 160 on my Mac. I need to cancel it and start again.

[enzyme69]

I do getting this "warning" everytime I import turicreate module:

/Users/jimmygunawan/anaconda3/lib/python3.5/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters

[enzyme69]

And also:
Unsupported image format. Supported formats are JPEG and PNG file: style/.DS_Store In [3]:

[enzyme69]

One question: Does style and content image need to be at certain dimensions? Is square format necessary?

[enzyme69]

I have also have similar issue the processing is hanging when I do this:

• Content: my cat
• Style: a bunch of style of same artists

Stuck at iteration 1 or 2.

[enzyme69]

From my observation:
Default 5000 is taking too long on CPU. But I remember at 5000, it seems to go up to 70 after around 1 hour.

So I tried changing the iteration to 1000 and 2000. To my surprise, it is taking much longer to go from 0 to 70 iterations.

I tried 10 iterations, and it is even taking longer from 0 to 1....

[shreyasvj25]

@enzyme69 Thank you for providing the detailed information.
We will try to repro your issue and investigate further.

If I understand correctly you are using just 1 image (cat) as the content image during training?
Generally, training requires a good amount of content images.
Can you try training with a few content images (say 100-200) from your photo gallery or images from the web?
Please report if you still see the training hanging.

[gustavla]

The hi-res images could be causing memory issues. During training, the model is fed images resized down to 256x256, so there will be no benefit of providing content or style images larger than this. You can still stylize images of any size.

So, one thing you could try is to scale the images down. You can do that for instance doing:

images = tc.load_images('images/')
images['image'] = images['image'].apply(lambda img: tc.image_analysis.resize(img, 256, 256, 3))
images.save('resized.sframe')

Now, you can load that SFrame directly in your training script. This is actually what we do at the onset of training, but I'm worried that we are still keeping the hi-res images around in memory.

Also, the warnings that you mentioned are not important. The h5py seems unrelated to Turi Create, since we do not depend on that package. When loading images, it will warn every time it sees a file that was not PNG or JPEG, but this is safe to ignore. We shouldn't be warning about hidden files like .DS_Store at all though, so I filed an issue for this in #641 (thanks for pointing this out!).

[enzyme69]

@gustavla I tried resizing using that method but still getting it hanging at iteration 2, from total 10 iterations I assigned. Hmmm..

[enzyme69]

style = tc.load_images('style/')
style['image'] = style['image'].apply(lambda img: tc.image_analysis.resize(img, 256, 256, 3))
style.save('resized.sframe')

path | image
-- | --
style/image_1250.jpeg | Height: 256 Width: 256
style/image_3690.jpeg | Height: 256 Width: 256
style/images-1.jpeg | Height: 256 Width: 256
style/images.jpeg | Height: 256 Width: 256

Am I doing this the right thing:

• CONTENTS: I currently have only 1 image for test
• STYLE: I have multiple images with similar "Style"

I expected it to just work, but looks like the iteration process failed at around 10-15%. No matter I specify default 5000, or 100, or 10, it will hang.

Would be nice if we can stop the process at any stage and then just use that?

[shreyasvj25]

@enzyme69 I was able to repro your error.

It happens if you use number of content images less than default batch_size (which is 6 currently).

Reason is due to raising StopIteration in style_transfer/_sframe_loader.py:

        if self.cur_epoch == self.num_epochs or len(self.sframe) == 0:  
            raise StopIteration

We should use try and except block on StopIteration as per https://www.python.org/dev/peps/pep-0479/
I filed an issue See #647

For you to get unblocked:
_[Like I said before, please use more content images (around 100-200) during training]_

[enzyme69]

@shreyasvj25 This tips seems to do the job, I manage to finally get 10 iterations working to completion! Am I still unblocked?

Ok I will try 100 and 1000 iterations with CPU! Exciting module this one, thanks~

[talyn]

@enzyme69 May I ask what specific changes you made to get it to work? I've increased my number of content images to 125, but the training process still hangs. Did you make changes to style_transfer/_sframe_loader.py?

[shreyasvj25]

@talyn are you still hitting this issue?
Can you provide your settings (number of content, style images, and other specifics)? so that we can repro the issue and find the root cause

[talyn]

@shreyasvj25 I just got it working! The problem was, I only had one style image. It seems to work fine if I have 8 style images and 8 model images. Thanks!

Do you know if we'll ever be able to run this script with one style image? Just curious.

[srikris]

@talyn It should work on one style image. This may be a bug if it doesn't.

[znation]

The title says "1 style image", but I think the bug here happens when it's 1 content image (regardless of how many style images).