Dlib: Using mmod_dog_histerizer.dat, analyzing one 12MB jpg on iPhone X tries to std::alloc 3.3GB

[Kenster999]

Expected Behavior

I am trying to use the mmod_dog_hipsterizer.cpp code (modified) in an iPhone X app (iOS 11.4.1), along with the mmod_dog_hipsterizer.dat file, to identify any dog faces in an image from my phone's photo library. I removed all of the code to draw glasses and a mustache, as my project won't be doing that (and it's nice to avoid all the X11 stuff).

Current Behavior

When I run my code, I get this failure notice in the console:

DogTest(21658,0x1b3e0fb40) malloc: *** mach_vm_map(size=3,313,926,144) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
std::bad_alloc

Why is it trying to allocate 3.3GB? (I inserted some commas in the error message size to make it easier to parse by eye.)

Steps to Reproduce

I am trying to use mmod_dog_hipsterizer.cpp and .dat that I got from github. It works fine when I compile it on my MacBook Pro (macOS High Sierra 10.13.6, 16GB RAM). I want to use it on my iPhone X (iOS 11.4.1, 3GB RAM, 256GB "disk" capacity). After fiddling around with various flags and other Xcode settings I have gotten dlib to build (libdlib.a), using these flags:

GCC_PREPROCESSOR_DEFINITIONS = 'CMAKE_INTDIR="$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME)"' DLIB_JPEG_STATIC DLIB_NO_GUI_SUPPORT

OTHER_CPLUSPLUSFLAGS = -DDLIB_JPEG_SUPPORT -DDLIB_USE_BLAS -DDLIB_USE_LAPACK -DLAPACK_FORCE_UNDERSCORE '-ftemplate-depth=500' '-std=gnu++11'

I then used the mmod_dog_hipsterizer.cpp as the basis for an Objective-C test app ("DogTest"), adding libdlib.a to the project. I managed to get everything compiling and seeming to work (except for this one problem). The compiler flags are:

GCC_PREPROCESSOR_DEFINITIONS = DEBUG=1 DLIB_NO_GUI_SUPPORT $(inherited)

OTHER_CPLUSPLUSFLAGS = $(OTHER_CFLAGS) -DDLIB_JPEG_SUPPORT -DDLIB_USE_BLAS -DDLIB_USE_LAPACK -DENABLE_ASSERTS

I also linked in Accelerate.framework to get BLAS and LAPACK support.

(BTW, all of this is using the Debug configuration. I can also compile using Release for both libdlib.a and my DogTest.app, but I get the same problem.)

My DogTest.app uses UIImagePickerController to let me pick a picture from my photo library. I pick the latest picture, which actually happens to be a dog. :) The picture is in portrait orientation, and is 3024 x 4032, with size = 2.9MB. I get the full path to the .jpeg file of the picture, and pass it to these lines of code within my modified dog_hipsterizer code:

net_type net;
shape_predictor sp;
deserialize(pDatPath) >> net >> sp;  // pDatPath is the path to mmod_dog_hipsterizer.dat
matrix<rgb_pixel> img;
load_image(img, pJpgPath);  // pJpgPath is the path to the jpeg selected by user
auto dets = net(img);

When I run this code, everything works through the call to load_image, but then the call to net(img) outputs this failure notice in the console:

DogTest(21658,0x1b3e0fb40) malloc: *** mach_vm_map(size=3,313,926,144) failed (error code=3)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
std::bad_alloc

Why is it trying to allocate 3.3GB?

I stepped through the code (although I haven't used c++ much, and this is all my first exposure to machine learning, tensor, etc). I found this line in cuda/cpu_dlib.cpp, in the function void img2col:

        output.set_size(out_nr*out_nc, 
                        data.k()*filter_nr*filter_nc);

When this is called, the values are effectively:

        output.set_size(5455 * 2025, 
                        3 * 5 * 5);

(in other words, out_nr == 5455, out_nc == 2025, data.k() == 3, filter_nr == 5, and filter_nc == 5)

* Note: is it wrong that my code is using code in the cuda folder? I don't have any CUDA files on my Mac, and of course there are no libraries on the iPhone. *

Then the call which actually fails is in memory_manager_stateless/memory_manager_stateless_kernel_1.h, in the function allocate_array (reformatted for prettiness):

T* allocate_array (size_t size)
{
    return new T[size];
}

In conclusion: what am I doing wrong? Is this all correct given the data that I'm working with? Is the large size (5455 * 2025) coming from the trained data set? Can/should I reduce that somehow (and would I lose much accuracy)? Is there another solution that will enable this to work on an iPhone?

I tried looking into the set_size() function but I couldn't find it in the documentation index. http://dlib.net/faq.html and http://dlib.net/matrix_ex.cpp.html weren't enough to help me figure out the problem, probably due to my lack of experience in this area.

Thanks for any help, insights, or comments! (And thanks for an amazing library and data sets!)

• Version
  I'm using dlib 19.15, which I downloaded from https://github.com/davisking/dlib

• Platform
  iOS 11.4.1

• Compiler
  Xcode version 9.4.1

[davisking]

Big images require a lot of ram to process. Make your image smaller before running the program on it.

[Kenster999]

OK, thank you!

For anyone interested: I had to resize the image down to 150x200. (Even 300x400 was too big.) And note that the retina device has a 3.0 scale, so the image was actually 450x600.

[ay8s]

@Kenster999 I've been checking out Dlib for a while, didn't have much luck with running the hipsterizer on iOS, I wondered if you had any example code you'd be up for sharing? Wanting to piece together a little project for my girlfriend.

[Kenster999]

Is your girlfriend a dog? 🤔
Just kidding, obviously 🤣
My code is incomplete & imperfect, but usable. I'll see if I can put up something this weekend. You're just targeting iOS, nothing unusual, right? What would be the image input: selecting from camera roll, or taking live picture with camera?

[ay8s]

Haha, could have worded that better for sure. 3AM here :P

Just iOS, right now would be just a UIImage from a camera snapshot taken within the app. Processed then shown to the user in a preview. 👍 No worries if its WIP, anything would help.

[Kenster999]

Sorry I couldn't get to it until now, but here's a starter project that should help:

https://github.com/Kenster999/dlib_dog_hipsterizer

Let me know if you have any questions, although hopefully the README.md is pretty clear. You'll definitely want/need to edit the code for your own purposes. "Enjoy!"