Sharp: Custom EXIF/XMP metadata

[sachin-walia]

Is it possible to add custom metadata such as "Exif.Photo.CopyrightInfo" during one of the image transformation operation - convert/compress.

thanks.

[lovell]

Hello, the only EXIF field currently supported is Orientation, which can be set via withMetadata.

To support more fields, I think the best approach would be to have the withMetadata() function also accept an optional Buffer via an exif attribute. This would allow people to use another module such as piexif to generate/manipulate the EXIF blob.

[zekth]

So i've worked on this feature but now i'm stuck due to my lack of knowledge in c++ especially v8.

So i take the buffer from fs in javascript (just the exif part).

In pipeline.cc i get the buffer by:

#include <node.h>
#include <node_buffer.h>
  if (HasAttr(options, "withMetadataExifBuffer")) {
    bool withBuffer = AttrTo<bool>(options, "withMetadataExifBuffer");
    if (withBuffer) {
      printf("Buffer is set ");
      v8::Local<v8::Object> ExifBuffer = AttrAs<v8::Object>(options, "withMetadataExifBuffer");
      baton->withMetadataExifBuffer = node::Buffer::Data(ExifBuffer);
      // FYI baton->withMetadataExifBuffer is char*
    }
  }

Using this function in common.cc

  void SetExifBuffer(VImage image, char *exifData) {
    image.set(VIPS_META_EXIF_NAME, exifData);
  }

And i got the error:

Uncaught Error: VipsImage: field "exif-data" is of type VipsRefString, not VipsBlob
VipsImage: field "exif-data" is of type VipsRefString, not VipsBlob

In final in think i have to overloard VOption::set with a VipsRefString parameter but i don't know how to manage the code in it.

I've tried to direclty use vips_set_string with the VIPS_META_EXIF_NAME and exifData as a string using std::string s(exifData) but Vips keep saying me its a VipsBlob :/

Any clue?

[lovell]

@zekth Thank you for working on this, try something like:

VipsBlob *exif = vips_blob_new(nullptr, exifData, exifDataLength);

[zekth]

@lovell it's not exactly what i was meaning.

I'm doing this to set the exif data

  void SetExifBuffer(VImage image, const char *exifData) {
    vips_image_set_string(image.get_image(), VIPS_META_EXIF_NAME, exifData);
  }

Signature of the method is this

void vips_image_set_string( VipsImage *image, 
    const char *name, const char *str )

But i still have the issue:

Uncaught Error: VipsImage: field "exif-data" is of type VipsRefString, not VipsBlob
VipsImage: field "exif-data" is of type VipsRefString, not VipsBlob

So am i missing something? Maybe @jcupitt have an idea?

NOTE: i've pushed everything on my fork if you want to take a look. Obviously my UT does not work but is written.

[jcupitt]

Hello @zekth,

The argument to _set_string() is a UTF-8 string, so you can't use it for binary data (like EXIF data blocks).

Try vips_image_set_blob():

http://jcupitt.github.io/libvips/API/current/libvips-header.html#vips-image-set-blob

[jcupitt]

vips 8.7 adds (finally!) support for setting string-valued exif data items, so you can also just write:

vips_image_set_string (image, "exif-ifd0-Copyright", "my great copyright notice");

Though I think that would need support in sharp.

[zekth]

@jcupitt i have tried with vips_image_set_blob and seems to fail again:

  void SetExifBuffer(VImage image, char *exifData, size_t exifDataBufferSize) {
    vips_image_remove(image.get_image(), VIPS_META_EXIF_NAME);
    void* dataToCopy = static_cast<void*>(exifData);
    vips_image_set_blob(image.get_image(), VIPS_META_EXIF_NAME,
     (VipsCallbackFn) vips_free, dataToCopy, exifDataBufferSize);
  }

I have a crash after the unit tests. maybe i'm using the wrong callback? Even i comment the line for vips_image_remove.

FYI: datas coming from JS are casted with:"

baton->withMetadataExifBufferSize = node::Buffer::Length(ExifBuffer);
baton->withMetadataExifBuffer = node::Buffer::Data(ExifBuffer);

[jcupitt]

Hello @zekth, yes, vips_free() is probably not correct there.

You give vips_image_set_blob() a pointer to a memory area, plus a callback to free the memory when libvips is finished with it.

You could make a callback that used the node memory manager to free the memory, but that sounds very difficult and dangerous. You would need to make sure that the relevant node memory pool was still active, for example.

It would be simpler to make a copy of the memory area for libvips, and then just use vips_free, as you have here. Something like (untested):

void *data_copy = vips_malloc (NULL, size);
memcpy (data_copy, data, size);
vips_image_set_blob (image, VIPS_META_EXIF_NAME, (VipsCallbackFn) vips_free, data_copy, size);

Now you can be sure that the memory will be freed by the same thing that allocated it.

[lovell]

sharp is using libvips' C++ API so I'd use an approach a bit like the following (untested):

VipsBlob *exif = vips_blob_new(nullptr, exifData, exifDataLength);
image.set(VIPS_META_EXIF_NAME, exif);

The buffer will already be freed by Node when it is eligible for GC so we don't have to worry about memory management. (You will need to add its reference to the buffersToPersist vector to prevent the possibility of V8 moving it during GC memory compaction.)

[zekth]

Mmmm seems i found something guys.

When i do :

vips_image_remove(image.get_image(), VIPS_META_EXIF_NAME)

I still got the exif into the output buffer and the method returns TRUE.
exif: <Buffer 45 78 69 66 00 00 49 49 2a 00 08 00 00 00 0a 00 0f 01 02 00 07 00 00 00 96 00 00 00 10 01 02 00 07 00 00 00 9e 00 00 00 12 01 03 00 01 00 00 00 01 00 ... >,
and it's the full exif datas from the input file:

{ image:
   { Make: 'Canon',
     Model: 'Canon EOS 700D',
     Orientation: 1,
     XResolution: 240,
     YResolution: 240,
     ResolutionUnit: 2,
     Software: 'Adobe Photoshop Lightroom 6.0 (Windows)',
     ModifyDate: 2017-11-14T21:27:27.000Z,
     ExifOffset: 220 },
  thumbnail:
   { Compression: 6,
     XResolution: 72,
     YResolution: 72,
     ResolutionUnit: 2,
     ThumbnailOffset: 756,
     ThumbnailLength: 14786 },
  exif:
   { ExposureTime: 0.16666666666666666,
     FNumber: 3.5,
     ExposureProgram: 1,
     ISO: 100,
     ExifVersion: <Buffer 30 32 33 30>,
     DateTimeOriginal: 2017-11-14T22:25:54.000Z,
     DateTimeDigitized: 2017-11-14T22:25:54.000Z,
     ShutterSpeedValue: 2.584963,
     ApertureValue: 3.61471,
     ExposureBiasValue: 0,
     MaxApertureValue: 1.75,
     MeteringMode: 2,
     Flash: 16,
     FocalLength: 50,
     SubSecTimeOriginal: '37',
     SubSecTimeDigitized: '37',
     FlashpixVersion: <Buffer 30 31 30 30>,
     ColorSpace: 1,
     PixelXDimension: 1778,
     PixelYDimension: 1000,
     FocalPlaneXResolution: 5798.657718120805,
     FocalPlaneYResolution: 5788.94472361809,
     FocalPlaneResolutionUnit: 2,
     CustomRendered: 0,
     ExposureMode: 1,
     WhiteBalance: 0,
     SceneCaptureType: 0 } }

And if i do :
vips_image_remove(image.get_image(), VIPS_META_ICC_NAME)

The output metadata has no property for icc.

Is there anything wrong into the libvips side?

BTW i've tried your code @jcupitt it compiles but does not work. Maybe the case i've explained is the key of all my issues? Could you please check on your side?

  void SetExifBuffer(VImage image, char *exifData, size_t exifDataBufferSize) {
    if (vips_image_remove(image.get_image(), VIPS_META_ICC_NAME)) {
      printf("Exif section Removed \r\n");
    } else {
      printf("Exif section Not Removed \r\n");
    }
    void* data = static_cast<void*>(exifData);
    void *data_copy = vips_malloc(NULL, exifDataBufferSize);
    memcpy(data_copy, data, exifDataBufferSize);
    vips_image_set_blob(image.get_image(), VIPS_META_EXIF_NAME,
     (VipsCallbackFn) vips_free, data_copy, exifDataBufferSize);
}

[jcupitt]

It's working, it's just rebuilt the exif from the other tags. Sorry @zekth I think I should have read this issue more carefully from the start, I think you are conflicting with libvips exif handling.

libvips supports exif editing: you can manipulate tags by setting/removing any of the exif-ifdN-xxx fields. Here's how it works:

On load, libvips extracts the exif data block, parses it, and makes a set of metadata items for each field. It knows about all the numeric types, and the three types of string valued tags. It also attaches the original exif data block (this is the thing that you've changed) as exif-data.

On save, libvips parses the exif-data block (or makes a minimal compliant set of exif data, if there is none), then does a diff against the exif-ifdN- tags.

1. libvips metadata field, but no matching tag in exif means add tag to exif
2. libvips metadata field, matching tag in exif means update exif tag from metadata value
3. no libvips metadata field, tag exists in exif means remove tag from exif

Finally, it reserialises and writes. Things like embedded thumbnails, XMP, ICC profiles etc. all work the same way.

Summary: you should be able to do most EXIF manipulations with a simple get/set/remove on the image. If there's something you want to do that is not supported by that, I'd love to hear and I'll add it as a feature if possible.

If you really want to handle exif yourself, I think you will need to do this:

1. make a unique copy of the image (you must not change an image that someone else might be using)
2. attach a new exif data block (I don't think you need to remove the old one first)
3. remove any metadata items that start exif-ifd (optional)
4. call the internal function int vips__exif_parse (VipsImage *image); --- this will walk the exif block on the image and set (or update) the image metadata from that, where possible

Point 3 above is optional, since you might want to keep fields the user has set previously.

Obviously calling internal API is not great, but we could move it to the public API very easily.

[zekth]

Thanks for this. I think i now have a better understanding of the workflow inside vips. I'll work on that when i get back home in few weeks. I'll do a Proof Of Concept using the internal API and if it properly works i think it would be a good step for considering moving this part to the public API.

As you mentionned in https://github.com/lovell/sharp/issues/650#issuecomment-410176951 there is a new feature for setting string exif. @lovell what do you think of this? it would be a good addition to sharp too?

[lovell]

I'd prefer to get a Buffer-based version working first as that should be simpler and will allow people to use the existing piexifjs module (in their own code, not in sharp directly).

[zekth]

No problem i was thinking about another feature after the one with the buffer is done.

[jadrake75]

This looks very promising for use in my app. I am liking my choice of going with Sharp more and more. Setting some of these properties in my output files would really help!

[claytonrothschild]

Has anyone created a successful implementation of this?