Hi,
This is not really an issue with Parse-Server, rather a cry for help with using Imagemagick instead of Parse-Image in my cloud code functions. Can anyone point me in the right direction to achieve the same as expressed in my old cloud code:
Parse.Cloud.beforeSave("TheClass", function(request, response) {
var theObject= request.object;
Parse.Cloud.httpRequest({
url: theObject.get("img").url()
}).then(function(response) {
var image = new Image();
return image.setData(response.buffer);
}).then(function(image) {
var currentWidth = image.width();
var currentHeight = image.height();
var ratio = currentHeight / currentWidth;
if (currentWidth > 640) {
return image.scale({
width: 640,
height: 640 * ratio
});
} else {
return image;
}
}).then(function(image) {
return image.setFormat("JPEG");
}).then(function(image) {
return image.data();
}).then(function(buffer) {
var base64 = buffer.toString("base64");
var cropped = new Parse.File("thumb.jpeg", { base64: base64 });
return cropped.save();
}).then(function(cropped) {
theObject.set("thumb", cropped);
}).then(function(result) {
response.success();
}, function(error) {
response.error(error);
});
});
Any help will be much appreciated!
@christianmarth Thanks! E.g. in the first link example (EasyImage), how would you initialize an image from the response.buffer like I do in line 8 of my pasted code?
@oyvindvol I wrote a wrapper for Parse.Image that has 100% compatibility with the original parse-image:
I was just getting to the point where I needed this too. Thanks @flovilmart !
@nitrag @oyvindvol the installation is automated on debian and OSX. Installation in Windows is not working yet. For heroku, you need to add a build pack (as explained in the README)
@flovilmart Nice! So, to use this on a heroku based install of parse-server, i just add imagemagick , graphicsmagick and your parse-image to my package.json, and add var Image = require("parse-image") to the top of my cloud code file?
@oyvindvol follow the instructions here: https://github.com/flovilmart/parse-image#heroku-installation
then:
var Image = require("parse-image");
var image = new Image();
image.setData(buffer).then(function(){
});
Given the code you provided, it seems that it should work right away.
@flovilmart Ok, so now I did only the "Heroku installation" part, and after deploy the heroku logs say "Cannot find module 'parse-image'". Then I added 'parse-image' as a dependency in package.json, and I still get "cannot find module 'parse-image'" in the logs.
Did you run npm install --save parse-image?
@flovilmart On Heroku server?
does it work locally with foreman?
I haven`t tested locally, I am just trying to set it up on my parse-server-example that I deployed directly to Heroku. I configured the heroku-buildpack-graphicsmagick via git bash, and push to heroku master. I can see that the buildpack is installed, but am I also supposed to put something in my package.json for it to work in my cloud code?
the build pack is installing the binaries required to parse-image to work correctly.
When you push to heroku, you you see parse-image being installed? Can you give me the heroku logs?
Ok, starting from fresh. I added the buildpack in git bash. I remove everything regarding parse-image from my package.json, then i do git push heroku master. This shows in the console:
$ git push heroku master
Counting objects: 4, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (4/4), 328 bytes | 0 bytes/s, done.
Total 4 (delta 2), reused 0 (delta 0)
remote: Compressing source files... done.
remote: Building source:
remote:
remote: -----> Fetching set buildpack https://github.com/mcollina/her
remote: -----> GraphicsMagick app detected
remote: -----> Installing zlib 1.2.8
remote: -----> Installing libpng 1.5.17
remote: -----> Installing nasm 2.10.09
remote: -----> Installing libjpeg-turbo 1.3.0
remote: -----> Installing graphicsmagick 1.3.23
remote: -----> Building runtime environment for graphicsmagick
remote: -----> Using set buildpack heroku/nodejs
remote: -----> Node.js app detected
remote:
remote: -----> Creating runtime environment
remote:
remote: NPM_CONFIG_LOGLEVEL=error
remote: NPM_CONFIG_PRODUCTION=true
remote: NODE_ENV=production
remote: NODE_MODULES_CACHE=true
remote:
remote: -----> Installing binaries
remote: engines.node (package.json): >=4.1
remote: engines.npm (package.json): unspecified (use default
remote:
remote: Resolving node version >=4.1 via semver.io...
remote: Downloading and installing node 5.6.0...
remote: Using default npm version: 3.6.0
remote:
remote: -----> Restoring cache
remote: Loading 2 from cacheDirectories (default):
remote: - node_modules
remote: - bower_components (not cached - skipping)
remote:
remote: -----> Building dependencies
remote: Pruning any extraneous modules
remote: Installing node modules (package.json)
remote:
remote: -----> Caching build
remote: Clearing previous node cache
remote: Saving 2 cacheDirectories (default):
remote: - node_modules
remote: - bower_components (nothing to cache)
remote:
remote: -----> Build succeeded!
remote: ├── [email protected]
remote: ├── [email protected]
remote: ├── [email protected]
remote: └── [email protected]
remote:
remote:
remote: -----> Discovering process types
remote: Procfile declares types -> (none)
remote: Default types for buildpack -> web
remote:
remote: -----> Compressing...
remote: Done: 33.8M
remote: -----> Launching...
remote: Released v140
remote: https://<appname>.herokuapp.com/ deployed to Heroku
remote:
remote: Verifying deploy... done.
To https://git.heroku.com/<appname>.git
241bb07..94bc7b3 master -> master
After deploy finished, this is the heroku logs:
2016-02-25T13:44:26.959982+00:00 heroku[api]: Release v140 created by
2016-02-25T13:44:27.098280+00:00 heroku[slug-compiler]: Slug compilation started
2016-02-25T13:44:27.098284+00:00 heroku[slug-compiler]: Slug compilation finished
2016-02-25T13:44:27.472830+00:00 heroku[web.1]: State changed from crashed to starting
2016-02-25T13:44:31.180828+00:00 heroku[web.1]: Starting process with command `npm start`
2016-02-25T13:44:34.223438+00:00 app[web.1]:
2016-02-25T13:44:34.223454+00:00 app[web.1]: > [email protected] start /app
2016-02-25T13:44:34.223455+00:00 app[web.1]: > node index.js
2016-02-25T13:44:34.223456+00:00 app[web.1]:
2016-02-25T13:44:35.351267+00:00 app[web.1]: npm ERR! Linux 3.13.0-77-generic
2016-02-25T13:44:35.351727+00:00 app[web.1]: npm ERR! argv "/app/.heroku/node/bin/node" "/app/.heroku/node/bin/npm" "start"
2016-02-25T13:44:35.352006+00:00 app[web.1]: npm ERR! node v5.6.0
2016-02-25T13:44:35.352641+00:00 app[web.1]: npm ERR! npm v3.6.0
2016-02-25T13:44:35.352863+00:00 app[web.1]: npm ERR! code ELIFECYCLE
2016-02-25T13:44:35.353074+00:00 app[web.1]: npm ERR! [email protected] start: `node index.js`
2016-02-25T13:44:35.353260+00:00 app[web.1]: npm ERR! Exit status 1
2016-02-25T13:44:35.353469+00:00 app[web.1]: npm ERR!
2016-02-25T13:44:35.353663+00:00 app[web.1]: npm ERR! Failed at the [email protected] start script 'node index.js'.
2016-02-25T13:44:35.353868+00:00 app[web.1]: npm ERR! Make sure you have the latest version of node.js and npm installed.
2016-02-25T13:44:35.354073+00:00 app[web.1]: npm ERR! If you do, this is most likely a problem with the parse-server-example package,
2016-02-25T13:44:35.354285+00:00 app[web.1]: npm ERR! not with npm itself.
2016-02-25T13:44:35.354466+00:00 app[web.1]: npm ERR! Tell the author that this fails on your system:
2016-02-25T13:44:35.354682+00:00 app[web.1]: npm ERR! node index.js
2016-02-25T13:44:35.354864+00:00 app[web.1]: npm ERR! You can get information on how to open an issue for this project with:
2016-02-25T13:44:35.355304+00:00 app[web.1]: npm ERR! Or if that isn't available, you can get their info via:
2016-02-25T13:44:35.361157+00:00 app[web.1]: npm ERR! Please include the following file with any support request:
2016-02-25T13:44:35.361327+00:00 app[web.1]: npm ERR! /app/npm-debug.log
2016-02-25T13:44:35.360891+00:00 app[web.1]:
2016-02-25T13:44:35.355700+00:00 app[web.1]: npm ERR! There is likely additional logging output above.
2016-02-25T13:44:35.355092+00:00 app[web.1]: npm ERR! npm bugs parse-server-example
2016-02-25T13:44:35.355515+00:00 app[web.1]: npm ERR! npm owner ls parse-server-example
2016-02-25T13:44:35.329120+00:00 app[web.1]: module.js:341
2016-02-25T13:44:35.329124+00:00 app[web.1]: throw err;
2016-02-25T13:44:35.329125+00:00 app[web.1]: ^
2016-02-25T13:44:35.329125+00:00 app[web.1]:
2016-02-25T13:44:35.329126+00:00 app[web.1]: Error: Cannot find module 'parse-image'
2016-02-25T13:44:35.329128+00:00 app[web.1]: at Function.Module._load (module.js:290:25)
2016-02-25T13:44:35.329127+00:00 app[web.1]: at Function.Module._resolveFilename (module.js:339:15)
2016-02-25T13:44:35.329129+00:00 app[web.1]: at Module.require (module.js:367:17)
2016-02-25T13:44:35.329129+00:00 app[web.1]: at require (internal/module.js:16:19)
2016-02-25T13:44:35.329131+00:00 app[web.1]: at Module._compile (module.js:413:34)
2016-02-25T13:44:35.329130+00:00 app[web.1]: at Object.<anonymous> (/app/cloud/main.js:1:75)
2016-02-25T13:44:35.329131+00:00 app[web.1]: at Object.Module._extensions..js (module.js:422:10)
2016-02-25T13:44:35.329132+00:00 app[web.1]: at Module.load (module.js:357:32)
2016-02-25T13:44:35.329132+00:00 app[web.1]: at Function.Module._load (module.js:314:12)
2016-02-25T13:44:35.329133+00:00 app[web.1]: at Module.require (module.js:367:17)
2016-02-25T13:44:35.340090+00:00 app[web.1]:
2016-02-25T13:44:36.035081+00:00 heroku[web.1]: State changed from crashed to starting
2016-02-25T13:44:36.033934+00:00 heroku[web.1]: State changed from starting to crashed
2016-02-25T13:44:36.030078+00:00 heroku[web.1]: Process exited with status 1
2016-02-25T13:45:58.111674+00:00 heroku[web.1]: Starting process with command `npm start`
2016-02-25T13:46:00.447983+00:00 app[web.1]: > node index.js
2016-02-25T13:46:00.447984+00:00 app[web.1]:
2016-02-25T13:46:00.447981+00:00 app[web.1]: > [email protected] start /app
2016-02-25T13:46:00.447964+00:00 app[web.1]:
2016-02-25T13:46:01.919098+00:00 app[web.1]: npm ERR! Linux 3.13.0-77-generic
2016-02-25T13:46:01.901180+00:00 app[web.1]: module.js:341
2016-02-25T13:46:01.901184+00:00 app[web.1]:
2016-02-25T13:46:01.901186+00:00 app[web.1]: at Function.Module._load (module.js:290:25)
2016-02-25T13:46:01.901190+00:00 app[web.1]: at Module.require (module.js:367:17)
2016-02-25T13:46:01.919662+00:00 app[web.1]: npm ERR! node v5.6.0
2016-02-25T13:46:01.920572+00:00 app[web.1]: npm ERR!
2016-02-25T13:46:01.901189+00:00 app[web.1]: at Module.load (module.js:357:32)
2016-02-25T13:46:01.920799+00:00 app[web.1]: npm ERR! Make sure you have the latest version of node.js and npm installed.
2016-02-25T13:46:01.921630+00:00 app[web.1]: npm ERR! Or if that isn't available, you can get their info via:
2016-02-25T13:46:01.920445+00:00 app[web.1]: npm ERR! Exit status 1
2016-02-25T13:46:01.901183+00:00 app[web.1]: ^
2016-02-25T13:46:01.901186+00:00 app[web.1]: at Module.require (module.js:367:17)
2016-02-25T13:46:01.921130+00:00 app[web.1]: npm ERR! Tell the author that this fails on your system:
2016-02-25T13:46:01.901188+00:00 app[web.1]: at Module._compile (module.js:413:34)
2016-02-25T13:46:01.901185+00:00 app[web.1]: at Function.Module._resolveFilename (module.js:339:15)
2016-02-25T13:46:01.920340+00:00 app[web.1]: npm ERR! [email protected] start: `node index.js`
2016-02-25T13:46:01.901187+00:00 app[web.1]: at require (internal/module.js:16:19)
2016-02-25T13:46:01.921412+00:00 app[web.1]: npm ERR! You can get information on how to open an issue for this project with:
2016-02-25T13:46:01.901185+00:00 app[web.1]: Error: Cannot find module 'parse-image'
2016-02-25T13:46:01.920061+00:00 app[web.1]: npm ERR! npm v3.6.0
2016-02-25T13:46:01.920218+00:00 app[web.1]: npm ERR! code ELIFECYCLE
2016-02-25T13:46:01.901182+00:00 app[web.1]: throw err;
2016-02-25T13:46:01.921287+00:00 app[web.1]: npm ERR! node index.js
2016-02-25T13:46:01.901190+00:00 app[web.1]: at Function.Module._load (module.js:314:12)
2016-02-25T13:46:01.901188+00:00 app[web.1]: at Object.<anonymous> (/app/cloud/main.js:1:75)
2016-02-25T13:46:01.921847+00:00 app[web.1]: npm ERR! There is likely additional logging output above.
2016-02-25T13:46:01.901189+00:00 app[web.1]: at Object.Module._extensions..js (module.js:422:10)
2016-02-25T13:46:01.919493+00:00 app[web.1]: npm ERR! argv "/app/.heroku/node/bin/node" "/app/.heroku/node/bin/npm" "start"
2016-02-25T13:46:01.911119+00:00 app[web.1]:
2016-02-25T13:46:01.921026+00:00 app[web.1]: npm ERR! not with npm itself.
2016-02-25T13:46:01.921739+00:00 app[web.1]: npm ERR! npm owner ls parse-server-example
2016-02-25T13:46:01.920685+00:00 app[web.1]: npm ERR! Failed at the [email protected] start script 'node index.js'.
2016-02-25T13:46:01.921524+00:00 app[web.1]: npm ERR! npm bugs parse-server-example
2016-02-25T13:46:01.925610+00:00 app[web.1]: npm ERR! /app/npm-debug.log
2016-02-25T13:46:01.920921+00:00 app[web.1]: npm ERR! If you do, this is most likely a problem with the parse-server-example package,
2016-02-25T13:46:01.925509+00:00 app[web.1]: npm ERR! Please include the following file with any support request:
2016-02-25T13:46:01.925339+00:00 app[web.1]:
2016-02-25T13:46:02.554694+00:00 heroku[web.1]: Process exited with status 1
2016-02-25T13:46:02.564724+00:00 heroku[web.1]: State changed from starting to crashed
it seems that your package.json is not properly updated, or that heroku don't like it..
Build succeeded!
remote: ├── [email protected]
remote: ├── [email protected]
remote: ├── [email protected]
remote: └── [email protected]
Try to run on your machine: npm install --save parse-image and then deploy to heroku
After adding parse-image to my package.json the module is found.
But now I get this error:
[Error: Command failed: identify.im6: no decode delegate for this image format `/tmp/magick-x4qqyfba'
Ok, thanks a lot so far @flovilmart!
I tried console.logging the url to my uploaded ParseFile, console.log(theObject.get("img").url());. The url was http, not https. Then i ran this command in the heroku bash identify -verbose <URL-TO-FILE>, and there the identify seemed to work fine at least.
identify -verbose http://<app>/parse/files/sv78uRNzD4RPUKW9
Image: 5ef1d74591c83c189609957ee1f18860_portal.PNG
Base filename: 5ef1d74591c83c189609957ee1f18860_portal.PNG
Format: PNG (Portable Network Graphics)
Class: DirectClass
Geometry: 2438x1400+0+0
Resolution: 75.59x75.59
...
...
and so on
Is your response.buffer set?
Yes
Uhm... that's very very odd indeed. Did you try on you local machine, see if it's heroku related or something else?
You should probably open an issue on parse-image instead of there. Closing for now.
Well, initially this issue was about using the recommended Imagemagick in parse server cloud code. I am still interested in examples on that though..
OK, can you post an issue on parse-image I'll re-open in that case.
I used jimp to replace parse-image when I migrated our app. It is written entirely in JavaScript for Node so no external dependencies needed (no special install on heroku or windows). It probably has worse performance than imagemagick, but I have yet to run any benchmarks. We are scaling a lot of images in our app however and so far it looks promising.
@simonbengtsson That looks lightweight and good. Are you using jimp to e.g. save thumbnails via cloud code? I am having trouble getting that work. With the code provided below, everything seems OK, and the thumb is saved on the "Message" object, but viewing the thumb in the browser with the file URL is not working.
var Jimp = require("jimp");
Parse.Cloud.beforeSave("Message", function(request, response) {
var obj = request.object;
if (!obj .get("img")) {
response.success();
return;
}
Jimp.read(obj.get("img").url())
.then(function(image) {
var currentWidth = image.bitmap.width;
var currentHeight = image.bitmap.height;
var ratio = currentHeight / currentWidth;
if (currentWidth > 640) {
return image.resize(640, 640 * ratio)
} else {
return image;
}
})
.then(function(image) {
console.log("### 1");
var base64 = image.hash();
var cropped = new Parse.File("thumb.jpeg", { base64: base64 });
return cropped.save();
})
.then(function(cropped) {
console.log("### 2");
obj.set("thumb", cropped);
})
.then(function(result) {
console.log("### 3");
response.success();
}, function(error) {
console.log("### 4 ERROR: " + error);
response.error(error);
})
.catch(function (err) {
console.log("### 5 ERROR: " + err);
});
});
Yes we are generating thumbnails in cloud code using jimp. Your issue is probably that the string from image.hash() cannot be used as data for parse files. Try something like this instead:
return new Parse.Promise((resolve, reject) => {
image.resize(100, 100).getBuffer(Jimp.MIME_JPEG, (err, buffer) => {
if (err) return reject(err);
var file = new Parse.File('picture.jpg', {base64: buffer});
resolve(file.save());
});
});
@simonbengtsson Thanks! :) That worked..
Since it took me several hours to get this correct I'm posting here so the next guy has an easier time:
Parse.Cloud.beforeSave("Photos", function(request, response) {
Parse.Cloud.useMasterKey();
createThumbnail(request.object.get("Image").url(), 360, 360).then(function(shrunk) {
request.object.set("ImageThumbnail", shrunk);
console.log("Thumbnail generated/added: " + JSON.stringify(shrunk));
}).then(function(result) {
response.success("thumbnail saved");
}, function(error) {
response.error("thumbnail creation error: " + error.code + "("+ error.message + ")");
});
});
function createThumbnail(url, maxWidth, maxHeight){
var Jimp = require("jimp");
Parse.Cloud.useMasterKey();
if(url === ""){
console.log("url empty");
return;
}
console.log("Generating Thumbnail...");
return Jimp.read(url).then(function(image) {
console.log("Buffer read, resizing...");
var ratioX = maxWidth / image.bitmap.width;
var ratioY = maxHeight / image.bitmap.height;
var ratio = Math.min(ratioX, ratioY);
var w = image.bitmap.width * ratio;
var h = image.bitmap.height * ratio;
return image.resize(w, h );
}).then(function(image) {
console.log("Resized, setting quality...");
return image.quality(90);
}).then(function(image) {
console.log("Quality set, return buffer");
var promise = new Parse.Promise();
image.getBuffer(Jimp.MIME_JPEG, function(err, buffer){
if (err) return promise.reject(err);
promise.resolve(buffer);
});
return promise;
}).then(function(buffer) {
console.log("Buffer returned, creating Parse File");
var shrunk = new Parse.File("thumbnail.jpg", { base64: buffer }, "image/jpeg");
return shrunk.save();
}).then(function(shrunk) {
return shrunk;
}, function(error) {
console.log("thumbnail generation error: " + error.code + "("+ error.message + ")");
return;
});
}
Thanks for sharing this pure js image module, here is the benchmarks I found for jimp in comparison with other libraries.
https://github.com/lovell/sharp/issues/275#issuecomment-143526723
It seems enough for common image processing cases.
@flovilmart I tried your parse-image module on Heroku for a simple thumbnail generation. I did not need to change a single line of code, only added "parse-image": "~0.1.1" in my package.json dependencies, and it works fine so far.
Great job!
@nitrag Thanks for your post. Fwiw, I had to add a buffer.toString conversion before I could get everything to work.
var base64 = buffer.toString("base64");
var shrunk = new Parse.File("thumbnail.jpg", { base64: base64 }, "image/jpeg");
return shrunk.save({useMasterKey: true});
This all seems to be working great for me using jimp until the part where I try to set it to the object in the beforeSave hook. I see the images showing up in S3 and it comes back correctly. I have other attributes that are getting set correctly as well. It is just the 'pictureThumbnail' that isn't. For some reason it just seems as if it's not updating the request.object. @rickkrauland Was there something you had to do different then what is documented in @nitrag 's example?
createThumbnail(url, 300, 300).then(function(shrunk) {
restaurant.set("pictureThumbnail", shrunk);
console.log("Thumbnail generated/added: " + JSON.stringify(shrunk));
}).then(function(result) {
response.success("thumbnail saved");
}, function(error) {
response.error("thumbnail creation error: " + error.code + "("+ error.message + ")");
});
So... turns out that response.success doesn't want a string sent along with it. I removed "thumbnail saved" from response.success() and it worked.
Most helpful comment
Since it took me several hours to get this correct I'm posting here so the next guy has an easier time: