P5.js: add GIF support for image() function

[lmccart]

Currently, animated .gif files loaded via loadImage() and displayed via image() will display as static images. A current workaround is to use the createImg() function, but this adds complexity and it would be nice for gifs to work with the core image() function.

[verma-varsha]

Hi! I would like to work on this issue

[vedhant]

@lmccart can I work on this issue if @verma-varsha is not working on it currently?

[shiffman]

As a reference @wenheLI and @NHibiki created this library as part of an open source class at ITP!

https://github.com/wenheLI/p5.gif/

[Ajayneethikannan]

Hello @lmccart @shiffman @meiamsome !

I would love to solve this issue !

Since we are trying to implement gifs, can we consider this as an ad hoc solution ,
we can use gifs in the canvas by converting the gif to a video format, and draw the video to the canvas ,
using the already robust video support in the library.
This has its own advantages, as converting a gif to a video drastically reduces its size,and even recommended in some places .
Gifs in video format are also available directly online (eg: Giphy).

As a permanent solution, parsing the gif into separate image frames and drawing them to the canvas in a loop seems to be the only way. If this is agreed upon, I will start working immediately !

@shiffman ( hello ! very excited to meet you !)
I went through the library mentioned by you and I think parsing the gif to separate images is the way to go. Since this is strikingly similar to the concept of sprites ,
I believe , implementing a sprite function , which can be overloaded with gifs or a spritesheet ,
would be a good idea and the first step .

Looking forward to the suggestions ,
Thank you !

[meiamsome]

I think this issue needs to be more scoped out than it currently is...
If I have a gif with 10 frames that have a number from 0 to 9 written on them in order, with 1 second of delay on them then:

• Should the first frame rendered from the gif always be the 0 frame?

• If I have a sketch running at 30fps, do I expect to see the number change:
  a. Every Frame
  b. Every Second

• Assuming B, If I have a sketch running at 30fps, where the first 30 frames are rendered with the gif, then 30 without, then 30 on do I expect
  a. To see all numbers sequentially
  b. To only see the even numbers (0, 2, 4, 6, 8, 0, ...etc)

• If I render two of the same gif at one time in one sketch do they:
  a. Each have their own timings so they can be out of sync (this would imply some sort of gif proxy class you can instantiate multiple times from one gif)
  b. Have to be in sync with eachother

In addition, in cases where the gif is set not to repeat at the end:

• With a sketch that renders it every frame, do I see:
  a. It goes through all the frames and stops on the last one
  b. It goes through all the frames and then displays nothing after the last one
  c. It goes through all the frames and then restarts automatically despite not being a repeating GIF

• Is there a good way to tell when the GIF has ended? (A callback perhaps?)

That's a bunch of questions that would help guide implementing this, because depending upon the feature set the implementations would be far different

[Ajayneethikannan]

@meiamsome Thank you for suggesting the different cases that need to be handled !

On the basis of your questions, I believe creating a sprite class would be helpful, where handling gifs would be one of its functionality . I would love to hear people's thoughts on this

• The first frame to be rendered must be able to be given as a parameter ( perhaps in the constructor function )
• I think the change in the image frames should be dependent on the frameRate, and this can be implemented as the gif file contains details about the time delay between the images
• One gif should be able to create several instances on the canvas , as parsing the gif file is an expensive activity, maybe the frames created as a result can be passed between different instances ?
  And I believe every instance must be able to have their own independent timings
• I am not sure about the case of non repeating gifs, but maybe the callback function(as you suggested ) can handle it.

Waiting to hear more about this ,
Thank you!

[limzykenneth]

My thoughts: I think the behaviour should be as close to DOM behaviour as possible and where undefined be what the user would expect out of the gif they loaded. To elaborate:

• Should the first frame rendered from the gif always be the 0 frame?

Yes, I least that's what I would expect intuitively.

• If I have a sketch running at 30fps, do I expect to see the number change:
  a. Every Frame
  b. Every Second

I would go for "b", as mentioned at the beginnning, that's what I expect the gif to do (when opening it up directly in a browser or viewer, that's what it does)

• Assuming B, If I have a sketch running at 30fps, where the first 30 frames are rendered with the gif, then 30 without, then 30 on do I expect
  a. To see all numbers sequentially
  b. To only see the even numbers (0, 2, 4, 6, 8, 0, ...etc)

Again "b" here, to keep the framerate of the gif tied to real time and not p5.js framerate.

• If I render two of the same gif at one time in one sketch do they:
  a. Each have their own timings so they can be out of sync (this would imply some sort of gif proxy class you can instantiate multiple times from one gif)
  b. Have to be in sync with eachother

I would go for "a" for this as that provides more flexibility.

• With a sketch that renders it every frame, do I see:
  a. It goes through all the frames and stops on the last one
  b. It goes through all the frames and then displays nothing after the last one
  c. It goes through all the frames and then restarts automatically despite not being a repeating GIF

It definitely should stop on the last frame, if the user wants it to loop in p5 there can be a loop() function to make it looping but if the gif itself is defined to not loop then it shouldn't loop.

• Is there a good way to tell when the GIF has ended? (A callback perhaps?)

Could this be achieved by a user defined check that the current playhead is at the length of the number of frames in the gif? Much like how you would check if a video has finished playing or not.

One thing to note here is that the user will most likely already seen the gif playing before they decide to load it into p5 (either when they download it from a website or when they created it in something like Photoshop) as such they would have expectation about what the gif would do and I think what we should try to do is to match those expectation as much as possible by default, while providing API that can control them.

[Ajayneethikannan]

@limzykenneth thank you very much for your suggestions !

The current implementation I have in mind, after referring online and looking into the library referenced by @shiffman , is to parse the gif into its constituent images, and store them together, to be displayed in the canvas , mostly by using the image function.

If I can continue on this discussion,

Again "b" here, to keep the framerate of the gif tied to real time and not p5.js framerate.

I understand the importance of this , but I also have a doubt, since the rate at which the canvas is updated is controlled by the frameRate(), we cannot have the gif run separately at its own frameRate ( to my knowledge ) , as its also drawn on the canvas.

This means when the frameRate is very low (very large elapsed time ), it can result in the skipping of the frames of the gif. Also , a situation , though quite rare, is that the time delay between different frames of the gif can be different, adding to the complexity.

• My doubt is should we allow for the skipping of the frames , or is it to be avoided ?
• Or is there a better way to implement this altogether ?

A crude formula for calculating which frame to render can be,
suppose index points at some frame,

(if elapsed time from last canvas update > time delay) index ++;
currentFrame = frames[index];

This will prevent skipping of frames, but will appear slow in low frame rates.

• Should this be the intended behavior, or skipping the frames , while maintaining a constant speed is the better approach ?

I would go for "a" for this as that provides more flexibility.

I am in support of this.

It definitely should stop on the last frame, if the user wants it to loop in p5 there can be a loop() function to make it looping but if the gif itself is defined to not loop then it shouldn't loop.

I am in support of this.

Could this be achieved by a user defined check that the current playhead is at the length of the number of frames in the gif? Much like how you would check if a video has finished playing or not.

Can we do this by adding an extra parameter to the class which will most likely be created ?

One thing to note here is that the user will most likely already seen the gif playing before they decide to load it into p5 (either when they download it from a website or when they created it in something like Photoshop) as such they would have expectation about what the gif would do and I think what we should try to do is to match those expectation as much as possible by default, while providing API that can control them.

I am in support of this. But also if you believe that the user already has control of which gif they are going to use, imho, converting it into a video will give the user the support of the already existing video functionality, as well be more efficient as a video's size is drastically less than the gif. Could this be implemented as the temporary solution ? The only adverse effect of using this strategy is that the user cannot use gifs from the internet easily, but considering that giphy, the biggest source of gifs , serves every gif in video format also, maybe this can be implemented ?

Currently I am considering the creation of a sprite class feature as the initial step , which is very similar to what we are doing here. But I believe if that is to be implemented, it will be frameRate dependent,
and gif will be not. What should be the basic implementation strategy ?

I would start working on it if I have the approval of the basic implementation by the maintainers .

Would love to hear your thoughts on it ,
Thank you !

[limzykenneth]

I understand the importance of this , but I also have a doubt, since the rate at which the canvas is updated is controlled by the frameRate(), we cannot have the gif run separately at its own frameRate ( to my knowledge ) , as its also drawn on the canvas.
This means when the frameRate is very low (very large elapsed time ), it can result in the skipping of the frames of the gif. Also , a situation , though quite rare, is that the time delay between different frames of the gif can be different, adding to the complexity.

For me in this case I think of the loaded gif as you mentioned in the beginning to be a series of images (a data structure) and p5.js is responsible for displaying them. To display the gif in realtime would just be controlling the current frame displayed based on the delta time of the last p5 frame which we kinda already have the mechanism to do it already with millis().

Skipping gif frames here to me seemed reasonable since if your p5.js sketch is operating at 1fps you wouldn't be expecting something that is in the sketch itself to be updating faster than 1fps. And with a function under the gif instance to change the playback fps the gif playback speed can be matched to the sketch's fps as well if desired.

Can we do this by adding an extra parameter to the class which will most likely be created ?

The API p5 users would be used to would probably not be anything that uses any kind of constructor with new keyword, so it is best to still provide the explicit API to control it even if it is more verbose.

I am in support of this. But also if you believe that the user already has control of which gif they are going to use, imho, converting it into a video will give the user the support of the already existing video functionality, as well be more efficient as a video's size is drastically less than the gif. Could this be implemented as the temporary solution ? The only adverse effect of using this strategy is that the user cannot use gifs from the internet easily, but considering that giphy, the biggest source of gifs , serves every gif in video format also, maybe this can be implemented?

I think giphy still do serve pure gifs and in addition to video format, same goes for imgur. I'm not sure how well transparency is supported by videos but if they do support it it would still be rather different from gifs (gif transparency are on or off only, no inbetween). Also it is important to not have the user be tied to a particular source of data, as mentioned in my last comment, people do make their own gifs sometimes.

[Ajayneethikannan]

Got it :+1: Thank you @limzykenneth !

[Ajayneethikannan]

Upon doing some research on the problem,
this is the implementation I have in my mind:

• A gif parser (inspired by https://stackoverflow.com/questions/48234696/how-to-put-a-gif-with-canvas)
  The parser will convert each frame of the gif into p5.Image instances

Reason: Currently , I am planning to write the parser based on the referenced code, modified to suit p5,
since installing any package may increase the dependency of the library.

• A new p5.Gif class, such that it can store the frames as p5.Image instances,
  and hold additional properties, such as variable to check whether it is over, looping or non looping, functions to change properties, etc.

Reason: The reason for creating a new class p5.Gif is because it becomes easier to manipulate the existing features, and introduce new features such as changing the play mode (real time vs matching the canvas frameRate), setting offsets, etc.
(Basic implementation will be real time updation, with skipping of frame for very low frameRate)

for eg:

for the basic functionality

function preload()
{
    let a = loadGif("link_to_gif");  //creates an instance of p5.Gif class
}
function draw()
{
    gif(a, 0, 0, 100, 100);  //new function to draw the Gif instance on canvas, similar to image()
}

This method can be included in the image function , but imho I think separating the functionalities makes the usage more clear

Known shortcomings:

• Parsing a gif image is an expensive process, and can take long durations, if the file size/ number of frames is huge

I request you to check the implementation idea, if you agree to it , I will start working on the basic features immediately.
If not, please provide your suggestions
Thank you :smile:

[limzykenneth]

I would prefer not to have a separate class for gifs as that adds additional difficulty for the user to use this functionality. Also the code referenced in the stackoverflow answer have a caveat of "NOT for commercial use" which although I don't know how enforceable it is, I'm not comfortable with recommending its usage here.

I would suggest to hold off on actual implementation for the moment as so far I've been voicing my opnion and that does not necessarily reflect the overall concensus of the community or the project maintainer. We should start the implementation only when we have agreed on a rough outline or just as a proof of concept.

[Ajayneethikannan]

I would prefer not to have a separate class for gifs as that adds additional difficulty for the user to use this functionality. Also the code referenced in the stackoverflow answer have a caveat of "NOT for commercial use" which although I don't know how enforceable it is, I'm not comfortable with recommending its usage here.

@limzykenneth , I am sorry for wording it incorrectly, what I meant was that it is better to write our own parser based on GIF89 standards (not copy the code in the stack overflow answer, rather using its logic so to speak) , such that it is highly flexible to our needs, removing any dependency issues if we use any external library specifically for this.

I would suggest to hold off on actual implementation for the moment as so far I've been voicing my opnion and that does not necessarily reflect the overall concensus of the community or the project maintainer. We should start the implementation only when we have agreed on a rough outline or just as a proof of concept.

Yes I understand your concerns regarding the overall consensus, and I am ready to hear more about it.

Thank you for clarifying !

[Ajayneethikannan]

Continuing the discussion, if we consider parsing the gifs into the individual frames (I am unaware of any other implementation at the moment ) , what is the general view on using external libraries to achieve this ? I was able to find a few good libraries such as

• gifuct.js
• jsgif

which handle this part well

Imho,

The advantage would be that the unimportant implementation details (such as the decompression function) would not be present directly in the code, allowing us to write only the important parts, and the code will still remain easier for people to read and understand (abstraction over gif parsing).

The disadvantage would be the dependency on the external library. If one feature breaks in the dependent library it would be difficult to fix here, whereas if we write it on our own, it would be highly customisable to our needs. I think the final file size would also be increased significantly.

Would love to hear the community's thoughts on this.

[Ajayneethikannan]

Hello everyone !

I would love to take up this issue to work with the processing foundation this summer , and I have written in the processing forum about some of the features I would like to implement :
Introduction
Feature details

It would be extremely helpful if you could guide me and provide feedback, which would help me immensely in contributing to this wonderful organisation .

Thank you for your support !