Stdlib: Reading and writing common image formats, ppm, tiff, jpeg, png

[certik]

Here is an example implementation of loading and saving ppm: https://github.com/certik/fortran-utils/blob/b43bd24cd421509a5bc6d3b9c3eeae8ce856ed88/src/ppm.f90. The advantage of the ppm format is that it is simple to write such readers and writers. Then one can use external tools (such as pnmtopng) to convert to more common formats. So perhaps tiff, jpeg and png are not initially needed and ppm might be enough to allow to work with images in Fortran.

Prior art:

• Matlab: imread can read BMP, JPEG, PNG, CUR, PPM, GIF, PBM, RAS, HDF4, PCX, TIFF, ICO, PGM and XWD files. See also imwrite.
• SciPy: imread uses Python Imaging Library (PIL) to read the image; PIL supports: BMP, DIB, EPS, GIF, ICNS, ICO, IM, JPEG, MSP, PCX, PNG, PPM, SGI, SPIDER, TGA, TIFF, WebP, XBM. See also imsave.
• Julia: Images.jl package has a nice comparison page with SciPy and Matlab; can read at least PNG, GIF, TIFF, JPEG.

[certik]

If we only do PPM, that can be easily done in pure Fortran. For almost all the other formats we would have to depend on a 3rd party library.

It feels like supporting all the formats might be a better fit for a separate library from stdlib, because at least initially I think we should stay in pure Fortran and only have minimal external dependencies to make the distribution of stdlib easier.

But since PPM can be supported easily in pure Fortran, it would allow codes to export and import images by using some external tool to convert to and from PPM first, which seems like it might be valuable enough to include in stdlib.

[milancurcic]

PPM is good enough for start! I agree on the advantage of pure Fortran implementation. We can adopt your implementations, they're good. I also have a P3 writer (same as P6, but ascii).

[certik]

Perfect, let's do it. Yes, we can support both ascii and binary PPM and a few related formats.

[milancurcic]

What module would this belong to? stdlib_experimental_io or its own stdlib_experimental_image? On one hand, I think this fits thematically in io. However, we anticipate that io will grow further and would be split in smaller modules eventually, so maybe a dedicated module is more fitting.

I'm happy to prepare a PR with loadppm and saveppm implementations for P3 and P6 formats -- basically take your code and add P3 implementations. Other formats may be implemented later if desired.

How should we handle choosing a format? For loadppm I think we can have just one specific procedure, because the format is specified in the file header and the user doesn't have to specify which format it is. However, for saveppm, user should somehow specify which format we're storing into. We can just use different procedures directly:

• saveppm_p3
• saveppm_p6

We could also have a thin wrapper and specify it with the format argument:

subroutine saveppm(filename, img, format)
  ...
  if (format == 'P3') then
    call saveppm_p3(filename, img)
  else if (format == 'P6') then
    call saveppm_p3(filename, img)
  else
    call error_stop()
  end if
end subroutine saveppm

[certik]

Let's use stdlib_experimental_io for now. I would use stdlib_experimental_image if we plan to support more image routines. In SciPy it is in scipy.misc. Also, SciPy and Matlab calls this imread. However, their imread can read all kinds of image formats. Since we will only do PPM at first, I think it's fine to call it load / saveppm. We can later add imread as we support more formats.

Regarding the API, I agree with your choice. I would further make the format argument optional, and if it is not present, probably choose P6 (binary) as the default.

P.S. Isn't format a "reserved" word? If so, you can use fmt instead.

[milancurcic]

Isn't format a "reserved" word? If so, you can use fmt instead.

I don't think Fortran has reserved words. fmt is also a keyword in read() and write() statements. I do prefer fmt here because it's shorter and clear what it means.

I like the idea of P6 as default.

[ivan-pi]

For some applications I had in the past a graymap PPM (P2 or P5) would have sufficed. Specifically, I could use a ppm as a porosity map in a Lattice Boltzmann simulation. Since these only require a rank 2 array (i and j image coordinates) vs a rank 3 array (i, j, and rgb) it would be easy to support these too.

[jacobwilliams]

FYI: pure Fortran gif (even animated) library here: https://github.com/jacobwilliams/FGIF

[certik]

@jacobwilliams this is great! It might not be that difficult to provide pure Fortran readers / writers to other image formats also.

[gronki]

IMHO this is easily implemented as part of the ecosystem with iso_c_binding to currently existing libraries. There is no need complicating stuff by making competing solutions in stdlib (even for ppm). And one would need unsigned types to make it work in any decent way.

[zbeekman]

There is no need complicating stuff by making competing solutions in stdlib (even for ppm). And one would need unsigned types to make it work in any decent way.

Often there is a compelling case to write something in Fortran so that the community can contribute and fix it, but for certain cases, especially file formats, if there is a widely used existing library that stdlib can provide a Fortran interface for then that reduces complexity and reduces maintenance burden, while ensuring a higher quality of implementation.

In this particular case we should be pragmatic and include implementations only where:

1. There is already a popular and well tested Fortran version we can adopt
2. It is easy to write and maintain (e.g., PPM)
3. There is no suitable extant implementation for which a Fortran interface can be created

[milancurcic]

@gronki Can you list C library candidates? I know of stb and libpng. What do you have in mind?

We should absolutely weigh-out the complexity of pure Fortran vs. C-interop implementations. In case of ppm, the pure Fortran implementation is so simple that I can't imagine a similar Fortran-C interface would be any simpler. I may be wrong though.

Involving any external C library would add the complexity of a dependency. While I'm not against external dependencies in stdlib (Python has them), I think we should consider them as a last resort.

[gronki]

But is reading and writing ppm's worth the burden of the maintenance? Don't get me wrong but it's not a serious image format. PPM is short for "i don't feel like googling libpng manual". I see no value of having a ppm implementation but maybe my opinion is isolated.

I have never heard about stb before. I personally used libgdal, libpng and cfitsio in my field (from Fortran). All of them mature, stable and insanely easy to use by iso_c_binding (or even to wrap in a derived type). (edit: Actually, cfitsio has a f77 interface included.) I think it would be nice to have a stable and up-to-date bindings as a part of the ecosystem (whether they make it into stdlib or not) but I see absolutely no need to put any effort in implementing a worse version of what is already done.

One a side note, I feel that not having certain bindings for popular format handling libraries is the actual problem we are trying to solve here. Fortran does have PNG or JPG handling. But it takes too much effort (one has to write their own binding rather than have a ready-to-use interface such as C or Python).

[zbeekman]

We should absolutely weigh-out the complexity of pure Fortran vs. C-interop implementations. In case of ppm, the pure Fortran implementation is so simple that I can't imagine a similar Fortran-C interface would be any simpler. I may be wrong though.

@milancurcic Exactly how I feel. Well said.

Involving any external C library would add the complexity of a dependency. While I'm not against external dependencies in stdlib (Python has them), I think we should consider them as a last resort.

This is very true. But in the case where an external library already does it---and does it well---and adding a roll-your-own implementation in the standard library adds unacceptable complexity, then I think a choice should be made about either:

1. Creating an interface for the extant external library
2. Not including it in the standard library

Elon Musk recently said something like this when talking about SpaceX Starship:

The best design is an un-design. It has no moving parts, no points of failure, no maintenance and development costs, no documentation to write or protocols to create and follow.

Obviously we want convenience and utility in a standard library but it needs to be provided in a sustainable way.

(And, yes, I know Musk is a bit mad, but he has some moments of wisdom/genius.)

[milancurcic]

But in the case where an external library already does it---and does it well---and adding a roll-your-own implementation in the standard library adds unacceptable complexity, then I think a choice should be made about either:

1. Creating an interface for the extant external library
2. Not including it in the standard library

I agree! It seems to me that any format that would need compression like zlib would fall into one of these two categories. (if we need to include zlib as a dependency, this lowers the bar to entry to an external image library as a dependency as well).

@gronki Let's focus on PNG and libpng -- do you have example interfaces that we can look at? Would you like to contribute them to stdlib?

[certik]

The idea of PPM is that they made the format simple enough so that anybody can easily read and write it from any language. Then you can use external tools to convert it to more common formats.

The cost of any dependency should not be underestimated. For example, I would suggest to make such dependencies optional, at least initially. Not every (future) user of stdlib wants to use images, and having them require to install 10 libraries for the 10 image formats that we will support (but they don't need), that I think is not worth it.

[jvdp1]

I would suggest to avoid non-optional dependencies as much as possible, except if there are installed on several OSes per default (or are very common and expected to be already installed).
Otherwise, if a user wants to use stdlib that depends on (exotic) dependencies, on a machine on which he is not an admin (e.g. on a HPC), that user may have issues to install the depencies and stdlib itself. stdlib should help the user with Fortran, not to learn him how to install various libraries (automatically or not;locally or not).
Having said that, I also think we should avoid to "translate" existing and well-developed C-librairies into Fortran.
Optional dependencies would then be a good solution IMHO.

[jacobwilliams]

I also agree that we shouldn't re-implement rock solid libs written in other languages (I was also thinking of something like libcurl). The stdlib CMake project needs to be set up so that if you don't have some external lib installed, it builds it without that component.

But, also keep in mind that this is a volunteer project. If somebody wants to spend their time writing the world's most amazing Fortran PPM interface, I have absolutely no objection to them doing that. :)

[certik]

Looks like we are all in agreement on the following:

1) Let's figure out the natural API, no matter how it is actually implemented

2) Let's start using external libraries optionally to implement as many formats as we can. This should always be available as an option.

3) Let's also have pure Fortran backends, as people voluntarily write them. (E.g., the PPM one is already written.) Setup our CMake / Make so that people can choose which one to use (whether our "reference" pure Fortran implementation if available, or a 3rd party library if available).

Let me know if you all agree with the above, and if you think this will satisfy everybody.

[urbanjost]

M_pixel

was designed as a simple self-contained vector library for PPM files for use in a graphics class. The Fortran-based GIF utilities mentioned previously worked great in conjunction with it for making simple GIF and animated GIF examples. PPM is very good (of course) for programs that are basically manipulating pixels; this could use some polish and needs a higher-level axis routine added, and prettier fonts but if anyone ones something in self-contained Fortran to start with feel free to take a look.

In a batch environment the venerable NetPBM and ImageMagik filters worked seamlessly to convert the PPM files to just about any image file format around. With command-level packages like that the PPM format was basically just as good for single-image graphics files as any other, and easiest to convert to other formats. It is a bit verbose so it was almost always converted to something else on the spot. We actually let the programs write multiple images sequentially to a single file and then split them with a post-processor to make multi-plot formats like Adobe PDF. The main irritation with PPM files is they are only designed to represent one frame.

It was designed to be compatible (Originally) with the VOGLE/M_graph library, which supports a lot of vector output formats but very little pixel-level graphics; so anyone that used VOGLE in the past
should find it very familiar.

[vmagnin]

2\. Let's start using external libraries optionally to implement as many formats as we can.

The C GdkPixbuf library can save files in "jpeg", "png", "tiff", "ico" or "bmp":
https://developer.gnome.org/gdk-pixbuf/stable/

See my Discourse post for a Fortran example:
https://fortran-lang.discourse.group/t/making-computer-graphics-in-fortran-without-gui-just-creating-a-png/