Essentials: Camera API (take photo)

[jamesmontemagno]

Description of the Feature

Ability to take a photo with the default camera. First round could take a photo without saving it to disk and returning a stream.

API Declaration

• MediaFile Camera.TakePhotoAsync();

Additional Classes or Structs needed

• Media File is an option here with the stream, information?

Documentation links for supported platforms

• Android Document
• iOS Document
• UWP Document

VS bug #735678

[mattleibow]

Does Android support passing a stream around?

[harindaka]

@mattleibow I tried the same thing a while back and couldn't find a direct way to obtain a byte array or stream. You could write the bitmap to the DCIM folder and obtain the stream from that. The file can be removed immediately afterwards without broadcasting to gallery apps.

[harindaka]

Would be good to have image selection from gallery as well.

[Mr-Pearce]

I tried the same thing a while back and couldn't find a direct way to obtain a byte array or stream. You could write the bitmap to the DCIM folder and obtain the stream from that. The file can be removed immediately afterwards without broadcasting to gallery apps.

Is this Really impossible on Android?@mattleibow @harindaka
We have a (droid/ios) project in the pipeline that has to take photos with the camera without saving them on the device(for security reasons and also to don't trigger some photo auto backup stuff like google photos)

It would be really nice if you could chose if the photo will be saved to the file system or just lives in memory.

[bertt]

here some (device specific) code for taking pictures app https://github.com/xamarin/xamarin-forms-samples/tree/master/CustomRenderers/ContentPage

[Redth]

Thanks, we are planning on implementing this in the next feature update _after_ 1.0.0

[ederbond]

Camera is one of the most common features we encounter while building an app. Please take to Xamarin.Essentials ASAP.
Also an inline camera view would be considered, to enable us to place a custom live previewer on our app.

[harindaka]

We recently did implement a similar API for both platforms in of my projects and one thing I understood was that a Camera API would not be complete without some post processing helper methods for stuff like image compression (lossy jpg and lossless png), image resizing, image rotation, image cropping, etc. Often we do not use the raw images output by the camera. Android tends to produce huge bitmaps which if not recycled properly causes the app to crash due to memory leaks / insufficient memory. If they are not resized (width and height reduced with aspect ratio preserved) before actual use, it can cause slow processing following the actual capture. The API should give developers control on how these post process actions are performed. Some post process actions would also be broadcasting the image to Gallery Apps, writing the image to a custom gallery album, etc. A camera API without these would not be of much value as it's not just as simple as calling a method and getting a bitmap byte array back. I'm sure @jamesmontemagno and the team is already on top of things. Just wanted to share my two cents based on the issues I faced doing the same. On an off note, we just love the awesome work you are doing on the Xamarin.Essentials @jamesmontemagno and team. Something like this is long overdue for the Xamarin stack.

[Redth]

Images are definitely challenging for at least some of the reasons mentioned here. Any time we bring raw bitmap data back over to the managed .NET side, we potentially incur some problems.

I'm curious @harindaka what you would expect this cross platform API to look like? My assumption is that most developers have varying post processing needs and giving them access to the raw data would allow them to do whatever else they need (albeit maybe less _conveniently_).

If I'm understanding correctly, you're advocating for some control on the capture side (capture resolution, orientation, format) as to start with a more accurate representation of what you're trying to capture rather than trying to post process it completely.

[ederbond]

I guess it could be something very close to this:
https://github.com/jamesmontemagno/MediaPlugin

[Redth]

So MediaPlugin allows some things like resolution selection before capture. I think if I'm reading into this thread correctly, that's one of the big ones, and the idea is that we do this before we even capture the image so we are limiting resource usage.

[harindaka]

@Redth yes. The point I tried to convey is that a Camera API which just captures an image and just provides access to the raw bitmap does not add much value since 99% of the time developers end up having to post process. Here's what the implemented API looks like on my end.

    public interface IMediaUtility
    {
        IPermission CreateCameraPermission();
        IPermission CreateCameraImageCaptureWritePermission();
        IPermission CreatePickImagePermission();
        IPermission CreateGalleryWritePermission();

        Task<ImageCaptureResult> ImageFromCamera(IViewContext viewContext, ImageFromCameraOptions options);

        Task<IEnumerable<ImageHandle>> ImagesFromGallery(IViewContext viewContext,ImagesFromGalleryOptions imagesFromGalleryOptions);
        Task<byte[]> ReadImageContent(IViewContext viewContext,ImageHandle imageHandle, ProcessImageOptions processImageOptions);

        ImageResizeResult ResizeBitmap(byte[] bitmapImage, int widthRequestPx, int heightRequestPx);
        ImageResizeResult ResizeBitmap(byte[] bitmapImage, int widthRequestPx, int heightRequestPx, bool preserveAspectRatio);

        ImageCompressionResult CompressBitmapToJpegOfSize(byte[] bitmapImage);
        ImageCompressionResult CompressBitmapToJpegOfSize(byte[] bitmapImage, double maxSizeMB, int minQualityPercentage, int qualityReductionPercentagePerStep);
        ImageCompressionResult CompressBitmapToJpegOfQuality(byte[] bitmapImage, int qualityPercentage);

        ImageProcessResult OnPostProcessImage(byte[] originalBitmapData, ProcessImageOptions imageOptions);

        Task<byte[]> ShowImageEditor(IViewContext viewContext,byte[] originalBitmapData, ImageEditOptions imageEditOptions);

    }

[harindaka]

Note that output from either the image selection (from gallery) or image capture (from camera) can be further processed (resized, compressed, etc.) via the other helper methods. There's rotation logic somewhere in the implementation too which I haven't got around to adding a similar helper in the interface. But this should give you an idea. ImageCaptureResult, ImageCompressionResult and ImageResizeResult expose corresponding byte arrays of the processed image to the shared code.

[Redth]

I think I'd like to see Resizing and Compression/Format all in the options specified when capturing an image. I'll draft up an API spec soon that we can discuss further here.

[ederbond]

Give us the ability to bypass the confirmation page that is shown to the user on @jamesmontemagno's MediaPlugin.

[apolloLegends]

One consideration I would like to see taken into account would be to grab the location of the device at the time of the photo being taken. In the MediaPlugin this has to be specified before hand (at which point the user could move which they typically do in our usage scenario).

I would consider working on such functionality when the time presents itself. I don't imagine that this is a straightforward request. However, I have derived a lot of value from this and other James Montemagno's libraries (thank you) would like to give back!

[jamesmontemagno]

Once CameraX is out on Android we will start the implementation of this API.

[jrahma]

user should be able to take specific size photos / video by setting the aspect ratio, for example if they want to take square photos then set the frame aspect ratio to 1:1

Also, it should have custom header template and custom footer or it can be placed on any ContentPage or ContentView

Finally we should be able to add / remove any button, e..g take photo, record video, pause, cancel, etc as well as using own custom buttons / images as buttons

[eman1986]

would love to know the status on this, if I had a simple API I could use for both iOS & Android, that'd be amazing.

[ederbond]

Seems to be planned to XF 4.5 https://github.com/xamarin/Xamarin.Forms/wiki/Feature-Roadmap

[muffmolch]

Seems to be planned to XF 4.5 https://github.com/xamarin/Xamarin.Forms/wiki/Feature-Roadmap

I can't see it!

[ederbond]

XF’s team have updated the roadmap and removed it again. So for now, no hope from the built in framework.

[lazmeister]

I can't believe the XF team does not see this as a basic need for the framework. It really shocks me!

[lazmeister]

This is a good start: https://github.com/fferegrino/Camera2Basic and https://github.com/xamarin/xamarin-forms-samples/tree/master/CustomRenderers/ContentPage/CustomRenderer

[LeoJHarris]

Massive pain point here without access to the camera on Xamarin mobile apps. While https://github.com/jamesmontemagno/MediaPlugin works on some android version it doesnt work for all. 2 years ago issue was created and we still dont have a reliable camera plugin??

[aktxyz]

have been getting into xamarin lately ... and just got to the point of wanting to take a picture ... and am stunned that there does not seem to be a out-of-the-box way to do that? is this really the case or did I miss something in googling

+1

[harindaka]

@aktxyz You are right. This is a big miss in Xamarin but at the same time, a difficult one to get right such that, the API suits atleast 90% of the expected use cases. As you can see from this thread some like me would like it to be a high level API with support for image compression, quality settings, cropping, overlay support, etc. But such an API would also be highly opinionated I guess. Others would like a low level API which is not so opinionated which would mean less features. I had to build my own in the past using Xamarin.iOS and Xamarin.Android. That's when you realize that it's not so simple as calling the camera and getting an image returned. So I guess you've got two options, build your own or see what other options are out there. It's been a while since I last worked on Xamarin so I guess things must have improved a lot since then. There may be other libraries out there. But would still love to see this in Xamarin Essentials.

[LeoJHarris]

@harindaka @aktxyz have you guys tried the xlabs media plugin - https://github.com/XLabs/Xamarin-Forms-Labs for taking photos. The source code is all there and seems to work fine. Ive had problems with https://github.com/jamesmontemagno/MediaPlugin on certain android versions that were not present on the xlabs media plugin.

[aktxyz]

I have tried MediaPlugin, will give the other one a shot as well.

And I get that getting it right is difficult, but a mobile framework that cannot access the camera in 2020 is very surprising ... when I was revisiting xamarin this last month after not looking at it for years I was super pleased (delighted even) at how far it's come and how slick things could be ... it never even dawned on me to check if you can access the camera.

One good place to start I think would be to do whatever flutter did.

[LeoJHarris]

@aktxyz if you need proper source code for it I could provide it, but have a look at xlabs.

[lazmeister]

@aktxyz I agree, I was stunned to learn there was nothing out of the box from Xamarin. We are almost at v5 and still nothing. I don't think it will happen or even worth the time as .NET MAUI is coming and will replace Xamarin in a year and going forward anyway.

[eman1986]

@lazmeister Xamarin isn't being replaced, Xamarin.Forms is being replaced with MAUI. Xamarin itself is just not going to be using mono once version 5 comes out which will be nice as mono is limiting the Xamarin framework, I'm excited to see it using netcore under the hood going forward.

I actually don't have any issues using the camera with Xamarin.Native, I made a binding of both an iOS and Android library for cropping and I'm good to go now. I honestly like XN over XF for that very reason (among many other).

[lazmeister]

Sorry yes, I meant to say Xamarin Forms :)

[jorgomezga]

There's a pull request on a component for camera preview and recording:
https://github.com/xamarin/Xamarin.Forms/pull/4084

Hopefully it will make it, someday, to XF, as the mediaelement did.

[mattleibow]

This was added with the new MediaPicker API and supports selecting pictures, videos and using the camera.

[luis-cortina]

@mattleibow Is there any ability to compress images here? This was a big ask in the thread since 99% of the time devs don't want to handle 17MB pictures...

[mattleibow]

@luis-cortina Adding a link to your issue for reference: https://github.com/xamarin/Essentials/issues/1541

The discussion can take place there.

[fzany]

Once CameraX is out on Android we will start the implementation of this API.

I look forward to this James.

[tartulab]

I have same problem in Red mi A7 phone. While taking a photo, application refresh itself and goes to main page. Does anyone find a proper solution?

[lazmeister]

Once CameraX is out on Android we will start the implementation of this API.

I look forward to this James.

@fzany They already published the mediapicker, it's not a full replacement for James' media plugin but close enough to start with: https://docs.microsoft.com/en-us/xamarin/essentials/media-picker?context=xamarin%2Fxamarin-forms&tabs=android

[mattleibow]

Hey folks, for these new questions and problems, can we get new issues so we don't miss them? I just was looking through the notifications and saw these.