Mapsui: Coloring countries possible in Xamarin.Forms?

[FelixKainz]

I am asking since Forms support seems to be in beta.
I was searching for a map API that supports Xamarin.Forms (UWP and Android).

My plan is to color all countries globally representing a trait of them and to show certain information when the user clicks/presses on a country.

Can someone please tell me whether Mapsui in Forms is mature enough to reach these goals?
Since I have a tight deadline it would be absolutely great if you could sum up for me what to do if it is even possible.

Many thanks!

[charlenni]

The Forms implementation of Mapsui isn‘t anymore in beta state. Since version 2 it is full supported.

What you want to do has nothing to do with Forms, but with Mapsui. So the question is, in what a format you have this country outlines.

[FelixKainz]

Ah, ok. On the website it still says that forms is in beta.

So, to make countries clickable/colorable I need them seperated as files?
I thought providers such as openstreetmap would make each country accessible within Mapsui. Or does it just download them es not entities but rather rasterimages?
Nevermind. What I wanted to say is that I don't have any country outlines. I just followed these instructions. I guess I have to get "country outlines" from somewhere and then display them another way?

Thank you for your help!

[charlenni]

OpenStreetMap data are vector data. You have to extract country boarders by your own. Or you try to get them from someone else. Perhaps as Shapefile (*.shp). That then could be displayed in Mapsui.

[FelixKainz]

Ok, I've heard about shapefiles. I'll go hunt for some tomorrow then. Do you have some kind of tutorial for me on how to work with those?

[charlenni]

Look at issue #914. There is a sample project.

[pauldendulk]

Ah, ok. On the website it still says that forms is in beta.

Thanks for pointing that out. I will fix it.

[FelixKainz]

Look at issue #914. There is a sample project.

Ok, thank you very much! That is a starting point.

However, I hape problems porting that sample project to Forms.
I ported the way the map is generated from

// Create a theme style for shape file

            //Set a gradient theme on the countries layer, based on Population density
            //First create two styles that specify min and max styles
            //In this case we will just use the default values and override the fill-colors
            //using a colorblender. If different line-widths, line- and fill-colors where used
            //in the min and max styles, these would automatically get linearly interpolated.
            var min = new VectorStyle { Outline = new Mapsui.Styles.Pen { Color = Mapsui.Styles.Color.Black } };
            var max = new VectorStyle { Outline = new Mapsui.Styles.Pen { Color = Mapsui.Styles.Color.Black } };

            //Create theme using a density from 0 (min) to 400 (max)
            var style = new GradientTheme("PopDens", 0, 400, min, max) { FillColorBlend = ColorBlend.Rainbow5 };

            var map = new Map
            {
                CRS = "EPSG:3857",
                Transformation = new MinimalTransformation()
            };

            map.Layers.Add(new Layer
            {
                Name = "Countries",
                DataSource = new ShapeFile(".\\ShapeFiles\\countries.shp", true),
                Style = style,
            });

            MyMapControl.Map = map;
            MyMapControl.RenderMode = Mapsui.UI.Wpf.RenderMode.Skia;

to

private void CreateMapFromShapefile()
        {
            // Create a theme style for shape file

            //Set a gradient theme on the countries layer, based on Population density
            //First create two styles that specify min and max styles
            //In this case we will just use the default values and override the fill-colors
            //using a colorblender. If different line-widths, line- and fill-colors where used
            //in the min and max styles, these would automatically get linearly interpolated.
            var min = new VectorStyle { Outline = new Mapsui.Styles.Pen { Color = Mapsui.Styles.Color.Black } };
            var max = new VectorStyle { Outline = new Mapsui.Styles.Pen { Color = Mapsui.Styles.Color.Black } };

            //Create theme using a density from 0 (min) to 400 (max)
            var style = new GradientTheme("PopDens", 0, 400, min, max) { FillColorBlend = ColorBlend.Rainbow5 };

            var map = new Map
            {
                CRS = "EPSG:3857",
                Transformation = new MinimalTransformation()
            };

            map.Layers.Add(new Layer
            {
                Name = "Countries",
                DataSource = new ShapeFile(".\\ShapeFiles\\countries.shp", true),
                Style = style,
            });

            //MyMapControl.Map = map;
            //MyMapControl.RenderMode = Mapsui.UI.Wpf.RenderMode.Skia;
            //MyMapControl.RenderMode = Mapsui.Rendering.Skia;

            //map.Widgets.Add(new Mapsui.Widgets.ScaleBar.ScaleBarWidget(map) { TextAlignment = Mapsui.Widgets.Alignment.Center, HorizontalAlignment = Mapsui.Widgets.HorizontalAlignment.Left, VerticalAlignment = Mapsui.Widgets.VerticalAlignment.Bottom });
            mapView.Map = map;
        }

VS is satisfied with that after I copied over the Shapefile and Shapefiles folders, but in the Shapefile.cs I get errors because using System.Web.Caching; could not be found and thus HttpContext in the methods LoadSpatialIndex(bool forceRebuild, bool loadFromFile) and RebuildSpatialIndex() dont can not be resolved.
I assume these issues are due to the change from WPF to Forms.
I thought I can get away with just commenting those methods and imports out and it compiled, but then I got another error.
For better readability, I'll put it in the next comment.

[FelixKainz]

After commenting those methods out, I got this error when I run it as UWP app:
System.IO.DirectoryNotFoundException: 'Could not find a part of the path 'C:\Users\Joker\source\repos\Project1\Project1\Project1.UWP\bin\x86\Debug\AppX\ShapeFiles\countries.shx'.'
I have the Shapefiles folder in my Project1 Project folder, but it searches for it in Project1\Project1.UWP\bin\x86\Debug\AppX\ShapeFiles\countries.shx'.' for some reason.
When I manually copy the folder over to that directory, I get this error when running via UWP:

Exception thrown: 'System.NotSupportedException' in System.Private.CoreLib.dll
An exception of type 'System.NotSupportedException' occurred in System.Private.CoreLib.dll but was not handled in user code
No data is available for encoding 1252. For information on defining a custom encoding, see the documentation for the Encoding.RegisterProvider method.

The program '[22408] Project1.UWP.exe' has exited with code -1 (0xffffffff).

It throws this exception in DbaseReader.cs (line 300) while trying to encode with ANSI.
I can imagine this is due to my naive 'solution' with commenting out the methods mentioned in the comment before.

I am so sorry for bombarding you guys with questions but you would really help me! :D

[pauldendulk]

The shapefile samples are part of a .net framework project, we never got around to porting it to .netstandard, which will be necessary to get it to work on UWP. What you are trying to do is to port it, we never tried this ourselves. The file access issues are part of the porting process.

Another, and perhaps better, approach is to use the shapefile support of NTS:
https://github.com/NetTopologySuite/NetTopologySuite.IO.ShapeFile
This will return the features as NTS features which will have to be converted to Mapsui Geometries. If your data contains only one or a few data types (e.g. only polygons) then this may not be so hard.

I expect we will make use of NTS's shapefile support in the future.

[FelixKainz]

Oof, ok. That's getting more complicated than I expected... Especially since I never worked with any of those packages. I don't even know what a Mapsui Geometry is.
Is there a doc I can use to see how exactly I convert something into Mapsui Geometries? Do I need any previous knowledge about some topic? Geometries sound like maths...

And NTS will be able to read my shapefiles from UWP as well as Android through Xamarin.Forms? That would be important.

Thank you very much!

[pauldendulk]

Converting a polygon is not that hard. It' is one on one. Iterate through the NTS ExteriorRing and add the points to the Mapsui polygon ExteriorRing and you are halfway there.

But yes, perhaps first checkout if you can use that NTS shapefile library on UWP.

EDIT: replaced OutterRing with ExteriorRing

[FelixKainz]

I hope I can figure out what an ExteriorRing is myself :P

[pauldendulk]

There it is:
https://github.com/Mapsui/Mapsui/blob/master/Mapsui.Geometries/Polygon.cs#L61

Just a list of points.

[FelixKainz]

OK, I'll try to do that now.

Just to be sure, my goal is to fill the DataSource in the Layer Object here:

map.Layers.Add(new Layer
{
     Name = "Countries",
     DataSource = new ShapeFile(".\\ShapeFiles\\countries.shp", true),
     Style = style,
});

with the data in the shapefile.
The problem is that the ShapeFile constructor is not able to read the file, so I try to give the ShapeFile object here the data manually using NTS, correct?

[FelixKainz]

But the ShapeFile class seems to only have one constructor requiring the path. So how am I supposed to give it that data?

[FelixKainz]

Or did I understand you wrong and you propose to completely ditch the ShapeFile.cs?

[FelixKainz]

NTS seems to be compatible with .netstandard and thus UWP and Android I hope:
https://github.com/NetTopologySuite/NetTopologySuite.IO.ShapeFile/issues/35#issuecomment-631682583

[FelixKainz]

I was just about to begin implementing my NetTopologySuite.Geometries.Polygon to Mapsui.Geometries.Polygon converter but then I encountered a problem: NTS' LinearRing's consist of Coordinates, but it does not make those accessible through a field or method. It only contains these methods:

namespace NetTopologySuite.Geometries
{
    public class LinearRing : LineString
    {
        public const int MinimumValidSize = 4;

        public LinearRing(Coordinate[] points);
        public LinearRing(CoordinateSequence points, GeometryFactory factory);

        public override Dimension BoundaryDimension { get; }
        public override bool IsClosed { get; }
        public override string GeometryType { get; }
        public bool IsCCW { get; }
        protected override SortIndexValue SortIndex { get; }

        public override Geometry Reverse();
        protected override Geometry CopyInternal();
    }
}

So, do I have to rewrite this class to expose the coordinates or am I missing something here since I am still quite new in C# and especially those modules :) ?

I am still sorry for annyoing you with my questions!

[charlenni]

LinearRing derives from LinearString, which has an array Coordinates. See here.

[FelixKainz]

LinearRing derives from LinearString, which has an array Coordinates. See here.

Oh cool. I didn't think about looking into it's base(es). I'll look into that tomorrow (it's midnight here now :o ).

But what I am still confuesed about is how @pauldendulk intended how I get those polygons into the map as I asked in this commet earlyer...

[charlenni]

Now it is after midnight 😉

If there time tomorrow, I try to look into this.

[FelixKainz]

That would be amazing!

Just wanted to mention that I need them implemented into the map in a way I can reach my initial goal: Color them according to a trait of them (which they don't have yet and I need to implement at some point) and make them clickable.

[charlenni]

I made a sample project. Works for UWP, but throws an error on Android. There is a problem with NTS, so it would be greate to ask there.

ShapeFileForms.zip

[FelixKainz]

I made a sample project. Works for UWP, but throws an error on Android. There is a problem with NTS, so it would be greate to ask there.

ShapeFileForms.zip

Oh wow! That looks like exactly what I wanted!
But the crashing on Android is definitely an issue for me. Tested it with Android 8.1, 9 and 10 VMs.

I'll ask over at NTS Shapefile repo.

[FelixKainz]

I tried to integrate the MainPage.* files into my existing project and after fixing the namespaces and the string containing the project name in CopyFileLocal it compiles, but crashes when run on both UWP as well as Android.
On UWP it crashes when it tries to retrieve the write the variable resource to the cached file in CopyFileLocal (at resource.CopyTo(file);). It says that resource is Null. The interesting thing is that it only happens with countires.shx which is handled after .shp and .dbf.

On Android, it crashes in the same line but when calling CopyFileLocal for the first time with the .shp.

Sometimes, I think computers are just cursed.
I just observed all this and tried multiple things and then reverted to how it was before and now I get an exception at the inizialization of shapeFileReader in CreateDrawables. ALTHOUGH I DID NOT CHANGE ANYTHING!

System.IO.EndOfStreamException
  HResult=0x80070026
  Message=Unable to read beyond the end of the stream.
  Source=System.Private.CoreLib
  StackTrace:
   at System.IO.BinaryReader.FillBuffer(Int32 numBytes)
   at System.IO.BinaryReader.ReadInt32()
   at NetTopologySuite.IO.ShapefileHeader..ctor(BigEndianBinaryReader shpBinaryReader)
   at NetTopologySuite.IO.ShapefileReader..ctor(IStreamProviderRegistry shapeStreamProviderRegistry, GeometryFactory geometryFactory)
   at NetTopologySuite.IO.ShapefileDataReader..ctor(IStreamProviderRegistry streamProviderRegistry, GeometryFactory geometryFactory)
   at NetTopologySuite.IO.Shapefile.CreateDataReader(IStreamProviderRegistry streamProviderRegistry, GeometryFactory geometryFactory)
   at NetTopologySuite.IO.Shapefile.CreateDataReader(String filename, GeometryFactory geometryFactory)
   at CoronaStatistics.MainPage.CreateDrawables(String shapeFile) in C:\Users\Joker\source\repos\CoronaStatistics\CoronaStatistics\CoronaStatistics\MainPage.xaml.cs:line 83
   at CoronaStatistics.MainPage..ctor() in C:\Users\Joker\source\repos\CoronaStatistics\CoronaStatistics\CoronaStatistics\MainPage.xaml.cs:line 61
   at CoronaStatistics.App..ctor() in C:\Users\Joker\source\repos\CoronaStatistics\CoronaStatistics\CoronaStatistics\App.xaml.cs:line 13
   at CoronaStatistics.UWP.MainPage..ctor() in C:\Users\Joker\source\repos\CoronaStatistics\CoronaStatistics\CoronaStatistics.UWP\MainPage.xaml.cs:line 24

  This exception was originally thrown at this call stack:
    [External Code]
    CoronaStatistics.MainPage.CreateDrawables(string) in MainPage.xaml.cs
    CoronaStatistics.MainPage.MainPage() in MainPage.xaml.cs
    CoronaStatistics.App.App() in App.xaml.cs
    CoronaStatistics.UWP.MainPage.MainPage() in MainPage.xaml.cs

I am so annoyed by this and so sorry to drag you through this with me. But I have to get this going until next week >.<

[charlenni]

For the UWP crash, I would say, that you have a wrong setting for the file. You should set it to embedded resource. Your it is a typo.

On Android I would say, that it is the same problem.

[FelixKainz]

After setting the files to embedded, it still didn't work but it did once I deleted the package directory at C:\Users\Joker\AppData\Local\Packages\. Good to know that something like that can help sometimes. These package directories seem to not get cleaned when one cleans the solution from within VS.
However, I get another error in the same line in my Project:

System.IO.EndOfStreamException
  Message=Unable to read beyond the end of the stream.

Where in your sample Project I get

System.ArgumentNullException
  Message=Value cannot be null.
Parameter name: key

Do I have to change something else about my project other than copying over MainPage.xaml.cs, Extensions, namespace fixes and markind the shapefiles as embedded?

I asked over at NTS about the Android issue here.

[charlenni]

What shapefiles do you use? Your own or that from my sample?

[FelixKainz]

I use the same ones you do, I thought you took them out of the sample I linked a few days ago, so we took the same as base.
Just checked, both of our ShapeFiles folders are 479.688 Bytes in size, so they should be identical.

I could also try the other way around (to merge my project into yours) but I am not sure how git likes that when merging.
If you don't know what I missed spontaniously I guess I'll try that next...

[charlenni]

All this things have nothing to do with Mapsui. The biggest problems are to bring the files at a place, where NTS could get them.

Did you tried a File.Exists()?

[FelixKainz]

I added a debug output of the files in the chached ShapeFiles dir in the top of the problematic method CreateDrawables:

DirectoryInfo di = new DirectoryInfo(shapeFile.Substring(0, shapeFile.Length - 9));
foreach (FileInfo fi in di.GetFiles())
{
    System.Diagnostics.Debug.WriteLine(string.Format("{0} ({1} bytes)", fi.FullName, fi.Length));
}

It gives the following output on UWP:

C:\Users\Joker\AppData\Local\Packages\a81b00f1-04c5-4770-b9bc-bd427610f6cf_x1waj26dt6j1y\LocalCache\ShapeFiles\countries.dbf (7301 bytes)
C:\Users\Joker\AppData\Local\Packages\a81b00f1-04c5-4770-b9bc-bd427610f6cf_x1waj26dt6j1y\LocalCache\ShapeFiles\countries.shp (459332 bytes)
C:\Users\Joker\AppData\Local\Packages\a81b00f1-04c5-4770-b9bc-bd427610f6cf_x1waj26dt6j1y\LocalCache\ShapeFiles\countries.shx (1276 bytes)

And on Android:

[0:] /data/user/0/com.companyname.shapefileforms/cache/ShapeFiles/countries.shp (459332 bytes)
[0:] /data/user/0/com.companyname.shapefileforms/cache/ShapeFiles/countries.dbf (7301 bytes)
[0:] /data/user/0/com.companyname.shapefileforms/cache/ShapeFiles/countries.shx (1276 bytes)

These are all identical with respect to IDs across both of our projects and UWP and Android 8.1, 9, 10.
However, because of this idea of you I was able to fix my issue earlyer with the EndOfStreamException in my project that didn't happen in your's. It turned out that two of those files in my 8.1 VM were 0 bytes in size. It worked after resetting it to factory defaults. So at least that is fixed now :D

So, to conclude: all the DBF, SHP and SHX files are there on any target platform and at least superficially (only judging by filesize) intact.

[charlenni]

Do you get still the ArgumentNullException? Even in my sample. Even on UWP? Or any other error?

Sorry, but I have lost the overview. Was a long day.

[FelixKainz]

Do you get still the ArgumentNullException? Even in my sample. Even on UWP? Or any other error?

Sorry, but I have lost the overview. Was a long day.

That is absolutely OK! :)
I can't expect you to spent your free time on helping me after having to work all day.
Just wanted to say again how thankful I am to you for helping me so much! I wish I could return this somehow but I guess that will be difficult to do with code since you seem to be much more experienced in coding than me :D

So, the current state is:
Our projects seem to be in sync now. We both get this error on Android 8.1 and 10, so I guess 9 too:

System.ArgumentNullException
  Message=Value cannot be null.
Parameter name: key

at line var shapeFileReader = Shapefile.CreateDataReader(shapeFile, new NetTopologySuite.Geometries.GeometryFactory(new NetTopologySuite.Geometries.PrecisionModel(1000000.0))); in method CreateDrawables.
I can't step into this line in the debugger. I guess because I/we installed NTS.IO.Shapefile via Nuget, which only downloaded it compiled rather than importing their code? I am not sure how this works in C#.
This is the same one you had in your initial sample. I spent the meantime with debugging the issues I had in my project after merging your code into it and other projects.
Call Stack:

{System.ArgumentNullException: Value cannot be null.Parameter name: key  
at System.Collections.Generic.Dictionary`2[TKey,TValue].FindEntry (TKey key) [0x00008] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/Common/src/CoreLib/System/Collections/Generic/Dictionary.cs:380   
at System.Collections.Generic.Dictionary`2[TKey,TValue].ContainsKey (TKey key) [0x00000] in /Users/builder/jenkins/workspace/archive-mono/2020-02/android/release/external/corefx/src/Common/src/CoreLib/System/Collections/Generic/Dictionary.cs:286   
at NetTopologySuite.IO.DbaseEncodingUtility.AddLdidEncodingPair (System.Byte ldid, System.Text.Encoding encodingToAdd) [0x0000c] in <0e9c02ea98264091a344714de10def9b>:0   
at NetTopologySuite.IO.DbaseEncodingUtility.RegisterEncodings (System.Object[][] ldidCodePagePairs) [0x00027] in <0e9c02ea98264091a344714de10def9b>:0   
at NetTopologySuite.IO.DbaseEncodingUtility..cctor () [0x00799] in <0e9c02ea98264091a344714de10def9b>:0 }

UWP works fine in both projects.

The only code I added is the debugging code I was tallking about in my previous post.
Conclusion of this debugging code: all the files are there, accessible and seemingly intact on all platforms.

I also posted about this issue over at NTS but they don't seem to have as many as helpful people as you two :P.

[Dali2579]

Ah, ok. On the website it still says that forms is in beta.

Thanks for pointing that out. I will fix it.

MapsUI still showing beta version of mapsui 2.0

[pauldendulk]

Mapsui still showing beta version of mapsui 2.0

I just removed a reference here:
https://mapsui.com/documentation/home.html?q=beta

If I search for 'beta' in the website and in the source code I can not find any hits on 'beta'. Where do did you see the 'beta' reference?