Mono: mkbundle - Use of any Web/Http-Client causes "The URI prefix is not recognized." exception unless mono is installed

[FredMangelsdorf]

I'm trying to do an HTTP request as a client from an application build with mkbundle so it can be deployed to many Linux systems without a mono installation. After facing issues, I wrote a small test program to isolate this issue and also to find out if a particular .NET class/approach works.

This is my test program:

//define USE_HTTPCLIENT
//define USE_WEBREQUEST
#define USE_WEBCLIENT

using System;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Text;
using System.Threading.Tasks;

namespace HttpClientInspector {
  class Program {
    public static List< string> GetAssemblies( ) {
      List< string> Result = new List< string>( );
      foreach ( Assembly a in AppDomain.CurrentDomain.GetAssemblies( )) {
        string Name = a.FullName.Split( ',')[ 0];
        Result.Add( Name);
      }
      return Result;
    } // GetAssemblies

    protected static List< List< string>> AssemblyStatus;

    public static void GetPage( ) {
      try {
        Console.WriteLine( "Creating a Uri");
        Uri u = new Uri( "http://192.168.3.112/"); // TODO: change this !!!
        if ( u == null) Console.WriteLine( "The Uri is null");
        else {
          /*
          Console.WriteLine( "Creating a ServicePoint");
          ServicePoint sp = ServicePointManager.FindServicePoint( u, null);
          if ( sp == null) Console.WriteLine( "The ServicePoint is null");
          else Console.WriteLine( "Got the ServicePoint");
          */
#if USE_HTTPCLIENT
          Console.WriteLine( "Creating an HttpClient");
          HttpClient hc = new HttpClient( );
          Console.WriteLine( "Calling GetStringAsync");
          Task< string> tp = hc.GetStringAsync( u);
#endif // USE_HTTPCLIENT
#if USE_WEBREQUEST
          Console.WriteLine( "Creating a WebRequest");
          HttpWebRequest hwr = WebRequest.CreateHttp( u);
          Console.WriteLine( "Calling BeginGetResponse");
          IAsyncResult iar = hwr.BeginGetResponse( null, null);
#endif // USE_WEBREQUEST
#if USE_WEBCLIENT
          Console.WriteLine( "Creating a WebClient");
          WebClient wc = new WebClient( );
          Console.WriteLine( "Calling DownloadString");
          string Page = wc.DownloadString( u);
#endif // USE_WEBCLIENT
          Console.WriteLine( "Fetching all loaded Assemblies");
          AssemblyStatus.Add( GetAssemblies( ));
#if USE_HTTPCLIENT
          Console.WriteLine( "Waiting for the Task to complete.");
          if ( tp.Wait( 500)) {
            string Page = tp.Result;
            Console.WriteLine( "The response has a Length of {0}",
                                                                Page.Length);
          } else Console.WriteLine( "Timed-out waiting for a response.");
#endif // USE_HTTPCLIENT
#if USE_WEBREQUEST
          Console.WriteLine( "Waiting for the WebRequest to complete.");
          if ( iar.AsyncWaitHandle.WaitOne( 500)) {
            WebResponse we = hwr.EndGetResponse( iar);
            Stream s = we.GetResponseStream( );
            StreamReader sr = new StreamReader( s);
            string Page = sr.ReadToEnd( );
            Console.WriteLine( "The response has a Length of {0}",
                                                                Page.Length);
          } else Console.WriteLine( "Timed-out waiting for a response.");
#endif // USE_WEBREQUEST
#if USE_WEBCLIENT
          Console.WriteLine( "The response has a Length of {0}",
                                                                Page.Length);
#endif // USE_WEBCLIENT
        }
      } catch ( Exception ex) {
        if ( ex is AggregateException ae) {
          Console.WriteLine( "Caught an AggregateException with:");
          foreach ( Exception e in ae.InnerExceptions)
            Console.WriteLine( "  {0}: {1}", e.GetType( ).Name, e.Message);
          Console.WriteLine( "{0}", ex.StackTrace);
        } else
        Console.WriteLine( "Caught unexpected {0}: {1}\n{2}",
                                ex.GetType( ).Name, ex.Message, ex.StackTrace);
        if ( ex.InnerException != null) Console.WriteLine(
                                                        "  - inner {0}: {1}",
                                        ex.InnerException.GetType( ).Name,
                                                ex.InnerException.Message);
      }
    } // GetPage

    static void Main( string[ ] args) {
      /*
      Console.WriteLine( "Starting Main - apply workaround");
      try {
        Assembly.Load( "System.Configuration")
        ?.GetType( "System.Configuration.ConfigurationManager")
        ?.GetMethod( "GetSection", BindingFlags.Static | BindingFlags.Public)
        ?.Invoke( null, new [ ] { "configuration" });
      } catch ( Exception ex) {
        Console.WriteLine("Caught unexpected {0}: {1}\n{2}",
                                ex.GetType( ).Name, ex.Message, ex.StackTrace);
      }
      */
      object cs = ConfigurationManager.GetSection(
                                            "system.net/connectionManagement");

      Console.WriteLine( "Started Main - fetching all loaded Assemblies");
      AssemblyStatus = new List< List< string>> { GetAssemblies( ) };
      Console.WriteLine( "Done fetching all loaded Assemblies - calling GetPage");
      GetPage( );
      Console.Write( "Inspect the loaded so's with\n  cat /proc/$pid/maps\n" +
                                "and then hit ENTER to finish this program.");
      Console.ReadLine( );
      AssemblyStatus.Add( GetAssemblies( ));

      int MaxAsy = AssemblyStatus[ AssemblyStatus.Count - 1].Count;
      for ( int i = 0; i < AssemblyStatus.Count - 1; i++)
        if ( AssemblyStatus[ i].Count > MaxAsy)
          MaxAsy = AssemblyStatus[ i].Count;

      int Width = 20;
      for ( int i = 0; i < AssemblyStatus.Count; i++)
        for ( int j = 0; j < MaxAsy; j++)
          if ( j < AssemblyStatus[ i].Count &&
               AssemblyStatus[ i][ j].Length > Width)
            Width = AssemblyStatus[ i][ j].Length;

      if ( Width > 32) Width = 32;
      string Fmt = string.Format( "{{0,{0}}} {{1,{0}}} {{2,{0}}}", Width);

      Console.WriteLine( Fmt, "Before", "During", "After");
      for ( int l = 0; l < MaxAsy; l++) {
        string c1 = l < AssemblyStatus[ 0].Count ? AssemblyStatus[ 0][ l] : "";
        string c2 = l < AssemblyStatus[ 1].Count ? AssemblyStatus[ 1][ l] : "";
        string c3 = l < AssemblyStatus[ 2].Count ? AssemblyStatus[ 2][ l] : "";
        if ( c1.Length > Width || c2.Length > Width) {
          Console.WriteLine( Fmt, c1, "", "");
          Console.WriteLine( Fmt, "", c2, "");
          Console.WriteLine( Fmt, "", "", c3);
        } else Console.WriteLine( Fmt, c1, c2, c3);
      }  
    } // Main
  } // class Program
} // namespace HttpClientInspector

None of the 3 different approaches work, however. I did manage to find issue #6187 which looks a lot like my issue. With this help (and some more googling), I build using:

mkbundle -o HttpClientInspector --simple           \
--deps HttpClientInspector.exe                     \
--config /etc/mono/config                          \
--library /usr/lib64/libc-2.17.so                  \
--library /usr/lib64/libgcc_s-4.8.5-20150702.so.1  \
--library /usr/lib64/libresolv-2.17.so             \
--library /usr/lib64/libnss_files-2.17.so          \
--library /usr/lib64/libnss_dns-2.17.so            \
--library /usr/lib64/ld-linux-x86-64.so.2          \
--library /usr/lib64/libpthread-2.17.so            \
--library /usr/lib64/libdl-2.17.so                 \
--library /usr/lib64/librt-2.17.so                 \
--library /usr/lib64/libm-2.17.so                  \
--library /usr/lib64/libdl-2.17.so                 \
--library /usr/lib64/libmonosgen-2.0.so.1.0.0

This is the result of running this program with some additional verbosity:

[fred@nomono ~]$ MONO_LOG_MASK=asm
[fred@nomono ~]$ MONO_LOG_LEVEL=debug
[fred@nomono ~]$ export MONO_LOG_MASK
[fred@nomono ~]$ export MONO_LOG_LEVEL
[fred@nomono ~]$ ./HttpClientInspector 
Mono: Assembly Loader loaded assembly from bundle: 'HttpClientInspector.exe'.
Mono: Assembly Loader probing location: '/usr/local/lib/mono/4.5/mscorlib.dll'.
Mono: Assembly Loader loaded assembly from bundle: 'mscorlib.dll'.
Mono: Image addref mscorlib[0xfe9180] -> mscorlib.dll[0xfe7cf0]: 3
Mono: Prepared to set up assembly 'mscorlib' (mscorlib.dll)
Mono: Assembly mscorlib[0xfe9180] added to domain HttpClientInspector.exe, ref_count=1
Mono: Assembly Loader probing location: 'HttpClientInspector.exe'.
Mono: Unloading image HttpClientInspector.exe [0x104f290].
Mono: Assembly Loader loaded assembly from bundle: 'HttpClientInspector.exe'.
Mono: Image addref HttpClientInspector[0x104f380] -> HttpClientInspector.exe[0xfe67d0]: 5
Mono: Prepared to set up assembly 'HttpClientInspector' (HttpClientInspector.exe)
Mono: Assembly HttpClientInspector[0x104f380] added to domain HttpClientInspector.exe, ref_count=1
Mono: Assembly Loader probing location: 'HttpClientInspector.exe'.
Mono: Unloading image HttpClientInspector.exe [0x10524d0].
Mono: Assembly Loader loaded assembly from bundle: 'HttpClientInspector.exe'.
Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0
Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0
Mono: Assembly Ref addref HttpClientInspector[0x104f380] -> mscorlib[0xfe9180]: 2
Mono: Found assembly remapping for System.Configuration and was for the same version 4.0.0.0
Mono: Found assembly remapping for System.Configuration and was for the same version 4.0.0.0
Mono: Domain HttpClientInspector.exe search path is:
Mono:   path[0] = '/home/fred/'
Mono: End of domain HttpClientInspector.exe search path.
Mono: Assembly Loader probing location: '/usr/local/lib/mono/gac/System.Configuration/4.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll'.
Mono: Assembly Loader loaded assembly from bundle: 'System.Configuration.dll'.
Mono: Image addref System.Configuration[0x1061960] -> System.Configuration.dll[0x1060810]: 3
Mono: Prepared to set up assembly 'System.Configuration' (System.Configuration.dll)
Mono: Assembly System.Configuration[0x1061960] added to domain HttpClientInspector.exe, ref_count=1
Mono: Assembly Ref addref HttpClientInspector[0x104f380] -> System.Configuration[0x1061960]: 2
Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0
Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0
Mono: Assembly Ref addref System.Configuration[0x1061960] -> mscorlib[0xfe9180]: 3
Mono: Found assembly remapping for System and was for the same version 4.0.0.0
Mono: Found assembly remapping for System and was for the same version 4.0.0.0
Mono: Domain HttpClientInspector.exe search path is:
Mono:   path[0] = '/home/fred/'
Mono: End of domain HttpClientInspector.exe search path.
Mono: Assembly Loader probing location: '/usr/local/lib/mono/gac/System/4.0.0.0__b77a5c561934e089/System.dll'.
Mono: Assembly Loader loaded assembly from bundle: 'System.dll'.
Mono: Image addref System[0x106f3b0] -> System.dll[0x106e4b0]: 3
Mono: Prepared to set up assembly 'System' (System.dll)
Mono: Assembly System[0x106f3b0] added to domain HttpClientInspector.exe, ref_count=1
Mono: Assembly Ref addref System.Configuration[0x1061960] -> System[0x106f3b0]: 2
Mono: Found assembly remapping for System.Xml and was for the same version 4.0.0.0
Mono: Found assembly remapping for System.Xml and was for the same version 4.0.0.0
Mono: Domain HttpClientInspector.exe search path is:
Mono:   path[0] = '/home/fred/'
Mono: End of domain HttpClientInspector.exe search path.
Mono: Assembly Loader probing location: '/usr/local/lib/mono/gac/System.Xml/4.0.0.0__b77a5c561934e089/System.Xml.dll'.
Mono: Assembly Loader loaded assembly from bundle: 'System.Xml.dll'.
Mono: Image addref System.Xml[0x107d4d0] -> System.Xml.dll[0x107c170]: 3
Mono: Prepared to set up assembly 'System.Xml' (System.Xml.dll)
Mono: Assembly System.Xml[0x107d4d0] added to domain HttpClientInspector.exe, ref_count=1
Mono: Assembly Ref addref System.Configuration[0x1061960] -> System.Xml[0x107d4d0]: 2
Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0
Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0
Mono: Assembly Ref addref System.Xml[0x107d4d0] -> mscorlib[0xfe9180]: 4
Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0
Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0
Mono: Assembly Ref addref System[0x106f3b0] -> mscorlib[0xfe9180]: 5
Started Main - fetching all loaded Assemblies
Done fetching all loaded Assemblies - calling GetPage
Mono: Found assembly remapping for System and was for the same version 4.0.0.0
Mono: Found assembly remapping for System and was for the same version 4.0.0.0
Mono: Assembly Ref addref HttpClientInspector[0x104f380] -> System[0x106f3b0]: 3
Creating a Uri
Creating a WebClient
Calling DownloadString
Mono: Found assembly remapping for System.Configuration and was for the same version 4.0.0.0
Mono: Found assembly remapping for System.Configuration and was for the same version 4.0.0.0
Mono: Assembly Ref addref System[0x106f3b0] -> System.Configuration[0x1061960]: 3
Caught unexpected WebException: An exception occurred during a WebClient request.
  at System.Net.WebClient.DownloadDataInternal (System.Uri address, System.Net.WebRequest& request) [0x00072] in <c78e65c37a9c4932b53d91283b8b4db1>:0 
  at System.Net.WebClient.DownloadString (System.Uri address) [0x00020] in <c78e65c37a9c4932b53d91283b8b4db1>:0 
  at (wrapper remoting-invoke-with-check) System.Net.WebClient.DownloadString(System.Uri)
  at HttpClientInspector.Program.GetPage () [0x0004d] in <50379b3594614b63b70a5bf4fce9110a>:0 
  - inner NotSupportedException: The URI prefix is not recognized.
Inspect the loaded so's with
  cat /proc/$pid/maps
and then hit ENTER to finish this program.

I've run this program on CentOS Linux release 7.4.1708 (Core), without mono installed.
I've run mkbundle on CentOS Linux release 7.4.1708 (Core),
with a mono --version:

Mono JIT compiler version 5.9.0 (master/f58ba94 Mon Nov  6 00:00:09 CET 2017)
Copyright (C) 2002-2014 Novell, Inc, Xamarin Inc and Contributors. www.mono-project.com
    TLS:           __thread
    SIGSEGV:       altstack
    Notifications: epoll
    Architecture:  amd64
    Disabled:      none
    Misc:          softdebug 
    LLVM:          supported, not enabled.
    GC:            sgen (concurrent by default)

using the default target

Any help and/or hints are very much appreciated.

[marek-safar]

IIRC URI prefixes come from machine.config so you need to bundle it as well

[FredMangelsdorf]

Yes !!!!

Thank you very much, that fixes it. Problem was solved by adding "--machine-config /etc/mono/4.5/machine.config" to the mkbundle command.

Is there also an alternative way to do this through code ? I'm asking just out of curiosity (and to better understand the mechanisms playing here).

[marek-safar]

By code you mean mkbundle? This is an optional file so it'll be quite hard to add it automatically.

[FredMangelsdorf]

No, I meant is there a way to wire the URI prefix to the class to be instanciated (at least that's what I assume happens through the machine.config file) from c# code.