I have been using the Microsoft ClearScript.V8 library since 2013 and consider it the most reliable and functional a .NET wrapper around the V8 JavaScript engine. I was very happy when Linux and macOS support was implemented in version 7.0 RC, but, unfortunately, ClearScript.V8 library cannot be called fully cross-platform yet. The main problem is as follows: there are two versions of the ClearScript library for .NET Core (one targets to Windows and the other targets to Unix-like operating systems). For this reason, there is currently no universal NuGet package for the ClearScript library, but there are three packages at once:
ClearScript.dll): one for .NET Framework and other for .NET Core. In addition to wrapper for the V8 JavaScript engine, this assembly also contains wrappers for working with the Windows Script Host (supports JScript and VBScript). This package also contains native assemblies for different processor architectures: ClearScriptV8-32.dll and ClearScriptV8-64.dll.ClearScript.dll) and one native assembly (ClearScriptV8.so). The ClearScript.dll assembly has a stripped down functionality compared to version for Windows and uses stubs instead of wrappers around the WSH.ClearScript.dll) and one native assembly (ClearScriptV8.dylib). The ClearScript.dll assembly is exactly same as in the package for Linux.This library distribution system is great for developing desktop applications, but it can cause problems when developing web applications. Web applications can be developed on Windows or Mac computers, and deployed on Linux servers. To avoid these problems, you can use the approach that I used when designing the LibSassHost library and the ChakraCore module of JavaScript Engine Switcher library. The essence of this approach is as follows: we create a main package, which contains a managed assembly, and several additional packages, which contain native assemblies for each platform. Moreover, a managed assembly for .NET Core should not contain any platform-dependent code.
I know that this approach is difficult to implement for a ClearScript library, but in principle it is possible. I managed to make a cross-platform implementation of the ClearScript.V8 library (see the ClearScript-Experimental repository and the clearscript-v8-cross-platform-experimental branch of the JavaScriptEngineSwitcher repository) and even publish a pre-release of the V8 module. Therefore, I suggest that you familiarize yourself with my plan for creating a fully cross-platform version of the ClearScript.V8 library.
Currently, the managed library is a “monolith” in which the V8 is adjacent to the WSH. I don’t know people who would need such different scripting engines at the same time. Therefore, I propose to split the ClearScript.dll assembly into three parts:
ClearScript.V8.dll. Move all types from the Microsoft.ClearScript.V8 namespace to this assembly. In addition, all platform-dependent code should be removed from .NET Core version.ClearScript.Windows.dll. Move all types from the Microsoft.ClearScript.Windows namespace to this assembly.ClearScript.Common.dll. All common types must be placed in this assembly. Version for .NET Core should also not contain platform-dependent code. Accordingly, previous two assemblies will depend on this assembly.Then need to publish each of the listed libraries as a separate NuGet package: Microsoft.ClearScript.V8, Microsoft.ClearScript.Windows and Microsoft.ClearScript.Common.
First of all, need to unify all the assembly names. Rename the native assemblies ClearScriptV8-32.dll and ClearScriptV8-64.dll to ClearScriptV8.dll.
Then need to create four NuGet packages:
ClearScriptV8.dll assembly.ClearScriptV8.dll assembly.ClearScriptV8.so assembly.ClearScriptV8.dylib assembly.Since for the V8 JavaScript engine, we mainly focus on .NET Core, then need to use the runtimes directories and RID catalogs to place assemblies in NuGet packages. To do this, add the following code to .nuspec-files:
Microsoft.ClearScript.V8.Native.win-x86.nuspec
<package …>
…
<files>
…
<file src="path-to-directory-with-native-assembly-for-32-bit-windows/ClearScriptV8.dll" target="runtimes/win-x86/native/" />
…
</files>
…
</package>
Microsoft.ClearScript.V8.Native.win-x64.nuspec
<package …>
…
<files>
…
<file src="path-to-directory-with-native-assembly-for-64-bit-windows/ClearScriptV8.dll" target="runtimes/win-x64/native/" />
…
</files>
…
</package>
Microsoft.ClearScript.V8.Native.linux-x64.nuspec
<package …>
…
<files>
…
<file src="path-to-directory-with-native-assembly-for-64-bit-linux/ClearScriptV8.so" target="runtimes/linux-x64/native/" />
…
</files>
…
</package>
Microsoft.ClearScript.V8.Native.osx-x64.nuspec
<package …>
…
<files>
…
<file src="path-to-directory-with-native-assembly-for-64-bit-macos/ClearScriptV8.dylib" target="runtimes/osx-x64/native/" />
…
</files>
…
</package>
The main advantage of the mechanism based on RID catalogs is automation of the native assembly deployment. Also in this case, we don't need to write any code to load the assemblies into the process, because .NET Core runtime will do it for us.
Now need to implement a mechanism for deploying the native assemblies for .NET Framework. Since we can no longer use the -32 and -64 suffixes for the native assemblies intended for Windows, we will place them in the x86 and x64 subdirectories of the bin\[Debug|Release] directory (for web applications and sites just the bin directory). To do this, create MSBuild scripts:
build/Microsoft.ClearScript.V8.Native.win-x86.props
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Condition=" '$(TargetFramework.TrimEnd(`0123456789.`))' != 'netcoreapp' And '$(TargetFramework)' != 'net5.0' ">
<None Include="$(MSBuildThisFileDirectory)..\runtimes\win-x86\native\ClearScriptV8.dll">
<Link>x86\ClearScriptV8.dll</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<Visible>False</Visible>
</None>
</ItemGroup>
</Project>
build/Microsoft.ClearScript.V8.Native.win-x64.props
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Condition=" '$(TargetFramework.TrimEnd(`0123456789.`))' != 'netcoreapp' And '$(TargetFramework)' != 'net5.0' ">
<None Include="$(MSBuildThisFileDirectory)..\runtimes\win-x64\native\ClearScriptV8.dll">
<Link>x64\ClearScriptV8.dll</Link>
<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
<Visible>False</Visible>
</None>
</ItemGroup>
</Project>
During project building, these scripts copy the native assemblies from RID catalogs to corresponding subdirectories of the bin directory. To avoid such copying when building of .NET Core applications in the ItemGroup element a special condition is specified.
Then need to include these MSBuild scripts in .nuspec-files:
Microsoft.ClearScript.V8.Native.win-x86.nuspec
<package …>
…
<files>
…
<file src="build/Microsoft.ClearScript.V8.Native.win-x86.props" target="build/" />
…
</files>
…
</package>
Microsoft.ClearScript.V8.Native.win-x64.nuspec
<package …>
…
<files>
…
<file src="build/Microsoft.ClearScript.V8.Native.win-x64.props" target="build/" />
…
</files>
…
</package>
Unfortunately, MSBuild scripts are not suitable for all use cases of .NET Framework. For example, ASP.NET 4.X web sites don't have project files and need to use a PowerShell scripts to deploy the native assemblies in the bin directory.
Firstly, need to create the Install.ps1 files that run during the installation of NuGet packages. These scripts makes corresponding subdirectories in the bin directory, and then copies the native assemblies there:
tools/Install.ps1 for the Microsoft.ClearScript.V8.Native.win-x86 package
param($installPath, $toolsPath, $package, $project)
if ($project.Type -eq "Web Site") {
$runtimeDirectoryPath = Join-Path $installPath "runtimes/win-x86/"
$projectDirectoryPath = $project.Properties.Item("FullPath").Value
$binDirectoryPath = Join-Path $projectDirectoryPath "bin"
$assemblyFileName = "ClearScriptV8.dll"
$assemblyDestDirectoryPath = Join-Path $binDirectoryPath "x86"
if (!(Test-Path $assemblyDestDirectoryPath)) {
New-Item -ItemType Directory -Force -Path $assemblyDestDirectoryPath
}
$assemblySourceFilePath = Join-Path $runtimeDirectoryPath ("native/" + $assemblyFileName)
Copy-Item $assemblySourceFilePath $assemblyDestDirectoryPath -Force
}
tools/Install.ps1 for the Microsoft.ClearScript.V8.Native.win-x64 package
param($installPath, $toolsPath, $package, $project)
if ($project.Type -eq "Web Site") {
$runtimeDirectoryPath = Join-Path $installPath "runtimes/win-x64/"
$projectDirectoryPath = $project.Properties.Item("FullPath").Value
$binDirectoryPath = Join-Path $projectDirectoryPath "bin"
$assemblyFileName = "ClearScriptV8.dll"
$assemblyDestDirectoryPath = Join-Path $binDirectoryPath "x64"
if (!(Test-Path $assemblyDestDirectoryPath)) {
New-Item -ItemType Directory -Force -Path $assemblyDestDirectoryPath
}
$assemblySourceFilePath = Join-Path $runtimeDirectoryPath ("native/" + $assemblyFileName)
Copy-Item $assemblySourceFilePath $assemblyDestDirectoryPath -Force
}
Secondly, need to create the Uninstall.ps1 files that run during the uninstallation of NuGet packages. These scripts remove previously copied the native assemblies:
tools/Uninstall.ps1 for the Microsoft.ClearScript.V8.Native.win-x86 package
param($installPath, $toolsPath, $package, $project)
if ($project.Type -eq "Web Site") {
$projectDirectoryPath = $project.Properties.Item("FullPath").Value
$binDirectoryPath = Join-Path $projectDirectoryPath "bin"
$assemblyFileName = "ClearScriptV8.dll"
$assemblyDirectoryPath = Join-Path $binDirectoryPath "x86"
$assemblyFilePath = Join-Path $assemblyDirectoryPath $assemblyFileName
if (Test-Path $assemblyFilePath) {
Remove-Item $assemblyFilePath -Force
}
}
tools/Uninstall.ps1 for the Microsoft.ClearScript.V8.Native.win-x64 package
param($installPath, $toolsPath, $package, $project)
if ($project.Type -eq "Web Site") {
$projectDirectoryPath = $project.Properties.Item("FullPath").Value
$binDirectoryPath = Join-Path $projectDirectoryPath "bin"
$assemblyFileName = "ClearScriptV8.dll"
$assemblyDirectoryPath = Join-Path $binDirectoryPath "x64"
$assemblyFilePath = Join-Path $assemblyDirectoryPath $assemblyFileName
if (Test-Path $assemblyFilePath) {
Remove-Item $assemblyFilePath -Force
}
}
Thirdly, then need to include these PowerShell scripts in .nuspec-files:
Microsoft.ClearScript.V8.Native.win-x86.nuspec and Microsoft.ClearScript.V8.Native.win-x64.nuspec
<package …>
…
<files>
…
<file src="tools/Install.ps1" target="tools/" />
<file src="tools/Uninstall.ps1" target="tools/" />
…
</files>
…
</package>
After creating packages with the native assemblies, I recommend that you implement an handler for the DllNotFoundException exception in a .NET wrapper around the V8 JavaScript engine. This handler should notify the developer about need to install NuGet packages with the native assemblies (see the example).
An example of implementation of packages that contains the native assemblies can be seen in the clearscript-v8-cross-platform-experimental branch of the JavaScriptEngineSwitcher repository.
In order to use the native assemblies that loaded by using the mechanism based on RID catalogs, DllImport attributes must use universal assembly names (without the lib prefix and file extension):
[DllImport("ClearScriptV8", …)]
Therefore, we should remove the V8SplitProxyNative.Windows.cs, V8SplitProxyNative.Windows.32.cs and V8SplitProxyNative.Windows.64.cs files and use the V8SplitProxyNative.Unix.cs file as the primary file for all versions of the ClearScript.V8 managed library.
Since we no longer load the native assemblies manually, then you also need to remove the V8Proxy.Windows.cs and V8Proxy.Unix.cs files.
To load the native assemblies in version for .NET Framework, you need to create a V8AssemblyResolver class that uses the SetDllDirectory method of Win32 API to specify the directory from which the assembly should be loaded. The Initialize method of V8AssemblyResolver class must be called before the first access to one of the native assembly methods (see the V8Proxy.cs file).
All the listed changes in the ClearScript.V8 managed library can be found in the ClearScript-Experimental repository.
Since many existing projects use the Microsoft.ClearScript package, you can convert it to metapackage with the following dependencies:
UPDATE: In the second step added a information about PowerShell scripts.
Hi Andrey,
Thanks for your recommendations. We'll take a closer look, but first we'd like to get a better understanding of the problems you're seeing with our current offerings.
You mentioned Windows/macOS development and Linux deployment, but we aren't clear on the advantages of your proposal in that scenario.
With our approach, deployment on Linux – like deployment on any of the supported platforms – requires one OS-specific package. With yours, if we understand it correctly, the same deployment would require an OS-specific package and two cross-platform packages. Why is that preferable? Sorry if we're missing something obvious.
We do agree that aspects of our code and package breakdown are less than optimal at the moment. You're right that most users don't need both V8 and Windows script engines. We're not happy with with the three similar versions of V8SplitProxyNative.cs either and are considering merging the Unix version with the 64-bit Windows version in an upcoming release.
We're also evaluating the possibility of offering one more NuGet package – a cross-platform V8-only ClearScript assembly and all the native V8 assemblies in one place – for convenience.
But again, since all these combinations add up to the same API and ABI, we aren't sure that any major effort along these lines is really worth it. What are we missing?
Thanks again!
Hello!
It seems that you misunderstood me.
At first, I will try to explain the disadvantages of your approach from the point of view of a library developer. I have the JavaScriptEngineSwitcher.V8 library that acts as an adapter for the V8 JavaScript engine (a kind of analogue of specific DBMS provider for ORM) from the Microsoft ClearScript library. Suppose I want to make it compatible with all three operating systems: Windows, Linux and macOS.
If I will use your NuGet packages from version 7.0 RC3, then I also need to create three packages:
JavaScriptEngineSwitcher.V8.win-x86-x64
|--Microsoft.ClearScript
…
JavaScriptEngineSwitcher.V8.linux-x64
|--Microsoft.ClearScript.linux-x64
…
JavaScriptEngineSwitcher.V8.osx-x64
|--Microsoft.ClearScript.osx-x64
…
If you organize NuGet packages according to the approach I suggested:
Packages with the managed assemblies
====================================
Microsoft.ClearScript.Common
Microsoft.ClearScript.Windows
|--Microsoft.ClearScript.Common
Microsoft.ClearScript.V8
|--Microsoft.ClearScript.Common
Packages with the native assemblies
===================================
Microsoft.ClearScript.V8.Native.win-x86
Microsoft.ClearScript.V8.Native.win-x64
Microsoft.ClearScript.V8.Native.linux-x64
Microsoft.ClearScript.V8.Native.osx-x64
Then I will have to create only one package:
JavaScriptEngineSwitcher.V8
|--Microsoft.ClearScript.V8
|--Microsoft.ClearScript.Common
And developers using my library will decide for themselves which packages with the native assemblies they need to install.
Now let's look at both approaches from a web developer's point of view. Suppose a web developer develops a ASP.NET Core 3.1 application on a computer running 64-bit Windows 10, and then deploys it on a server running Ubuntu 20.04.1. To perform some task, web developer needed the V8 JavaScript engine.
If using current version of the ClearScript library, web developer will need to install the Microsoft.ClearScript package. When task is completed and need to deploy a new version of web application on the server, then will need to perform the following steps:
Now consider the variant where NuGet packages are organized according to new approach. Web developer immediately installs three packages: Microsoft.ClearScript.V8, Microsoft.ClearScript.V8.Native.win-x64 and Microsoft.ClearScript.V8.Native.linux-x64. And when the deployment stage comes, web developer does not need to perform any additional steps.
In case a web developer uses the ClearScript library not directly, but as dependency of another library and wants to use a newer version of the V8, then can simply build a new version of the native assemblies and create own NuGet packages based on them. This eliminates the need to rebuild the managed assemblies of ClearScript library and third-party library.
Hi Andrey,
Thanks for your informative response. A few more questions:
If you organize NuGet packages according to the approach I suggested: [...] Then I will have to create only one package:
We'd like to facilitate your scenario, but we'd also like to avoid an explosion of NuGet packages. What if we replaced the current non-Windows packages with a single new package, perhaps Microsoft.ClearScript.Core, that was cross-platform but self-sufficient – a portable V8-only ClearScript assembly plus all the native files in one package – would that work for you?
And when the deployment stage comes, web developer does not need to perform any additional steps.
To help us optimize and test this scenario, could you describe the method of cross-platform deployment you're envisioning?
One other thing. For various reasons, we'd prefer _not_ to use the SetDllDirectory scheme, even it means adding a suffix to the native non-Windows assembly names. Do you see any problems with that?
Thanks again!
In the second step added a information about PowerShell scripts.
What if we replaced the current non-Windows packages with a single new package, perhaps Microsoft.ClearScript.Core, that was cross-platform but self-sufficient – a portable V8-only ClearScript assembly plus all the native files in one package – would that work for you?
Of course, this is also a good variant, but why would a developer download the native assemblies that he does not need?
… but we'd also like to avoid an explosion of NuGet packages.
I do not see this as a particular problem, since greatest amount of work occurs during the initial creation of NuGet package. Then you just need to update a version numbers and release notes.
To help us optimize and test this scenario, could you describe the method of cross-platform deployment you're envisioning?
Standard deployment of ASP.NET Core web application. I think there will still be people who can write about their experience of deploying web applications in Docker containers.
One other thing. For various reasons, we'd prefer not to use the SetDllDirectory scheme, even it means adding a suffix to the native non-Windows assembly names. Do you see any problems with that?
If you leave suffixes in assembly names, you will not be able to take advantage of automatic loading of the native assemblies by using the mechanism based on runtimes directories and RID catalogs. And in this case, you will have to write own code to load the native assemblies, which will make the managed assemblies for .NET Core is platform-dependent.
Hi Andrey,
Of course, this is also a good variant, but why would a developer download the native assemblies that he does not need?
ClearScript has always shipped both 32-bit and 64-bit mixed and native assemblies. It's an extension of the recommended practice of bundling managed libraries for multiple .NET runtimes in the same package. Since we aren't talking about multi-gigabyte files, it's a tradeoff that makes sense.
I do not see this as a particular problem, since greatest amount of work occurs during the initial creation of NuGet package.
By minimizing the package count, we get a simpler integration and deployment story, a smaller test and support matrix, and a more predictable deployment environment. Unless they're particularly huge, we see no problem with bundling files that may go unused.
However, we agree that there's a problem when multiple packages contain the _same_ file – especially if it's different variants of the same file – if developers might want to combine those packages. This is definitely a problem with ClearScript 7.0 RC, and we thank you for bringing it to our attention.
If you leave suffixes in assembly names, you will not be able to take advantage of automatic loading of the native assemblies by using the mechanism based on runtimes directories and RID catalogs.
Hmm, could you clarify this? Do you actually mean loading assemblies at runtime, or are you talking about deployment? Not being experts in cross-platform web deployment, we're not aware of the mechanism to which you're referring or of any deployment issues with our current packages.
And in this case, you will have to write own code to load the native assemblies, which will make the managed assemblies for .NET Core is platform-dependent.
We have to have that code anyway for historical reasons, but the native binding now happens via DllImportAttribute, which is cross-platform. We'll need both 32-bit and 64-bit entry points and invocation paths, as we have today on Windows, but the ClearScript assembly can still be portable.
Given our aggregate concerns, we're leaning toward the following plan:
Please send us your thoughts!
Hello!
Hmm, could you clarify this? Do you actually mean loading assemblies at runtime, or are you talking about deployment? Not being experts in cross-platform web deployment, we're not aware of the mechanism to which you're referring …
I refer to this mechanism and also to this aspect of creating NuGet packages.
Hello!
I recommend that you look at the Alexandre Mutel's SharpScss library. Currently, this library is only compatible with .NET Core. Using this library, you can understand how to work a automatic deploying and loading of the native assemblies based on RID mechanism in .NET Core (see the “Runtime” section of the readme.md file).
Prior to version 1.5.0, compatibility with the .NET Framework was implemented by using a special MSBuild script.
Thanks, Andrey. Can you confirm that JavaScriptEngineSwitcher will be able to use Microsoft.ClearScript.Core as described above across platforms without much difficulty?
It is difficult to say anything without seeing a concrete implementation. Current version of the JavaScriptEngineSwitcher.V8 module implements three deployment scenarios at once:
I implemented support for these deployment scenarios in my packages long before the ClearScript library had an official NuGet package (support for the RID mechanism appeared in version 2.0.0, and extraction of the native assemblies into separate packages occurred in version 2.1.0).
I think that if the Microsoft.ClearScript.Core for .NET Core will rely on the RID mechanism, and in the version for .NET Framework it will be possible to override the mechanism for loading the native assemblies, then I can build my own versions of packages without any problems.
It is difficult to say anything without seeing a concrete implementation.
Could we send you packages to test before we publish them? We'd like very much to provide cross-platform support for JSES.V8.
I implemented support for these deployment scenarios in my packages long before the ClearScript library had an official NuGet package [...].
Understood. Out of curiosity, when you prepare a new JSES.V8 release, do you extract binaries from our NuGet package, or do you build ClearScript yourself? We're asking about your normal, preferred procedure, not about the fork you're using for the current preview release.
Also, we'd like to be absolutely clear about the issue(s) preventing you from using our mainline code and/or NuGet packages to build cross-platform JSES.V8. The root issue, as we understand it, is that there's currently no cross-platform version of the managed ClearScript assembly. Is that correct?
Understood. Out of curiosity, when you prepare a new JSES.V8 release, do you extract binaries from our NuGet package, or do you build ClearScript yourself? We're asking about your normal, preferred procedure, not about the fork you're using for the current preview release.
For stable version, I build all the assemblies myself, because I sign the managed assemblies with my own key (it happened historically).
For cross-platform preview version, I built the managed assemblies and the native assemblies for Windows myself. I took the native assemblies for Linux and macOS from official packages, because I couldn't build them myself.
The root issue, as we understand it, is that there's currently no cross-platform version of the managed ClearScript assembly. Is that correct?
This is one of the main reason, but there are others: usage of separate packages for the native assemblies, own scenarios for deploying and loading the native assemblies.
ClearScript 7.0 follows the recommended project and package breakdown. Thanks again!
Thanks! I will try to make a new release of the JavaScript Engine Switcher in the coming days.