Botframework-solutions: appsettings.json read error on macos

[nathanlindorff]

What project is affected?

Virtual Assistant Template

What language is this in?

TypeScript

What happens?

When running npm run start the app crashes with the error SyntaxError: <project path>/lib/appsettings.json: Unexpected token � in JSON at position 0

What are the steps to reproduce this issue?

Generate with yo template
add code to workaround #1923
run deploy.ps1 & publish.ps1
run npm run start

What were you expecting to happen?

Virtual assistant to start

Can you share any logs, error output, etc.?

> [email protected] copy-templates <project path>
> copyfiles --up 1 "./src/**/*.json" "./lib"

internal/modules/cjs/loader.js:723
    throw err;
    ^

SyntaxError: <project path>/lib/appsettings.json: Unexpected token � in JSON at position 0
    at JSON.parse (<anonymous>)
    at Object.Module._extensions..json (internal/modules/cjs/loader.js:720:27)
    at Module.load (internal/modules/cjs/loader.js:600:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
    at Function.Module._load (internal/modules/cjs/loader.js:531:3)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at Module.patchedRequire [as require] (<project path>/node_modules/diagnostic-channel/dist/src/patchRequire.js:14:46)
    at require (internal/modules/cjs/helpers.js:22:18)
    at Object.<anonymous> (<project path>/lib/index.js:26:34)
    at Module._compile (internal/modules/cjs/loader.js:701:30)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `npm run build && node ./lib/index.js NODE_ENV=development`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/<username>/.npm/_logs/2019-08-01T00_21_58_214Z-debug.log

Any screenshots or additional context?

Running on Macos

I was able to fix this by changing the encoding of the json file from Unicode (UTF-16 Little-Endian) to Unicode (UTF-8).

[nathanlindorff]

I think it may be Powershell writing in that format, as I had to re-run the deploy, and the appsettings was rewritten as 'Unicode (UTF-16 Little-Endian)' (According to TextWrangler)

[darrenj]

Thanks for reporting and advising on the solution. We'll take a look on macos.

[Batta32]

Thanks @nathanlindorff for reporting the issue!

Which version of PowerShell are you using? If it's not the latest, can you download it and try again?

Also, if you want to use the changes from master install the latest _preview_ version of the generator-botbuilder-assistant, following this steps:

npm uninstall --global generator-botbuilder-assistant
npm install --global [email protected] --registry https://botbuilder.myget.org/F/aitemplates/npm

We have been researching about the encoding for the cmdlets and we found out that between PowerShell and PowerShell Core (for cross platforms) exists inconsistences in the encoding for the cmdlets, like in Out-File which creates the files in UTF-16LE.

Please, read this answer for further information.

@darrenj should we update the PowerShell scripts adding the -Encoding UTF8 argument for those PowerShell commands which handle cognitivemodels.json and appsettings.json files?
The encoding of the following cmdlets were affected in the latest version of PowerShell Core.

[nathanlindorff]

I am running PowerShell Core v6.0.0-beta.3 on MacOS.

I'll run the upgrade for powershell and the template, and let you know.

[nathanlindorff]

The issue looks to be fixed. I ran through a complete generation using the preview and the json files were written as UTF-8. Thanks very much!