Meshcentral: Problem with chat

[rmgrmg]

Hi I'm having a problem with the chat function. I've recently upgraded and am running the latest version of meshcentral, but this was an issue I was experiencing with earlier versions as well.

I am using Meshcentral with a caddy server, and am forwarding from caddy server to an internal IP, forwarding port 4430. The entry in the caddy server is

mesh.xyz.com {
reverse_proxy 192.168.1.223:4430 {
}
}

All is working well except the chat feature. When chosen from the console, a browser window opens, with "Waiting for the other user..." message, but nothing appears on the target PC. After a short period the message on the console webpage indicates "connection closed."

The anonymized config.json file for mesh is as follows
{
"settings": {
"selfupdate": true,
"Cert": "mesh.xyz.com",
"Port": 4430,
"AliasPort": 443,
"RedirPort": 81,
"TlsOffload": "192.168.1.223",
"DesktopMultiplex": true
},
"domains": {
"": {
"certurl": "https://mesh.xyz.comt",
"title2": "",
"footer": "my footer",
"_welcomeText": "Sample Text on Login Page.",
"newaccounts": false
}
}
}

Might you know why I'm experiencing this?

[LPJon]

I have been randomly seeing this as well. Seems that sometimes it works and sometimes it doesn't. Mainly what I notice is that no browser is opened on the client system. I think this is related to antivirus software or maybe even windows defender. I assume that the command line execution is stopped for some reason but I really have no idea. Try shutting down all antivirus security and windows security and see if the browser window opens for you. If it does....well then we know it's security related.

[rmgrmg]

AV is just windows 10 defender on test machine. Disabling as much as possible, including turning off public, private and domain firewalls and still no go on the chat. However, I did try it on a server that has no AV (running windows server 2012 I believe) and the chat page did come up, so I agree it may be something client related, However making it as vanilla as possible on a test win10 PC but still same results with no chat window.

[LPJon]

AV is just windows 10 defender on test machine. Disabling as much as possible, including turning off public, private and domain firewalls and still no go on the chat. However, I did try it on a server that has no AV (running windows server 2012 I believe) and the chat page did come up, so I agree it may be something client related, However making it as vanilla as possible on a test win10 PC but still same results with no chat window.

Have you tried setting a default app for the web browser in Windows 10?

[krayon007]

Is the logged in user on that machine an active directory user?

[LPJon]

Is the logged in user on that machine an active directory user?

@krayon007 In my case the ones that work yes. The ones that don't are stand alone machines. On the stand alone machines, setting the default web browser seems to fix this issue.

[rmgrmg]

Thanks for the suggestions. This is a machine that is connected to Azure AD. The user logged in is the global administrator, and I've tried setting the default browser to IE, Chrome and Edge. Unfortunately still striking out.

[Ylianst]

When you hit the "Chat" button, the agent is told to open a web browser to the remote device for the currently logged in users with a special URL that connects back to MeshCentral. If the agent can't open the URL because no users are logged it, nothing will happen. No sure all the situations where the browser window can't open on the remote device.

[rmgrmg]

A user is logged in on the client. With your explanation of the this function in mind, I tested the option for opening a webpage on the client, which also fails to bring up a web page.

[LPJon]

When you hit the "Chat" button, the agent is told to open a web browser to the remote device for the currently logged in users with a special URL that connects back to MeshCentral. If the agent can't open the URL because no users are logged it, nothing will happen. No sure all the situations where the browser window can't open on the remote device.

@Ylianst This is happening with a user that is logged in. It's as though windows does not know how, or with what app, to open the URL. Maybe specifying, or the ability to specify, a specific browser app installed in it's default location would help this problem. For instance, specifying an application path such as%programfiles%\mozilla firefox\firefox.exe https://meshcentral.domain/ this would open firefox 64bit or 32bit based on the OS with system variables. Or %programfiles(x86)%\Microsoft\Edge\Application\msedge.exe https://meshcentral.domain/ for Microsoft Edge on Windows 10 x64. Of course for the x32 version it would be %programfiles%

Another option may be to set a default app for the HTTPS protocol in Windows 10

Just trying to help not tell you how to come at this.....

[krayon007]

Another issue I have seen that would prevent this, that I still cannot explain why or how to fix, is the following...

I use the windows task scheduler to spawn a process onto the user's desktop session, as I cannot use an impersonation token, because the process needs to be owned by the user in order for the proxy/bookmarks/etc to work correctly.

Anyways, I have found that on some systems where the logged in user is part of a joined domain, the task scheduler fails to spawn the process... The part that I cannot explain is this:

On some domain systems, the task scheduler requires me to specify the fully qualified domain name, such as: mydomain\\user while on others it will only work if I do the opposite and omit the domain name, like: user

The closest I've come is that I think on the systems where it doesn't work, the domain name and netbios name does not match. For example, if the AD domain is "MyDomain.com", but the netbios name is "MyWorkDomain". I have not been able to find (or reproduce) a consistent scenario when this will fail, as many people with domain setups report that it works fine, so I am not certain what configuration causes this problem.

To test, open a terminal to the target windows machine, and type the following two commands, substituting XXX with the username of the logged in user.... You can try both: XXX and domain\xxx
If it works, it should popup a command prompt on the user's desktop. From what I can tell, when it fails, the task creation will return an error.

SCHTASKS /CREATE /TN MyTest /SC ONCE /ST 00:00 /RU XXXX /TR "C:\Windows\System32\Cmd.exe"
SCHTASKS /RUN /TN MyTest

[LPJon]

Another issue I have seen that would prevent this, that I still cannot explain why or how to fix, is the following...

I use the windows task scheduler to spawn a process onto the user's desktop session, as I cannot use an impersonation token, because the process needs to be owned by the user in order for the proxy/bookmarks/etc to work correctly.

Anyways, I have found that on some systems where the logged in user is part of a joined domain, the task scheduler fails to spawn the process... The part that I cannot explain is this:

On some domain systems, the task scheduler requires me to specify the fully qualified domain name, such as: mydomain\\user while on others it will only work if I do the opposite and omit the domain name, like: user

The closest I've come is that I think on the systems where it doesn't work, the domain name and netbios name does not match. For example, if the AD domain is "MyDomain.com", but the netbios name is "MyWorkDomain". I have not been able to find (or reproduce) a consistent scenario when this will fail, as many people with domain setups report that it works fine, so I am not certain what configuration causes this problem.

To test, open a terminal to the target windows machine, and type the following two commands, substituting XXX with the username of the logged in user.... You can try both: XXX and domainxxx
If it works, it should popup a command prompt on the user's desktop. From what I can tell, when it fails, the task creation will return an error.

SCHTASKS /CREATE /TN MyTest /SC ONCE /ST 00:00 /RU XXXX /TR "C:\Windows\System32\Cmd.exe"
SCHTASKS /RUN /TN MyTest

@krayon007 Running your test in a "User Shell" on a non domain connected Windows 10 20H2 system originally failed. However, the username had a space in it. When I quoted the username everything worked perfectly. Is it possible that the times this is not working is when there is an un-quoted space in the username when this command is run? Most domain names do not have spaces in the usernames but I know some that do. I do not believe that when ran locally on a domain connected machine you must specify the domain at all as it will should always look to AD for the domain user context.

[krayon007]

Another issue I have seen that would prevent this, that I still cannot explain why or how to fix, is the following...
I use the windows task scheduler to spawn a process onto the user's desktop session, as I cannot use an impersonation token, because the process needs to be owned by the user in order for the proxy/bookmarks/etc to work correctly.
Anyways, I have found that on some systems where the logged in user is part of a joined domain, the task scheduler fails to spawn the process... The part that I cannot explain is this:
On some domain systems, the task scheduler requires me to specify the fully qualified domain name, such as: mydomain\\user while on others it will only work if I do the opposite and omit the domain name, like: user
The closest I've come is that I think on the systems where it doesn't work, the domain name and netbios name does not match. For example, if the AD domain is "MyDomain.com", but the netbios name is "MyWorkDomain". I have not been able to find (or reproduce) a consistent scenario when this will fail, as many people with domain setups report that it works fine, so I am not certain what configuration causes this problem.
To test, open a terminal to the target windows machine, and type the following two commands, substituting XXX with the username of the logged in user.... You can try both: XXX and domainxxx
If it works, it should popup a command prompt on the user's desktop. From what I can tell, when it fails, the task creation will return an error.
SCHTASKS /CREATE /TN MyTest /SC ONCE /ST 00:00 /RU XXXX /TR "C:\Windows\System32\Cmd.exe"
SCHTASKS /RUN /TN MyTest

@krayon007 Running your test in a "User Shell" on a non domain connected Windows 10 20H2 system originally failed. However, the username had a space in it. When I quoted the username everything worked perfectly. Is it possible that the times this is not working is when there is an un-quoted space in the username when this command is run? Most domain names do not have spaces in the usernames but I know some that do. I do not believe that when ran locally on a domain connected machine you must specify the domain at all as it will should always look to AD for the domain user context.

Interesting, I did not know usernames could have spaces in it. I'll make sure they are quoted, and have ylian push an update, to see if this resolves the issue for you. I know for the cases that were brought to my attention so far, did not have spaces in the domain or user names. Thanks for catching this issue!

[krayon007]

On a side note, I have also seen issues when there are name collisions between local and remote user names. And in those cases, sometimes specifying the domain fixes it, and other times not....

[krayon007]

@LPJon I updated meshcore.js in the server repo, so @Ylianst should be able to push it out with the next server push, so that you can test to see if this resolves your issues.

[LPJon]

On a side note, I have also seen issues when there are name collisions between local and remote user names. And in those cases, sometimes specifying the domain fixes it, and other times not....

@krayon007 That is 100% correct! Name collisions are difficult because the local machine names the users domain folder differently locally and this seems to also change the local username in an AD situation. Even though the domain name is different in AD some how the local machine associates the two locally to sort out the collision. E.g. AD name is pjohnson so instead of creating the local user folder pjohnson, because that user already exists locally, it will name the folder pjohnson.DomainServerName and it's reference will be DomainServerNamepjohnson. Does that track?

[krayon007]

On a side note, I have also seen issues when there are name collisions between local and remote user names. And in those cases, sometimes specifying the domain fixes it, and other times not....

@krayon007 That is 100% correct! Name collisions are difficult because the local machine names the users domain folder differently locally and this seems to also change the local username in an AD situation. Even though the domain name is different in AD some how the local machine associates the two locally to sort out the collision. E.g. AD name is pjohnson so instead of creating the local user folder pjohnson, because that user already exists locally, it will name the folder pjohnson.DomainServerName and it's reference will be DomainServerNamepjohnson. Does that track?

That could explain a bunch of things with the apparent irregularity. Thanks! On a side note, I forgot to mention that I recently fixed an issue where the scheduled tasks did not run if you were on battery power. That fix wasn't applied to the chat window thing, because that particular scheduled task was not created using my dispatcher module. I'll go fix that now...

[LPJon]

On a side note, I have also seen issues when there are name collisions between local and remote user names. And in those cases, sometimes specifying the domain fixes it, and other times not....

@krayon007 That is 100% correct! Name collisions are difficult because the local machine names the users domain folder differently locally and this seems to also change the local username in an AD situation. Even though the domain name is different in AD some how the local machine associates the two locally to sort out the collision. E.g. AD name is pjohnson so instead of creating the local user folder pjohnson, because that user already exists locally, it will name the folder pjohnson.DomainServerName and it's reference will be DomainServerNamepjohnson. Does that track?

That could explain a bunch of things with the apparent irregularity. Thanks! On a side note, I forgot to mention that I recently fixed an issue where the scheduled tasks did not run if you were on battery power. That fix wasn't applied to the chat window thing, because that particular scheduled task was not created using my dispatcher module. I'll go fix that now...

@krayon007 A useful way of sorting the Username thing out may be to specify the pc's name in place of the AD domain name when it's not a domain tied machine and specify the domain name when it is with the quotes on the username in either case.

[krayon007]

On a side note, I have also seen issues when there are name collisions between local and remote user names. And in those cases, sometimes specifying the domain fixes it, and other times not....

@krayon007 That is 100% correct! Name collisions are difficult because the local machine names the users domain folder differently locally and this seems to also change the local username in an AD situation. Even though the domain name is different in AD some how the local machine associates the two locally to sort out the collision. E.g. AD name is pjohnson so instead of creating the local user folder pjohnson, because that user already exists locally, it will name the folder pjohnson.DomainServerName and it's reference will be DomainServerNamepjohnson. Does that track?

That could explain a bunch of things with the apparent irregularity. Thanks! On a side note, I forgot to mention that I recently fixed an issue where the scheduled tasks did not run if you were on battery power. That fix wasn't applied to the chat window thing, because that particular scheduled task was not created using my dispatcher module. I'll go fix that now...

@krayon007 A useful way of sorting the Username thing out may be to specify the pc's name in place of the AD domain name when it's not a domain tied machine and specify the domain name when it is with the quotes on the username in either case.

That's the original approach I had, but found that failed on some people's setups. Granted, I didn't have quotes at the time... I'll have to give that another go around.

[LPJon]

@krayon007 I have also noticed another issue. When a Windows 10 machine has "Internet Explorer" setup as it's default browser (often for compatibility issues) the chat window will come up but not function. Is there a way to force the default of Microsoft Edge in this type of case scenario?

[rmgrmg]

Here's my update from testing - When logged in as an AD (Azure AD) user/administrator, the chat window will not come up. On the same machine, if I log in as a local user, the chat window comes up fine. Both of these accounts are single word user names, and both have full access to the computer. So it does seem to be entwined somehow in the particular user account used, i.e. local vs AAD account.

[LPJon]

Here's my update from testing - When logged in as an AD (Azure AD) user/administrator, the chat window will not come up. On the same machine, if I log in as a local user, the chat window comes up fine. Both of these accounts are single word user names, and both have full access to the computer. So it does seem to be entwined somehow in the particular user account used, i.e. local vs AAD account.

@rmgrmg In your example test, the administrator account IS an account collision because there is a local administrator account and an AD administrator account. @krayon007 this is starting more and more to sound like the account collision issue you described. So maybe it's a combination of the two like we think it might be. Account collisions and quoted usernames.

[rmgrmg]

Are they really "colliding" if there are different names? Is the collision simply because on might access the same pc with different names, both being administrator accounts on the same machine? The user folders created/present under c:user are simple enough, no extended domains, just simply the user name, e.g. localuser and domainuser

[LPJon]

Are they really "colliding" if there are different names? Is the collision simply because on might access the same pc with different names, both being administrator accounts on the same machine? The user folders created/present under c:user are simple enough, no extended domains, just simply the user name, e.g. localuser and domainuser

@rmgrmg I think I mis-read...sorry. I originally understood it as the "Administrator" account not the administrator account context. Did you try running @krayon007 's test using:
SCHTASKS /CREATE /TN MyTest /SC ONCE /ST 00:00 /RU "XXXX" /TR "C:\Windows\System32\Cmd.exe"
SCHTASKS /RUN /TN MyTest

Where "XXXX" is the local username and "domainXXXX" is the AD username including the quotes? If not, can you give that a try in the same testing scenario you previously used and see if it makes a difference? This should create a task in Task Scheduler called "MyTest" and then try to run the task and bring up a simple command shell. You'd need to test it in both a user context shell and an administrator context shell. Once your done testing the following command will delete the created testing task "MyTask":
SCHTASKS /DELETE /TN MyTest

[krayon007]

Are they really "colliding" if there are different names? Is the collision simply because on might access the same pc with different names, both being administrator accounts on the same machine? The user folders created/present under c:user are simple enough, no extended domains, just simply the user name, e.g. localuser and domainuser

@rmgrmg I think I mis-read...sorry. I originally understood it as the "Administrator" account not the administrator account context. Did you try running @krayon007 's test using:
SCHTASKS /CREATE /TN MyTest /SC ONCE /ST 00:00 /RU "XXXX" /TR "C:\Windows\System32\Cmd.exe"
SCHTASKS /RUN /TN MyTest

Where "XXXX" is the local username and "domainXXXX" is the AD username including the quotes? If not, can you give that a try in the same testing scenario you previously used and see if it makes a difference? This should create a task in Task Scheduler called "MyTest" and then try to run the task and bring up a simple command shell. You'd need to test it in both a user context shell and an administrator context shell. Once your done testing the following command will delete the created testing task "MyTask":
SCHTASKS /DELETE /TN MyTest

This was actually one of the exact use cases that was brought to my attention... AD domain admin conflicting with local admin. The person that brought it up was even stumped by it, as he eventually got it to work, but the solution he used didn't work with his other AD user accounts... You may be able to find the conversation on GIT if you search for it.

[flyRV]

You may be able to find the conversation on GIT if you search for it.

Maybe it was me? Just went searching again to see if there was any update on this and found this thread that appears to be discussing the same issue but has some newer activity.

It's been a few months since I've dealt with it or tried any further testing but in general, my issue was replicated by having a local admin user "administrator" match the same username as a domain user "[email protected]" and I was able to "fix" by renaming the local "administrator" user. More details in that previous link. I'll update this clients instance and do some additional testing but imagine it's still an on-going issue.

[edit]
As of 2/27/2021 and version 0.7.78 the issue remains. When logged into a device as [email protected]_ when a local/built-in user of the same name _administrator_ exists, I cannot launch a chat session. Rename the local/built-in user to something different like _administrator1_ and the chat session launches immediately.