Cli: Windows Drive Path Casing Issue

[Szandor72]

Hello,

I have a scratch org create script that does a few things including data import.

What I have noticed: The first time I push changes from VS Code to the newly created scratch org, VS Code does a full push. Even if I changed only one file, I get the whole source push which takes quite some time.

In summary, I wait once for a complete push in my script and then have to wait for another full push in VS Code.

Is this behaviour to be expected?

[lcampos]

Hi @Szandor72 , can you clarify something first. Is your scratch org create script also doing a code deploy? If it is, is it using sfdx cli ?

I'm assuming you are running this script on the same machine you are coding using vscode.

[ntotten]

Closing due to lack of response. Please reopen if you can provide more info @Szandor72. :)

[Szandor72]

Sorry for the late response, will not happen again.

Yes the script does a full code / metadata deploy. I just doublechecked, it is done via source:push (not mdapi:deploy). Some parts are deployed via mdapi:deploy after the initial push

Yes the script and vscode run on the same machine, mostly from the same dir as well.

[Szandor72]

We're losing some time around this:

• we have a command line createScratch Org script that pushes code and also some changes via mdapi
• first push from vs code is a complete push as indicated already (a time consuming step for large packages like ours)
• this overwrites our adaptions, so we need to do the mdapi deployment manually after vs code finished its initial push

From there, everything works like a charm again.

[lcampos]

Hi @Szandor72 , sorry for dropping the ball on this question. I've followed the steps you previously described by creating a script that 1) creates a scratch org 2) pushes metadata using source:push 3) deploys more metadata using mdapi:deploy 4) create data. After that I open the project in vscode and I run SFDX: Push Source to Default Scratch Org but I'm not able to replicate what you are running into (the results in VSCode's Output view show that nothing was pushed).

Below are more details to what I've tried so far, can you confirm if I'm missing something ? One thing worth pointing out is that the metadata deployed using mdapi:deploy is not the same as the one used for source:push, which I'm assuming is the same as what you are doing.

Script

sfdx force:org:create -f config/project-scratch-def.json --setalias GH521 --setdefaultusername
sfdx force:source:push -u GH521
sfdx force:mdapi:deploy -d ./md-deploy/src -u GH521
sfdx force:data:tree:import --plan sfdx-out/export-demo-Account-plan.json

Project structure

- config
    - project-scratch-def.json
- force-app
    - main
        - default
            - applications
            - aura
            - classes
            - documents
            - layouts
            - objects
            - pages
            - permissionsets
            - tabs
            - triggers
- md-deploy
    - src
        - classes
        package.xml
- sfdx-out
    - export-demo-Account-plan.json
    - export-demo-Accounts.json

VSCode output when running SFDX: Push Source to Default Scratch Org

sfdx force:source:push
=== Pushed Source
No results found
sfdx force:source:push ended with exit code 0

Details

SFDX cli version: 6.41.1-44a593d6a3
VSCode version: 1.29.1
SalesforceDX VSCode extensions : 44.8.0
OS: Mac OS High Sierra (10.13.6)

[no-response[bot]]

This issue has been automatically closed because there has been no response to our request for more information from the original author. Currently, there is not enough information provided for us to take action. Please reply and reopen this issue if you need additional assistance.

[Szandor72]

Have been on PTO, sorry for the delay.

• Behaviour does persist in current build for my Win10 machine, I just tried
• my script does a source:push (Production Code) and a mdapi:deploy (helper stuff like FeatureParameters) as you did as well.
• my script then runs data imports via talend which I believe should also be unrelated.

After I created the org via script and go to VSCode my first push includes the whole production code.

This is cumbersome since after this initial push, I need to do my mdapi:deploy again as FeatureParameters are overwritten.

Should we do a screen sharing session, @lcampos ?

[Szandor72]

The same occurs if I switch push operations from VSCode to powershell. That is: if I push a change from VS Code, and do a push from powershell after, powershell does a complete push.
If I push something from powershell, then push in VS Code, VS Code does a whole push.

[lcampos]

Hi @Szandor72 , yes let's schedule something to go over what you're experiencing since I'm not able to replicate it on my side. Here's a link (https://calendly.com/lcamposguajardo) with the available days/times I'm free to meet, feel free to schedule something. If non of those times work for you let me know.

[Szandor72]

Done, thank you!

[Szandor72]

Observation: When a full push happens, I need to deploy some additional metadata to get my dev environment working. this happens via powershell and mdapi:deploy. Going back to VS code, I can continue working normally. The mdapi deploy will not trigger another full push.

[lcampos]

@Szandor72 I want to provide an update on this. So far I've been able to replicate this issue on my side. I'm still trying to figure out where the fix needs to be done but I can confirm this is unrelated to you using a powershell scripts since I was able to replicate this using a bash script. I'll post an update on this once I have more details.

[lcampos]

Hi @Szandor72 , I've been able to determine the root cause of this issue being a combination of how the sfdx cli keeps track of local file changes for source:push & source:pull and the way we spawn sfdx cli commands from vscode.

The sfdx cli considers a file's system path as part of the information used to track changes that would have to be considered when doing a source:push or source:pull. I'm seeing the file path's casing change based on how the cli gets spawned e.g. C:\\Users\\myuser\\project\\TestMyCode.cls vs c:\\Users\\myuser\\project\\TestMyCode.cls.

From vscode's side, we spawn cli commands using vscode's workspace api. In the windows scenario, this api returns file paths with the hard drive's letter as lower case. When we trigger a source:push from vscode, it causes the sfdx cli to track the file paths with a lower case. When the sfdx cli is triggered outside of vscode the casing of the hard drive letter is usually capitalized (but that could vary based on how the machine is configured).

In the case described in this issue, this difference in path casing is enough to make the cli consider everything in a project needing to be deployed when a user triggers source:push both inside and outside of vscode for the same project and same scratch org.

In order for us to fix this we'll need to make changes in the sfdx cli in order to handle different system configurations that are now affecting the way we run the file diffs.

cc @ntotten

[Szandor72]

Thanks!

Any nasty, undocumented, all warranty void workaround I could use in the meantime? Because I just pushed > 1000 cmps more times than I'd care for. I need to be very careful what I do where.

[lcampos]

As an undocumented workaround, the salesforce cli uses file sourcePathInfos.json, located under /.sfdx/orgs/, to keep track of local file changes that would have to be considered for deployment when using source:push. You could add a script to your process where it updates the hard drive's casing in that file to the correct one depending on the tool you want to use. VS Code lower cases it while other command line clients usually use upper case (this heavily depends on how the machine was configured).

[Szandor72]

Before writing such a script, I will simply try to rename my hard drive
first.

[berrymd]

I am experiencing this issue as well and have found something that may work as a workaround.

When switching from command line sfdx to running it through VS Code, if you first run
SFDX: View Local Changes
while it will list all local files as a pending change, it will also update the casing in sourcePathInfos.json so that subsequent source related commands detect only items that have actually changed.

The reverse works too, if switching from VS Code to command line sfdx, run
sfdx force:source:status
first before running any other force:source commands

[Szandor72]

Awesome tip! Will try

On Wed, 15 May 2019, 17:44 berrymd, [e-mail] wrote:

I am experiencing this issue as well and have found something that may
work as a workaround.

When switching from command line sfdx to running it through VS Code, if
you first run
SFDX: View Local Changes
while it will list all local files as a pending change, it will also
update the casing in sourcePathInfos.json so that subsequent source related
commands detect only items that have actually changed.

The reverse works too, if switching from VS Code to command line sfdx, run
sfdx force:source:status
first before running any other force:source commands

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/forcedotcom/salesforcedx-vscode/issues/922?email_source=notifications&email_token=AEAGS6XYKFONQYH7J4GPL53PVQVXHA5CNFSM4GSZVOLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVPCS4I#issuecomment-492710257,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEAGS6RITIC5QTRIZLXYL6LPVQVXHANCNFSM4GSZVOLA
.

[Szandor72]

Aha! This is were my issue ended up. :) I am still hoping to get this sorted out.

[nbendjamaa]

Hi, do you know when this issue will be fixed?

[Szandor72]

Since api v49 the situation got worse for me.

The mentioned workaround (view local changes) stopped working for me altogether.

It seems, I am forced to do two full pushes since I am running my setup from a powershell script, that means pushing 2000 components twice - once from my command line, once in VS Code if I want to use it. I am not happy.

[aheber]

It would seem that a way forward here would be for the DX plugin to store relative paths instead of full directory paths. That would drop the drive letter entirely.

This could also have a better workaround if it the VS Code plugins exposed the the new source:tracking:reset command. Running that after the build script from within VS Code could take care of the miss-match.

(There are ways to get your CLI path onto c instead of C but it takes switching drive directories and some other stuff that isn't fun or consistent)

[Szandor72]

Is this issue being actively investigated? It's been two years and I still have to push 2000 Metadata Components twice into scratch orgs.

[lnoellemorgan]

This bug is being tracked by work item: W-6208564

[mdonnalley]

In order to properly fix this, we'd have to normalize file paths across multiple packages and plugins. There are simply too many places that we inspect and compare file paths to implement this safely. Because of that, we're not going to fix this now. However, we may revisit this issue when we migrate the source commands to their own plugin