Themekit: Theme.lock preventing uploads unnecessarily

[joshiefishbein]

What happened

409 Conflict: There have been changes to this file made remotely.

Honestly this theme.lock has caused significantly more trouble that it's saved since I've started using this new recent update. In multiple stores I am having trouble with both multiple-developer workflows and single-developer workflows on themes. theme.lock is doing the opposite of what it is (apparently) designed to do: prevent conflicts. Instead, it causes conflicts as we have to pull down individual files that it wrongly has issues with and re-apply changes.

Thank goodness for git keeping a more reliable record of what/when changes occur.

What I expected to happen

Files to upload.

Steps to reproduce

1. Update to latest version of ThemeKit
2. Duplicate master/published theme and run as a development env in themekit
3. Make change to a compiled/dist asset
4. Publish theme
5. Repeat step 2 and 3 to find the 409 error when theme kit is watching for changes to assets

My Environment

OS (including version): macOS Sierra (10.12.6)
Themekit version: 0.7.5
Ignore Files(if any):
- src/
- emails/
- node_modules/
- bower_components
- bower.json
- package.json
- Gruntfile.js

[joshiefishbein]

I'm seeing a few other issues on this subject, all of which are closed for some reason.

As a technical director it's been hard to recommend Shopify for enterprise because of this very reason: enterprise tools on Shopify have been half baked if baked at all. The theme.lock feature is exemplary of this major problem. All the while Shopify still pushes hard on gaining Magento, DemandWare, Spree/Solidus clients. Dev shops need enterprise workflows to be appropriate for team work, stable version control, and reliable distribution. Shopify has yet to deliver on this in any substantial way.

With that being said, I understand Themekit is just a single team. And a forward thinking, appreciated one at that. This grumbling above should be sent up the chain and not taken to heart here.

[tanema]

I think I will need more information from you to properly help with this issue.

• Are you running watch and this happens?
• Does this only happen when deploying? Are you using replace or upload?
• If you dislike the lock why are you not using --force

theme.lock is doing the opposite of what it is (apparently) designed to do: prevent conflicts. Instead, it causes conflicts as we have to pull down individual files that it wrongly has issues with and re-apply changes.

themekit cannot cause a conflict. It never changes the contents of your files. So if there is a conflict reported by themekit, this means that the files that are on shopify's server have changed since you last downloaded changes. You are right, git is a great tool for identifying changes that have happened. themekit can only identify that a change has happened, it is suggested that you use other tool to identify what changes happened.

Lastly I have very little avenues for feedback on this tool. The only way I can make this tool better or less "half baked" is by developers giving feedback and suggestions rather than their opinion of the company who supports it. This project has a code of conduct, so going forward please have your feedback focused on the desired outcomes and not the team maintaining the project.

[joshiefishbein]

• I am running watch when it happens, but this also happens if watch is turned off while I switch contexts/themes on the same store
• It happens when I use upload or remove via watch
• This issue wasn't opened because I dislike lock. I'm reporting this because I think lock for ThemeKit was a good idea and want this product to work well. If I should be using --force why did we even make a lock? Please note I am not the only one who's opened a ticket about this. Also note that in many forums/slack channels others are reporting faultiness and unexpected behavior with theme.lock.

theme.lock may not be causing a conflict directly as it stops a conflicting procedure from taking place. But it forces a developer to download a file they may not have wanted to, causing conflicts to occur on their local files. The only other way to get around this is to use --force, as you noted, but doesn't this then defeat the purpose of a lock? Let me be clear that this isn't being reported as a weird edge-case. This happens almost each time I try to branch off of my own theme and make minor changes.

In other environments that use a lock file (Javascript and Ruby apps for example) version control works very differently, or at least it doesn't have to strike a balance between what may be stored in git and what may be stored in the theme files. Those workflows are much more straightforward and their relationship to version is more 1:1. Shopify's case has a little bit more to juggle. There isn't much information on https://shopify.github.io/themekit/theme.lock/ that relates to the workflow it is most compatible with – other than Shopify suggesting that developers use git or mercurial.

Any additional feedback I provided is a response to this lack of formal workflow and confusion on how the ThemeKit team envisioned this feature, like others, to be used by developers. I understand your code of conduct and I apologize if you or any one else here feels harassed or insulted by my comments.

@tanema

[tanema]

theme.lock may not be causing a conflict directly as it stops a conflicting procedure from taking place. But it forces a developer to download a file they may not have wanted to, causing conflicts to occur on their local files.

How would you imagine this be resolved in your ideal workflow? Currently, the developer would run theme download assets/file_with_conflict.liquid to get changes made to that file.

Personally, I would recommend either run a theme download or a theme replace --force on your development store before starting theme watch as that will make sure that the theme is in a state that you expect. However I know some developers run theme watch with the --force when developing because they know they are the only person touching the current theme and don't care about version checking.

There isn't much information on shopify.github.io/themekit/theme.lock that relates to the workflow it is most compatible with

It seems like the root of this issue is that you lack direction in workflow, and I can see if I can help with that. I can add more documentation however generally themekit fits a lot of different workflows so it will hard to document just one.

Lastly what forums and slack channels are you referring to. I would love to provide support and have a wider audience to discuss how they would see theme development made easier.

[joshiefishbein]

Most developers I know use git as a source of truth. Not files stored on Shopify – being that these can be edited by anyone who has access to the backend with no version control (or version history that disappears whenever you duplicate a theme).

Our workflows vary from team to team and from project to project to meet the needs of the project. If you have Shopify-authored documentation regarding team workflows please link me to it.

Is there anything I can add to my config to disable theme.lock? Regardless the workflow, the feature isn't working well for us.

Thanks!

[tanema]

While developing, git is your source of truth yes. If this was always the case then the theme.lock would not exist. However, the reason that this was developed is that often when deploying a theme to a clients website it becomes apparent that the client made changes to their own website without communication to you. This is their right, however, it makes your job harder. This is the reason the theme.lock exists. When deploying, Shopify is your client's source of truth meaning it is your source of truth.

What the theme.lock does, is make sure you do not have to run theme download before deploying to see what has changed. It adds a set of checks to make sure what you "know" is on the client's site is what is actually there.

As for a mass disabling of the theme.lock file please use the --force flag. I have purposefully not provided a way to disable it entirely so that I could get feedback on the feature. I will add one in the future but most likely after what I feel is a stability in the feature.

[joshiefishbein]

That's all understandable.

I'm going to use the --force flag going forward. I look forward to the stability you described. Feel free to close this ticket if it doesn't help you get there.

Thanks again @tanema

[marjansdg]

We are in the same boat as joshiefishbein. It would be nice to disable it, cause it's unnecessary with our workflow too.
So, looking forward when we can disable this option.

[cameroncowden]

For what it's worth, we're encountering the same issue in our workflow, which goes like this...

Master git branch is production theme. (Sync'd via theme download -e=production)
Duplicate theme in Shopify admin -> update config.yml with new theme ID for env "staging"
Create new branch in Git, commit new config.yml to init new branch for dev of feature.
Running theme watch -e=staging results in 409 conflicts for all files, even though they're identical on Shopify (freshly duplicated theme), and locally (fresh branch off master).

Running it with --force works fine, or re-downloading the file(s) I'm working on will allow upload without force, but can be a bit annoying since I'll usually have already made changes and need to stash in temp file, redownload to overwrite my local working copy, then replace back with the updated contents.

Another thing I'm a bit concerned about is that I'm often discarding changes to theme.lock while changing git branches, reverting it to a previous state instead of deleting it outright and starting fresh. Could that potentially cause false-positives when it comes to versioning? (e.g. would fail to warn on an upload that was modified via an alt git branch?)

[tanema]

I have completed initial work on a new version of themekit without the asset manifest. For those who wish to live without the manifest, please try out v0.8.0 prerelease by running theme update --version=v0.8.0-prerelease

It has removed the manifest but it has also added better error reporting. Also theme bootstrap has been renamed to theme new as well as a command theme get has been added that will create a config.yml file for you and download. (basically theme configure with auto downloading)

It would really help me to ensure the deploy of the new version if you would try it out and report any issues you encounter.

[tanema]

I will be releasing v0.8.0 at the beginning of next week unless there is any feedback of weird behavior.

[tanema]

This will no longer be a problem in v0.8.0 (released today) since the asset manifest has been removed from themekit.

[agentlewis]

Hi @tanema I have just started a new project and was very confused when there was no theme.lock :smile_cat: than i found this thread and groked what was going on. I too had some troubles with theme.lock but than I worked out how to work with it and the only annoyance I had was that it was a thing I to commit into git often (a compromise i was happy with)

Funnily enough now that I understand what its was doing for me I miss it. Namely faster uploads and simple diff checking. Obviously with software like this it is most important that it works well for the majority of users.

Anyway - this is just a quick note to say I liked the feature and it worked for me :smile: I'm now just running a script that downloads before uploading. This "should" mean that i get any changes my clients make via the online editor, which i discourage - but it happens.

Thanks to the team for the great work - you make my work easier and more pleasont