Shards: Local overrides (was: Shards Link)

[crisward]

Npm, bower and other package manages have the concept of linking dependencies. This is useful when developing app dependencies. Allowing you to have two projects in different folders, then one is sym-linked into the libs folder of the other.

I think it'd be useful to have this in shards. What do you think?

[ysbaddaden]

Maybe use the path resolver?

[RX14]

The problem with the path resolver is that you need to run shards update to copy the new source code every time you make a change. It also required you to modify the shard.yml every time you want to bugfix in a dependency.

[ysbaddaden]

Huh? AFAIK path creates a symlink...

[RX14]

@ysbaddaden Oh, I thought it didn't. I should have checked, sorry.

[crisward]

What is the path resolver?

[ysbaddaden]

Instead of using a git ot github, you may:

dependencies:
  my_shard_name:
    path: /absolute/path/to/dependency

[crisward]

got it. Thanks.

[crisward]

Still not quite as nice as the link option, with link I don't have to remember to change by path before pushing live... Maybe I could add that to a deploy script..

[RX14]

Just tried using path on a transitive dependency, it didn't work:

dependencies:
  db:
    github: crystal-lang/crystal-db
    path: /data/programming/crystal-db/
  pg:
    github: will/crystal-pg
    version: ~> 0.13.1

This links crystal-db from github like usual, whichever way I order the listings.

[bcardiff]

In bundler this is done by local overrides. It was discussed a bit in #105. I do manual link in ./lib/shard_name. It is error prone but it has been working for me until now.

[ysbaddaden]

@RX14 you can't have multiple resolvers for the same dependency. You must drop/comment the github line.

[RX14]

@ysbaddaden Apparently i'm so used to keeping github: and changing branch:, version: or commit: I didn't even think about it! Thanks for pointing out my mistake, it works great now.

[jhass]

+1 for something like this, bundler's way seems most appropriate for us. Having a dirty shard.yml and paying attention to not accidentally committing it is just too much of a PITA.

Additionally we obviously should fail hard and early if there's more than one resolver given.

[RX14]

Very much :+1: to my mistake failing hard.

[ysbaddaden]

@jhass the current solution ain't great. I never had the need for that, so I admit I didn't search much to address the issue.

Maybe local overrides, so we could keep git: in shard.yml but have a path: locally, which could be automated with shards link /path/to/dependency for example, only applying to the current project, with warnings to the check, install and update commands that it's using a local override.

[watzon]

Anyone put more thought into this? This is definitely a feature that's missing.

[ysbaddaden]

I haven't.

I needed this the other day, and I simply changed git: to path: and ran shards update. Tested things out, then ran git checkout shard.*. No big deal.

Having a different, git ignored, source for dependencies, such as local overrides has always been confusing in my experience.

[anamba]

To add another data point, I still feel the pain of this quite often. Updating a Gemfile and running bundle is more or less instant, but running shards update typically takes 30-40 seconds (~1s per repo), so I actively avoid using shards when possible. I made a shell script to symlink dependencies into lib when I need to test something.

I did notice that once, when I tried running shards from an EC2 instance located in the US, it was astonishingly quick. Is that is how it works for other people normally??

[ysbaddaden]

You can use --local in the 0.9.0 beta release to make commands instant, as long as the cache is up to date, as is the case here.

[straight-shoota]

This was resolved by #422