Sentence-transformers: Transformers 4.x support

[davidmezzetti]

First off, thank you for this package. It's very exciting to see how it has developed over the last year especially, great work!

With pip 20.3+ being pushed out more and more now, the hard requirement cap on transformers is going to be more problematic. The new dependency resolver is harsh and it's not always apparent what the issue is. For example, I was trying to upgrade to sentence-transformers 0.3.9 + transformers 4.0.0 and pip 20.3.1 was what was in what looked to be an infinite loop.

Now I know this isn't really a sentence-transformer issue and upgrading automatically to a major release isn't a great idea. But just thinking ahead when transformers 4.0.1 or 4.1, 4.2 etc comes out. The Hugging Face team is growing rapidly and I expect they'll continue releasing at a faster clip.

Have you guys considered more automated tests with something like GitHub Actions? For my use case, transformers 4.0.0 + sentence-transformers 0.3.9 works well (I forced the install using the legacy pip resolver) but I am only using a portion of the full package. The suite of tests could be run on check ins, pull requests and scheduled nightly for example.

[nreimers]

Testing transformers 4 is on my agenda and a new version will be released soon. There are some breaking changes which I think need some updates to make it compatible for all use cases.

So far we have not thought about automatic checks.

[davidmezzetti]

Great, I appreciate the quick response!

In terms of automatic tests, the yaml snippet below is something that you could add to this repo to automatically run the tests in the tests directory using a GitHub Action workflow.

If you create the following file in the root of the project: .github/workflows/build.yml

name: build

on: ["push", "pull_request"]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - uses: actions/setup-python@v2
        with:
          python-version: 3.6

      - name: Build
        run: |
          pip install -U pip wheel
          pip install .
          cd tests
          python -m unittest -v -s .

Each check in to master or a pull request will run this workflow on GitHub's infrastructure.

You can also add builds for macOS and/or Windows. I've found it very helpful as people will report strange errors occasionally on other platforms that I don't have a dev environment for.

Builds can also be scheduled. If a build fails, the repo maintainers will get an email. Something like this could help raise transformers conflicts with new releases.

If helpful, here is a build script I have with some of the functionality mentioned above.

I cloned sentence-transformers and tried this script but I was unable to get sts-test.csv into the format the tests expected. But you can easily add a wget command to pull whatever data you need for the tests.

[nreimers]

Thanks for the example and the links. I will have a look and integrate it into the repository with unit tests for the base functionality.

[nreimers]

@davidmezzetti I tested it with Transformers 4 and it appears to work well. I updated the requirements here and a new release what allows Transformers 4 will be release soon.

I also started to add better unit test.

[davidmezzetti]

Thank you for the update! Glad to hear everything is working with 4.x.

[davidmezzetti]

Thank you for the new release. I've upgraded to 0.4.0 and everything is working great! I'll go ahead and close the issue.