Virtual-environments: brew update/brew upgrade failing with Error: Not a directory @ dir_s_rmdir error

[traversaro]

Description
On some scheduled jobs on macOS, since few days the brew update and brew upgrade are failing.

Area for Triage:

Question, Bug, or Feature?:

Bug

Virtual environments affected

• [x] macOS 10.15
• [ ] Ubuntu 16.04 LTS
• [ ] Ubuntu 18.04 LTS
• [ ] Ubuntu 20.04 LTS
• [ ] Windows Server 2016 R2
• [ ] Windows Server 2019

Expected behavior
The commands:

• brew update
• brew upgrade

Should work fine.

Actual behavior

• brew update
• brew upgrade
  commands fail with error:
  ~
  ==> Summary
  🍺 /usr/local/Cellar/node@12/12.19.0_1: 4,304 files, 59.5MB
  Removing: /usr/local/Cellar/node@12/12.18.4... (4,296 files, 58.9MB)
  Removing: /Users/runner/Library/Caches/Homebrew/[email protected]... (15.8MB)
  Error: Not a directory @ dir_s_rmdir - /usr/local/Cellar/openssl
  ==> Upgrading [email protected] 1.1.1h -> 1.1.1h
  Removing: /usr/local/Cellar/openssl/1.0.2t... (1,787 files, 12.4MB)
  Error: Process completed with exit code 1.
  ~

Repro steps
A description with steps to reproduce the issue. If your have a public example or repo to share,
please provide the link.

1. Run brew update
2. Run brew upgrade

[traversaro]

I do not have a mac so I can't try to reproduce this, but I checked online and it seems that it is affecting other CI systems: https://github.com/dotnet/runtime/pull/43299 .

[traversaro]

Example of failing build: https://github.com/robotology/robotology-superbuild/runs/1248677157 .

[LeonidLapshin]

Hey, @traversaro !
It seems that openssl (Cellar/openssl/1.0.2t) that is the cause of the error will be removed from macOS images soon, the link to corresponding PR
for now you can workaround this issue with

brew uninstall [email protected]

command before

brew update
brew upgrade

I have tried to install your dependencies from https://github.com/robotology/robotology-superbuild/blob/master/.github/workflows/ci.yml and

          brew uninstall [email protected]
          brew cask install xquartz
          brew update
          brew upgrade
          brew install ace boost cmake eigen gsl ipopt jpeg libedit opencv pkg-config qt5 sqlite swig tinyxml
          brew install libmatio
          brew tap osrf/simulation
          brew install osrf/simulation/gazebo11
          brew install ninja

works correctly.
Could you please check the provided workaround?
Thank you!

[traversaro]

Thanks!

[traversaro]

Thanks a lot! I tried the workaround in https://github.com/robotology/robotology-superbuild/pull/493 , but doing that in my CI results in this failure:
~~~
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Error: Fetching /usr/local/Homebrew/Library/Taps/local/homebrew-openssl failed!
Fetching /usr/local/Homebrew/Library/Taps/local/homebrew-python2 failed!
fatal: invalid upstream 'origin/master'
fatal: invalid upstream 'origin/master'
~~~

[LeonidLapshin]

@traversaro , seems that happens because of homebrew version, could you please try to run:

brew uninstall [email protected]
brew uninstall [email protected]
brew untap local/openssl
brew untap local/python2
brew cask install xquartz
brew update
brew upgrade
brew install ace boost cmake eigen gsl ipopt jpeg libedit opencv pkg-config qt5 sqlite swig tinyxml
...

it should help.
Thank you!

[traversaro]

Thanks a lot @LeonidLapshin , your workaround is working fine!

[ppamorim]

@traversaro This still not fixed and the issue was re-introduced yesterday, any idea when this will be fixed again?

[rdzman]

Workaround is not working for me. That is, doing

brew uninstall [email protected]

before

brew update

simply results in

Uninstalling /usr/local/Cellar/[email protected]/1.0.2t... (1,787 files, 12.4MB)
Warning: The following may be [email protected] configuration files and have not been removed!
If desired, remove them manually with `rm -rf`:
  /usr/local/etc/openssl
  /usr/local/etc/[email protected]

And, whether or not a manually rm -rf those directories, it is all followed by the same error I was getting before, namely ...

fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: 'origin' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Error: Fetching /usr/local/Homebrew/Library/Taps/local/homebrew-openssl failed!
Fetching /usr/local/Homebrew/Library/Taps/local/homebrew-python2 failed!
fatal: invalid upstream 'origin/master'
fatal: invalid upstream 'origin/master'

[al-cheb]

@rdzman,

brew untap local/homebrew-openssl
brew untap local/homebrew-python2
brew update 

or

brew uninstall [email protected]
brew uninstall [email protected]
brew untap local/openssl
brew untap local/python2
brew update

https://github.com/actions/virtual-environments/issues/1811#issuecomment-708480190

[rdzman]

Thanks @al-cheb. That worked.

[bagder]

So there's no fix for this, just the workaround for everyone to do themselves?

[rdzman]

So there's no fix for this, just the workaround for everyone to do themselves?

And if that's the case, shouldn't this issue be left open until there is a real fix?

[maxim-lobanov]

Reopening.
Fix was done in https://github.com/actions/virtual-environments/pull/1889 and will be deployed everywhere in 3-4 days

[traversaro]

Even with the workaround suggested in this issue, now the brew upgrade step is failing with:
~
==> Pouring [email protected]_1.catalina.bottle.tar.gz
Error: The brew link step did not complete successfully
Warning: These files were overwritten during brew link step:
The formula built, but is not symlinked into /usr/local
Could not symlink lib/pkgconfig/python3-embed.pc
Target /usr/local/lib/pkgconfig/python3-embed.pc
is a symlink belonging to [email protected]. You can unlink it:
brew unlink [email protected]
~

I am not sure if this is related to https://github.com/actions/virtual-environments/issues/1929 .

[traversaro]

The same error occurs even by removing the workaround mentioned in this issue, so it is probably a different problem, but it is anyhow still related to brew update/brew upgrade failing.

[traversaro]

For the new problem mentioned in https://github.com/actions/virtual-environments/issues/1811#issuecomment-718475660, adding brew unlink [email protected] as suggested by the Homebrew error seems to be effective, see https://github.com/robotology/robotology-superbuild/pull/509#issuecomment-718494055 .

[rdzman]

@traversaro I believe the new issue you mention (reported in Homebrew/homebrew-core#63619) has also been fixed now, in Homebrew/homebrew-core#63665.

[miketimofeev]

@traversaro the new image has been deployed and

brew update
brew upgrade

works just fine!
I'm going to close the issue, but please feel free to contact us if you have any concerns.
Thank you!

[luigiberrettini]

The same error shows up with:

brew cleanup
brew update

[nunorafaelrocha]

@miketimofeev the problem isn't because of brew update or upgrade.

Just by running

brew cleanup

I get the following error:

Run brew cleanup
Error: Not a directory @ dir_s_rmdir - /usr/local/Cellar/openssl
Removing: /usr/local/Cellar/openssl/1.0.2t... (1,787 files, 12.4MB)
Error: Process completed with exit code 1.

The action doesn't run anything else... https://github.com/nunorafaelrocha/dotfiles/runs/1402373738?check_suite_focus=true

[miketimofeev]

@nunorafaelrocha Sorry for the late answer — this is also related to the outdated openssl formula, which will be removed in the following month. Could you please create a new issue if it's a severe problem for you and we'll try to find a temporary workaround?

[nunorafaelrocha]

Hi @miketimofeev 👋

Forgot to post here, I've founded a workaround 😄

Added a step where I delete openssl

      - name: Workarounds
         run: |
           brew uninstall [email protected]
           rm -rf /usr/local/etc/openssl
           rm -rf /usr/local/etc/[email protected]