Redash: Upgrade script fails with invalid SemVer string

[vitorbaptista]

Issue Summary

Inside the docker image redash/redash:1.0.0.b2588, I downloaded the upgrade script as explained on https://redash.io/help-onpremise/maintenance/how-to-upgrade-redash.html. When running the script with the REDASH_DATABASE_URL pointing to a database on 0.12, I got:

Starting Redash upgrade:
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
Found version: 0.12.0
Traceback (most recent call last):
  File "./upgrade", line 237, in <module>
    deploy_release(args.channel)
  File "./upgrade", line 217, in deploy_release
    verify_newer_version(release)
  File "./upgrade", line 203, in verify_newer_version
    if not release.is_newer(current_version()):
  File "./upgrade", line 69, in is_newer
    return semver.compare(self.version, version) > 0
  File "/usr/local/lib/python2.7/dist-packages/semver.py", line 54, in compare
    v1, v2 = parse(ver1), parse(ver2)
  File "/usr/local/lib/python2.7/dist-packages/semver.py", line 21, in parse
    raise ValueError('%s is not valid SemVer string' % version)
ValueError: current is not valid SemVer string

On the upgrade script, the CURRENT_VERSION_PATH is defined as ${REDASH_HOME}/current. Then, on the same script, you have the current_version() method defined as:

# https://github.com/getredash/redash/blob/01a8075a6796653f046901a6d302caec17affdb1/bin/upgrade#L179-L181
def current_version():
    real_current_path = os.path.realpath(CURRENT_VERSION_PATH).replace('.b', '+b')
    # real_current_path == '/opt/redash/current'
    return real_current_path.replace(REDASH_HOME + '/', '').replace('redash.', '')
    # the returned value is 'current'

The problem is that current isn't a valid SemVer version, so as far as I can see this code will always fail.

Steps to Reproduce

1. docker run --name redash -e 'REDASH_DATABASE_URL=postgresql://redash@postgres/redash' -d redash/redash:1.0.0.b2588
2. docker exec -it redash bash
3. wget https://raw.githubusercontent.com/getredash/redash/master/bin/upgrade
4. chmod +x upgrade
5. ./upgrade

Technical details:

• Upgrade script: https://github.com/getredash/redash/blob/01a8075a6796653f046901a6d302caec17affdb1/bin/upgrade

[arikfr]

On "regular" servers current is a symlink to the current version, which has correct version name. In the Docker images current is the folder itself.

The script is not designed to work with the Docker image. I assume that the Docker image users will just use newer image instead of upgrading the existing one.

[vitorbaptista]

@arikfr I was using re:dash 0.12 and now would like to upgrade to 1.0.0. As I'm using a Docker container, my thought was that I needed to get into the container and run the upgrade script. If that's not the case, what would be the recommended way of migrating?

Edit: Answering myself, I'd need to run the migrations via sudo -u redash bin/run ./manage.py db upgrade

[arikfr]

Yes. Also note that there is no recent v1.0.0 container yet (there will be in day or two).

Btw, how are you running your containers?

[vitorbaptista]

On Docker Cloud, we created a Stackfile based on your https://github.com/getredash/redash/blob/master/docker-compose.production.yml. We've built a new container based on the latest code (https://hub.docker.com/r/okibot/redash), but am hoping to use your offical ones when they're available.

[arikfr]

👍 looks like you build your images in almost the same way that I'm going to, so the transition will be easy.

[sadavarti]

@vitorbaptista Can you please explain how to migrate data from docker redash container v1.0.0to v1.0.1
I had read your answer but I am get you. Sir, could you please elaborate.
Thanks

[vitorbaptista]

@sadavarti I ended up having a lot of issues running the migrations. My guess was that as previously redash was using a DIY migration tool that didn't guarantee that migrations wouldn't be run twice (AFAIK), I ended up messing the DB. I ended up having to check the migration files, analyzing what I had applied already, and fixing the errors as we went. All too manual and error-prone. Gladly, as redash now uses a good migration tool, this shouldn't happen anymore.

[myoung34]

This is still an issue.
Need to test LDAP fix in V4. Running upgrade from redash:latest to 4.0.0 beta results in:

root@b8191f03ed19:/app# gosu redash bin/upgrade --channel beta
Starting Redash upgrade:
Found version: 4.0.0-beta
Current version: current
Traceback (most recent call last):
  File "bin/upgrade", line 242, in <module>
    deploy_release(args.channel)
  File "bin/upgrade", line 220, in deploy_release
    verify_minimum_version()
  File "bin/upgrade", line 190, in verify_minimum_version
    if semver.compare(current_version(), '0.12.0') < 0:
  File "/usr/local/lib/python2.7/dist-packages/semver.py", line 54, in compare
    v1, v2 = parse(ver1), parse(ver2)
  File "/usr/local/lib/python2.7/dist-packages/semver.py", line 21, in parse
    raise ValueError('%s is not valid SemVer string' % version)
ValueError: current is not valid SemVer string

[padewitte]

I ended up finding the solution in issue 2445 and run the command
docker exec -ti redash_server_1 ./manage.py db upgrade

[rexuli]

@padewitte

I tried the script and its not working.
Seems there is no obvious to migrate the data from 3.0 to 4.0.

[jolly7481]

Agree with @rexuli script doesn't work and no matter how I try, upgrading docker image using available resources/"guide" fails. Always having the same error in Postgres.
"ERROR: column queries.search_vector does not exist at character 792"