Migrate: File Source Driver produces incorrect paths on Windows

[bsomes]

Describe the Bug
When trying to run a migration in Windows using the "NewWithDatabaseInstance" function, the source driver does not parse the file URL correctly.

when using

if migration, err  = migrate.NewWithDatabaseInstance("file://" + "rel/path/to/source/dir", "sqlite3", driver); err != nil {
    return
}

I get the error "parse file file://path/to/source/dir: invalid character "\" in host name"

When I do 3 forward slashes, like

if migration, err = migrate.NewWithDatabaseInstance("file:///" + "abs/path/to/source/dir", "sqlite3", driver); err != nil {
    return
}

I get the error "open /path/to/source/dir: The filename, directory name, or volume label syntax is incorrect." Note that in this error message there is a forward slash prepended to the file path, so the path "C:\Users" would be "/C:\Users" in this case.

Steps to Reproduce

1. On Windows, make a directory containing a single up and down migration.
2. Make an empty sqlite database
3. Create a go command line utility containing the following:

import (
        _ "github.com/mattn/go-sqlite3"

       migrate "github.com/golang-migrate/migrate/v4"
    "github.com/golang-migrate/migrate/v4/database"
    "github.com/golang-migrate/migrate/v4/database/sqlite3"
)

func TestMigrate(db *sql.DB, migrationsPath string) {
    var (
        migration *migrate.Migrate
        driver    database.Driver
        err       error
    )

    if driver, err = sqlite3.WithInstance(db, &sqlite3.Config{}); err != nil {
        log.Fatal(err)
    }
    // code from above bug description here
}

Expected Behavior
I would expect this to execute the migrations against the database without error.

Migrate Version
4.4.0

Loaded Source Drivers
file

Loaded Database Drivers
sqlite3

Go Version
go1.12.5 linux/amd64

[dhui]

Unfortunately, I don't have access to a Windows machine, so will be unable to reproduce and fix the issue. You could try slapping on a filepath.Clean() before the ioutil.ReadDir() call in source/file/file.go.

[puellanivis]

This fundamentally has to do with a bug/feature in how this package handles file: URLs. There is no proper way to do relative paths with the file: URL scheme, and especially not with file://.

According to URL specifications, after the scheme (file:), if there is then the sequence // then an authority section MUST follow, which ends with any of /, ?, #, or end of string. The only characters allowed that do not need to be percent encoded, are: /[-A-Za-z0-9._~!$&'()*+,;=]/ note that this character class does not include \. So, if you start with the string: file://this\is\all\actually\the\authority\section the URL is immediately “unhealthy”, because there is an unescaped \ in the authority section.

Instead, file: URLs normally have an authority section that is empty: file:/// (though some systems with simplified OS-level multihost file access, such as Windows, can allow for different hostnames), this means that the common-case file: URL should always begin with file:/// and the path should always be absolute.

The proper way to get a relative file: URL path is the same way you always get a relative URL: you do not specify either a scheme or authority section at all: relative/url/path/ Note: url.Parse does work on these relative URLs.

As well, these file: URLs are supposed to use / regardless of the normal file path separator from the OS (which is on the caller’s side of the code), which means even concatenating file:/// with a raw windows path will immediately result in an unhealthy URL, because file:///C:\Windows is not file:///C:/Windows.

In order to properly build a file: URL compatible with Windows, you need: rawurl := "file:///" + filepath.ToSlash(filepath.Abs(relpath)) then when you are processing the URL (in this package), you need something like: uri := url.Parse(rawurl); osPath := filepath.Clean(uri.Path) (which will also perform a filepath.FromSlash). However, there might need to be some cleanup to make sure that there is not accidentally a preceding /.

Also, additionally note that adding a filepath.Clean will not help the current code at least of healthy absolute paths, because filepath.Abs is already defined to call filepath.Clean.

[dhui]

@raynjamin's approach might work

Any fixes to the file source driver would be appreciated. Don't forget to add tests!