Isort: Local imports being identified as THIRDPARTY?

[averypmc]

Hello! Thanks for the tool. I arrived here after setting up pre-config and using the seed-isort-config tool which successfully identified a sensible set of known third parties. However, when I then run isort, it identifies nearly all of my local imports as THIRDPARTY

Minimum example:

from app.create import (
    create_frontend_exposed_config,
    create_helpers,
    create_resources,
    create_services,
    create_stores,
)
from app.init import (
    initialize_additional_extensions,
    initialize_config,
    initialize_db,
    initialize_logging,
    initialize_scheduler,
    initialize_security,
)
from app.mount import mount_api, mount_lifecycle_events
from apscheduler.schedulers.background import BackgroundScheduler
from flask import Flask
from models.common import db as models_db

isort output:

from-type place_module for app.create returned THIRDPARTY
from-type place_module for app.init returned THIRDPARTY
from-type place_module for app.mount returned THIRDPARTY
from-type place_module for apscheduler.schedulers.background returned THIRDPARTY
from-type place_module for flask returned THIRDPARTY
from-type place_module for models.common returned THIRDPARTY
ERROR: /server.py Imports are incorrectly sorted and/or formatted.

Only apscheduler and Flask here are third party imports. Everything else is local. I'm sure I'm just doing something dumb, but not sure what. Any help is appreciated.

[averypmc]

For now, I've just updated my config to specify some of these imports as known first party. Is that correct?

[timothycrosley]

Hi @averypmc thanks for raising this issue! known_first_party is a good solution, as documented in the 5.0.0 upgrade guide: https://timothycrosley.github.io/isort/docs/upgrade_guides/5.0.0/#module-placement-changes-known_third_party-known_first_party-default_section-etc. However, it strictly speaking should rarely be necessary if everything is working correctly. A few questions if you are willing to help me diagnose the issue:

• Can you confirm you are using version 5.0.0 or greater?
• Are you still using seed-isort-config? This should not be necessary on the 5.0.0 release and could lead to weird interactions.
• Can you provide the command you use to run isort, and the output of isort --show-config?

Thanks!

~Timothy

[averypmc]

Thanks for the quick response!

• Using version 5.0.5
• Yes, still using seed-isort-config.
• isort is running as part of pre-commit without any additional arguments. So I'd assume just isort?

If I run --show-config directly from the command line (e.g. not using --show-config), this is my output:

    "skip_glob": [],
    "line_length": 79,
    "wrap_length": 0,
    "line_ending": "",
    "sections": [
        "FUTURE",
        "STDLIB",
        "THIRDPARTY",
        "FIRSTPARTY",
        "LOCALFOLDER"
    ],
    "no_sections": false,
    "known_future_library": [
        "__future__"
    ],
    "known_third_party": [
        "pylru",
        "werkzeug",
        "google",
        "flask_restful",
        "flask_uuid",
        "sqlalchemy",
        "flask_sqlalchemy",
        "sqlalchemy_utils",
        "datadog",
        "alembic",
        "requests",
        "environs",
        "boto3",
        "apispec",
        "factory",
        "flask_migrate",
        "pytz",
        "pytest",
        "structlog",
        "transitions",
        "geojson",
        "jwt",
        "flask",
        "marshmallow_enum",
        "apscheduler",
        "freezegun",
        "Crypto",
        "jsonschema",
        "marshmallow",
        "yaml",
        "dateutil"
    ],
    "known_first_party": [
        "common",
        "flights",
        "server",
        "utils",
        "models",
        "resources",
        "security",
        "app",
        "loggers",
        "tests",
        "alert",
        "temp",
        "integrations",
        "background_jobs",
        "settings",
        "detections"
    ],
    "known_standard_library": [
        "modulefinder",
        "sunau",
        "stringprep",
        "rlcompleter",
        "warnings",
        "resource",
        "imghdr",
        "zipapp",
        "secrets",
        "mailbox",
        "spwd",
        "argparse",
        "decimal",
        "io",
        "termios",
        "dbm",
        "abc",
        "contextvars",
        "trace",
        "syslog",
        "formatter",
        "ssl",
        "pprint",
        "posixpath",
        "tabnanny",
        "csv",
        "audioop",
        "copyreg",
        "linecache",
        "filecmp",
        "queue",
        "code",
        "typing",
        "colorsys",
        "grp",
        "dummy_threading",
        "fractions",
        "bisect",
        "cmd",
        "msvcrt",
        "nntplib",
        "tkinter",
        "fnmatch",
        "tty",
        "ensurepip",
        "shelve",
        "functools",
        "encodings",
        "tempfile",
        "faulthandler",
        "xdrlib",
        "unicodedata",
        "distutils",
        "ast",
        "itertools",
        "zipimport",
        "compileall",
        "urllib",
        "unittest",
        "py_compile",
        "copy",
        "timeit",
        "os",
        "token",
        "concurrent",
        "selectors",
        "traceback",
        "socket",
        "bz2",
        "tarfile",
        "pty",
        "bdb",
        "codecs",
        "glob",
        "difflib",
        "pipes",
        "numbers",
        "lzma",
        "dataclasses",
        "base64",
        "multiprocessing",
        "curses",
        "optparse",
        "stat",
        "lib2to3",
        "atexit",
        "zlib",
        "configparser",
        "xmlrpc",
        "pickle",
        "ctypes",
        "plistlib",
        "reprlib",
        "msilib",
        "quopri",
        "getopt",
        "binhex",
        "crypt",
        "smtpd",
        "sysconfig",
        "marshal",
        "uuid",
        "builtins",
        "mimetypes",
        "uu",
        "shutil",
        "pstats",
        "statistics",
        "test",
        "pyclbr",
        "telnetlib",
        "math",
        "gettext",
        "fileinput",
        "inspect",
        "sys",
        "importlib",
        "wave",
        "pwd",
        "pydoc",
        "asyncore",
        "ntpath",
        "types",
        "pathlib",
        "winsound",
        "imaplib",
        "signal",
        "tokenize",
        "posix",
        "chunk",
        "keyword",
        "struct",
        "email",
        "smtplib",
        "locale",
        "pdb",
        "threading",
        "winreg",
        "fcntl",
        "ipaddress",
        "textwrap",
        "webbrowser",
        "hashlib",
        "json",
        "weakref",
        "runpy",
        "poplib",
        "symbol",
        "heapq",
        "nis",
        "imp",
        "asynchat",
        "wsgiref",
        "time",
        "aifc",
        "select",
        "_dummy_thread",
        "datetime",
        "mailcap",
        "re",
        "collections",
        "http",
        "readline",
        "venv",
        "profile",
        "macpath",
        "symtable",
        "sqlite3",
        "gc",
        "cProfile",
        "shlex",
        "platform",
        "ftplib",
        "getpass",
        "operator",
        "netrc",
        "ossaudiodev",
        "random",
        "tracemalloc",
        "zipfile",
        "gzip",
        "_thread",
        "turtle",
        "binascii",
        "xml",
        "sre_constants",
        "asyncio",
        "cmath",
        "cgi",
        "logging",
        "array",
        "pkgutil",
        "cgitb",
        "html",
        "site",
        "sched",
        "mmap",
        "string",
        "calendar",
        "parser",
        "dis",
        "contextlib",
        "enum",
        "doctest",
        "sndhdr",
        "pickletools",
        "socketserver",
        "errno",
        "subprocess",
        "hmac",
        "fpectl",
        "turtledemo",
        "codeop"
    ],
    "extra_standard_library": [],
    "known_other": {},
    "multi_line_output": "VERTICAL_HANGING_INDENT",
    "forced_separate": [],
    "indent": "    ",
    "comment_prefix": "  #",
    "length_sort": false,
    "length_sort_sections": [],
    "add_imports": [],
    "remove_imports": [],
    "reverse_relative": false,
    "force_single_line": false,
    "single_line_exclusions": [],
    "default_section": "THIRDPARTY",
    "import_headings": {},
    "balanced_wrapping": false,
    "use_parentheses": true,
    "order_by_type": true,
    "atomic": false,
    "lines_after_imports": -1,
    "lines_between_sections": 1,
    "lines_between_types": 0,
    "combine_as_imports": false,
    "combine_star": false,
    "keep_direct_and_as_imports": true,
    "include_trailing_comma": true,
    "from_first": false,
    "verbose": false,
    "quiet": false,
    "force_adds": false,
    "force_alphabetical_sort_within_sections": false,
    "force_alphabetical_sort": false,
    "force_grid_wrap": 0,
    "force_sort_within_sections": false,
    "lexicographical": false,
    "ignore_whitespace": false,
    "no_lines_before": [],
    "no_inline_sort": false,
    "ignore_comments": false,
    "case_sensitive": false,
    "sources": [
        {
            "py_version": "py3",
            "force_to_top": [],
            "skip": [
                ".tox",
                ".git",
                "node_modules",
                "venv",
                ".pants.d",
                "buck-out",
                "_build",
                ".eggs",
                ".venv",
                ".mypy_cache",
                "build",
                "dist",
                ".nox",
                ".hg"
            ],
            "skip_glob": [],
            "line_length": 79,
            "wrap_length": 0,
            "line_ending": "",
            "sections": [
                "FUTURE",
                "STDLIB",
                "THIRDPARTY",
                "FIRSTPARTY",
                "LOCALFOLDER"
            ],
            "no_sections": false,
            "known_future_library": [
                "__future__"
            ],
            "known_third_party": [
                "google.appengine.api"
            ],
            "known_first_party": [],
            "known_standard_library": [
                "modulefinder",
                "sunau",
                "stringprep",
                "rlcompleter",
                "warnings",
                "resource",
                "imghdr",
                "zipapp",
                "secrets",
                "mailbox",
                "spwd",
                "argparse",
                "decimal",
                "io",
                "termios",
                "dbm",
                "abc",
                "contextvars",
                "trace",
                "syslog",
                "formatter",
                "ssl",
                "pprint",
                "posixpath",
                "tabnanny",
                "csv",
                "audioop",
                "copyreg",
                "linecache",
                "filecmp",
                "queue",
                "code",
                "typing",
                "colorsys",
                "grp",
                "dummy_threading",
                "fractions",
                "bisect",
                "cmd",
                "msvcrt",
                "nntplib",
                "tkinter",
                "fnmatch",
                "tty",
                "ensurepip",
                "shelve",
                "functools",
                "encodings",
                "tempfile",
                "faulthandler",
                "xdrlib",
                "unicodedata",
                "distutils",
                "ast",
                "itertools",
                "zipimport",
                "compileall",
                "urllib",
                "unittest",
                "py_compile",
                "copy",
                "timeit",
                "os",
                "token",
                "concurrent",
                "selectors",
                "traceback",
                "socket",
                "bz2",
                "tarfile",
                "pty",
                "bdb",
                "codecs",
                "glob",
                "difflib",
                "pipes",
                "numbers",
                "lzma",
                "dataclasses",
                "base64",
                "multiprocessing",
                "curses",
                "optparse",
                "stat",
                "lib2to3",
                "atexit",
                "zlib",
                "configparser",
                "xmlrpc",
                "pickle",
                "ctypes",
                "plistlib",
                "reprlib",
                "msilib",
                "quopri",
                "getopt",
                "binhex",
                "crypt",
                "smtpd",
                "sysconfig",
                "marshal",
                "uuid",
                "builtins",
                "mimetypes",
                "uu",
                "shutil",
                "pstats",
                "statistics",
                "test",
                "pyclbr",
                "telnetlib",
                "math",
                "gettext",
                "fileinput",
                "inspect",
                "sys",
                "importlib",
                "wave",
                "pwd",
                "pydoc",
                "asyncore",
                "ntpath",
                "types",
                "pathlib",
                "winsound",
                "imaplib",
                "signal",
                "tokenize",
                "posix",
                "chunk",
                "keyword",
                "struct",
                "email",
                "smtplib",
                "locale",
                "pdb",
                "threading",
                "winreg",
                "fcntl",
                "ipaddress",
                "textwrap",
                "webbrowser",
                "hashlib",
                "json",
                "weakref",
                "runpy",
                "poplib",
                "symbol",
                "heapq",
                "nis",
                "imp",
                "asynchat",
                "wsgiref",
                "time",
                "aifc",
                "select",
                "_dummy_thread",
                "datetime",
                "mailcap",
                "re",
                "collections",
                "http",
                "readline",
                "venv",
                "profile",
                "macpath",
                "symtable",
                "sqlite3",
                "gc",
                "cProfile",
                "shlex",
                "platform",
                "ftplib",
                "getpass",
                "operator",
                "netrc",
                "ossaudiodev",
                "random",
                "tracemalloc",
                "zipfile",
                "gzip",
                "_thread",
                "turtle",
                "binascii",
                "xml",
                "sre_constants",
                "asyncio",
                "cmath",
                "cgi",
                "logging",
                "array",
                "pkgutil",
                "cgitb",
                "html",
                "site",
                "sched",
                "mmap",
                "string",
                "calendar",
                "parser",
                "dis",
                "contextlib",
                "enum",
                "doctest",
                "sndhdr",
                "pickletools",
                "socketserver",
                "errno",
                "subprocess",
                "hmac",
                "fpectl",
                "turtledemo",
                "codeop"
            ],
            "extra_standard_library": [],
            "known_other": {},
            "multi_line_output": "GRID",
            "forced_separate": [],
            "indent": "    ",
            "comment_prefix": "  #",
            "length_sort": false,
            "length_sort_sections": [],
            "add_imports": [],
            "remove_imports": [],
            "reverse_relative": false,
            "force_single_line": false,
            "single_line_exclusions": [],
            "default_section": "THIRDPARTY",
            "import_headings": {},
            "balanced_wrapping": false,
            "use_parentheses": false,
            "order_by_type": true,
            "atomic": false,
            "lines_after_imports": -1,
            "lines_between_sections": 1,
            "lines_between_types": 0,
            "combine_as_imports": false,
            "combine_star": false,
            "keep_direct_and_as_imports": true,
            "include_trailing_comma": false,
            "from_first": false,
            "verbose": false,
            "quiet": false,
            "force_adds": false,
            "force_alphabetical_sort_within_sections": false,
            "force_alphabetical_sort": false,
            "force_grid_wrap": 0,
            "force_sort_within_sections": false,
            "lexicographical": false,
            "ignore_whitespace": false,
            "no_lines_before": [],
            "no_inline_sort": false,
            "ignore_comments": false,
            "case_sensitive": false,
            "sources": [],
            "virtual_env": "",
            "conda_env": "",
            "ensure_newline_before_comments": false,
            "directory": "",
            "profile": "",
            "src_paths": [],
            "old_finders": false,
            "source": "defaults"
        },
        {
            "multi_line_output": 3,
            "include_trailing_comma": true,
            "force_grid_wrap": 0,
            "use_parentheses": true,
            "ensure_newline_before_comments": true,
            "source": "black profile"
        },
        {
            "known_third_party": [
                "pylru",
                "werkzeug",
                "google",
                "flask_restful",
                "flask_uuid",
                "sqlalchemy",
                "flask_sqlalchemy",
                "sqlalchemy_utils",
                "datadog",
                "alembic",
                "requests",
                "environs",
                "boto3",
                "apispec",
                "factory",
                "flask_migrate",
                "pytz",
                "pytest",
                "structlog",
                "transitions",
                "geojson",
                "jwt",
                "flask",
                "marshmallow_enum",
                "apscheduler",
                "freezegun",
                "Crypto",
                "jsonschema",
                "marshmallow",
                "yaml",
                "dateutil"
            ],
            "known_first_party": [
                "common",
                "flights",
                "server",
                "utils",
                "models",
                "resources",
                "security",
                "app",
                "loggers",
                "tests",
                "alert",
                "temp",
                "integrations",
                "background_jobs",
                "settings",
                "detections"
            ],
            "profile": "black",
            "source": "<redacted>/.isort.cfg"
        }
    ],
    "virtual_env": "",
    "conda_env": "",
    "ensure_newline_before_comments": true,
    "directory": "<redacted>",
    "profile": "black",
    "src_paths": [
        "<redacted>/src",
        "<redacted>"
    ],
    "old_finders": false
}

[averypmc]

And the .isort.cfg that's generated from seed-isort-config with my additions to the first party section.

[settings]
known_third_party = Crypto,alembic,apispec,apscheduler,boto3,datadog,dateutil,environs,factory,flask,flask_migrate,flask_restful,flask_sqlalchemy,flask_uuid,freezegun,geojson,google,jsonschema,jwt,marshmallow,marshmallow_enum,pylru,pytest,pytz,requests,sqlalchemy,sqlalchemy_utils,structlog,transitions,werkzeug,yaml
known_first_party = <lots of first party packages>

[averypmc]

If I remove seed-isort-config and .isort.cfg and use only isort via pre-commit, I get the following for the section of code quoted above:

from app.create import (create_frontend_exposed_config, create_helpers,
                        create_resources, create_services, create_stores)
from app.init import (initialize_additional_extensions, initialize_config,
                      initialize_db, initialize_logging, initialize_scheduler,
                      initialize_security)
from app.mount import mount_api, mount_lifecycle_events
from apscheduler.schedulers.background import BackgroundScheduler
from flask import Flask
from models.common import db as models_db

So a number of local packages identified as third party it seems like.

                 _                 _
                (_) ___  ___  _ __| |_
                | |/ _/ / _ \/ '__  _/
                | |\__ \/\_\/| |  | |_
                |_|\___/\___/\_/   \_/

      isort your imports, so you don't have to.

                    VERSION 5.0.5

from-type place_module for app.create returned THIRDPARTY
from-type place_module for app.init returned THIRDPARTY
from-type place_module for app.mount returned THIRDPARTY
from-type place_module for apscheduler.schedulers.background returned THIRDPARTY
from-type place_module for flask returned THIRDPARTY
from-type place_module for models.common returned THIRDPARTY

[averypmc]

Ah interesting. Reading the link you posted it looks like things are default THIRDPARTY. So then I guess two follow up questions:

1) Is there a recommended/reliable way to detect first party/local imports in lieu of having to keep updating known_first_party?
2) If not, what do you think about updating the landing page for the docs (https://timothycrosley.github.io/isort/), to better explain what magic isort can and cannot do? E.g. if it's the case that you need to call out first party imports, would be nice to show that in the minimal example. Happy to help.

[timothycrosley]

@averypmc where does your .isort.cfg file live? In general if you have a directory structure like:

root\
    .isort.cfg
    module_1\
       _init_.py
    module_2\
       __init__.py

OR:

root\
    .isort.cfg\
    src\
      module_1\
         _init_.py
      module_2\
         __init__.py

isort can and will automatically find module_1 and module_2 and identify them as first party. However, if your modules are more scattered, or in a non typical src folder, you can simply do --src DIRECTORY_ABOVE_MODULES and isort can then again find it. You can add it multiple times, and you can set it in the config file as well. See: https://timothycrosley.github.io/isort/docs/configuration/options/#src-paths.

[averypmc]

Nice! Specifying the src and removing all of the known_first_party that I had added results in what I'd expect.

from apscheduler.schedulers.background import BackgroundScheduler
from flask import Flask

from app.create import (
    create_frontend_exposed_config,
    create_helpers,
    create_resources,
    create_services,
    create_stores,
)
from app.init import (
    initialize_additional_extensions,
    initialize_config,
    initialize_db,
    initialize_logging,
    initialize_scheduler,
    initialize_security,
)
from app.mount import mount_api, mount_lifecycle_events
from models.common import db as models_db

My directory structure:

.
├── .isort.cfg
├── .pre-commit-config.yaml
├── backend # Had to specify this as src
│   ├── ...
└── frontend
    ├── ...

Thanks for all the help @timothycrosley!

[timothycrosley]

Happy that worked for you! Thanks for working with me to diagnose it, this will help me know how to best improve the documentation :)