Channels: Install channels without daphne

[ostcar]

Currently channels has daphne as dependency. So it is not possible to install channels on one machine without installing daphne with its dependencies.

The only place in code of channels, which needs daphne, is the runserver command, which kind of a development command that should not be used in production. So there is no need to require daphne as a requirment for a production setup.

Could it be possible to create a package, that you can install without installing daphne as well?

Our background is as follows: We have a django-channels app, that has be used also on windows from low-tech-users. But it is quite hard to install a current version of twisted on windows. So we build geiss as an replacement for daphne, which does not need twisted. But with the current package of channels, it is not possible to get rid of twisted, even when we don't want to use daphne.

But I think, that there are other use cases where someone wants to install channels without daphne. For example if you want to separate the worker on different containers/hardware then the protocol server.

[andrewgodwin]

This is a good point, but I also want to ensure that people who pip install channels still end up with a working runserver command, because if we change it to pip install channels[daphne] it's going to result in a lot more people being very confused.

Do you know of a way we could have a default dependency in Python that's possible to exclude?

[ostcar]

I have no idea if it is possible to have optional default dependencies in python.

Another way to solve this would be, if renamed the current package to something like channels-solo and removed daphne as dependency and created a new (meta-)package, which as daphne and channels-solo as dependency.

[proofit404]

My guess is that making runserver command implementation independent of Daphne code base will be the right decision. For example we can make Daphne itself depends on runserver at this moment and eventually move it totally into fully async twisted stack.

[andrewgodwin]

That will still destroy the first time user experience though. I even think having runserver soft-depend on daphne and print Please install daphne the first time is a bit annoying. Let me see if I can think of another way.

[proofit404]

Maybe I wasn't clean. runserver will work out of the box without installing Daphne. So first impression still be the same. But when you deploy to production you'll install Daphne explicitly.

[ostcar]

@andrewgodwin Did you find time to think about this issue?

My preference would be, if you created two packages. A package called "channels" that has no dependency to daphne and does not ship the runserver command. Another package called for example "channels-environment", ships the runserver command and has all the dependencies.

[andrewgodwin]

I have given it some thought, but there's not an ideal solution yet. I don't really want to make two packages as channels is already 5 packages as it is , but if I did, it would be something like channels-light and channels, so that pip install channels still does the expected thing.

It's possible this can be done with a single setup.py even, just with environment variables that switch to the -light build during the release process, but that still makes things a bit over-complicated.

The other part of the issue is that so far, you're the only person requesting the change, so it's quite low priority at the moment compared to other issues that are around.

[ThaJay]

I like the idea of runserver not depending on daphne because ws4redis does not run uwsgi directly, instead it runs runserver which does not work any more once channels is installed and runserver gets hijacked to use daphne instead. This makes it very difficult to run both channels and ws4redis (as long as our users need to get their apps up to date).

edit: we will just uninstall ws4redis

[rixx]

I like the idea of a separate package version (pip install channels[standalone]). This would be way more doable if setuptools would allow for one of the defined extras to be default, but there does not seem to be a way to achieve this. But I don't think that "Please install channels[standalone]" or a similar message is too much of a stretch, and I'd prefer it a lot over having to install a separate package.

[proofit404]

The only way I know how to do it is some think like have empty PyPI package called channels which depends on daphne and channels-standalone. Python package channels will be provided by channels-standalone.

[andrewgodwin]

The pip install channels[standalone] idea is that standalone would have runserver support? As in, when you run runserver, that's what prints if you installed non-standalone?

[rixx]

That was my idea, yes. I know that having a channels[bare] with channels as-is including runserver would be much nicer, but apparently that's just not possible with plain setuptools.

[creeon2003]

Great idea ...

[bastbnl]

@ostcar did you consider that you'll need to run manage.py runworkers in production in order to get the websockets running? Or did you find a way around that as well?

[ostcar]

Yes, I have to call manage.py runworkers but the runworker could be started without daphne. The the import statements in the files for runworker and runserver.

So I am not sure, if I understand your question correctly.

[bastbnl]

Thanks @ostcar. Your statement concerning manage.py in production got me wondering if you've found a way to run the workers in some other way.

My question was totally unrelated to daphne, indeed.

[andrewgodwin]

I haven't found a good solution to this even in the Channels 2.0 refactor, so I am going to close this as an effective WONTFIX. If someone can think of a clever solution that preserves pip install channels as working and installing Daphne, we can re-open.

[proofit404]

Maybe we can have two separate setup.py for classic channels and channels-bare packages. The second one is channels without dependencies and the first one depends on the second one and Daphne and ASGI and etc..

But I don't know if we really want to support this complexity.

[andrewgodwin]

I just don't think it's worth the overhead to maintain and publish two separate packages. If someone cannot install Daphne for some reason, they can download and build their own package from the source plus a patch easily.

[reorx]

Currently the solution workaround for me is to install channels with --no-deps option, and then install its dependencies without daphne separately:

pip install --no-deps channels
pip install asgiref

In practice I do use channels without daphne, because uvicorn seems to be a better choice than daphne, and since ASGI technology is evolving actively, I think make channels a pure library without server could becoming more and more considerable.

One thing also worth noticing is that daphne requires twisted, which is very bloated comparing to uvloop required by uvicorn...

[rneilson]

If I could make an argument to reconsider this issue: the problem with requiring daphne is its dependency on Twisted. At least the current version (20.3.0) requires a compiler, which when using eg uvicorn instead, and creating a container using eg python:3.8-slim, means extra build steps and bloat. The workaround above - installing channels without dependencies - is very much a _workaround_, not a solution.

[carltongibson]

So looking on the pip issue tracker, there's this closed issue, "Way to specify --no-deps option in requirements.txt?"

There's a suggested approach:

If you still wanted to maintain a single file, I don't think it would be hard to just pre-process your requirements.txt and invoke pip separately as suggested by @pfmoore above, like

grep -v -e --no-deps requirements.txt > requirements-with-deps.txt
grep -e --no-deps requirements.txt > requirements-without-deps.txt
pip install -r requirements-without-deps.txt --no-deps
pip install -r requirements-with-deps.txt

Looking at uvicorn, they've now got $ pip install uvicorn[standard] as the default, because you can't specify just the base package as an extra... — which is a bit 😬 — it would be nicer if there were a better approach.

Maybe we can think about this in the future, but don't have the bandwidth to think about re-opening it right now. (_Just use --no-deps_ seems a fairer division of labour.)

[oTree-org]

How about we have just 1 channels package, but allow users to set an env var that skips installation of Daphne. Channels would check for it in setup.py:

install_requires=[
    'Django>=2.2',
    'asgiref>=3.2.10,<4',
]

if not os.getenv('CHANNELS_SLIM_INSTALL'):
    install_requires.append('daphne>=3.0,<4')

setup(
    name='channels',
    packages=find_packages(exclude=['tests']),
    install_requires=install_requires,
    # etc...
)

(The env var could also be called CHANNELS_INSTALL_WITHOUT_DAPHNE etc.)

Also in apps.py, we would wrap the daphne import in a try/except.

[oTree-org]

Here is a diff with the changes I propose: https://github.com/django/channels/compare/3.0.2...oTree-org:master

[oTree-org]

Hi, any feedback on my above approach and linked PR? To summarize:

• It addresses the use case of (1) people wanting a slimmer install for their own projects, and (2) package authors who want to use Channels as a dependency but don't want the extra weight of all its dependencies, especially Twisted.
• It doesn't affect the "first time" user experience
• Installing different packages based on env vars is a commonly used technique, for example: https://github.com/certbot/certbot/blob/master/certbot-dns-google/setup.py

[carltongibson]

Hey @oTree-org. I haven't had bandwidth to look at it properly yet, but it does look like it resolves the concerns at first pass yes.

[oTree-org]

OK thank you :)

[andersk]

@oTree-org That doesn’t work. pip resolves requirements not by running setup.py for every package, but by reading a metadata file (channels.egg-info/requires.txt) included in the PyPI upload. Any conditionals need to be expressed there.

[riconnon]

I would recommend setting daphne as an extras_require perhaps