Pyatv: Scan is broken after update?

[strunker]

import asyncio
import time

import pyatv #for controlling apple tv
from pyatv import conf, connect, scan #for controlling apple tv
from pyatv.const import Protocol #for getting port

async def test():
    loop = asyncio.get_event_loop()
    atvs = await pyatv.scan(loop, timeout=5)
    print(atvs)

asyncio.run(test())

the above was working fine prior to the last update? Any idea what would have changed to break it?

[strunker]

Looks like as per the new docs instead of just calling scan and passing in the event loop, you need to specify protocol.

However, If I try the below I get back 0 results. I should have paid attention to the version I was on prior because scan worked without flaw but not sure what version to back rev to now.

atvs = await pyatv.scan(loop, protocol=Protocol.MRP,timeout=5)

[postlund]

You should not have to pass protocol, it should work and return all found devices. Can you roll back to last version that you know works, run atvremote --debug scan and then to the same with the latest version (or reverse order if you like).

[strunker]

This is from the most recent 0.7.4 version
c:\Python38\Scripts>atvremote.exe --debug scan
2020-11-02 12:49:17 DEBUG: Binding on *:5353
2020-11-02 12:49:17 DEBUG: Binding on 10.173.1.24:5353
2020-11-02 12:49:17 DEBUG: Binding on 192.168.56.1:5353
2020-11-02 12:49:17 DEBUG: Binding on 169.254.96.167:5353
2020-11-02 12:49:17 DEBUG: Binding on 10.1.1.26:5353
2020-11-02 12:49:17 DEBUG: Binding on 127.0.0.1:5353
2020-11-02 12:49:18 DEBUG: Error during MDNS lookup: [WinError 1231] The network location cannot be reached. For information about network troubleshooting, see Windows Help
2020-11-02 12:49:18 DEBUG: Error during MDNS lookup: [WinError 1231] The network location cannot be reached. For information about network troubleshooting, see Windows Help
2020-11-02 12:49:18 DEBUG: Failed to decode message (Msg=0000840000000012000000010b4c6976696e6720526f6f6d0f5f636f6d70616e696f6e2d6c696e6b045f746370056c6f63616c000010800100001194006416727042413d32443a35333a34463a36343a41433a323711727041443d3163633562336362326335610c7270466c3d30783336373832117270484e3d6638316536383334616636300f72704d643d4170706c655456362c320b727056723d3230302e3231095f7365727669636573075f646e732d7364045f756470c02d000c0001000011940002c018c018000c0001000011940002c00c0b4c6976696e6720526f6f6d0c5f6465766963652d696e666fc0280010000100001194000e0d6d6f646...)
2020-11-02 12:49:19 DEBUG: Error during MDNS lookup: [WinError 1231] The network location cannot be reached. For information about network troubleshooting, see Windows Help
2020-11-02 12:49:19 DEBUG: Error during MDNS lookup: [WinError 1231] The network location cannot be reached. For information about network troubleshooting, see Windows Help
Scan Results
========================================

c:\Python38\Scripts>

[strunker]

So. I spent some time going through all your recent versions. The below version is when it broke. If I had to guess it has to do with the change to zeroconf. But 0.7.4 and 0.7.3 are both broken when it comes to scan.

v0.7.3

[strunker]

Here is a debug from 0.7.3

c:\Python38\Scripts>atvremote.exe --debug scan
2020-11-02 13:17:10 DEBUG: Binding on *:5353
2020-11-02 13:17:10 DEBUG: Adding socket <socket.socket fd=804, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('0.0.0.0', 5353)>
2020-11-02 13:17:10 DEBUG: Binding on 10.173.1.24:5353
2020-11-02 13:17:10 DEBUG: Adding socket <socket.socket fd=968, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('10.173.1.24', 5353)>
2020-11-02 13:17:10 DEBUG: Binding on 192.168.56.1:5353
2020-11-02 13:17:10 DEBUG: Adding socket <socket.socket fd=792, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('192.168.56.1', 5353)>
2020-11-02 13:17:10 DEBUG: Binding on 169.254.96.167:5353
2020-11-02 13:17:10 DEBUG: Adding socket <socket.socket fd=156, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('169.254.96.167', 5353)>
2020-11-02 13:17:10 DEBUG: Binding on 10.1.1.26:5353
2020-11-02 13:17:10 DEBUG: Adding socket <socket.socket fd=148, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('10.1.1.26', 5353)>
2020-11-02 13:17:10 DEBUG: Binding on 127.0.0.1:5353
2020-11-02 13:17:10 DEBUG: Adding socket <socket.socket fd=992, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('127.0.0.1', 5353)>
2020-11-02 13:17:10 DEBUG: Failed to decode message (Msg=0000840000000011000000010b4c6976696e6720526f6f6d0f5f636f6d70616e696f6e2d6c696e6b045f746370056c6f63616c000010800100001194006416727042413d44363a46383a45343a35313a30343a393311727041443d3039316235343263326235320c7270466c3d30783336373832117270484e3d3663363165613035383834620f72704d643d4170706c655456362c320b727056723d3230302e3231c018000c0001000011940002c00c0b4c6976696e6720526f6f6d0c5f6465766963652d696e666fc0280010000100001194000e0d6d6f64656c3d4a313035614150c00c0021800100000078001400000000c0000b4c6976696e672d526...)
2020-11-02 13:17:11 DEBUG: Error during MDNS lookup: [WinError 1231] The network location cannot be reached. For information about network troubleshooting, see Windows Help
2020-11-02 13:17:11 DEBUG: Error during MDNS lookup: [WinError 1231] The network location cannot be reached. For information about network troubleshooting, see Windows Help
2020-11-02 13:17:11 DEBUG: Error during MDNS lookup: [WinError 1231] The network location cannot be reached. For information about network troubleshooting, see Windows Help
2020-11-02 13:17:11 DEBUG: Error during MDNS lookup: [WinError 1214] The format of the specified network name is invalid
2020-11-02 13:17:11 DEBUG: Error during MDNS lookup: [WinError 1214] The format of the specified network name is invalid
2020-11-02 13:17:11 DEBUG: Failed to decode message (Msg=0000840000000012000000010b4c6976696e6720526f6f6d0f5f636f6d70616e696f6e2d6c696e6b045f746370056c6f63616c000010800100001194006416727042413d44363a46383a45343a35313a30343a393311727041443d3039316235343263326235320c7270466c3d30783336373832117270484e3d3663363165613035383834620f72704d643d4170706c655456362c320b727056723d3230302e3231095f7365727669636573075f646e732d7364045f756470c02d000c0001000011940002c018c018000c0001000011940002c00c0b4c6976696e6720526f6f6d0c5f6465766963652d696e666fc0280010000100001194000e0d6d6f646...)
2020-11-02 13:17:12 DEBUG: Error during MDNS lookup: [WinError 1231] The network location cannot be reached. For information about network troubleshooting, see Windows Help
2020-11-02 13:17:12 DEBUG: Error during MDNS lookup: [WinError 1231] The network location cannot be reached. For information about network troubleshooting, see Windows Help
2020-11-02 13:17:12 DEBUG: Error during MDNS lookup: [WinError 1214] The format of the specified network name is invalid
2020-11-02 13:17:12 DEBUG: Error during MDNS lookup: [WinError 1214] The format of the specified network name is invalid
2020-11-02 13:17:12 DEBUG: Error during MDNS lookup: [WinError 1214] The format of the specified network name is invalid
Scan Results
========================================

[strunker]

And finally here it is working fine in the version prior 0.7.2

2020-11-02 13:19:48 DEBUG: Binding on *:5353
2020-11-02 13:19:48 DEBUG: Adding socket <socket.socket fd=492, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('0.0.0.0', 5353)>
2020-11-02 13:19:48 DEBUG: Binding on 10.173.1.24:0
2020-11-02 13:19:48 DEBUG: Adding socket <socket.socket fd=844, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('10.173.1.24', 56491)>
2020-11-02 13:19:48 DEBUG: Binding on 192.168.56.1:0
2020-11-02 13:19:48 DEBUG: Adding socket <socket.socket fd=892, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('192.168.56.1', 56492)>
2020-11-02 13:19:48 DEBUG: Binding on 169.254.96.167:0
2020-11-02 13:19:48 DEBUG: Adding socket <socket.socket fd=744, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('169.254.96.167', 56493)>
2020-11-02 13:19:48 DEBUG: Binding on 10.1.1.26:0
2020-11-02 13:19:48 DEBUG: Adding socket <socket.socket fd=496, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('10.1.1.26', 56494)>
2020-11-02 13:19:48 DEBUG: Binding on 127.0.0.1:0
2020-11-02 13:19:48 DEBUG: Adding socket <socket.socket fd=1032, family=AddressFamily.AF_INET, type=SocketKind.SOCK_DGRAM, proto=0, laddr=('127.0.0.1', 56495)>
2020-11-02 13:19:51 DEBUG: Auto-discovered Bedroom at 10.1.1.72:49153 (Protocol.MRP)
2020-11-02 13:19:51 DEBUG: Auto-discovered Bedroom at 10.1.1.72:7000 (Protocol.AirPlay)
2020-11-02 13:19:51 DEBUG: Auto-discovered Living Room at 10.1.1.184:49153 (Protocol.MRP)
2020-11-02 13:19:51 DEBUG: Auto-discovered Living Room at 10.1.1.184:7000 (Protocol.AirPlay)
Scan Results
========================================
       Name: Bedroom
   Model/SW: 4 tvOS 14.0.2 build 18J411
    Address: 10.1.1.72
        MAC: 08:66:98:A0:4A:98
 Deep Sleep: False
Identifiers:
 - 79D090C9-8983-45D2-985C-0EB41BED0DE3
 - 08:66:98:A0:4A:98
Services:
 - Protocol: MRP, Port: 49153, Credentials: None
 - Protocol: AirPlay, Port: 7000, Credentials: None

       Name: Living Room
   Model/SW: 4K tvOS 14.0.2 build 18J411
    Address: 10.1.1.184
        MAC: C8:D0:83:BC:B1:B9
 Deep Sleep: False
Identifiers:
 - B3CBB9AD-8902-466F-8D33-A7B1990CD335
 - C8:D0:83:BC:B1:B9
Services:
 - Protocol: MRP, Port: 49153, Credentials: None
 - Protocol: AirPlay, Port: 7000, Credentials: None

[strunker]

The difference there I am sure you will also note is the port. When it is set to 0 it looks to properly do discovery when it is set to 5353 it fails.

[oviano]

I am seeing exactly the same thing.

I'm switching between 0.7.2 and 0.7.3 using "pip3 install --upgrade git+https://github.com/postlund/pyatv.git@refs/tags/v0.7.X" and with 0.7.2. scan works reliably, with 0.7.3 it mostly returns no results and the same with 0.7.4.

[postlund]

I suspect this is because I changed the port I bind to from random to 5353, which I would expect should work (that what python-zeroconf seems to do). But I might have to revert that.

[strunker]

It is honestly faster, and arguably better, to statically assign your apple devices an address (can do this on router, and probably in the tv interface) and just direct connect to them. Personally, knowing that I can do this now and having it work properly with direct connect, I probably won't swap back to scan no real reason to do so.

[oviano]

Is the port number guaranteed though?

I already fix the IP but I assumed the port number could change so a scan was necessary.

[strunker]

Should be it is part of their protocol spec.

see below
https://support.apple.com/en-us/HT202944

5353 | UDP | Multicast DNS (MDNS) | 3927 | mdns | Bonjour, AirPlay, Home Sharing, Printer Discovery
-- | -- | -- | -- | -- | --

[oviano]

But that's just the mDNS, the actual port used when you connect to MRP comes up as 49152 for one of my ATVs and 49153 for another.

Unless I'm misunderstanding how it works, wouldn't it be the 49152/3 I'd need to specify for a direct connection.

[postlund]

You should not assume ports to be static, although some protocols do use static ports (e.g. 3389 for DMAP and 7000 for AirPlay). If you assign a static IP, you can use unicast scanning instead of regular scanning based on multicast. In that mode, MDNS requests are sent directly to the device IP instead of the global multicast address. This will yield a fast and accurate response, assuming the IP is correct and the host is up. Unicast scanning corresponds to passing -s to atvremote. I advise you to not fill in details manually, e.g. with --address, --port, etc. (again, assuming atvremote) as you will miss a lot of metadata, like device model, unique identifiers and such.

[oviano]

Right I’m with you, thanks for the tip.

[strunker]

Did not hardcode ports. Simply did the below. On my router I setup the below addresses to be static in DHCP, so even after reboot or arp reset these devices will pick up the same addresses. Its fast, works, and runs inside its own thread updating a global if the device settings were ever to change for some reason, or the ports to shift, it would automatically update the global and the rest of the code would be aware.

async def FindAppleTV():
    global AppleTVInfo
    while(1):
        try:
            loop = asyncio.get_event_loop()
            atvs = await pyatv.scan(loop, hosts=['10.1.1.72', '10.1.1.184'],timeout=5) #can no longer scan by just protocol.
            #Get TVs update ports
            for tv in atvs:
                tvName = tv.name
                tvPort = tv.get_service(Protocol.MRP).port
                AppleTVInfo[tvName]= tvPort
            print(Fore.BLUE + "\nApple TV Info: " + str(AppleTVInfo))
        except Exception as E:
            print("Error Apple TV Thread: " + str(E))
        time.sleep(60)

[oviano]

Yep, thanks I understand now. I hadn't noticed that the scan had a unicast option.

[postlund]

If I get time for it, I will revert the port change tonight. I will have to look through a few issues first, but I will try to make a release soon and bump in Home Assistant for the 1.0 release.

[strunker]

Thanks Pierre.

[postlund]

@strunker Can you try if #885 works better?

[strunker]

Looking good :) I get back the two objects and can iterate them for their info. Below is sample code that works.

import pyatv #for controlling apple tv
import asyncio
from pyatv import conf, connect, scan #for controlling apple tv
from pyatv.const import Protocol #for getting port


async def test():
    try:
        loop = asyncio.get_event_loop()
        atvs = await pyatv.scan(loop,timeout=5)
        print(atvs)
        for atv in atvs:
            print("Name: {0}, Address: {1}".format(atv.name, atv.address))
    except Exception as E:
        print("Error",E)

asyncio.run(test())

[, ]
Name: Living Room, Address: 10.1.1.184
Name: Bedroom, Address: 10.1.1.72
[Finished in 6.2s]

[postlund]

Thanks @strunker! I will merge it and make a release very soon.