Pyfa: Mac OS - Beta 3 bug

[gscottnelson]

Good afternoon,

Thanks for all your hard work on making this a great tool.

I am on Mac OS 10.13.3 (latest version)

I have run a clean install of this, and when I try to obtain my API's I get the following error on the screen.

Not sure what the issue is. I am getting this on my Mac laptop and my iMac as well..same error

THanks
Scott

[blitzmann]

Thanks for the report! I will try to investigate this this weekend :)

[Neugeniko]

I am unable to reproduce on Win 10 - py3 branch, from code or executable.

[blitzmann]

Thanks @Neugeniko, saved me a step!

[pabliedung]

Hi, I can confirm the same behavior as OP. Also on OSX 10.13.13 with same error message.
Thanks, Phil

[blitzmann]

Can't reproduce while running from source, I expect it may be an issue with how the requests module is packaged in the mac builds. Still investigating

[blitzmann]

Can't reproduce from beta 3 client.

Are you guys trying to do it from behind a firewall, maybe in a work environment? I know some companies manipulate SSL as part of their packet inspection and that could cause SSL errors. I'll continue looking into it best I can, but what I can do is limited if I can't reproduce the issue. :/

EDIT: I have 10.10 (because that's the lowest version I want pyfa to support), that might have something to do with it, but I also doubt it

[blitzmann]

@gscottnelson @pabliedung Can you check for log files? should be in ~/.pyfa/*.log, just attach them here, they may provide a bit more information

[pabliedung]

@blitzmann -- please see below:

pyfa-2018-03-02.log

[blitzmann]

@pabliedung Thanks!

[2018-03-02 18:11:51.545612] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:11:51.545899] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable
[2018-03-02 18:12:29.910305] WARNING: service.network: Timed out or other URL error:
[2018-03-02 18:12:29.910661] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:12:29.910896] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable
[2018-03-02 18:12:31.055959] WARNING: service.network: Timed out or other URL error:
[2018-03-02 18:12:31.056292] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:12:31.056524] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable
[2018-03-02 18:12:31.730094] WARNING: service.network: Timed out or other URL error:
[2018-03-02 18:12:31.730447] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:12:31.730687] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable
[2018-03-02 18:12:49.396311] WARNING: service.network: Timed out or other URL error:
[2018-03-02 18:12:49.396953] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:12:49.397231] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable
[2018-03-02 18:12:50.446231] WARNING: service.network: Timed out or other URL error:
[2018-03-02 18:12:50.446582] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:12:50.446816] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable
[2018-03-02 18:13:01.681911] WARNING: service.network: Timed out or other URL error:
[2018-03-02 18:13:01.682255] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:13:01.682487] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable
[2018-03-02 18:13:58.835874] WARNING: service.network: Timed out or other URL error:
[2018-03-02 18:13:58.837851] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:13:58.838159] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable
[2018-03-02 18:14:00.250584] WARNING: service.network: Timed out or other URL error:
[2018-03-02 18:14:00.250949] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:14:00.252123] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable
[2018-03-02 18:14:00.823879] WARNING: service.network: Timed out or other URL error:
[2018-03-02 18:14:00.824212] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:14:00.824444] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable
[2018-03-02 18:14:01.112000] WARNING: service.network: Timed out or other URL error:
[2018-03-02 18:14:01.112348] WARNING: service.network: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>
[2018-03-02 18:14:01.112583] ERROR: gui.characterEditor: argument of type 'ssl.SSLError' is not iterable

Bunch of SSL errors, it not being able to verify the certificate. This is usually due to being behind a corporate firewall that mucks with SSL certificates. Are you on a home computer? Can you check the 1.35.x releases to see if they have a similar problem?

[blitzmann]

It could also be due to 10.13 having different system-trusted certs than 10.10 (which is what I test with). I'm not really sure on that or how to test for that. Would be nice to get a few more mac user inputs with a variety of OS X versions

[pabliedung]

@blitzmann - the previous versions (been using pyfa for past 6 months) did not show this particular behavior. Been working fine on multiple Mac computers.

I initially installed beta3 on my MacBook Pro while at coffee shop (public WiFi) when I noticed the behavior. Also tried at home (Comcast xfinity) with same behavior. Also get the same error on my iMac (on 10.13.13 as well).

[blitzmann]

:/ kk, I'll have to dig into it a bit more. Are you able to retrieve pricing information on things? That also utilizes HTTPS, so I'm wondering if it's just the api requests or something more low level

[pabliedung]

no, it's not pulling any pricing in.

pyfa-2018-03-03.log

[gscottnelson]

My other computer has 10.12 so it gets same error there

Sent from my iPhone

On Mar 3, 2018, at 4:33 PM, Ryan Holmes <[email protected][e-mail]> wrote:

It could also be due to 10.13 having different system-trusted certs than 10.10 (which is what I test with). I'm not really sure on that or how to test for that. Would be nice to get a few more mac user inputs with a variety of OS X versions

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpyfa-org%2FPyfa%2Fissues%2F1452%23issuecomment-370181542&data=02%7C01%7Cscott.nelson%40realpage.com%7C78dd6c85b71b404a29e208d5814e6118%7C2c94bed6d6754d3da53b7b461fd6acc2%7C1%7C0%7C636557095996131141&sdata=sMkYDlUPs25%2FXmewAgGMOumvPb1g3n8S80IneEqd8fI%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAJjBBRj4r71eUoTGonDmwMHiocqlMPA9ks5tawwcgaJpZM4SY6an&data=02%7C01%7Cscott.nelson%40realpage.com%7C78dd6c85b71b404a29e208d5814e6118%7C2c94bed6d6754d3da53b7b461fd6acc2%7C1%7C0%7C636557095996131141&sdata=wDsCX%2FtHlrQ0O%2F2gqdgHXIIWmJUSQqVM4KRXTBHU2fY%3D&reserved=0.

---

This message is intended only for the use of the individual(s) or entity to which it is addressed and may contain information that is privileged, confidential, and/or proprietary to RealPage and its affiliated companies. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, forwarding or copying of this communication is prohibited without the express permission of the sender. If you have received this communication in error, please notify the sender immediately and delete the original message.

[gscottnelson]

I get it from work and home where I had no issue previously with the latest official release

Sent from my iPhone

On Mar 3, 2018, at 3:23 PM, Ryan Holmes <[email protected][e-mail]> wrote:

Can't reproduce from beta 3 client.

Are you guys trying to do it from behind a firewall, maybe in a work environment? I know some companies manipulate SSL as part of their packet inspection and that could cause SSL errors. I'll continue looking into it best I can, but what I can do is limited if I can't reproduce the issue. :/

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpyfa-org%2FPyfa%2Fissues%2F1452%23issuecomment-370177011&data=02%7C01%7Cscott.nelson%40realpage.com%7C10d0843c392c429909d308d58144933b%7C2c94bed6d6754d3da53b7b461fd6acc2%7C1%7C0%7C636557053881613455&sdata=nSsbR5j9wZRpFk2w1P4vjQnnxNVQZQ1W8mdOLEHpLuI%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAJjBBSrcR0_KGNlCCwPPkw8D9p4rYOyPks5tavuqgaJpZM4SY6an&data=02%7C01%7Cscott.nelson%40realpage.com%7C10d0843c392c429909d308d58144933b%7C2c94bed6d6754d3da53b7b461fd6acc2%7C1%7C0%7C636557053881613455&sdata=Ozw3dqTOFaw9YFKhvYVD4LsqAU5L0I1%2BFDS6PV5UHtA%3D&reserved=0.

---

This message is intended only for the use of the individual(s) or entity to which it is addressed and may contain information that is privileged, confidential, and/or proprietary to RealPage and its affiliated companies. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, forwarding or copying of this communication is prohibited without the express permission of the sender. If you have received this communication in error, please notify the sender immediately and delete the original message.

[blitzmann]

kk, thanks for the updates guys.

@pabliedung since it's not pulling prices in, this is most likely a cert problem. We used to include a cert file with pyfa when shipping the builds - I removed that functionality with 2.x due to hopes that our new packager did it for us. I'm running a few tests and will probably ask you guys to run a test build that I upload sometime tonight.

[blitzmann]

https://nofile.io/f/xSmg0kqJGpv/pyfa+2.zip

Can you guys open this build and then go to preferences. Take screenshots of the Database and Logging screens (the logging screen has a new field that's only for testing this issue, it's not going to be permanent). Post screenshots here.

[pabliedung]

[blitzmann]

Thanks. :)

I loaded beta 3 to my 10.11, and can reproduce with the beta 3 build. I haven't tried running from source yet, but I suspect it would work. Not really sure how to proceed from here though. Will have to play around with it a bit.

[Seavert]

Is it worth to fix XML which will be deleted in May 8?

[blitzmann]

Is it worth to fix XML which will be deleted in May 8?

This isn't an XML API-specific thing - this affects most HTTP requests in the mac build, so it is a high priority to get this done.

Good new though, I believe I found a workaround. I was able to get a small, simple test application together that didn't have all the additional fluff of pyfa to test with: built on my 10.10 machine, and tried it on my 10.11 machine. It failed, woo! I noticed that we are using urllib instead of requests, which I thought we were using. The reason behind this is long and convoluted, but changing the test app to use the requests library seems to work after building it on both 10.10 and 10.11. I'm not sure, but I suspect pyinstaller isn't bundling the certs correctly for the mac build when using urllib - I can make a bug report on that later to get more information.

There are some other problems with this though. If I were to change this over, most of the network settings would have to be disabled until a proper fix for proxy support can be worked out. Additionally, the EVE XML library that we use uses urllib, so to get that to work I would need to modify the library. I am not sure of the amount of work involved, and it very well may be doable, just wanted to make a quick update as to the progress of this issue.

[blitzmann]

@gscottnelson @pabliedungPlease try this build out, I switched everything over to use the requests module and it has worked for my tests. I wasn't able to get it done for beta 4 release earlier today, but if this works, feel free to use this build until beta 5 comes out :)

https://nofile.io/f/HE1UmtGEUyT/pyfa.zip

[pabliedung]

@blitzmann the build you shared is working. I was able to add characters and fetch skills properly. Market prices for ships, fittings etc are also pulled in properly again.

[blitzmann]

Oh happy days 😄 Thanks for the feedback!

[gscottnelson]

It worked for me. Connected was able to pull characters and skills.

From: Ryan Holmes [e-mail]
Reply-To: pyfa-org/Pyfa [e-mail]
Date: Sunday, March 4, 2018 at 5:26 PM
To: pyfa-org/Pyfa [e-mail]
Cc: Scott Nelson [e-mail], Mention [e-mail]
Subject: Re: [pyfa-org/Pyfa] Mac OS - Beta 3 bug (#1452)

@gscottnelsonhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgscottnelson&data=02%7C01%7Cscott.nelson%40realpage.com%7Cf74106a82e544afb241808d582274f62%7C2c94bed6d6754d3da53b7b461fd6acc2%7C1%7C0%7C636558027708407990&sdata=UMJd8vW32%2FVPMJ57DCspURATKwttesJOCHDNYIerAPQ%3D&reserved=0 @pabliedungplease try this build out, I switched everything over to use the requests module and it has worked for my tests. I wasn't able to get it done for beta 4 release earlier today, but if this works, feel free to use this build until beta 5 comes out :)

https://nofile.io/f/HE1UmtGEUyT/pyfa.ziphttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnofile.io%2Ff%2FHE1UmtGEUyT%2Fpyfa.zip&data=02%7C01%7Cscott.nelson%40realpage.com%7Cf74106a82e544afb241808d582274f62%7C2c94bed6d6754d3da53b7b461fd6acc2%7C1%7C0%7C636558027708407990&sdata=UK97fA5zDsFzTrA%2Fn8%2FaYJhqM94cOsZnA6KTlkgpcK8%3D&reserved=0

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpyfa-org%2FPyfa%2Fissues%2F1452%23issuecomment-370274433&data=02%7C01%7Cscott.nelson%40realpage.com%7Cf74106a82e544afb241808d582274f62%7C2c94bed6d6754d3da53b7b461fd6acc2%7C1%7C0%7C636558027708407990&sdata=GyzX7oRNfdOKP%2BgF5gCVlyO1jnfeOhZr0Vfa3BBrK1A%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAJjBBWiUjg7wqbt_TE2LbUNZvuQz4KdVks5tbHgPgaJpZM4SY6an&data=02%7C01%7Cscott.nelson%40realpage.com%7Cf74106a82e544afb241808d582274f62%7C2c94bed6d6754d3da53b7b461fd6acc2%7C1%7C0%7C636558027708407990&sdata=q29eqdBkC%2BwmVhf0y8hQrlAHwTfRWNdSM91H8pwOiBs%3D&reserved=0.

---

This message is intended only for the use of the individual(s) or entity to which it is addressed and may contain information that is privileged, confidential, and/or proprietary to RealPage and its affiliated companies. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, forwarding or copying of this communication is prohibited without the express permission of the sender. If you have received this communication in error, please notify the sender immediately and delete the original message.