Pandas-profiling: Disabling Correlation Computation

Created on 8 Jun 2018  ·  5Comments  ·  Source: pandas-profiling/pandas-profiling

Hi,

My understanding is that the argument “check_correlation = False” still computes correlations in the background (because I see a correlation matrix at the bottom of a report) but doesn’t reject any highly correlated variables. I often need to run this report on datasets with several hundred variables and I suspect that correlation piece adds quite a bit to execution time.

Is there a way to completely disable correlation computation using the current set of arguments?

PS: Thanks for all your work on this awesome package. This is the best open source tool for EDA that I found so far.

Thanks,
Roman

Most helpful comment

same here, whenever the data set does not contain any numeric columns, there is no correlation to be calculated and check_correlation=False does not stop the calculation. I created a fake numeric column and the report gets generated fine with the fake data being the only thing in the correlation matrix.

Is there a timeline when this will be fixed?

All 5 comments

I think I've also stumbled across this issue. My data set is causing a division by zero error in the correlation, so I tried to bypass it with “check_correlation = False”, but still get the error.

same here, whenever the data set does not contain any numeric columns, there is no correlation to be calculated and check_correlation=False does not stop the calculation. I created a fake numeric column and the report gets generated fine with the fake data being the only thing in the correlation matrix.

Is there a timeline when this will be fixed?

Maybe not the write place but anyone has tried cramers_corrected_stat !!!!
for the cat variables ….. any tip …..

profile=ProfileReport(data,title="first profile",html={'style':{'full_width':True}},correlations={"cramers": {"calculate": False},"pearson": {"calculate": False},"spearman": {"calculate": False},"kendall": {"calculate": False},"phi_k": {"calculate": False}}

Alternatively, simply use correlations=None as a shorthand for the above

Was this page helpful?
0 / 5 - 0 ratings