Handson-ml: Chapter 2 getting the Housing datasets, encountered file-not-found error

[vibhavk]

Hi Aurelien,

While preparing the housing data set in Chapter 2, I ran the function code in my Jupyter Notebook mentioned, to fetch the housing.tgz file it ran w/o any issues and then I ran the pandas function to return a Pandas dataframe, it ran without errors...but when I ran the following code to use DataFrame's head( ) method.

housing = load_housing_data( )
housing.head( )

the kernel throws a FileNotFoundError...and says this---->File b 'datasets/housing/housing.csv' not found.

I'm using virtualenv, Python3.5...

Following is the exact error I'm encountering....Please help me

FileNotFoundError                         Traceback (most recent call last)
<ipython-input-5-96e33597e42c> in <module>()
----> 1 housing = load_housing_data()
      2 housing.head

<ipython-input-4-6111fd061431> in load_housing_data(housing_path)
      2 def     load_housing_data(housing_path=HOUSING_PATH):
      3                                 csv_path        =       os.path.join(housing_path,      "housing.csv")
----> 4                                 return  pd.read_csv(csv_path)

~/env/lib/python3.5/site-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, skip_footer, doublequote, delim_whitespace, as_recarray, compact_ints, use_unsigned, low_memory, buffer_lines, memory_map, float_precision)
    703                     skip_blank_lines=skip_blank_lines)
    704 
--> 705         return _read(filepath_or_buffer, kwds)
    706 
    707     parser_f.__name__ = name

~/env/lib/python3.5/site-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds)
    443 
    444     # Create the parser.
--> 445     parser = TextFileReader(filepath_or_buffer, **kwds)
    446 
    447     if chunksize or iterator:

~/env/lib/python3.5/site-packages/pandas/io/parsers.py in __init__(self, f, engine, **kwds)
    812             self.options['has_index_names'] = kwds['has_index_names']
    813 
--> 814         self._make_engine(self.engine)
    815 
    816     def close(self):

~/env/lib/python3.5/site-packages/pandas/io/parsers.py in _make_engine(self, engine)
   1043     def _make_engine(self, engine='c'):
   1044         if engine == 'c':
-> 1045             self._engine = CParserWrapper(self.f, **self.options)
   1046         else:
   1047             if engine == 'python':

~/env/lib/python3.5/site-packages/pandas/io/parsers.py in __init__(self, src, **kwds)
   1682         kwds['allow_leading_cols'] = self.index_col is not False
   1683 
-> 1684         self._reader = parsers.TextReader(src, **kwds)
   1685 
   1686         # XXX

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader.__cinit__()

pandas/_libs/parsers.pyx in pandas._libs.parsers.TextReader._setup_parser_source()

FileNotFoundError: File b'datasets/housing/housing.csv' does not exist

[ageron]

Hi Vibhav,

Well it looks like the program does not find the file datasets/housing/housing.csv. Since this is a relative path, it means that the code is trying to find this file relative to the current directory, so please make sure that you started the Jupyter notebook from this project's root directory (i.e., the directory that contains the datasets directory, which should contain a housing directory, which should contain a housing.csv file).
Hope this helps.

[ganeshth]

Hi @vibhavk

Try running this code in the Jupyter notebook. You will get the current working directory of your Jupyter notebook. This is the path where the housing dataset gets downloaded and unzipped.
import os
os.getcwd()

Resolve the path conflicts based on the above command output.
(or)
Navigate to this directory and start the notebook. Hope this helps.

[Joe295159]

For people that have this problem and are dumb like me, make sure that you actually call(like below) before trying to load the data...

fetch_housing_data()

[ageron]

Thanks @ganeshth and @Joe295159 , very useful feedback. :)

[longbored]

I also ran into a problem at this function, and realized it was a macOS SSL error with the Python 3.6 installation. This tip here helped me: https://stackoverflow.com/a/42334357

(And only after looking at the readme file here do I see it's already mentioned. Doh!)

[datracker]

To solve the SSL Error for python 3.6 in macOS 10.6+, go to /Applications/Python 3.6 and open Install Certificates.command file, this will install a third party certifi package and solves the SSL error. You have to include fetch_housing_data() to run the function you wrote. When the function executes without error, you will see housing.csv and housing.tgz files are in the ~/datasets/housing folder.

[vibhavk]

Guys thanks you! issue resolved... :-)

[Awenjie10]

if you can't solve the problem,then you should try an easy way.
Download the cvs file from:https://github.com/ageron/handson-ml/tree/master/datasets/housing
then create the new folder /datasets/housing in mlbook.

[anthony-dk]

Thanks! But I tried the last option and got the following error
ParserError: Error tokenizing data. C error: Expected 1 fields in line 76, saw 6

Indeed (line 76 of the csv file):
\

And ignoring this line by making "load_housing_data" to return "pd.read_csv(csv_path, error_bad_lines=False)" doesn't help at all :(

Thanks in advance for your help!

[ageron]

Hi @anthony-dk ,
It looks like you downloaded an HTML page instead of the CSV file. On github, you need to click on "raw" to get the raw file instead of an HTML page that displays the contents nicely. Make sure to download the raw file:
https://raw.githubusercontent.com/ageron/handson-ml/master/datasets/housing/housing.csv
Hope this helps

[Amol1991]

import os
import tarfile
from six.moves import urllib

DOWNLOAD_ROOT = "https://raw.githubusercontent.com/ageron/handson-ml/master/"
HOUSING_PATH = os.path.join("datasets", "housing")
HOUSING_URL = DOWNLOAD_ROOT + "datasets/housing/housing.tgz"

def fetch_housing_data(housing_url=HOUSING_URL, housing_path=HOUSING_PATH):
    if not os.path.isdir(housing_path):
        os.makedirs(housing_path)
    tgz_path = os.path.join(housing_path, "housing.tgz")
    urllib.request.urlretrieve(housing_url, tgz_path)
    housing_tgz = tarfile.open(tgz_path)
    housing_tgz.extractall(path=housing_path)
    housing_tgz.close()

[Amol1991]

Dear Ageron sir,
can you explain the above coading its good but not undustandable.

[ageron]

Hi @Amol1991 ,
The function creates the ./datasets/housing directory if it does not exist yet (os.makedirs(...)), then it downloads the housing.tgz file at the URL "https://raw.githubusercontent.com/ageron/handson-ml/master/housing.tgz" and saves it in the ./datasets/housing directory (urlretrieve(...)), then it uncompresses it (extractall()).
Hope this helps,
Aurélien

[mathewsmutethia]

Hello @ageron
I am new in python and all this coding confuses me when trying to understand the book.
Will it hinder my capability of learning the content?
Regards
Mathews

[ageron]

Hi @mathewsmutethia ,
I tried to make the code as simple as possible, but I did assume that the reader would know how to program in Python. You can try to read on, but if you get confused too often with the code, I would suggest taking a break from the book, and getting a bit more experience with Python before you continue. There are tons of resources out there to learn Python, for example the official tutorial. There are also several books on the topic.
That said, most of the code I wrote is fairly straightforward. You need to know python basics, including lists, list comprehensions, dictionaries, strings, functions, lambdas, how to write and use a class, and the most common packages in the standard library (at least know what they do and where to get more documentation if needed), such as os, sys, json, time, logging, itertools, re, hashlib, zlib, tarfile and collections. I also assume that the reader knows scientific python libraries, including NumPy, SciPy, Matplotib and Pandas. If you don't know these, I wrote a few tutorials that you may find useful.
I hope this helps,
Aurélien

[Suveesh]

I had a same experience but then I realized the DOWNLOAD_ROOT = "https://raw.githubusercontent.com/ageron/handson-ml/master/" was copied directly from the e-book instead try to type it manually, that should work.
I just followed the instructions in the book and had no issues in retrieving the table head.

hope this helps!

[smt07]

For people that have this problem and are dumb like me, make sure that you actually call(like below) before trying to load the data...

fetch_housing_data()

fetch_housing_data()
housing = load_housing_data()
housing.head()

[smt07]

Good one :)

[rajnishkaushik]

I was missing the fetch_housing_data() as @smt07 has pointed out. Thanks much. However, now I am getting a 404 as the source file:
https://raw.githubusercontent.com/agreon/handson-ml2/master/datasets/housing/housing.tgz does not exist.

@ageron, please advise if the file location has changed. Thanks much