Python For Data Analysis learning path

In the introductory chapter, an example of processing the MovieLens 1M data set is introduced. The book introduces that the data set comes from GroupLens Research (), this address will jump directly to it, which provides various evaluation data sets from the MovieLens website, and you can download the corresponding compressed package. The MovieLens 1M data set we need is also there. in.

The downloaded and decompressed folder is as follows:

These three dat tables will be used in the example. The Chinese version (PDF) of "Python For Data Analysis" I read is the first edition in 2014. All the examples in it are written based on Python 2.7 and pandas 0.8.2, and I installed Python 3.5.2 and pandas 0.8.2. pandas 0.20.2, some functions and methods in it will be quite different. Some of them are parameters changed in the new version, while some are deprecated in the new version. This caused me to run according to the book When sample code, you will encounter some Errors and Warnings. When testing the MovieLens 1M data set code, under the same configuration environment as mine, I will encounter the following problems.

• When reading dat data into a pandas DataFrame object, the code given in the book is:

  users = pd.read_table('ml-1m/users.dat', sep='::', header=None, names=unames)
  
  rnames = ['user_id', 'movie_id', 'rating', 'timestamp']
  ratings = pd.read_table('ml-1m/ratings.dat', sep='::', header=None, names=rnames)
  
  mnames = ['movie_id', 'title', 'genres']
  movies = pd.read_table('ml-1m/movies.dat', sep='::', header=None, names=mnames)

  When running directly, a Warning will appear:

  F:/python/HelloWorld/DataAnalysisByPython-1.py:4: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.
    users = pd.read_table('ml-1m/users.dat', sep='::', header=None, names=unames)
  F:/python/HelloWorld/DataAnalysisByPython-1.py:7: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.
    ratings = pd.read_table('ml-1m/ratings.dat', sep='::', header=None, names=rnames)
  F:/python/HelloWorld/DataAnalysisByPython-1.py:10: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support regex separators (separators > 1 char and different from '\s+' are interpreted as regex); you can avoid this warning by specifying engine='python'.
    movies = pd.read_table('ml-1m/movies.dat', sep='::', header=None, names=mnames)

  Although it can also be run, as a perfect obsessive-compulsive disorder, I still want to solve this Warning . This warning means that because the 'C' engine does not support it, it can only fall back to the 'Python' engine, and there happens to be an engine parameter in the pandas.read_table method, which is used to set which parsing engine to use, including 'C' and 'Python' These two options. Since the 'C' engine does not support it, we only need to set the engine to 'Python'.

  users = pd.read_table('ml-1m/users.dat', sep='::', header=None, names=unames, engine = 'python')
  
  rnames = ['user_id', 'movie_id', 'rating', 'timestamp']
  ratings = pd.read_table('ml-1m/ratings.dat', sep='::', header=None, names=rnames, engine = 'python')
  
  mnames = ['movie_id', 'title', 'genres']
  movies = pd.read_table('ml-1m/movies.dat', sep='::', header=None, names=mnames, engine = 'python')

• Use the pivot_table method to calculate the average score of each movie by gender on the aggregated data. The code given in the book is:

  mean_ratings = data.pivot_table('rating', rows='title', cols='gender', aggfunc='mean')

  If you run it directly, an error will be reported and this code cannot be run:

  Traceback (most recent call last):
    File "F:/python/HelloWorld/DataAnalysisByPython-1.py", line 19, in <module>mean_ratings = data.pivot_table('rating', rows='title', cols='gender', aggfunc='mean')
  TypeError: pivot_table() got an unexpected keyword argument 'rows'</module>

  TypeError indicates that the 'rows' parameter here is not a keyword parameter available in the method. What is going on? I checked the pandas API usage documentation () on the official website and found that the keyword parameters in pandas.pivot_table have changed in version 0.20.2. In order to achieve the same effect, just replace rows with index. That's it, and there is no cols parameter, so use columns instead.

  mean_ratings = data.pivot_table('rating', index='title', columns='gender', aggfunc='mean')

• #In order to understand the favorite movies of female audiences, use the DataFrame method to sort column F in descending order , the sample code in the book is:

  top_female_ratings = mean_ratings.sort_index(by='F', ascending=False)

  This only gives a Warning and will not interfere with the program:

  F:/python/HelloWorld/DataAnalysisByPython-1.py:32: FutureWarning: by argument to sort_index is deprecated, pls use .sort_values(by=...)
    top_female_ratings = mean_ratings.sort_index(by='F', ascending=False)

  This means that the sort_index method for sorting may change in the language or library in the future, and it is recommended to use sort_values ​​instead. In the API usage documentation, the description of pandas.DataFrame.sort_index is "Sort object by labels (along an axis)", while the description of pandas.DataFrame.sort_values ​​is "Sort by the values ​​along either axis". Both can To achieve the same effect, then I will just replace it with sort_values. Sort_index will also be used in the following "Calculate score difference", and can also be replaced by sort_values.

  top_female_ratings = mean_ratings.sort_values(by='F', ascending=False)

• The last error is still related to sorting. After calculating the standard deviation of the score data in "Calculate Rating Difference", sort the Series in descending order according to the filtered value. The code in the book is:

  print(rating_std_by_title.order(ascending=False)[:10])

  这里的错误是：

  Traceback (most recent call last):
    File "F:/python/HelloWorld/DataAnalysisByPython-1.py", line 47, in <module>print(rating_std_by_title.order(ascending=False)[:10])
    File "E:\Program Files\Python35\lib\site-packages\pandas\core\generic.py", line 2970, in __getattr__return object.__getattribute__(self, name)
  AttributeError: 'Series' object has no attribute 'order'</module>

  居然已经没有这个order的方法了，只好去API文档中找替代的方法用。有两个，sort_index和sort_values，这和DataFrame中的方法一样，为了保险起见，我选择使用sort_values：

  print(rating_std_by_title.sort_values(ascending=False)[:10]

  得到的结果和数据展示的结果一样，可以放心使用。

第三方库不同版本间的差异还是挺明显的，建议是使用最新的版本，在使用时配合官网网站上的API使用文档，轻松解决各类问题~

The above is the detailed content of Python For Data Analysis learning path. For more information, please follow other related articles on the PHP Chinese website!