pandas
一、简介
是基于Numpy的一种工具,为了解决数据分析任务而创建的,引入了大量库和一些标准的数据模型,提供了高效地操作大型数据集所需的工具。pandas提供了大量能使我们快速便捷地处理数据的函数和方法。
官网 《10分钟入门》
二、基本操作
1.导入库
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
2.创建对象
可以通过传递一个list对象来创建一个Series。
import pandas as pd
s = pd.Series([1,3,5,np.nan,6,8])
print (s)
输出:
0 1.0
1 3.0
2 5.0
3 NaN
4 6.0
5 8.0
dtype: float64
自己构造一个series
series_custom = Series(rt_scores , index=film_names)
一般对dataFrame进行操作,dataFrame是由series组成,series底层是一个ndarray结构。
3.传递一个numpy array,时间索引创建DataFrame
import pandas as pd
dates = pd.date_range('20130101', periods=6)
print (dates)
输出:
DatetimeIndex([‘2013-01-01’, ‘2013-01-02’, ‘2013-01-03’, ‘2013-01-04’,
‘2013-01-05’, ‘2013-01-06’],
dtype=’datetime64[ns]’, freq=’D’)
4.传递一个字典对象创建DataFrame
import pandas as pd
df2 = pd.DataFrame({ 'A' : 1.,
'B' : pd.Timestamp('20130102'),
'C' : pd.Series(1,index=list(range(4)),dtype='float32'),
'D' : np.array([3] * 4,dtype='int32'),
'E' : pd.Categorical(["test","train","test","train"]),
'F' : 'foo' })
print (df2)
输出:
A B C D E F
0 1.0 2013-01-02 1.0 3 test foo
1 1.0 2013-01-02 1.0 3 train foo
2 1.0 2013-01-02 1.0 3 test foo
3 1.0 2013-01-02 1.0 3 train foo
5.查看不同列的数据类型
import pandas as pd
df2 = pd.DataFrame({ 'A' : 1.,
'B' : pd.Timestamp('20130102'),
'C' : pd.Series(1,index=list(range(4)),dtype='float32'),
'D' : np.array([3] * 4,dtype='int32'),
'E' : pd.Categorical(["test","train","test","train"]),
'F' : 'foo' })
print (df2.dtypes)
输出:
dtype: object
5.数据读取,加载csv
import pandas
food_info = pandas.read_csv("food_info.csv")
print (type(food_info))
print (food_info.dtpe)
print (help(pandas.read_csv)) 返回函数的说明
展示前三行
food_info.head(3)
默认第一行作为表头。
展示最后几行
food_info.tail(4)
列名
food_info.columns
shape
food_info.shape
返回一共多少行。
取一列
ndb_col = food_info['NDB_No"]
print (ndbcol)
取多个列
columns = ["col1","col2]
zinc_copper = food_info[columns]
找某些列
col_name = food_info.columns.tolist()
print (col_names)
gram_columns = []
for c in col_names:
if c.endswith("(g)"):
gram_columns.append(c)
gram_df = food_info[gram_columns]
print (gram_df.head(3))
运行结果:
新加一个列
food_info["Iron_(g)"] = iron_grams
print (food_info.shape)
求最大值
max()
排序
food_info.sort_values("Sodium_(mg)", inplace=True)
print (food_info)
food_info.sort_values("Sodium_(mg)",inplace= True,ascending=False)
print (food_info["Sodium_(mg)"])
运行结果:
示例
import pandas as pd
import numpy as np
titanicsurvival = pd.read_csv("titanic_train.csv")
titanic_suvival.head()
age = titanic_surival["Age"] #年龄拿出来
print (age.loc[0:10])
# 没有没缺失值
age_is_numm = pd.isnull(age)
age_is_null = pd.isnull(age)
age_null_true = age[age_is_null]
age_null_count = len(age_null_true)
print (age_null_count)
输出有多少缺失值
mean_age = sum(titanic_survival["Age"]) / len(titanic_survival["Age"])
print (mean_age)
返回 nan
筛选
good_ages = titanic_survival["Age"].mean()
print (corrent_mean_age)
统计各仓位价格
passenger_classes = [1,2,3]
fares_by_class = {}
for this_class in passenger_classes:
pclass_rows = titanic_survival[titanic_survival["Pclass"] == this_class]
pclass_fares = pclass_rows["Fare"]
fare_for_class=pclass_fares.mean()
fares_by_class[this_class] = fare_for_class
print fares_by_class
筛选列
指定按哪个列排序
titanic_reindexed = new_titanic_survival.reset_index(drop=True)
把乱序重新指定排序。
自定义函数
# This function returns the nundredth item from the pool
def hundredth_row(column):
# Extract the hundredth item
hundredth_item = column.loc[99]
return hundredth_item
# Return the hundredth item from each column
hundredth_row = titanic_survival.apply(hundredth_row)
print (hundredth_row)
另一个例子:
把1,2,3转为对应字符串:
def which_class(row):
pclass = row['Pclass']
if pd.isnull(pclass):
return "Unknown"
elif pclass ==1:
return "First Class"
elif pclass ==2:
return Second Class"
elif pclass ==3:
return "Third Class"
classes = titanic_survival.apply(which_class,axis=1)
print (classes)
计算获救比例的示例:
匿名函数
下节课:
怎么建立模型、建立标准、预处理方法 ,基于样本不均衡数据。