數(shù)據(jù)分析入門-以鳶尾花分類為例

1、以鳶尾花分類為例本以鳶尾花數(shù)據(jù)為例，總結(jié)數(shù)據(jù)分析般過程，python數(shù)據(jù)分析庫的部分法，并完成鳶尾花分類模型構(gòu)建數(shù)據(jù)獲取以及導(dǎo)import requestsimport numpy as np#request.get(URL)可以讀取站信息，返回respose對象,將其存變量r中。r =requests.get(/ml/machine-learning-databases/iris/iris.data )#這返回r，可以看出rResponse對象r#os.getcwd()可以返回當(dāng)前編輯錄path =os.getcwd()pathC:Users44587Python#pythonwith op

2、en法以write模式在當(dāng)前path下創(chuàng)建iris.data并將存儲于r中的數(shù)據(jù)寫#response.text表獲取response中的本信息with open(path+iris.data,w) as f:f.write(r.text)#數(shù)據(jù)寫后使pandasread_csv法讀取件，names參數(shù)可賦值個list以更改列名df =pd.read_csv(path +iris.data,names =sepal length,sepal width,petal length,petal width,Class)探索性數(shù)據(jù)分析這部分的的是對數(shù)據(jù)有個總體的認知，并發(fā)現(xiàn)些明顯的信息，并且對數(shù)據(jù)進清

3、洗()RangeIndex: 150 entries, 0 to 149Data columns (total 5 columns):sepal length 150 non-null float64sepal width 150 non-null float64petal length 150 non-null float64petal width 150 non-null float64Class150 non-null objectdtypes: float64(4), object(1)memory usage: 5.9+ KB可以看出，數(shù)據(jù)是分完整整齊的，沒有缺失值。#查看數(shù)據(jù)表的統(tǒng)

4、計信息df.describe()05.8433330.8280664.3000005.1000005.8000006.4000007.90000003.0540000.4335942.0000002.8000003.0000003.3000004.400000.03.7586671.7644201.0000001.6000004.3500005.1000006.90000001.1986670.7631610.1000000.3000001.3000001.8000002.500000min25%50%75%maxdf.head()Class01234.65.03.53.00.2Iris-se

5、tosaIris-setosaIris-setosaIris-setosaIris-setosadf.iloc:3,:4sepal lengthsepal widthpetal lengthpetal width03.53.00.2df.loc:3,sepal lengthName: sepal length, dtype: float64#查看類別,.unique()可以返回列中所有不同數(shù)據(jù)，類似SQL中的uniquedf.Class.unique()array(Iris-setosa, Iris-versicolor, Iris-virginica, dtype=object)#查看詳細分

6、組信息,可以看到類別前50為Setosa類，中間50為Versicolor，后50為Virginicadf.groupby(Class).groupsIris-virginica: Int64Index(100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112,113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125,126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138,139, 1

7、40, 141, 142, 143, 144, 145, 146, 147, 148, 149,dtype=int64)df.count()dtype: int64#取出特征組成列表labels =list(df.columns:4)labelssepal length, sepal width, petal length, petal width#取出類別為Virginica的數(shù)據(jù)并重置indexdf1 =dfdf.Class =Iris-virginicadf1.reset_index()index1001011021031041051061071081091101111121131141

8、15116Class01Iris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginica1.927.13.046.53.05.82.257.63.01.81.82.510111213141

9、52.02.55.02.03.05.51.8Class171819202122232425262728293031323334353637383940414243444546474849Iris-virginica1181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481497.76.06.02.83.03.02.53.03.43.06.95.02.02.02.02.31.8Iris-virginicaIris-virginicaIris-virginicaIris

10、-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virgi

11、nicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginicaIris-virginica#df.corr()dfdf.iloc:,:5df.corr()sepal length1.000000-0.1093690.8717540.817954sepal width-0.1093691.000000-0.420516-0.356544petal length0.871754-0.4205161.0000000.962757petal width0.817954-0.3565440.

12、9627571.000000sepal lengthsepal widthpetal lengthpetal width#Seaborn庫是基于matplotlib的階繪圖庫，可以簡潔優(yōu)美的繪制圖形sns.pairplot(df,hue =Class)從圖中可以觀察到，petallength以及petal width兩特征可以較好的對鳶尾花進分類，后的randomforest也證實了這點，兩特征分類貢獻近乎各占四成。觀察圖1,0可以發(fā)現(xiàn)，setosa的sepallength 與sepalwidth有定的線性關(guān)系，后對此進線性回歸分析。#繪制提琴圖，展各個特征分類數(shù)據(jù)分布fig,ax =plt.

13、subplots(2,2,figsize (8,8)sns.violinplot(x =dfClass,ydfsepal length,ax ax0,0)sns.violinplot(x =dfClass,ydfsepal width,ax ax0,1)sns.violinplot(x =dfClass,ydfpetal length,ax ax1,0)sns.violinplot(x =dfClass,ydfpetal width,ax ax1,1)plt.tight_layout#繪制直圖查看sepal width的分布plt.style.use(ggplot)fig,ax =plt.su

14、bplots(1,1,figsize(4,4)ax.hist(dfsepal width,color =black)ax.set_xlabel(sepal width)plt.tight_layout()Setosa 的Sepal Width 與Sepal Length線性相關(guān)性分析#繪制兩特征散點圖fig,axes =plt.subplots(figsize =(7,7)axes.set_ylabel(Sepal length)axes.set_title(Setosa Sepal Width vs. Sepal Length,y =1.02)Text(0.5, 1.02, Setosa S

15、epal Width vs. Sepal Length)result =sm.OLS(y,).fit()print(result.summary()D:Anacondalibsite-packagesnumpycorefromnumeric.py:2389: FutureWarning: Method .ptp is deprecated and will be removed in a future version. Usenumpy.ptp instead.return ptp(axis=axis, out=out, *kwargs)OLS Regression Results=Metho

16、d:Date:Time:1Covariance Type:nonrobusttOmnibus:Prob(Omnibus):Skew:Kurtosis:2.599 Cond. No.1 Standard Errors assume that the covariance matrix of the errors is correctly specified.可以看出，回歸程：sepal length = 0.6909*sepal width +2.6447回歸系數(shù)t檢驗p-val較，結(jié)果顯著，由于元線性回歸的局限性，R以及Adjusted R并不太，模型擬合程度較低，但是模型F檢驗顯著。#在散點

17、圖中繪制回歸線plt.legend(loc =Best)D:Anacondalibsite-packagesipykernel_launcher.py:6: MatplotlibDeprecationWarning: Unrecognized location Best. Falling back on best; valid locationsThis will raise an exception in 3.3.使隨機森林構(gòu)建分類模型from sklearn.ensemble import RandomForestClassifierfrom sklearn.model_selection import train_test_split#構(gòu)建并訓(xùn)練分類器X=df.iloc:,:4y =df.iloc:,4X_train,X_test,y_train,y_test =train_test_spli