工作时长和收入间的线性关系 Linear Regression

本文对KGSS 2021年的调查数据中的 Y RINCOM0收入变量 和 X WGWKHR工作时长间的线性关系进行统计检验

Data Source

• 分析工具: Python statsmodels统计包
• 数据来源: KGSS

Go statsmodel Document 📄 Go KGSS Data 📊

Method

• 假设 Y：RINCOM0 X：WGWKHR
• 使用Python包 statsmodels 中的 OLS 线性回归函数

Import Library

import statsmodels.api as sm
import statsmodels.graphics.regressionplots as sgr
import pandas as pd
import numpy as np

Load Data

df = pd.read_spss("2003-2021_KGSS_public_v3_20230210.sav", convert_categoricals=False)
df.describe()

提取2021年份的数据进行分析

df_2021 = df[df.YEAR==2021]
df_2021

Pre-process

WGWKHR

df_2021.WGWKHR.describe()

count    1205.000000
mean       13.727801
std        20.376129
min        -8.000000
25%        -1.000000
50%        -1.000000
75%        40.000000
max        80.000000
Name: WGWKHR, dtype: float64

统计异常值总和

sum(df_2021["WGWKHR"]==-8)

7

将异常值替换为空(np.nan)

WGWKHR = df_2021["WGWKHR"]
WGWKHR.replace((-8,-1), np.nan, inplace=True)
WGWKHR.describe()

/var/folders/pm/fx5ldf494tqd843k9mpzck740000gn/T/ipykernel_2294/3763845172.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  WGWKHR.replace((-8,-1), np.nan, inplace=True)





count    476.000000
mean      36.386555
std       14.185602
min        2.000000
25%       35.000000
50%       40.000000
75%       45.000000
max       80.000000
Name: WGWKHR, dtype: float64

RINCOM

RINCOM = df_2021["RINCOM0"]
RINCOM.describe()

count    1205.000000
mean      174.807469
std       211.070790
min        -8.000000
25%        -1.000000
50%       150.000000
75%       300.000000
max      2400.000000
Name: RINCOM0, dtype: float64

替换异常值

RINCOM.replace((-8,-1), np.nan, inplace=True)
RINCOM.describe()

/var/folders/pm/fx5ldf494tqd843k9mpzck740000gn/T/ipykernel_2294/3581387875.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  RINCOM.replace((-8,-1), np.nan, inplace=True)





count     730.000000
mean      289.547945
std       200.331463
min         0.000000
25%       190.000000
50%       250.000000
75%       350.000000
max      2400.000000
Name: RINCOM0, dtype: float64

收入数据方差过大，容易对线性模型产生影响，因此将RINCOMlog标准化

RINCOM = np.log(RINCOM).replace(-np.inf,0)
RINCOM.describe()

/Users/jmsu/anaconda3/envs/KGSS/lib/python3.10/site-packages/pandas/core/arraylike.py:402: RuntimeWarning: divide by zero encountered in log
  result = getattr(ufunc, method)(*inputs, **kwargs)





count    730.000000
mean       5.394227
std        1.001719
min        0.000000
25%        5.247024
50%        5.521461
75%        5.857933
max        7.783224
Name: RINCOM0, dtype: float64

Simaple Linear Regression （OLS）

建立模型，拟合并输出结果

model = sm.OLS(RINCOM, sm.add_constant(WGWKHR),missing='drop')
result = model.fit()
result.summary()

可以看出$R^2=0.117$ 说明模型的拟合程度不足，很好解释，因为很明显自变量工作时长不是影响因变量收入的唯一因素， 需要其他变量的介入，拟合多元线性模型以提高解释能力。

根据参数t检验结果，可以看出工作时长与收入间的存在着显著的线性关系

画个回归拟合图：

sgr.plot_fit(result, 1)