企业级实战项目:基于 pycaret 自动化预测公司是否破产

发布时间:2023年12月26日

本文系数据挖掘实战系列文章,我跟大家分享一个数据挖掘实战,与以往的数据实战不同的是,用自动机器学习方法完成模型构建与调优部分工作,深入理解由此带来的便利与效果。

1. Introduction

在这里插入图片描述

本文是一篇数据挖掘实战案例,详细探索了从台湾经济杂志收集的1999年到2009年的数据,看看在数据探索过程中,可以洞察出哪些有用的信息,判断哪一个模型能够最准确地预测公司是否破产。

公司破产的定义是根据台湾证券交易所的商业规则而定的。

该建模将尝试使用自动机器学习库pycaret来构建机器学习模型,pycaret是一个用python编写的开源低代码机器学习库,它将机器学习工作流程自动化。如果你想探索这个库并更好地理解它的功能。推荐查看

设置环境并读取数据

import pandas as pd  
import numpy as np  
import math  
import matplotlib.pyplot as plt  
import seaborn as sns  
  
bankruptcy_df = pd.read_csv("Bankruptcy.csv")    

bankruptcy_df.head()  

技术交流&源码获取

技术要学会交流、分享,不建议闭门造车。一个人可以走的很快、一堆人可以走的更远。

好的文章离不开粉丝的分享、推荐,资料干货、资料分享、数据、技术交流提升,均可加交流群获取,群友已超过2000人,添加时最好的备注方式为:来源+兴趣方向,方便找到志同道合的朋友。

本文数据&源码,技术交流、按照如下方式获取:

方式①、添加微信号:dkl88194,备注:资料
方式②、微信搜索公众号:Python学习与数据挖掘,后台回复:资料

资料1
在这里插入图片描述
资料2

我们打造了《100个超强算法模型》,特点:从0到1轻松学习,原理、代码、案例应有尽有,所有的算法模型都是按照这样的节奏进行表述,所以是一套完完整整的案例库。

很多初学者是有这么一个痛点,就是案例,案例的完整性直接影响同学的兴致。因此,我整理了 100个最常见的算法模型,在你的学习路上助推一把!

在这里插入图片描述

2. 理解数据

bankruptcy_df.info()
<class 'pandas.core.frame.DataFrame'>  
RangeIndex: 6819 entries, 0 to 6818  
Data columns (total 96 columns):  
 #   Column                                                    Non-Null Count  Dtype    
  
---  ------                                                    --------------  -----  
  
 0   Bankrupt?                                                 6819 non-null   int64    
 1    ROA(C) before interest and depreciation before interest  6819 non-null   float64  
 2    ROA(A) before interest and % after tax                   6819 non-null   float64  
 3    ROA(B) before interest and depreciation after tax        6819 non-null   float64  
 4    Operating Gross Margin                                   6819 non-null   float64  
 5    Realized Sales Gross Margin                              6819 non-null   float64  
 6    Operating Profit Rate                                    6819 non-null   float64  
 7    Pre-tax net Interest Rate                                6819 non-null   float64  
 8    After-tax net Interest Rate                              6819 non-null   float64  
 9    Non-industry income and expenditure/revenue              6819 non-null   float64  
 10   Continuous interest rate (after tax)                     6819 non-null   float64  
 11   Operating Expense Rate                                   6819 non-null   float64  
 12   Research and development expense rate                    6819 non-null   float64  
 13   Cash flow rate                                           6819 non-null   float64  
 14   Interest-bearing debt interest rate                      6819 non-null   float64  
 15   Tax rate (A)                                             6819 non-null   float64  
 16   Net Value Per Share (B)                                  6819 non-null   float64  
 17   Net Value Per Share (A)                                  6819 non-null   float64  
 18   Net Value Per Share (C)                                  6819 non-null   float64  
 19   Persistent EPS in the Last Four Seasons                  6819 non-null   float64  
 20   Cash Flow Per Share                                      6819 non-null   float64  
 21   Revenue Per Share (Yuan ¥)                               6819 non-null   float64  
 22   Operating Profit Per Share (Yuan ¥)                      6819 non-null   float64  
 23   Per Share Net profit before tax (Yuan ¥)                 6819 non-null   float64  
 24   Realized Sales Gross Profit Growth Rate                  6819 non-null   float64  
 25   Operating Profit Growth Rate                             6819 non-null   float64  
 26   After-tax Net Profit Growth Rate                         6819 non-null   float64  
 27   Regular Net Profit Growth Rate                           6819 non-null   float64  
 28   Continuous Net Profit Growth Rate                        6819 non-null   float64  
 29   Total Asset Growth Rate                                  6819 non-null   float64  
 30   Net Value Growth Rate                                    6819 non-null   float64  
 31   Total Asset Return Growth Rate Ratio                     6819 non-null   float64  
 32   Cash Reinvestment %                                      6819 non-null   float64  
 33   Current Ratio                                            6819 non-null   float64  
 34   Quick Ratio                                              6819 non-null   float64  
 35   Interest Expense Ratio                                   6819 non-null   float64  
 36   Total debt/Total net worth                               6819 non-null   float64  
 37   Debt ratio %                                             6819 non-null   float64  
 38   Net worth/Assets                                         6819 non-null   float64  
 39   Long-term fund suitability ratio (A)                     6819 non-null   float64  
 40   Borrowing dependency                                     6819 non-null   float64  
 41   Contingent liabilities/Net worth                         6819 non-null   float64  
 42   Operating profit/Paid-in capital                         6819 non-null   float64  
 43   Net profit before tax/Paid-in capital                    6819 non-null   float64  
 44   Inventory and accounts receivable/Net value              6819 non-null   float64  
 45   Total Asset Turnover                                     6819 non-null   float64  
 46   Accounts Receivable Turnover                             6819 non-null   float64  
 47   Average Collection Days                                  6819 non-null   float64  
 48   Inventory Turnover Rate (times)                          6819 non-null   float64  
 49   Fixed Assets Turnover Frequency                          6819 non-null   float64  
 50   Net Worth Turnover Rate (times)                          6819 non-null   float64  
 51   Revenue per person                                       6819 non-null   float64  
 52   Operating profit per person                              6819 non-null   float64  
 53   Allocation rate per person                               6819 non-null   float64  
 54   Working Capital to Total Assets                          6819 non-null   float64  
 55   Quick Assets/Total Assets                                6819 non-null   float64  
 56   Current Assets/Total Assets                              6819 non-null   float64  
 57   Cash/Total Assets                                        6819 non-null   float64  
 58   Quick Assets/Current Liability                           6819 non-null   float64  
 59   Cash/Current Liability                                   6819 non-null   float64  
 60   Current Liability to Assets                              6819 non-null   float64  
 61   Operating Funds to Liability                             6819 non-null   float64  
 62   Inventory/Working Capital                                6819 non-null   float64  
 63   Inventory/Current Liability                              6819 non-null   float64  
 64   Current Liabilities/Liability                            6819 non-null   float64  
 65   Working Capital/Equity                                   6819 non-null   float64  
 66   Current Liabilities/Equity                               6819 non-null   float64  
 67   Long-term Liability to Current Assets                    6819 non-null   float64  
 68   Retained Earnings to Total Assets                        6819 non-null   float64  
 69   Total income/Total expense                               6819 non-null   float64  
 70   Total expense/Assets                                     6819 non-null   float64  
 71   Current Asset Turnover Rate                              6819 non-null   float64  
 72   Quick Asset Turnover Rate                                6819 non-null   float64  
 73   Working capitcal Turnover Rate                           6819 non-null   float64  
 74   Cash Turnover Rate                                       6819 non-null   float64  
 75   Cash Flow to Sales                                       6819 non-null   float64  
 76   Fixed Assets to Assets                                   6819 non-null   float64  
 77   Current Liability to Liability                           6819 non-null   float64  
 78   Current Liability to Equity                              6819 non-null   float64  
 79   Equity to Long-term Liability                            6819 non-null   float64  
 80   Cash Flow to Total Assets                                6819 non-null   float64  
 81   Cash Flow to Liability                                   6819 non-null   float64  
 82   CFO to Assets                                            6819 non-null   float64  
 83   Cash Flow to Equity                                      6819 non-null   float64  
 84   Current Liability to Current Assets                      6819 non-null   float64  
 85   Liability-Assets Flag                                    6819 non-null   int64    
 86   Net Income to Total Assets                               6819 non-null   float64  
 87   Total assets to GNP price                                6819 non-null   float64  
 88   No-credit Interval                                       6819 non-null   float64  
 89   Gross Profit to Sales                                    6819 non-null   float64  
 90   Net Income to Stockholder's Equity                       6819 non-null   float64  
 91   Liability to Equity                                      6819 non-null   float64  
 92   Degree of Financial Leverage (DFL)                       6819 non-null   float64  
 93   Interest Coverage Ratio (Interest expense to EBIT)       6819 non-null   float64  
 94   Net Income Flag                                          6819 non-null   int64    
 95   Equity to Liability                                      6819 non-null   float64  
dtypes: float64(93), int64(3)  
memory usage: 5.0 MB
bankruptcy_df.shape  
(6819, 96)  
bankruptcy_df.describe()  

3. 数据探索与清洗

3.1 缺失值处理

bankruptcy_df.columns[bankruptcy_df.isna().any()]  
Index([], dtype='object')  

从结果看,改数据集非常完整,没有缺失值!

.any() 指的是有没有(缺失值),而与之对应的.all()指的是是否都是(缺失值)

调整数据列名
def clean_col_names(col_name):  
    col_name = (  
        col_name.strip()  
        .replace("?", "_")  
        .replace("(", "_")  
        .replace(")", "_")  
        .replace(" ", "_")  
        .replace("/", "_")  
        .replace("-", "_")  
        .replace("__", "_")  
        .replace("'", "")  
        .lower()  
    )  
    return col_name  
  
bank_columns = list(bankruptcy_df.columns)  
bank_columns = [clean_col_names(col_name) for col_name in bank_columns]  
bankruptcy_df.columns = bank_columns  
display(bankruptcy_df.columns)
Index(['bankrupt_', 'roa_c_before_interest_and_depreciation_before_interest',  
       'roa_a_before_interest_and_%_after_tax',  
       'roa_b_before_interest_and_depreciation_after_tax',  
       'operating_gross_margin', 'realized_sales_gross_margin',  
       'operating_profit_rate', 'pre_tax_net_interest_rate',  
       'after_tax_net_interest_rate',  
       'non_industry_income_and_expenditure_revenue',  
       'continuous_interest_rate_after_tax_', 'operating_expense_rate',  
       'research_and_development_expense_rate', 'cash_flow_rate',  
       'interest_bearing_debt_interest_rate', 'tax_rate_a_',  
       'net_value_per_share_b_', 'net_value_per_share_a_',  
       'net_value_per_share_c_', 'persistent_eps_in_the_last_four_seasons',  
       'cash_flow_per_share', 'revenue_per_share_yuan_¥_',  
       'operating_profit_per_share_yuan_¥_',  
       'per_share_net_profit_before_tax_yuan_¥_',  
       'realized_sales_gross_profit_growth_rate',  
       'operating_profit_growth_rate', 'after_tax_net_profit_growth_rate',  
       'regular_net_profit_growth_rate', 'continuous_net_profit_growth_rate',  
       'total_asset_growth_rate', 'net_value_growth_rate',  
       'total_asset_return_growth_rate_ratio', 'cash_reinvestment_%',  
       'current_ratio', 'quick_ratio', 'interest_expense_ratio',  
       'total_debt_total_net_worth', 'debt_ratio_%', 'net_worth_assets',  
       'long_term_fund_suitability_ratio_a_', 'borrowing_dependency',  
       'contingent_liabilities_net_worth', 'operating_profit_paid_in_capital',  
       'net_profit_before_tax_paid_in_capital',  
       'inventory_and_accounts_receivable_net_value', 'total_asset_turnover',  
       'accounts_receivable_turnover', 'average_collection_days',  
       'inventory_turnover_rate_times_', 'fixed_assets_turnover_frequency',  
       'net_worth_turnover_rate_times_', 'revenue_per_person',  
       'operating_profit_per_person', 'allocation_rate_per_person',  
       'working_capital_to_total_assets', 'quick_assets_total_assets',  
       'current_assets_total_assets', 'cash_total_assets',  
       'quick_assets_current_liability', 'cash_current_liability',  
       'current_liability_to_assets', 'operating_funds_to_liability',  
       'inventory_working_capital', 'inventory_current_liability',  
       'current_liabilities_liability', 'working_capital_equity',  
       'current_liabilities_equity', 'long_term_liability_to_current_assets',  
       'retained_earnings_to_total_assets', 'total_income_total_expense',  
       'total_expense_assets', 'current_asset_turnover_rate',  
       'quick_asset_turnover_rate', 'working_capitcal_turnover_rate',  
       'cash_turnover_rate', 'cash_flow_to_sales', 'fixed_assets_to_assets',  
       'current_liability_to_liability', 'current_liability_to_equity',  
       'equity_to_long_term_liability', 'cash_flow_to_total_assets',  
       'cash_flow_to_liability', 'cfo_to_assets', 'cash_flow_to_equity',  
       'current_liability_to_current_assets', 'liability_assets_flag',  
       'net_income_to_total_assets', 'total_assets_to_gnp_price',  
       'no_credit_interval', 'gross_profit_to_sales',  
       'net_income_to_stockholders_equity', 'liability_to_equity',  
       'degree_of_financial_leverage_dfl_',  
       'interest_coverage_ratio_interest_expense_to_ebit_', 'net_income_flag',  
       'equity_to_liability'],  
      dtype='object')
统计并绘制目标变量

该步骤的目的是查看目标变量是否平衡,如果不平衡,则需要针对性处理。

class_bar=sns.countplot(data=bankruptcy_df,x="bankrupt_")  
ax = plt.gca()  
for p in ax.patches:  
        ax.annotate('{:.1f}'.format(p.get_height()), (p.get_x()+0.3, p.get_height()+500))  
class_bar  

3.2 特征分布

检查偏态
# Return true/false if skewed  
import scipy.stats  
skew_df = pd.DataFrame(bankruptcy_df.select_dtypes(np.number).columns, columns = ['Feature'])  
  
skew_df['Skew'] = skew_df['Feature'].apply(lambda feature: scipy.stats.skew(bankruptcy_df[feature]))   
  
skew_df['Absolute Skew'] = skew_df['Skew'].apply(abs)    
# 得到与方向无关的倾斜幅度  
skew_df['Skewed']= skew_df['Absolute Skew'].apply(lambda x: True if x>= 0.5 else False)  
with pd.option_context("display.max_rows", 1000):  
    display(skew_df)

可视化分布
cols = list(bankruptcy_df.columns)  
ncols = 8  
nrows = math.ceil(len(cols) / ncols)  
  
fig, ax = plt.subplots(nrows, ncols, figsize = (4.5 * ncols, 4 * nrows))  
for i in range(len(cols)):  
    sns.kdeplot(bankruptcy_df[cols[i]], ax = ax[i // ncols, i % ncols])  
    if i % ncols != 0:  
        ax[i // ncols, i % ncols].set_ylabel(" ")  
plt.tight_layout()  
plt.show()  

在这里插入图片描述

查看有偏态的特征
query_skew=skew_df.query("Skewed == True")["Feature"]  
with pd.option_context("display.max_rows", 1000):  
    display(query_skew)
0                                             bankrupt_  
2                 roa_a_before_interest_and_%_after_tax  
3      roa_b_before_interest_and_depreciation_after_tax  
4                                operating_gross_margin  
5                           realized_sales_gross_margin  
6                                 operating_profit_rate  
7                             pre_tax_net_interest_rate  
8                           after_tax_net_interest_rate  
9           non_industry_income_and_expenditure_revenue  
10                  continuous_interest_rate_after_tax_  
11                               operating_expense_rate  
12                research_and_development_expense_rate  
13                                       cash_flow_rate  
14                  interest_bearing_debt_interest_rate  
15                                          tax_rate_a_  
16                               net_value_per_share_b_  
17                               net_value_per_share_a_  
18                               net_value_per_share_c_  
19              persistent_eps_in_the_last_four_seasons  
20                                  cash_flow_per_share  
21                            revenue_per_share_yuan_¥_  
22                   operating_profit_per_share_yuan_¥_  
23              per_share_net_profit_before_tax_yuan_¥_  
24              realized_sales_gross_profit_growth_rate  
25                         operating_profit_growth_rate  
26                     after_tax_net_profit_growth_rate  
27                       regular_net_profit_growth_rate  
28                    continuous_net_profit_growth_rate  
29                              total_asset_growth_rate  
30                                net_value_growth_rate  
31                 total_asset_return_growth_rate_ratio  
32                                  cash_reinvestment_%  
33                                        current_ratio  
34                                          quick_ratio  
35                               interest_expense_ratio  
36                           total_debt_total_net_worth  
37                                         debt_ratio_%  
38                                     net_worth_assets  
39                  long_term_fund_suitability_ratio_a_  
40                                 borrowing_dependency  
41                     contingent_liabilities_net_worth  
42                     operating_profit_paid_in_capital  
43                net_profit_before_tax_paid_in_capital  
44          inventory_and_accounts_receivable_net_value  
45                                 total_asset_turnover  
46                         accounts_receivable_turnover  
47                              average_collection_days  
48                       inventory_turnover_rate_times_  
49                      fixed_assets_turnover_frequency  
50                       net_worth_turnover_rate_times_  
51                                   revenue_per_person  
52                          operating_profit_per_person  
53                           allocation_rate_per_person  
57                                    cash_total_assets  
58                       quick_assets_current_liability  
59                               cash_current_liability  
60                          current_liability_to_assets  
61                         operating_funds_to_liability  
62                            inventory_working_capital  
63                          inventory_current_liability  
64                        current_liabilities_liability  
65                               working_capital_equity  
66                           current_liabilities_equity  
67                long_term_liability_to_current_assets  
68                    retained_earnings_to_total_assets  
69                           total_income_total_expense  
70                                 total_expense_assets  
71                          current_asset_turnover_rate  
72                            quick_asset_turnover_rate  
73                       working_capitcal_turnover_rate  
74                                   cash_turnover_rate  
75                                   cash_flow_to_sales  
76                               fixed_assets_to_assets  
77                       current_liability_to_liability  
78                          current_liability_to_equity  
79                        equity_to_long_term_liability  
81                               cash_flow_to_liability  
83                                  cash_flow_to_equity  
84                  current_liability_to_current_assets  
85                                liability_assets_flag  
86                           net_income_to_total_assets  
87                            total_assets_to_gnp_price  
88                                   no_credit_interval  
89                                gross_profit_to_sales  
90                    net_income_to_stockholders_equity  
91                                  liability_to_equity  
92                    degree_of_financial_leverage_dfl_  
93    interest_coverage_ratio_interest_expense_to_ebit_  
95                                  equity_to_liability  
Name: Feature, dtype: object

进行下采样,直至样本集中的破产与非破产比例为50/50。完成之后再次对数据进行偏态检查,决定是否需要做log转换,另外进行相关矩阵分析。

3.3 下采样

首先对数据集进行下采样,目标比例为bankrupt vs non bankrupt = 50 vs 50

bankruptcy_df2 = bankruptcy_df.sample(frac=1) #Shuffle Bankruptcy df  
  
bankruptcy_df_b = bankruptcy_df2.loc[bankruptcy_df2["bankrupt_"] == 1]  
bankruptcy_df_nb = bankruptcy_df2.loc[bankruptcy_df2["bankrupt_"] == 0][:220]  
  
bankruptcy_subdf_comb = pd.concat([bankruptcy_df_b,bankruptcy_df_nb])  
bankruptcy_subdf = bankruptcy_subdf_comb.sample(frac=1,random_state=42)  
  
bankruptcy_subdf  

再次绘图查看正负样本数。

sns.countplot(bankruptcy_subdf["bankrupt_"])  

随机选择220家非破产公司和220家破产公司。

4. 特征工程

bankruptcy_subdf2 = bankruptcy_subdf.drop(["net_income_flag"],axis=1)  
bankruptcy_subdf2.shape  

(440, 95)  

4.1 相关矩阵

fig = plt.figure(figsize=(30,20))  
ax1 = fig.add_subplot(1,1,1)  
sns.heatmap(bankruptcy_subdf2.corr(),ax=ax1,cmap="coolwarm")  
4.1.1 找出与破产相关的最高特征

根据对破产企业的基本认识,破产企业资产少、负债高、盈利能力低、现金流少。可以朝这个方向分析我们的数据集。

corr=bankruptcy_subdf2[bankruptcy_subdf2.columns[:-1]].corr()['bankrupt_'][:]  
  
corr_df = pd.DataFrame(corr)  
  
print("Correlations to Bankruptcy:")  
for index, row in corr_df["bankrupt_"].iteritems():  
    if row!=1.0 and row>=0.5:  
        print(f'Positive Correlation: {index}')  
    elif row!=1.0 and row<=-0.5:  
        print(f'Negative Correlation: {index}')  
Correlations to Bankruptcy:  
Negative Correlation: roa_c_before_interest_and_depreciation_before_interest  
Negative Correlation: roa_b_before_interest_and_depreciation_after_tax  
Negative Correlation: net_value_per_share_b_  
Negative Correlation: net_value_per_share_a_  
Negative Correlation: net_value_per_share_c_  
Negative Correlation: persistent_eps_in_the_last_four_seasons  
Negative Correlation: per_share_net_profit_before_tax_yuan_¥_  
Positive Correlation: debt_ratio_%  
Negative Correlation: net_worth_assets  
Negative Correlation: net_profit_before_tax_paid_in_capital  
Negative Correlation: total_income_total_expense  

这些特征代表什么

  • roa_c_before_interest_and_depreciation_before_interest息前资产收益率和息前折旧:总资产收益率–如果总资产收益率低,破产风险高

  • roa_a_before_interest_and_after_tax息前和税后利润:总资产回报率–如果总资产回报率较低,破产风险较高

  • roa_b_before_interest_and_depreciation_after_tax利润不计利息及税后折旧:总资产回报率–如果总资产回报率较低,破产风险较高

  • debt_ratio负债率:负债占总资产的比例–价值越高,负债占资产的比例越高,导致破产风险越高

  • net_worth_assets净资产:净资产越少,破产风险越高

  • retained_earnings_to_total_assets留存收益与总资产之比:留存收益越少,破产风险越高

  • total_income_total_expense总费用:收入与费用之比较低,破产风险较高

  • net_income_to_total_assets净收入与总资产之比:净收入越低,破产风险越高

从结果看,导致公司违约风险越高的特征,似乎与背景知识一致。

4.2 下采样后特征分布可视化

# Visualisation of distributions after sub-sampling  
cols = list(bankruptcy_subdf2.columns)  
ncols = 8  
nrows = math.ceil(len(cols) / ncols)  
  
fig, ax = plt.subplots(nrows, ncols, figsize = (4.5 * ncols, 4 * nrows))  
for i in range(len(cols)):  
    sns.kdeplot(bankruptcy_subdf2[cols[i]], ax = ax[i // ncols, i % ncols])  
    if i % ncols != 0:  
        ax[i // ncols, i % ncols].set_ylabel(" ")  
plt.tight_layout()  
plt.show()  

在这里插入图片描述

4.3 所有特征的箱线图

plt.figure(figsize=(30,20))  
boxplot=sns.boxplot(data=bankruptcy_subdf2,orient="h")  
boxplot.set(xscale="log")  
plt.show()  

在这里插入图片描述

4.4 异常值处理

quartile1 = bankruptcy_subdf2.quantile(q=0.25,axis=0)  
# display(quartile1)  
quartile3 = bankruptcy_subdf2.quantile(q=0.75,axis=0)  
# display(quartile3)  
IQR = quartile3 -quartile1  
lower_limit = quartile1-1.5*IQR  
upper_limit = quartile3+1.5*IQR  
  
lower_limit = lower_limit.drop(["bankrupt_"])  
upper_limit = upper_limit.drop(["bankrupt_"])  
# print(lower_limit)  
# print(" ")  
# print(upper_limit)  
  
bankruptcy_subdf2_out = bankruptcy_subdf2[((bankruptcy_subdf2<lower_limit) | (bankruptcy_subdf2>upper_limit)).any(axis=1)]  
display(bankruptcy_subdf2_out.shape)  
display(bankruptcy_subdf2.shape)  
(423, 95)  
  
(440, 95)  

额外复制一份表,供后续分析处理。

bankruptcy_subdf3 = bankruptcy_subdf2_out.copy()  
bankruptcy_subdf3  

下采样后且去除离群值后的分布可视化。

# Visualisation of distributions after sub-sampling after outlier removal  
cols = list(bankruptcy_subdf3.columns)  
ncols = 8  
nrows = math.ceil(len(cols) / ncols)  
  
fig, ax = plt.subplots(nrows, ncols, figsize = (4.5 * ncols, 4 * nrows))  
for i in range(len(cols)):  
    sns.kdeplot(bankruptcy_subdf3[cols[i]], ax = ax[i // ncols, i % ncols],fill=True,color="red")  
    sns.kdeplot(bankruptcy_subdf2[cols[i]], ax = ax[i // ncols, i % ncols],color="green")  
    if i % ncols != 0:  
        ax[i // ncols, i % ncols].set_ylabel(" ")  
plt.tight_layout()  
plt.show()  

在这里插入图片描述

5 数据预处理

5.1 特征编码

所有类别在基础数据中都已编码完成,因此这里不需要再次编码列。在实际工作中,这一步大概率是必不可少的,编码技术也是尤其重要,需要好好掌握。如果你还不了解或不是很了解,推荐查看:

5.2 Log转换

这一步是为了去除数据中的偏态分布。

# Log transform to remove skews  
target = bankruptcy_subdf3['bankrupt_']  
bankruptcy_subdf4 = bankruptcy_subdf3.drop(["bankrupt_"],axis=1)  
  
def log_trans(data):  
    for col in data:  
        skew = data[col].skew()  
        if skew>=0.5 or skew<=0.5:  
            data[col] = np.log1p(data[col])  
        else:  
            continue  
    return data  
  
bankruptcy_subdf4_log = log_trans(bankruptcy_subdf4)  
bankruptcy_subdf4_log.head()  

5.2.1 Log转换数据的箱线图
plt.figure(figsize=(30,20))  
boxplot=sns.boxplot(data=bankruptcy_subdf4_log,orient="h")  
boxplot.set(xscale="log")  
plt.show()  

在这里插入图片描述

5.2.2 Log转换后的数据分布可视化
# 在下采样后、去除离群值及log变换后的数据分布的可视化  
compare_subdf2 = bankruptcy_subdf2.drop(["bankrupt_"],axis=1)  
  
cols = list(bankruptcy_subdf4.columns)  
ncols = 8  
nrows = math.ceil(len(cols) / ncols)  
  
fig, ax = plt.subplots(nrows, ncols, figsize = (4.5 * ncols, 4 * nrows))  
for i in range(len(cols)):  
    sns.kdeplot(bankruptcy_subdf4_log[cols[i]], ax = ax[i // ncols, i % ncols],fill=True,color="red")  
    sns.kdeplot(bankruptcy_subdf2[cols[i]], ax = ax[i // ncols, i % ncols],color="green")  
    if i % ncols != 0:  
        ax[i // ncols, i % ncols].set_ylabel(" ")  
plt.tight_layout()  
plt.show()  
print("Red represents distributions after log transforms, green represents before log transform")  

在这里插入图片描述

红色表示Log变换后的分布,绿色表示Log变换前的分布。(完整数据集:关注@公众号:数据STUDIO,联系云朵君获取)

6 使用Pycaret构建模型

本次模型构建使用的是自动机器学习框架pycaret,如果你还没有安装,可使用下述命令安装即可。

pip install -U --ignore-installed --pre pycaret  

在pycaret中自动完成训练及测试数据的切分工作。

from pycaret.classification import *  
exp_name = setup(data = bankruptcy_subdf4,  target = bankruptcy_subdf3["bankrupt_"])  

compare_models()  

Pycaret显示,3种模型的准确性最高的是

  • LightGBM分类器

  • 梯度提升GBC分类器

  • XGBoost分类器

接下来将使用这5个模型进行超参数调优。

6.1 选定模型交叉验证

LightGBM
print("LGBM Model")  
lgb_clf = create_model("lightgbm")  
lgb_clf_scoregrid = pull()  
LGBM Model  

GBC
print("GBC Model")  
gbc_clf = create_model("gbc")  
gbc_clf_scoregrid = pull()  
GBC Model  

XGBoost
print("XGB Model")  
xgb_clf = create_model("xgboost")  
xgb_clf_scoregrid = pull()  
XGB Model  

7 使用Pycaret进行超参数调优

7.1 模型调优

LightGBM
print("Before Tuning")  
print(lgb_clf_scoregrid.loc[["Mean","Std"]])  
print("")  
lgb_clf = tune_model(lgb_clf,choose_better=True)  
print(lgb_clf)  
Before Tuning  
      Accuracy     AUC  Recall   Prec.      F1   Kappa     MCC  
Fold                                                            
Mean    0.8433  0.9233  0.8562  0.8497  0.8495  0.6866  0.6929  
Std     0.0524  0.0429  0.0802  0.0681  0.0506  0.1046  0.1048  

GBC
print("Before Tuning")  
print(gbc_clf_scoregrid.loc[["Mean","Std"]])  
print("")  
gbc_clf = tune_model(gbc_clf,choose_better=True)  
print(gbc_clf)  
Before Tuning  
      Accuracy     AUC  Recall   Prec.      F1   Kappa     MCC  
Fold                                                         
Mean    0.8329  0.9242  0.8558  0.8324  0.8419  0.6649  0.6691  
Std     0.0599  0.0403  0.0634  0.0750  0.0557  0.1204  0.1198  

XGBoost
print("Before Tuning")  
print(xgb_clf_scoregrid.loc[["Mean","Std"]])  
print("")  
xgb_clf = tune_model(xgb_clf,choose_better = True)  
print(xgb_clf)  
Before Tuning  
      Accuracy     AUC  Recall   Prec.      F1   Kappa     MCC  
Fold                                                            
Mean    0.8400  0.9270  0.8562  0.8410  0.8460  0.6797  0.6852  
Std     0.0582  0.0382  0.0906  0.0586  0.0583  0.1161  0.1187  

7.2 模型集成

  1. Bagged & Boosting 方法

  2. Blending

  3. Stacking

LightGBM
# Original  
print(lgb_clf_scoregrid.loc[['Mean', 'Std']])  
  
# Compare the original against bagged and boosted  
  
# Bagged  
lgb_clf = ensemble_model(lgb_clf,fold =5,choose_better = True)  
# Boosted  
lgb_clf = ensemble_model(lgb_clf,method="Boosting",choose_better = True)  
      Accuracy     AUC  Recall   Prec.      F1   Kappa     MCC  
Fold                                                            
Mean    0.8433  0.9233  0.8562  0.8497  0.8495  0.6866  0.6929  
Std     0.0524  0.0429  0.0802  0.0681  0.0506  0.1046  0.1048

GBC
# Original  
print(gbc_clf_scoregrid.loc[['Mean', 'Std']])  
  
# Compare the original against bagged and boosted  
  
# Bagged  
gbc_clf = ensemble_model(gbc_clf,fold =5,choose_better = True)  
# Boosted  
gbc_clf = ensemble_model(gbc_clf,method="Boosting",choose_better = True)  
      Accuracy     AUC  Recall   Prec.      F1   Kappa     MCC  
Fold                                                            
Mean    0.8329  0.9242  0.8558  0.8324  0.8419  0.6649  0.6691  
Std     0.0599  0.0403  0.0634  0.0750  0.0557  0.1204  0.1198

XGBoost
# Original  
print(xgb_clf_scoregrid.loc[['Mean', 'Std']])  
  
# Compare the original and boosted against bagged and boosted  
  
# Bagged  
xgb_clf = ensemble_model(xgb_clf,fold =5,choose_better = True)  
# Boosted  
xgb_clf = ensemble_model(xgb_clf,method="Boosting",choose_better = True)  
      Accuracy     AUC  Recall   Prec.      F1   Kappa     MCC  
Fold                                                            
Mean    0.8400  0.9270  0.8562  0.8410  0.8460  0.6797  0.6852  
Std     0.0582  0.0382  0.0906  0.0586  0.0583  0.1161  0.1187

7.3.1 Blend Models
blend_models([lgb_clf, gbc_clf, xgb_clf],choose_better=True)  

7.3.2 Stacking
stacker = stack_models(lgb_clf,gbc_clf)  #remove xgb as some issues  

print(stacker)  

8 模型评估

# evaluate_model(lgb_clf)  
# evaluate_model(gbc_clf)  
# evaluate_model(xgb_clf)  

8.1 ROC-AUC

plot_model(stacker, plot = 'auc')     
# Stacked classifier from ensembling  
plot_model(lgb_clf, plot = 'auc')     
# lgb最适合Bagging集成并被选中  
plot_model(gbc_clf, plot = 'auc')     
# gbc最适合Boosting集成并被选中  
plot_model(xgb_clf, plot = 'auc')     
# 基本的xgb分类器在经过调优和集成后仍然表现最好,因此选择了它  

8.2 混淆矩阵

plot_model(stacker,   
           plot = 'confusion_matrix',   
           plot_kwargs = {'percent' : True})  
plot_model(lgb_clf,   
           plot = 'confusion_matrix',   
           plot_kwargs = {'percent' : True})  
plot_model(gbc_clf,   
           plot = 'confusion_matrix',   
           plot_kwargs = {'percent' : True})  
plot_model(xgb_clf,  
           plot = 'confusion_matrix',   
           plot_kwargs = {'percent' : True})  

8.3 学习曲线

plot_model(stacker, plot = 'learning')  

plot_model(lgb_clf, plot = 'learning')  

就到这里了!

文章来源:https://blog.csdn.net/qq_34160248/article/details/135232277
本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。