Explaining the Predictions of an APL Gradient Boosting Model

[[{“value”:”

APL Gradient Boosting from the hana-ml Python package comes with a built-in HTML chart that explains the predictions made by the model. This blog illustrates how to generate such a graph whether your machine learning scenario is a classification or a regression scenario.

General Versus Local Explanations

In machine learning two different levels of explanations are considered: general and local.

General explanations are available as soon as the predictive model is trained (fit operation). Thanks to them the user can understand what the most important variables are overall.

Local explanations require predictions to be made (predict operation). We then see the contributions of the variables for each individual prediction.

In both cases, general and local, APL uses the SHAP framework (SHapley Additive exPlanations) and presents the results in a pre-canned HTML chart.

Training Dataset

The two coming examples involve the same dataset census, for the sake of simplicity.

First things first, we connect to HANA Cloud:

from hana_ml import dataframe as hd
conn = hd.ConnectionContext(
address = ‘Host_String’, port = 443,
user = ‘USER_APL’, password = ‘Password_String’,
encrypt = ‘true’, sslValidateCertificate = ‘false’ )
conn.connection.isconnected()

Then we define the HANA dataframe for the training data:

sql_cmd = ‘select * from apl_samples.census order by “id”‘
hdf_train = hd.DataFrame(conn, sql_cmd)
print(“Number of Rows:”, hdf_train.count())
print(“Number of Columns:”, hdf_train.shape[1])
print(“Columns name:”, hdf_train.columns)
hdf_train.tail(6).collect()

The order by clause on the table’s primary key permits a repeatable APL model.

Regression scenario

For our first example, we train an APL Gradient Boosting model with a continuous target.

ID_COLUMN = “id”
CONTINUOUS_TARGET = ‘age’
from hana_ml.algorithms.apl.gradient_boosting_regression import GradientBoostingRegressor
rg_model = GradientBoostingRegressor(variable_auto_selection = True)
rg_model.fit(hdf_train, label=CONTINUOUS_TARGET, key=ID_COLUMN)

Once the model is trained, we prepare the HANA dataframe for the inference data:

sql_cmd = ‘select * from apl_samples.census order by “id” limit 100’
hdf_inf_rg = hd.DataFrame(conn, sql_cmd).deselect(CONTINUOUS_TARGET)
print(“Number of Rows:”, hdf_inf_rg.count())
print(“Number of Columns:”, hdf_inf_rg.shape[1])
print(“Columns name:”, hdf_inf_rg.columns)

The predictions by the APL model are as follows:

rg_model.predict(hdf_inf_rg).head(3).collect()

You can see the explanations in a table format:

df_rg_expl = rg_model.predict(hdf_inf_rg, prediction_type=”Explanations”)
df_rg_expl.collect().sort_values([ID_COLUMN, “Explanation_Rank”]).head(20)

But the explanations are easier to interpret when plotted in a chart:

rg_model.build_report()
rg_model.generate_notebook_iframe_report()

Each variable contribution is expressed in the unit of the target. On top, the Baseline bar is the average of the target: Age. The black bar at the bottom gives the Age prediction. In between are the different variables impacting negatively the target (red bars) or positively (green bars).

To make the output digestible, APL limits the number of bars. If there are more than 10 predictors, APL sums the small positive contributions into a group called “Positive Others”, and the small negative into a second group called “Negative Others”.

You may have noticed, in the pandas dataframe, that the predicted age is an integer value, whereas in the waterfall chart it has decimals. If you want the predicted values with decimals everywhere, insert this code just before the fit operation:

rg_model.set_params(
variable_storages= {‘age’: ‘number’},
variable_value_types={‘age’: ‘continuous’}
)

To share the waterfall chart with your business user, save the report as an HTML file like this:

rg_model.generate_html_report(‘apl_chart’)

Classification scenario

Our second example follows the same steps as in the first example, but we will see at the end that the resulting chart for local explanations is of a different kind.

We train, this time, a binary classification model.

ID_COLUMN = “id”
BINARY_TARGET = ‘class’
from hana_ml.algorithms.apl.gradient_boosting_classification import GradientBoostingBinaryClassifier
bc_model = GradientBoostingBinaryClassifier(variable_auto_selection = True)
bc_model.fit(hdf_train, label=BINARY_TARGET, key=ID_COLUMN)

Then we build the inference HANA dataframe.

sql_cmd = ‘select * from apl_samples.census order by “id” limit 100’
hdf_inf_bc = hd.DataFrame(conn, sql_cmd).deselect(BINARY_TARGET)
print(“Number of Rows:”, hdf_inf_bc.count())
print(“Number of Columns:”, hdf_inf_bc.shape[1])
print(“Columns name:”, hdf_inf_bc.columns)

Now, we can predict the binary class.

bc_model.predict(hdf_inf_bc).head(5).collect()

Here are the explanations for each prediction:

df_bc_expl = bc_model.predict(hdf_inf_bc, prediction_type=”Explanations”)
df_bc_expl.collect().sort_values([ID_COLUMN, “Explanation_Rank”]).head(20)

The contribution (last column), in the case of a classification model, is an unbounded value, not easy to understand. That’s where the Strength indicator (second to last column) comes into play:

Strength = Contribution / Sigma
Sigma being the standard deviation of all the contributions.

This normalized indicator, varying roughly between -6 sigma and +6 sigma, estimates for each row the impact that a given variable has on the predicted value. A positive strength means the variable pushes the prediction toward the positive class.

From the strength value, one can obtain a qualitative strength level by defining ranges like the following:

Last, we generate the chart:

The same kind of chart is provided by APL if you predict a multi-class target instead of a binary target.

To know more about APL

“}]] 

 [[{“value”:”APL Gradient Boosting from the hana-ml Python package comes with a built-in HTML chart that explains the predictions made by the model. This blog illustrates how to generate such a graph whether your machine learning scenario is a classification or a regression scenario. General Versus Local Explanations In machine learning two different levels of explanations are considered: general and local.General explanations are available as soon as the predictive model is trained (fit operation). Thanks to them the user can understand what the most important variables are overall.Local explanations require predictions to be made (predict operation). We then see the contributions of the variables for each individual prediction.In both cases, general and local, APL uses the SHAP framework (SHapley Additive exPlanations) and presents the results in a pre-canned HTML chart. Training Dataset The two coming examples involve the same dataset census, for the sake of simplicity.First things first, we connect to HANA Cloud:from hana_ml import dataframe as hd
conn = hd.ConnectionContext(
address = ‘Host_String’, port = 443,
user = ‘USER_APL’, password = ‘Password_String’,
encrypt = ‘true’, sslValidateCertificate = ‘false’ )
conn.connection.isconnected()Then we define the HANA dataframe for the training data:sql_cmd = ‘select * from apl_samples.census order by “id”‘
hdf_train = hd.DataFrame(conn, sql_cmd)
print(“Number of Rows:”, hdf_train.count())
print(“Number of Columns:”, hdf_train.shape[1])
print(“Columns name:”, hdf_train.columns)
hdf_train.tail(6).collect()The order by clause on the table’s primary key permits a repeatable APL model. Regression scenario For our first example, we train an APL Gradient Boosting model with a continuous target.ID_COLUMN = “id”
CONTINUOUS_TARGET = ‘age’
from hana_ml.algorithms.apl.gradient_boosting_regression import GradientBoostingRegressor
rg_model = GradientBoostingRegressor(variable_auto_selection = True)
rg_model.fit(hdf_train, label=CONTINUOUS_TARGET, key=ID_COLUMN)Once the model is trained, we prepare the HANA dataframe for the inference data:sql_cmd = ‘select * from apl_samples.census order by “id” limit 100’
hdf_inf_rg = hd.DataFrame(conn, sql_cmd).deselect(CONTINUOUS_TARGET)
print(“Number of Rows:”, hdf_inf_rg.count())
print(“Number of Columns:”, hdf_inf_rg.shape[1])
print(“Columns name:”, hdf_inf_rg.columns) The predictions by the APL model are as follows:rg_model.predict(hdf_inf_rg).head(3).collect()   You can see the explanations in a table format:df_rg_expl = rg_model.predict(hdf_inf_rg, prediction_type=”Explanations”)
df_rg_expl.collect().sort_values([ID_COLUMN, “Explanation_Rank”]).head(20)But the explanations are easier to interpret when plotted in a chart:rg_model.build_report()
rg_model.generate_notebook_iframe_report() Each variable contribution is expressed in the unit of the target. On top, the Baseline bar is the average of the target: Age. The black bar at the bottom gives the Age prediction. In between are the different variables impacting negatively the target (red bars) or positively (green bars).To make the output digestible, APL limits the number of bars. If there are more than 10 predictors, APL sums the small positive contributions into a group called “Positive Others”, and the small negative into a second group called “Negative Others”.You may have noticed, in the pandas dataframe, that the predicted age is an integer value, whereas in the waterfall chart it has decimals. If you want the predicted values with decimals everywhere, insert this code just before the fit operation:rg_model.set_params(
variable_storages= {‘age’: ‘number’},
variable_value_types={‘age’: ‘continuous’}
)To share the waterfall chart with your business user, save the report as an HTML file like this:rg_model.generate_html_report(‘apl_chart’)  Classification scenario Our second example follows the same steps as in the first example, but we will see at the end that the resulting chart for local explanations is of a different kind.We train, this time, a binary classification model.ID_COLUMN = “id”
BINARY_TARGET = ‘class’
from hana_ml.algorithms.apl.gradient_boosting_classification import GradientBoostingBinaryClassifier
bc_model = GradientBoostingBinaryClassifier(variable_auto_selection = True)
bc_model.fit(hdf_train, label=BINARY_TARGET, key=ID_COLUMN)Then we build the inference HANA dataframe.sql_cmd = ‘select * from apl_samples.census order by “id” limit 100’
hdf_inf_bc = hd.DataFrame(conn, sql_cmd).deselect(BINARY_TARGET)
print(“Number of Rows:”, hdf_inf_bc.count())
print(“Number of Columns:”, hdf_inf_bc.shape[1])
print(“Columns name:”, hdf_inf_bc.columns) Now, we can predict the binary class.bc_model.predict(hdf_inf_bc).head(5).collect()    Here are the explanations for each prediction:df_bc_expl = bc_model.predict(hdf_inf_bc, prediction_type=”Explanations”)
df_bc_expl.collect().sort_values([ID_COLUMN, “Explanation_Rank”]).head(20) The contribution (last column), in the case of a classification model, is an unbounded value, not easy to understand. That’s where the Strength indicator (second to last column) comes into play:Strength = Contribution / SigmaSigma being the standard deviation of all the contributions.This normalized indicator, varying roughly between -6 sigma and +6 sigma, estimates for each row the impact that a given variable has on the predicted value. A positive strength means the variable pushes the prediction toward the positive class.From the strength value, one can obtain a qualitative strength level by defining ranges like the following:ConditionStrength level> 3Strong Positive> 1Meaningful Positive> 0Weak Positive≥ -1Weak Negative≥ -3Meaningful Negative< -3Strong Negative Last, we generate the chart:  The same kind of chart is provided by APL if you predict a multi-class target instead of a binary target. To know more about APL “}]] Read More Technology Blog Posts by SAP articles 

#SAPCHANNEL