Predict

XGBoost is great, but there may be concern that it results in models that are too good and overfitted. Are there ways to reduce the "tightness" of the model? In the estimation of the predictor coefficients? Related, is there any generally accepted measure of overfitting?

Within the PredictIQ model build settings the last tab called output allows you to ‘set population average for target’. This results in the predicted scores being scaled down to reflect the population average. It does not however make any adjustments to the actual results or the associated lifts from the model. This can be accomplished by appending model scores to the full modeling file and using the distribution of the targets and non-targets by decile proportionally increase the non-targets ba...

Building a revenue model and I see that Lift 1 over random =3.2 so decile 1 is 3.2 higher than the random. However, Numeric 1 vs 10 is only 2.83 -- what is numeric 1 vs 10 -- would expect it to be decile 1 vs decile 10, but of course it would be higher than 2.83 if that was the case.

Is there a way that I can tell how long a model run took?

If you took all responders and 1/3 of non-responders, I would historically build models using the Weight statement since not including it would affect the model calibration. In addition, the performance statistics such as lift need to be off the weighted data. How would you recommend this be handled in LityxIQ?

Can I output the scoring code for a model I create in LityxIQ?

Does PredictIQ have the ability to prune trees within models to avoid over fitting. I found this article which talks about this. http://www.statmethods.net/advstats/cart.html

What settings have the most impact on run times? I have already seen that if I cap the number of predictors from the default of 50 to 12 this has a big impact. I would assume looking for higher order terms is another one.

Does PREDICT have the ability to prune trees within models to avoid over fitting. I found this article which talks about this. http://www.statmethods.net/advstats/cart.html

User question received today. "When I’m running various iterations of a model (let’s saying Categorical Binning Yes/No and Numeric Binning Yes/No) there are 4 iterations. However, I can’t analyze each iteration separately. For example there are 4 iterations, with different numeric lifts. However, if I want to explore the model coefficients for the 4 iterations, I can’t. I am only left to look at iteration 2. I also couldn’t (for example) choose to put version 1.2.4 into production. I could only ...