Model Hyperparameter tuning XGBOOST REGRESSOR

AI Discussions
1

To improve the accuracy of XGBOOST Regressor I performed hyperparameter tuning-

  • Step 1: Adjusted parameters such as num_iterations and max_depth.
    • Step 2: Tuned learning_rate and subsample size.
    • Step 3: With the best values from the above steps, I applied regularization techniques using reg_lambda (L2) and reg_alpha (L1) to manage overfitting.

Despite these efforts, the performance improvements were marginal. The model showed only a 0.2% increase in AUC,. Please give me some suggestions how can I improve the accuracy with alternative solution or if I am missing something as I am just starting out my career in machine learning and AI

2

for one to response to your query, requires to know about your data, data frequency(variation), how to split the data, what type of data you are working(categorical/continuous, independent/dependent), data split
you also need to share screenshot of your first result and the result you stating only increased to 0.2 AUC.

Also check if your data classifiers are highly imbalanced.

3

I agree with the previous reply.

It’s possible that the results you got were as good as is possible for that dataset and model type.

You haven’t provided enough information to say otherwise.