Continue training in CLI if one iteration produces a single-leaf tree #5699
Conversation
As per discussion on GH-microsoft#5051 and GH-microsoft#5193 the python package does not stop training if a single leaf tree (stupm) is found and relies on early stopping methods to stop training. This commits removes the finish condition on training based on the result of `TrainOneIter()` and sets the `is_finished` flag on early stopping alone.
jameslamb
changed the title
There was a problem hiding this comment.
Thanks very much for returning to this!
I've updated the PR title to more closely match the intent of this PR. Please let me know if I've misunderstood.
I think this PR matches the outcome of the discussion in #5051, but I'm not qualified to review this. Hopefully @shiyu1994 or @guolinke can help with a review.
|
@shiyu1994 can you help with a review please? This has been open for several months now. |
| @@ -421,6 +419,8 @@ bool GBDT::TrainOneIter(const score_t* gradients, const score_t* hessians) { | |||
| models_.push_back(std::move(new_tree)); | |||
| } | |||
|
|
|||
| ++iter_; | |||
|
|
|||
| if (!should_continue) { | |||
There was a problem hiding this comment.
The one-leaf trees will be pop?
There was a problem hiding this comment.
Tested the result and stopping or using it as this is works the same (assumming there is no other built tree down the line). Incrementing the iter_ variable is needed for the boosting to take it as a finished tree (even if the tree is of no use). Otherwise it will continue forever.
There was a problem hiding this comment.
sorry for the late response. I think we should make the behavior the same as the python-package. I remember that the one-leaf tree will be kept in the python-package, but the current implementation will pop the one-leaf trees.
There was a problem hiding this comment.
@Samsagax what do you think about my comment?
|
It's been almost a year since the last activity on this PR. @guolinke what should we do with this? |
|
"I think we should make the behavior the same as the python-package". |
As per discussion on #5051 and #5193 the python package does not stop training if a single leaf tree (stupm) is found and relies on early stopping methods to stop training. This commits removes the finish condition on training based on the result of
TrainOneIter()and sets theis_finishedflag on early stopping alone.