Repository navigation
[FIX] Pass alpha from TweedieGLM to TweedieRegressor - #1663
Conversation
TweedieGLM stored alpha but built its TweedieRegressor without it, so every fit used scikit-learn's default penalty of 1.0. With alpha passed, TweedieGLM(alpha=0) on ukmotor reproduces the chain ladder ultimates. Closes casact#1584. Developed with an AI assistant (Árbol, running Muse).
TweedieGLM defaults to alpha=1.0, so the unpenalized ODP fit that matches chain ladder needs alpha=0 now that TweedieGLM passes alpha to TweedieRegressor. Developed with an AI assistant (Árbol, running Muse).
Pyright Type CompletenessView the full Project (full
Other symbols referenced but not exported by
Symbols without documentation:
Patch (exported symbols added or changed by this PR): 0.0% fully typed (0 / 1)
Patch symbol details
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1663 +/- ##
=======================================
Coverage 95.03% 95.03%
=======================================
Files 100 100
Lines 6460 6471 +11
Branches 819 819
=======================================
+ Hits 6139 6150 +11
Misses 203 203
Partials 118 118
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| glm = cl.TweedieGLM(alpha=0).fit(ukmotor) | ||
| assert glm.model_.estimator_ml.named_steps["model"].alpha == 0 | ||
| assert ( | ||
| abs( |
There was a problem hiding this comment.
i see this testing pattern pop up in this test file. it feels less precise than most other tests in the repo. i wonder if we can tighten up all the abs(relative diff) tests now alpha is being fed in.
There was a problem hiding this comment.
Done in 52b786e. test_basic_odp_cl, test_sample_weight and test_alpha_reaches_regressor now fit with alpha=0 and use np.testing.assert_allclose with rtol=1e-5. Each compares the ultimates with Chainladder, and the first and third also compare ldf_ with Development. Before, the bound on the largest ultimate gap over the latest diagonal was 1e-2 for the two genins tests and 1e-3 for ukmotor.
The largest relative gap I measured is 5.7e-07 on genins and 1.4e-07 on ukmotor, with or without the unit sample weight, so 1e-5 leaves room for solver noise across platforms. A higher max_iter or a smaller tol doesn't change either value. On main all three tests fail, with gaps of 1.2e-04 in the factors and 2.0e-04 in the ultimates on genins.
(This comment was written with Árbol, an AI assistant, under my direction.)
test_basic_odp_cl, test_sample_weight and test_alpha_reaches_regressor compared ultimates through max(|difference| / latest diagonal) with bounds of 1e-2 and 1e-3. Now that TweedieGLM passes alpha through, the three tests fit with alpha=0 and use np.testing.assert_allclose with rtol=1e-5 on the ultimates, and the genins and ukmotor tests also on ldf_ against Development. The largest measured relative differences are 5.7e-07 on genins and 1.4e-07 on ukmotor. On main all three fail. Developed with an AI assistant (Árbol, running Muse).
With the casact#1584 fix merged in casact#1663, TweedieGLM passes alpha to its regressor, so the example no longer needs a hand-built DevelopmentML pipeline. Both genins fits and the ultimate ratios are unchanged. Developed with an AI assistant (Árbol, running Muse).
Summary of Changes
TweedieGLMstoredalphabut built itsTweedieRegressorwithout it, so every fit used scikit-learn's default penalty of 1.0. The change passesalpha=self.alphato the regressor inglm.py.The
plot_glm_ldfgallery example now setsalpha=0, since the default penalty of 1.0 would keep its GLM factors from matchingDevelopment. With both changes the largest factor gap ongeninsdrops from 0.00042 to 0.0000008. The docs build executes notebooks, so the stored plot output stays as it is.test_alpha_reaches_regressorfitsTweedieGLM(alpha=0)toukmotorand checks that the regressor'salphais 0. With alpha passed through,test_basic_odp_cl,test_sample_weightandtest_alpha_reaches_regressornow fit withalpha=0and usenp.testing.assert_allclosewithrtol=1e-5on the ultimates againstChainladder, and the first and third also onldf_againstDevelopment. Before, they bounded the largest ultimate gap divided by the latest diagonal at 1e-2 or 1e-3. The largest measured relative gap is 5.7e-07 ongeninsand 1.4e-07 onukmotor. All three fail on main and pass with the change.Related GitHub Issue(s)
Closes #1584
AI/LLM Usage
This change was made with Árbol, an AI assistant running on Muse, working under my direction. It found the bug, wrote the change, the test and the gallery edit, ran the checks below and drafted this description.
Additional Context for Reviewers
C(development) + C(origin), log link, power 1:ukmotorDevelopmentandChainladderukmotorTweedieGLM(alpha=0)on mainukmotorTweedieGLM(alpha=0)with the changegeninsDevelopmentandChainladdergeninsTweedieGLM(alpha=0)on maingeninsTweedieGLM(alpha=0)with the changeThe default stays
alpha=1.0, so a fit that omitsalphagives the same numbers as before.With the change,
TweedieGLM(design_matrix=..., alpha=0, max_iter=1000)reproduces theDevelopmentML(..., TweedieRegressor(alpha=0, ...))fits in [DOCS] Show origin-varying ldf_ from DevelopmentML in the user guide #1621 exactly ongenins, for both formulas in that example.[CHORE] Change all project references from chainladder-python to chainladder (#1595) #1656 also edits
test_glm.py, and the two branches merge without conflict.On main at a59efc9,
chainladder/development/tests/test_glm.pygives 5 failed and 6 passed, and with the change 11 passed.chainladder/developmentgives 173 passed with the change, and ruff check and format are clean on the changed files.Submitter's Checklist
[FIX],[FEAT],[DOCS],[TST],[CHORE], or[BRK]).Reviewer's Checklist