Skip to content

BUG: Accept missing observations in string categorical inputs - #877

Open
jackiectl2 wants to merge 2 commits into
bashtage:mainfrom
jackiectl2:fix-string-categorical-missing
Open

jackiectl2 wants to merge 2 commits into
bashtage:mainfrom
jackiectl2:fix-string-categorical-missing

Conversation

@jackiectl2

Copy link
Copy Markdown
Contributor

IVData accepts string columns and converts them to categoricals, but its all-strings check also tests missing entries. A country/industry column containing None therefore raises a type error, while explicitly casting the same column to category correctly marks the observation as missing.

Check the observed entries when recognizing a string column. Keep the existing rejection of an entirely untyped missing column, and leave conversion/missing-row handling to the categorical path.

Validation (CPU, Python 3.12):

  • Six string/dtype regressions and a real IV2SLS fit fail before the fix.
  • Entire IV data test file: 31 passed.
  • Ruff lint/format and git diff --check pass.
  • Informational mypy reports 61 diagnostics in four files; normalized diagnostics match the unchanged base. This is not a clean type-check result.

The full estimator suite, documentation build and other pandas/Python versions were not run. No GPU is required.

@codecov

codecov Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.69%. Comparing base (d7980a7) to head (387189e).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #877   +/-   ##
=======================================
  Coverage   99.69%   99.69%           
=======================================
  Files         202      202           
  Lines       38871    38890   +19     
  Branches     2501     2501           
=======================================
+ Hits        38752    38771   +19     
  Misses         51       51           
  Partials       68       68           
Flag Coverage Δ
adder 99.68% <100.00%> (+<0.01%) ⬆️
subtractor 99.68% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant