Repository navigation
Read Portuguese given names a particle completes, in two tiers (#1118) - #1120
Merged
Merged
Conversation
rules/P10 read only Spanish-measured words, so Maria das Graças Silva and Maria de Lurdes Costa read the phrase as the start of the surname. Re-measured over the whole ORCID extract in Portuguese-speaking countries: eight words reach the 90% bar and are marked by default (lurdes, céu, glória, tarso, ribamar, kássia, amparo); seven more, below the bar but well above a surname's share, are marked default = false in pt.toml, read under locales.PT and locales.PT_BR alone (graças, graça, cássia, fatima, piedade). Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Mark unaccented fatima on its Spanish counts (91 to 0), on by default in es.toml as fátima is, where it had sat in pt.toml's pack tier on its Portuguese counts alone; María de Fatima López read family 'de Fatima López' beside María de Fátima López's middle 'de Fátima'. - State the pack tier's actual cut (75% up to the bar, ten given records) in decisions/P10's Standing and the vocab README, and record that an unaccented spelling is measured on its own (kassia parts from kássia). - Label the ORCID agreement rows as the capped corpus and give the population that could move; re-measure every row after the fatima move. Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Standing: an unaccented spelling under ten given records takes its twin's reading (ceu, graca, gracas), as the Log and pt.toml say. - Record that a strict Portuguese choice does not read the words es.toml marks (de Fátima, de Fatima, do Socorro, de Lourdes), accepted here. - Pin unaccented Fatima's default reading with a rules/P10 example; it fails with es.toml's reading removed. - State the 75% cut as a chosen threshold, with the words either side. Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
…tuguese-particle-given-names # Conflicts: # tests/v2/test_locales.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1118.
rules/P10 (a given name a particle completes) read only words marked on Spanish counts, so Portuguese compound given names read the phrase as the start of the surname:
Maria das Graças Silva→ familydas Graças Silva,Maria de Lurdes Costa→ familyde Lurdes Costa. #1057 had measured Portuguese-speaking countries on ORCID's capped sample (20,000 records a country) and found no word at its 90% bar. This PR re-measures over the whole extract and marks the Portuguese words in two tiers. This moves default parses.What changes
pt.tomlgainsgivenreadings writtenafter_particle, each recording its counts. The counts compare a given box ending in a particle run plus the word against a family box opening with one and holding more words, over BR PT AO MZ CV GW ST TL.lurdes42:1,céu31:1,gloria25:1,tarso53:3,ribamar17:1,kássia15:1,amparo11:1,glória38:4 (ceutakescéu's); plus unaccentedfatimaines.toml, on its Spanish counts (91:0), asfátimaisdefault = falsecássia462:64,cassia170:31,graça87:20,graças138:37,kassia19:5,piedade13:4 (graca,gracastake their accented words')locales.PTandlocales.PT_BR(apt-BRchoice selectspt.toml)nazaré70%,dores66%,penha62%,rosário61%,assunção,carmo53%,conceição47%,chagas,neves,luz;aparecida(written mostly with no particle)PT/PT_BRafterMaria de Lurdes Costade Lurdes Costade Lurdes, familyCostaJosé de Ribamar Sousade Ribamar Sousade RibamarMaria das Graças Silvadas Graças Silvadas Graças, familySilvaRITA DE CASSIA SOUZADE CASSIA SOUZADE CASSIAMaria das Dores Silva,José do Carmo SouzaWhy two tiers. A Brazilian filer often types a surname into the given box:
silvascores 32% "given" on the same count. That pulls every Portuguese share down, which is whyfátima(83%) andsocorro(85%), already marked on their Spanish counts, stand under the bar here. A word at 80% is far above a surname's share but below the default's bar. So it is read where the caller has said the names are Portuguese, and nowhere else. The alternative, the default tier alone, left the issue's owndas Graçasa surname for every caller.Accents.
_normalizedoes not fold them, and Brazilian registries write unaccented capitals, so each unaccented spelling is its own entry, measured on its own in each language's countries. So it can part from its accented twin:kassia(79%) is pack-tier wherekássiais default.Measurements
These are from 2026-11-06, against
e76cdad3with--base(agit archiveexport), on py3.11. The word counts above are over the whole ORCID extract. The ORCID agreement rows are over the capped corpus (raw.jsonl, at most 20,000 records a country, 1,831,533 records). Only 3,223 of those hold a default-tier word and 204 a pack-tier word, so the rows count moves among those.local_agreement.py orcid-researchers(capped), defaultlocal_agreement.py orcid-researchers --locale pt_br(capped, both tiers)local_agreement.py psc-individuals, defaultsnapshot.py --local tse-donors --basemoves 1,803 of 854,531 names, in family and middle only (DE FATIMA1,466,DA GLORIA115,DE TARSO73,DE LURDES47,DE RIBAMAR43,DO AMPARO20).PT_BR:config_diff.py diff --local tse-donors default plain:PT_BRover the 1,950 names holding a pack-tier phrase moves 1,218, nearly all family → middle (CASSIA568,GRACAS/GRAÇAS483).da Graça×4,das GraçasandDe Cássia; PSC's 2 areDa Gloria Vaz Ceita DaioandDe Fatima Zenti.corpus_rules.jsonland the snapshot as new names.Docs and tests
[pt]/[pt_br]examples and boundaries, and the marked-word bullet now names the Portuguese words.rules_budget.jsonis re-recorded. On axis 12, the new clause reads no new signal: it is the same lookup of the word behind the particle run, and only which readings a language choice turns on differs.customize.rst(the marked-word paragraph),locales.rst(a pack-tier paragraph with a doctest, and a newlanguage-packslabel), the vocab README'safter_particlerow, rootAGENTS.md'sPARTICLE_GIVEN_NAMESbullet, and the release log (folded into the Should a two-word given name likeMaría JoséorMin Hofillgiven, instead of given + middle? #704 bullet, since a user sees one change).test_es_and_pt_pack_contentsnow holds the PT surface with the pack-tier words, and_ROTATORS["pt"]/["pt_br"]gain a rotator for each.Checks: full suite, serial benchmarks, mypy, ruff,
build.py --check, sphinx doctest and html (no output), and README doctest all pass. No code changed, only vocabulary, tests and docs.Design-docs review (round 1, fixed in 6616c59)
All twelve axes ran on
fa47db4b. Every count in the entry and thept.tomlnotes reproduced on the whole extract, and a grid of 23 shapes × 17 words under every order, switch and pack moved only the P10 shape. Three findings, all fixed:fatimahad been weighed only on Portuguese counts and sat in the pack tier. On the Spanish counts that putfátimaon by default it is 91:0, soMaría de Fatima Lópezread familyde Fatima LópezbesideMaría de Fátima López's middlede Fátima. It is now a default reading ines.toml. This addsDE FATIMA's 1,466 TSE names to the default reach.doresandpenha, which the entry refused. Standing and the vocab README now state 75% up to the bar with ten given records, and that an unaccented spelling is measured on its own.🤖 Generated with Claude Code
Design-docs review (round 2, of 6616c59; fixed in 1d30f1a)
The reviewer confirmed all three round-1 fixes and recomputed every re-measured number. Four findings, all fixed:
ceu,gracaandgracas(2–4 given records) take their accented twin's reading. Standing now says a spelling under ten given records does.es.tomlmarks. Underparser_for(locales.PT, strict=True),de Fátima,de Fatima,do Socorro,de Lourdesanddo Rosarioread as family. Movingfatimajoined an existing class; it didn't create one. decisions/P10 now records this as accepted, and Shouldparser_for(locales.PT, strict=True)readMaria de Fátima Costaas middlede Fátima, as the default does? #1122 holds the question.fatimareading. rules/P10 now has"Maria de Fatima Costa" → middle="de Fatima", which fails withes.toml's reading removed (checked).piedade(76%) andnazaré(70%) on either side.