diff --git a/AGENTS.md b/AGENTS.md index db4d72570..08a63f43f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -242,7 +242,7 @@ The compiled sets, read by `Lexicon` (mostly as the `_sets` field of the same na - particles — `PARTICLES` (family-name particles, e.g. "de", "van") and `NON_GIVEN_NAME_PARTICLES`, the curated subset that is *never* a standalone given name (the particles whose word has no `given` reading, each stating `may_be_given = false`; a `given` reading on the word is what makes a particle possibly given, the phase 1 flag `may_be_given = true` refused since 2026-10-25). What the subset licenses is rules/P1's fold in `post_rules` (the "rule 1b" of older issue comments — decisions/legacy-rule-numbers maps the old numbers), and the fold has ONE site: the piece that OPENS the name. A lone never-given particle there, with at least one more name word behind it, starts the family. Titles do not move the opening position (rules/P4), so `Sir de Mesnil` reads family `de Mesnil` under every `name_order`, as `de Mesnil` does. How far the fold reaches depends on the order the name was read under: by default it takes the rest of the name (`de Mesnil Jean` → family `de Mesnil Jean`); under either family-first order it takes the particle run plus ONE name unit and lays the rest out by the order (`de Mesnil Jean` → family `de Mesnil`, given `Jean`); after a family comma it takes the rest of that part whatever the order (`Smith, de Mesnil Jean` → family `Smith de Mesnil Jean`). The leading particle stays a lone piece for the fold to find — `group` does not chain it forward, and the fold walks the particles after it itself — so `de la Vega` reads family `de la Vega` under all three orders. Behind a title-particle (`St`, `Freiherr`, P4's Except) it does chain, and under a family-first order the fold cuts that chain as it cuts a run (`St de Mesnil Jean` → family `de Mesnil`, given `Jean`, #828). The GIVEN position is not a site. It was a second one until #467, which removed it because under a family-first order that slot holds what the caller DECLARED to be the given name: `Mesnil de` under `FAMILY_FIRST` or `FAMILY_FIRST_GIVEN_LAST` reads given `de`, family `Mesnil`. Nor is a particle inside the name: `Juan de la Vega` under a family-first order reads family `Juan`, given `de la Vega`, with no ambiguity recorded — what the vocabulary forbids is the bare particle as a given name, not a name part that begins with one. A bare `de` has nothing to fold into and is placed by position like any one-word name: given `de` by default, family `de` under both family-first orders. A leading particle OUTSIDE the set is left to position — `Van Gogh` is given `Van`, family `Gogh` by default and family `Van`, given `Gogh` under both family-first orders — and records a `PARTICLE_OR_GIVEN` ambiguity under all three, save one written in lower case in a mixed-case name under a family-first order, which the fold takes and which records none (`van Gogh` → family `van Gogh`, rules/P11, #670); a never-given particle opening the name records none (`de Mesnil`), and neither does an ambiguous one inside it (`Vincent van Gogh`). The internal `Lexicon._sets.particles_ambiguous`, the particles holding a `given` reading, is its complement within `PARTICLES`, so the two mark OPPOSITE sets — see the flip warning in `docs/migrate.rst` before translating either - bound given names — `BOUND_GIVEN_NAMES` (`bound_given` readings, e.g. "abdul", "abu"; one that is also a particle needs a `given` reading, a build check); a group-stage rule joins the first non-title piece to its following piece before roles are assigned, reserving a family word unless a family comma has already fixed it or a given-name title stands ahead (#369, rules/P5) (ported from v1's `_join_bound_first_name`), and `BOUND_PHRASES` (#694, rules/P12, the two-word given names a bound word's `completed_by` lists, Arabic `عبد` before the names of God), each pair joined past the first name word, where P5's join does not reach - given names written short — `GIVEN_NAME_ABBREVIATIONS` (#789, a `given` reading carrying `same_as`, the full name: "chas", "wm", Filipino "ma"), the given names written short that rules/H2's shape yields to; nothing else reads it -- names a given name's second half reads — `PARTICLE_GIVEN_NAMES` (#704, a `given` reading written `after_particle`: "carmen", "ángeles", the Spanish words a compound given name writes behind a particle, which rules/P10 reads as a middle name in `María del Carmen García`, by default), `SAINT_PARTICLES` (the `saint` kind, `san`, `santa`, `st`, `ste`, `sta`, `sto`, whose run P10 refuses, `Santa María` being a surname) and `SHARED_NAMES` (#704, a name reading written `shared`: Sikh "kaur", "singh", which rules/O6 and O7 neither take nor complete and P10 does not complete, `Kaur del Carmen García`), private; each is read through classify's tags or the lexicon in `_post_rules` +- names a given name's second half reads — `PARTICLE_GIVEN_NAMES` (#704, a `given` reading written `after_particle`: "carmen", "ángeles", "lurdes", the Spanish and Portuguese words a compound given name writes behind a particle, which rules/P10 reads as a middle name in `María del Carmen García`, by default, and a few Portuguese ones, "graças", only under a Portuguese choice, #1118), `SAINT_PARTICLES` (the `saint` kind, `san`, `santa`, `st`, `ste`, `sta`, `sto`, whose run P10 refuses, `Santa María` being a surname) and `SHARED_NAMES` (#704, a name reading written `shared`: Sikh "kaur", "singh", which rules/O6 and O7 neither take nor complete and P10 does not complete, `Kaur del Carmen García`), private; each is read through classify's tags or the lexicon in `_post_rules` - conjunctions — `CONJUNCTIONS` (e.g. "and", "of") used to chain multi-word titles and to join name parts (rules/P3), plus `CONJUNCTIONS_AMBIGUOUS` (#383/#479, rules/P8, `may_be_initial`, refused by the build on anything but a cased single letter), the subset of single cased letters that read as an INITIAL in a name written wholly in one case — a marker set - maiden markers — `MAIDEN_MARKERS` (`birth_marker` readings, e.g. "née", "geb.") routing the following name to `maiden`; with an `addresses = "given"` title, the one kind a phrase may be (`z domu`) - `config/_ko_surnames.py`, private since 3.0 (`config/surnames.py` through 2.x, 2.8.2 warning of the move) and read as `Lexicon.default().surnames` — `KOREAN_SURNAMES`, the census list the 2.0 API splits unspaced hangul on (#271), still written in Python with an import-time `assert_normalized`: a segmentation list, outside the per-word model, like the locale packs' lists. diff --git a/docs/customize.rst b/docs/customize.rst index a1223388f..1d3276a3d 100644 --- a/docs/customize.rst +++ b/docs/customize.rst @@ -1187,12 +1187,15 @@ particle's words would otherwise run on into the family, the parser reads them as a middle name, with or without the switch, if the vocabulary marks the word behind the particle as a given name written there (``after_particle``), and the switch then takes them into the -given name. The Spanish ones are marked, those that ORCID's researchers -from Spain and Spanish-speaking America write in the given box at least -nine times for each ten they also open a family box with it (``del -Carmen``, ``de los Ángeles``, ``del Mar``, ``del Pilar``: 23 words in -all on 2026-11-03). A phrase too often a surname to mark, as Portuguese -``do Carmo`` is, stays the family, the same shape as ``"João da Silva +given name. The words marked are those that ORCID's researchers write +in the given box at least nine times for each ten they also open a +family box with it: from Spain and Spanish-speaking America (``del +Carmen``, ``de los Ángeles``, ``del Mar``, ``del Pilar``), and from +Portugal and Brazil (``de Lurdes``, ``do Céu``, ``da Glória``, ``de +Tarso``). Portuguese words written as given names less surely +(``das Graças``, ``de Cássia``) are read only under ``locales.PT`` and +``locales.PT_BR`` (see :ref:`language-packs`). A phrase too often a +surname to mark, as Portuguese ``do Carmo`` is, stays the family, the same shape as ``"João da Silva Santos"``, and so does one opening with a saint (``"Domingo Santa María González"``): diff --git a/docs/design/decisions/O7-two-given-names.md b/docs/design/decisions/O7-two-given-names.md index 35ad71d04..d060b55f6 100644 --- a/docs/design/decisions/O7-two-given-names.md +++ b/docs/design/decisions/O7-two-given-names.md @@ -43,7 +43,7 @@ - **Open:** which particle phrases complete a given name, and whether that is a reading on the word or a list the rule kind holds (the particle-chain bullet above), #704 (decided 2026-11-03, below), and whether this rule reads O6's given-name list for it (`María del Carmen` against `João da Silva`), #959 (declined 2026-11-03, below); deriving `DEVIATES` from the rules (decisions/vocabulary-model's 2026-10-29 4f entry, item 6), which this rule leaves as it found it, having no surface in the name to declare, #754. - 2026-10-30 #959 — THE WORD O6'S GIVEN-NAME LIST LEAVES IS THIS RULE'S. rules/O6 now leaves a listed given name where it stands (decisions/O6's entry of this date), so with both switches on the middle it leaves is this rule's to take where it is the unit beside the given name: `José Luis Pérez` reads given `José Luis`, family `Pérez`, where with the list empty O6 takes `Luis` first and the name keeps one given name. Where the given part holds two units before it, this rule takes the first and the listed word stays a middle (`Ana María Luis Pérez`: given `Ana María`, middle `Luis`), so a name of four words reads two and two only where O6 took its third (found by the #959 design-docs review). Nothing in this rule moved; its precedence clause states the reading with an example. Whether this rule reads the list itself is Open above. - 2026-10-30 #753, phase 3g, decided in chat — NEITHER `es` NOR `pt` TURNS THIS SWITCH ON. ORCID cannot score it: its given box holds the middle names too, so the switch moves no record's agreement (0 fixed, 0 broken in every Spanish- and Portuguese-speaking country measured). PSC, whose form has a middle-name box, scores it against the filer's own split, and filers put the second given name in that box: under a `es` choice the switch alone takes Spanish filers from 31.8% to 15.7% (1 fixed, 980 broken), and beside `locales.ES`'s `two_family_names` from 82.0% to 54.7%; Portuguese filers fall from 61.1% to 28.5%. CLDR's `es` renderings read it right (the 2026-10-30 #704 entry, 6 → 8 of 8). The two sources disagree about a convention, not about a misreading, and the pack follows the boxes callers fill: a caller whose data writes the compound given name whole turns the switch on beside the pack. Recompute: decisions/O6's phase 3g recipe with `PolicyPatch(two_given_names=True)` added. -- 2026-11-03 #704, decided in chat — (Superseded in part by decisions/P10's 2026-11-04 #704 entry: the particle phrase is read by default, as a middle name, and this rule takes it as any middle unit; so the properties no longer move nothing with the switches off, `after_particle` reading by default and `shared`, like a patronymic, keeping a given name P10 would complete.) A PARTICLE PHRASE COMPLETES THE GIVEN NAME WHERE THE VOCABULARY MARKS ITS WORD, AND KAUR AND SINGH ARE NO GIVEN NAME. Settles the 2026-10-30 entry's Open. Two name-reading properties, each read by this rule and rules/O6 and by no other, so a parse with both switches off moves nowhere (the snapshot gains the rule examples' new texts and moves no reading; a grid of 128 texts under every order and switch pairing, token readings and `Reading.languages` included, moved no default-policy parse against master `a1b3e186`). +- 2026-11-03 #704, decided in chat — (Superseded in part by decisions/P10's 2026-11-04 #704 entry: the particle phrase is read by default, as a middle name, and this rule takes it as any middle unit; so the properties no longer move nothing with the switches off, `after_particle` reading by default and `shared`, like a patronymic, keeping a given name P10 would complete. Its Portuguese clause, that no Portuguese-speaking word reaches the bar, is superseded by decisions/P10's 2026-11-06 #1118 entry: over the whole ORCID extract eight words do, and six more are marked for a Portuguese choice alone; and the unaccented `fatima`, which this entry did not weigh, passes on its Spanish counts.) A PARTICLE PHRASE COMPLETES THE GIVEN NAME WHERE THE VOCABULARY MARKS ITS WORD, AND KAUR AND SINGH ARE NO GIVEN NAME. Settles the 2026-10-30 entry's Open. Two name-reading properties, each read by this rule and rules/O6 and by no other, so a parse with both switches off moves nowhere (the snapshot gains the rule examples' new texts and moves no reading; a grid of 128 texts under every order and switch pairing, token readings and `Reading.languages` included, moved no default-policy parse against master `a1b3e186`). - **The shape: a property on the word behind the particle.** The particle cannot carry it (`del` opens `del Carmen` and `del Río` alike), and the question is a vocabulary one (the 2026-10-30 entry's particle-chain bullet). Weighed: (A) a phrase reading (`text = "del carmen"`), most precise, telling Spanish `de la Luz` from Portuguese `da Luz`, but a third kind of phrase reading beside a birth marker and a given-addressing title; (C) a list the rule holds as a `Policy` field or `[rules]` key, vocabulary in `Policy` that no choice of languages or strict parser reaches; (D) O6's given-name list (`Lexicon.given_names`), no new data, but it would split `de la Cruz` (175 family boxes in ORCID against 38 given), `de León` (151 against 5) and `de la Rosa`, and is empty by default and has no Portuguese list. B was chosen: `after_particle = true` on a `given` reading, one word, compiled into the internal `PARTICLE_GIVEN_NAMES`. Its cost against A is a word whose phrase reads both ways by particle, which is left out (`luz`). - **The reach.** Given-first, no family comma (a comma, and both family-first orders, already read `Silva, Maria do Carmo` given `Maria do Carmo` through the middle path), the given name one word with no middle between it and the chain, the chain opening with particles and then the marked word, and a name word left after them for the family, one with a letter that is no connective standing alone (a family of particles alone never reaches the path: assign leaves a trailing particle the family by itself, the phrase a middle, found by the second #1057 review), and no Turkic marker behind the phrase under that rule, as for the middle path (both found by the #1057 review: `María del Carmen y` and `Ali del Carmen oglu` had moved): `María del Carmen` alone keeps family `del Carmen`, as `Juan Carlos` keeps family `Carlos`. A given name of two words (`María-José del Carmen García`) keeps what it has, and a middle between leaves the chain to the family, this rule taking that middle instead (`Ana María del Carmen García` reads given `Ana María`). With rules/O6 on too, O6 leaves the phrase where it stands (decisions/O6, this date), so `María del Carmen de la Vega` reads given `María del Carmen` with both on, where the 2026-11-03 #959 entry in decisions/O6 recorded family `del Carmen de la Vega`; and the family the split leaves keeps its chained words as one unit, so `María del Carmen García López` gives no `surname2` (accepted in the rule). - **The words** (es.toml; each records its counts): `carmen`, `ángeles` and `angeles`, `pilar`, `rosario`, `valle`, `caridad`, `socorro`, `mar`, `rocío` and `rocio`, `lourdes`, `mercedes`, `maría` and `maria`, `milagro`, `milagros`, `cisne`, `fátima`, `concepción`, `nieves`, `consuelo`, `jesús` — 23. Measured 2026-11-03 on ORCID's `raw.jsonl` of the 2026-10-27 fetch (20,000 records a country at most): for each word, the records whose given box ends with a particle run (`Lexicon.default()`'s particles, NFC lower case) and the word, against those whose family box opens with a particle run and the word and holds more words after them, the shape that competes; kept where the given side is at least 90% with ten given records or more, counted over Spain and Spanish-speaking America (ES MX AR CO CL PE VE EC UY BO PY CR GT CU DO HN SV NI PA PR AD) and, separately, Portugal, Brazil and Portuguese-speaking Africa (BR PT AO MZ CV GW ST TL). `carmen` 1,261 to 5, `valle` 235 to 26 (exactly 90.0%), `rosario` 260 to 27. Left out: no Portuguese-speaking word reaches the bar (`carmo` 21 to 6, `rosário` 21 to 8, `conceição` 25 to 25, `fátima` 32 to 4, which the Spanish counts carry, 27 to 1); unaccented `jesus`, 203 to 16 in Spanish-speaking countries but 67 to 41 in Portuguese-speaking ones; `luz` (12 to 6, and Portuguese `da Luz` 4 to 5), `dios`, `paz`, `cruz`; and two that pass by count but are no compound given name, `c.` (an initial) and `piero` (Italian `del Piero`, 10 to 0). The Catalan phrases (`dels Àngels`, `de les Neus`, `de la Mercè`) barely occur, so no Catalan word is marked. The PSC register agrees where it can, its filers putting `del Carmen` in the middle-name box 21 times and the surname box 2, `del Mar` 11 and 0, `del Pilar` 10 and 1. `dels` stays out of `ca.toml` (the reason void since decisions/P10's 2026-11-04 entry, which holds the question Open): the property moves nothing with the switch off, so a `dels` particle would still move `Maria dels Àngels Puig` from middle `dels Àngels` to family `dels Àngels Puig` by default (the #704 comment of 2026-10-31). diff --git a/docs/design/decisions/P10-given-name-a-particle-completes.md b/docs/design/decisions/P10-given-name-a-particle-completes.md index bd15e3717..ef0702a3b 100644 --- a/docs/design/decisions/P10-given-name-a-particle-completes.md +++ b/docs/design/decisions/P10-given-name-a-particle-completes.md @@ -6,9 +6,14 @@ - A run holding a particle of the `saint` kind (`san`, `santa`, `st`, `ste`, and `sta` and `sto`, which decisions/H6 added for #725) is never such a phrase, `Santa María` being a surname — 2026-11-04 #704. - The rule reads before rules/O6 and O7, so two-given-name handling takes the phrase as the middle unit beside the given name and two-family-name handling leaves it; O7 no longer reads the phrase itself — 2026-11-04 #704. - The rule reports nothing — 2026-11-04 #704. +- The Portuguese words are marked on Portuguese-speaking countries' counts, by the same 90% bar and ten given records, over the whole ORCID extract; a word from 75% up to the bar, with ten given records or more, is marked `default = false` in `pt.toml`, read under a Portuguese choice alone (`graças`, `cássia`, `piedade`), and a word under 75% is not marked, though a surname's share on the same count is far lower — 2026-11-06 #1118, decided in chat. +- An unaccented spelling with ten given records or more is measured on its own, in each language's countries, and marked where those counts pass, so it may part from its accented twin: `fatima` is on by default on its Spanish counts, as `fátima` is, and `kassia` is read under a Portuguese choice alone where `kássia` is on by default. One with fewer is too rare to decide alone and takes its twin's reading (`ceu`, `graca`, `gracas`) — 2026-11-06 #1118. +- A word marked in `es.toml` is not read under a strict Portuguese choice, which does not compile `es.toml`, though Portuguese names write it too (`de Fátima`, `de Fatima`, `do Socorro`, `de Lourdes`): accepted here, since a strict choice reads only the files it names, and left for its own decision — 2026-11-06 #1118. Open: Catalan `dels` (`Maria dels Àngels Puig`). decisions/O7 kept it out of `ca.toml` because the property moved nothing with the switch off; under this rule a `dels` particle beside `àngels` written `after_particle` reads middle 'dels Àngels', family 'Puig' by default, today's reading, so the reason no longer holds and the words want their own measurement — #1059. +Open: whether a strict Portuguese choice should read the words `es.toml` marks that Portuguese names write too — #1122. + ## Log - 2026-11-04 #704, decided in chat — A GIVEN NAME A PARTICLE COMPLETES IS A MIDDLE NAME BY DEFAULT. The default reading decisions/O7's 2026-11-03 #704 entry left undecided ("Not decided here"): `María del Carmen García` read family 'del Carmen García' in 1.4.0 through 3.0's master, and reads given 'María', middle 'del Carmen', family 'García'. SUPERSEDES that entry's reach bullet where it put the phrase in O7: the particle phrase is this rule's, read whatever the switches, and O7 takes it as it takes any middle unit, so O7's own phrase path, its vocabulary clause, and the split-candidate line in decisions/O7's Standing go. @@ -25,3 +30,17 @@ Open: Catalan `dels` (`Maria dels Àngels Puig`). decisions/O7 kept it out of `c - **The reach** is O7's phrase path's, as reviewed on #1057: given-first, no family comma, a given name of one word (a bound pair counting two, `Abu Bakr`) that is no patronymic, shared name or mark, no middle between it and the family, and a name word left for the family that is no lone connective, no Turkic marker behind the phrase under that rule. - **An ambiguity's alternative** on a word behind a particle chain names the field of the chain's last word where that word stayed the family while the chain's head left it, P10's split (which O7 may then take into the given name), and the head's field otherwise, as before, a last word P6 attached to the family included (`Ungnad von Weissenwolf, David zu Sonnegg von` under `two_family_names` and `middle_as_family`, whose alternative's languages the second fix's first draft changed, found by comparing every corpus name's alternatives with master under every order and switch pairing: none moves): `JAIRO DO VALLE CARDOSO FILHO`'s `FILHO` read as a name would join family 'CARDOSO FILHO', where the head's field said 'middle', found by the TSE run (six names before the fix, none after). The first fix named the last word's field always, and #1058's second review found it wrong after rules/O6's family-first cut, which lays a chain's tail out by the order: under `FAMILY_FIRST` with `two_family_names` and `middle_as_family`, `Ana de María Pérez Filho` named 'given' where the fold makes it family, 42 rows newly wrong on a grid of 408 moves. Measured on that grid (20 chains, four heads, three agnomens, every order with each switch and `middle_as_family`, the field compared with where an unlisted word there lands), 132 alternatives move against master, each to the right field, and none away from it; `FAMILY_FIRST` with `two_family_names` alone stays as wrong as on master (`Pedro del Valle Llufrio Neto` names 'family' where the tail's order puts a name word in the middle). No committed corpus name moves. - Recompute: the commands above; the Portuguese-speaking split scores `local_agreement.read` per record over the countries BR, PT, AO, MZ, CV, GW, ST and TL in both trees (10 more records change); the TSE phrases count the moved names by their middle as written, all capitals; `--base` a `git worktree add --detach` checkout of `3b293777`; the saint counts over ORCID's `raw.jsonl` of the 2026-10-27 fetch, a given box ending with at least one word, a particle run and a word of `Lexicon.default()._sets.particle_given_names`, against a family box opening with a particle run and such a word and holding more words, grouped by the run. +- 2026-11-06 #1118, decided in chat — PORTUGUESE GIVEN NAMES A PARTICLE COMPLETES, IN TWO TIERS. decisions/O7's 2026-11-03 entry found no Portuguese-speaking word at the 90% bar on ORCID's capped sample (20,000 records a country), so `Maria das Graças Silva` and `Maria de Lurdes Costa` read family 'das Graças Silva' and 'de Lurdes Costa'. Re-measured over the whole extract (888,163 Brazilian records and 103,654 Portuguese where the sample held 20,000 each), with the recipe of that entry over Portugal, Brazil and Portuguese-speaking Africa (BR PT AO MZ CV GW ST TL). + - **The default tier**, the words at the 90% bar with ten given records or more: `lurdes` 42 to 1, `céu` 31 to 1, `gloria` 25 to 1, `tarso` 53 to 3, `ribamar` 17 to 1, `kássia` 15 to 1, `amparo` 11 to 1, `glória` 38 to 4. `ceu` (2 to 0) takes `céu`'s reading, too rare to decide alone. Each is new to the vocabulary, so no other reading clashes. The unaccented `fatima`, which decisions/O7's 2026-11-03 entry did not weigh, passes on the Spanish counts that mark `fátima` (91 to 0 over Spain and Spanish-speaking America), so it is on by default in `es.toml` though its Portuguese counts (185 to 45, 80%) would put it in the pack tier: by default `María de Fatima López` had read family 'de Fatima López' beside `María de Fátima López`'s middle 'de Fátima' (found by the design-docs review). + - **The pack tier.** A Brazilian filer often types a surname into the given box (`silva` scores 4,730 to 9,988 on the same count, 32%), so a Portuguese word's share runs lower than a Spanish one's, and the marked `fátima` (83%) and `socorro` (85%) stand under the bar here, marked on their Spanish counts. Words between 75% and the bar, with ten given records or more, are marked `default = false` in `pt.toml`, which `locales.PT` and `locales.PT_BR` turn on (a `pt-BR` choice selects `pt.toml` too): `cássia` 462 to 64, `cassia` 170 to 31, `graça` 87 to 20, `graças` 138 to 37, `kassia` 19 to 5, `piedade` 13 to 4; `graca` (3 to 0) and `gracas` (4 to 0) take their accented words'. `kassia`, measured on its own, parts from `kássia`, in the default tier, and no Spanish count weighs it. Weighed: (A) the default tier alone, which leaves the issue's own `das Graças` a surname for every caller; (B) both tiers, chosen, a weaker vocabulary signal read where the caller has said the names are Portuguese and nowhere else. + - **Left out**, under 75% though well above a surname's share (`silva` 32%, `santos` 28%). The 75% is a chosen threshold, not a gap in the data: the pack tier's lowest word, `piedade` 13 to 4 (76%), stands a few points above the highest left out, `nazaré` (70%), and a word measured later between them is judged against the number, not against the words beside it. Left out: `nazaré` 55 to 24 (70%), `dores` 38 to 20 (66%), `penha` 36 to 22 (62%), `rosário` 84 to 53 (61%), `assunção` 32 to 25, `carmo` 348 to 304, `conceição` 302 to 344, and `chagas`, `neves`, `luz`; and `aparecida`, written mostly with no particle (`Maria Aparecida Silva` reads middle 'Aparecida' already). + - **Measured** (2026-11-06, the tree against `origin/master` `e76cdad3` read with `--base`, py3.11). The word counts above are over the whole extract; the agreement rows are not: `local_agreement.py` scores ORCID's capped corpus (`raw.jsonl`, at most 20,000 records a country, 1,831,533 records), of which 3,223 hold a default-tier word in a box and 204 a pack-tier word, so the rows count moves among those. + + | Population | Fixed | Broken | Changed, agreeing in neither | + |---|---:|---:|---:| + | orcid-researchers (capped), default | 29 | 0 | 14 | + | orcid-researchers (capped), `--locale pt_br` (both tiers) | 56 | 6 | 28 | + | psc-individuals, default | 9 | 2 | 5 | + + `snapshot.py --local tse-donors --base` moves 1,803 of 854,531 names by default, in family and middle alone (`DE FATIMA` 1,466, `DA GLORIA` 115, `DE TARSO` 73, `DE LURDES` 47, `DE RIBAMAR` 43, `DO AMPARO` 20); under `PT_BR`, `config_diff.py diff --local tse-donors default plain:PT_BR` over the names holding a pack-tier phrase moves 1,218 of their 1,950 against the default, all but a few of them family to middle (`CASSIA` 568, `GRACAS` and `GRAÇAS` 483), the rest the pack's own `e` reading beside the phrase. ORCID's 6 broken are the pack-tier words borne as surnames, `da Graça` 4 of them, and PSC's two are a family box opening `Da Gloria` and one opening `De Fatima`. The committed corpora hold none of the phrases, so the committed snapshot moves nothing; the rule's new examples join it. + - Recompute: the phrase counts over `local_orcid.extract_path()`, the given box's last word behind a run of `Lexicon.default()._sets.particles` with at least one word before it, against the family box's first word behind such a run with more words after it, NFC lower case with edge periods and commas stripped, grouped by country; the measurements with the commands above, `--base` a `git archive` export of `e76cdad3`, and `local_agreement.py orcid-researchers --locale pt_br --base` for the pack row; the capped population, the `raw.jsonl` records whose given or family box, NFC lower case, holds a word of either tier; the TSE pack count with `--match '(?i)\b(DAS?|DE) (GRA[CÇ]AS?|C[AÁ]SSIA|K[AÁ]SSIA|PIEDADE)\b'`. diff --git a/docs/design/rules/P10-given-name-a-particle-completes.md b/docs/design/rules/P10-given-name-a-particle-completes.md index 75cf7fd67..e3c0d1062 100644 --- a/docs/design/rules/P10-given-name-a-particle-completes.md +++ b/docs/design/rules/P10-given-name-a-particle-completes.md @@ -4,16 +4,26 @@ **Then:** the particle run and that word are a middle name, so the chain's other words are the family. -- **The word is one a reading marks** (`after_particle`), the second of a compound given name written behind a particle: the Marian and devotional names (del Carmen, de los Ángeles, del Pilar, de Lourdes), marked on Spanish counts and written in Portuguese names too (do Socorro, de Fátima). +- **The word is one a reading marks** (`after_particle`), the second of a compound given name written behind a particle: the Marian and devotional names (del Carmen, de los Ángeles, del Pilar, de Lourdes), marked on Spanish counts and written in Portuguese names too (do Socorro, de Fátima), and the Portuguese ones marked on Portuguese counts (de Lurdes, do Céu, da Glória, de Tarso). ``` "María del Carmen García" → middle="del Carmen" "María del Carmen García" → family="García" "María de los Ángeles López Pérez" → family="López Pérez" "Ana de María Pérez" → middle="de María" "Maria do Socorro Silva" → middle="do Socorro" + "Maria de Lurdes Costa" → middle="de Lurdes" + "Maria de Fatima Costa" → middle="de Fatima" + "Paulo de Tarso Ribeiro" → family="Ribeiro" "Maria do Carmo Silva" → family="do Carmo Silva" · boundary "João da Silva Santos" → family="da Silva Santos" · boundary ``` +- **A word filers write as a given name less surely is marked for its language alone**, read where the caller chooses that language and left to the surname otherwise: Portuguese das Graças, de Cássia and da Piedade. + ``` + "Maria das Graças Silva" [pt] → middle="das Graças" + "RITA DE CASSIA SOUZA" [pt_br] → family="SOUZA" + "Maria das Graças Silva" → family="das Graças Silva" · boundary + "Maria das Graças Silva" [es] → family="das Graças Silva" · boundary + ``` - **A title, a suffix and a nickname are no name words**, so they stand outside the reading. ``` "Dr. María del Mar García Jr." → middle="del Mar" diff --git a/docs/locales.rst b/docs/locales.rst index 68e23b23e..b739e49f8 100644 --- a/docs/locales.rst +++ b/docs/locales.rst @@ -789,6 +789,8 @@ a middle name with the ending, so it follows the order: under and no patronymic. A marker and the word before it are read under every order. +.. _language-packs: + Packs that choose a language ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -819,6 +821,20 @@ stays the given name (``Nic Cage``), as ``Mac`` does by default; under a family-first order it is the first word, read as the family name. Each reports its doubt in ``ambiguities``. +``pt`` and ``pt_br`` also turn on the Portuguese given names written +behind a particle that filers write as given names less surely than +the default's bar (``das Graças``, ``de Cássia``, ``da Piedade``), so +the phrase reads as a middle name, as ``del Carmen`` does by default +(see :ref:`two-given-names`): + +.. doctest:: + + >>> parse("Maria das Graças Silva").family + 'das Graças Silva' + >>> gracas = parser_for(locales.PT).parse("Maria das Graças Silva") + >>> gracas.middle, gracas.family + ('das Graças', 'Silva') + A choice also settles every clash its language has a reading for, as `Choosing languages`_ shows, and that costs the other languages' readings of those words: diff --git a/docs/release_log.rst b/docs/release_log.rst index 9e8cf507b..09853d7ff 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -48,7 +48,7 @@ Release Log - **Read Graf, Gräfin, Fürst and Prinz inside a German surname as part of it, as Freiherr is.** ``parse("Otto Graf Lambsdorff")`` gives family ``Graf Lambsdorff``, where 1.4.0 through 2.8.1 gave middle ``Graf``, family ``Lambsdorff``; opening a name each is still a title (``Graf Zeppelin``). ``Herzog`` and ``Ritter``, common surnames, read as before. :issue:`709` - **Write St, San, Santa and Freiherr with their capital in case repair.** ``str(parse("JOHN ST CLAIR").capitalized())`` gives ``John St Clair``, where 1.4.0 through 2.8.1 gave ``John st Clair``; every saint and rank particle is written this way, so ``Graf``, ``Gräfin``, ``Fürst`` and ``Prinz``, particles since this release, keep the capital 2.8.1 gave them. A romanized honorific joined by a hyphen and spelled as one takes the capital too (``Suzuki-san`` repairs to ``Suzuki-San`` under a family-first order with ``force=True``, or written in one case). :issue:`1069` - **Read Sta. and Sto. as surname particles.** ``parse("Juan Sta. Maria Reyes")`` gives family ``Sta. Maria Reyes``, where 1.4.0 through 2.8.1 gave middle ``Sta. Maria``, family ``Reyes``; ``Juan Sto. Tomas`` gives family ``Sto. Tomas``, and case repair keeps them in title case. A bare ``Sta`` opening a name now starts the family (``Sta Maria Juan`` gives family ``Sta Maria Juan``). :issue:`725` - - **Read a compound given name written with a particle as a middle name.** ``parse("María del Carmen García")`` gives middle ``del Carmen``, family ``García``, where 1.4.0 through 2.8.0 gave family ``del Carmen García``. It reads 23 marked words behind ``de``, ``del``, ``de los`` and the like, Spanish and Portuguese alike (``del Pilar``, ``de los Ángeles``, ``do Socorro``); a surname written that way moves too (``Pedro del Valle Llufrio`` gives middle ``del Valle``), and ``Maria do Carmo Silva`` and ``Domingo Santa María González`` keep their family. :issue:`704` + - **Read a compound given name written with a particle as a middle name.** ``parse("María del Carmen García")`` gives middle ``del Carmen``, family ``García``, where 1.4.0 through 2.8.0 gave family ``del Carmen García``. It reads marked words behind ``de``, ``del``, ``de los`` and the like, Spanish and Portuguese alike (``del Pilar``, ``de los Ángeles``, ``do Socorro``, ``de Lurdes``, ``de Tarso``); a surname written that way moves too (``Pedro del Valle Llufrio`` gives middle ``del Valle``), and ``Maria do Carmo Silva`` and ``Domingo Santa María González`` keep their family. A few Portuguese words are read only under ``locales.PT`` or ``locales.PT_BR``: ``parser_for(locales.PT).parse("Maria das Graças Silva")`` gives middle ``das Graças``, where the default gives family ``das Graças Silva``. :issue:`704`, :issue:`1118` - **Fix a numeral VI to IX before a comma or ending a connective name costing the family name.** ``parse("Dumas VI, Jr.")`` gives family ``Dumas``, suffix ``VI, Jr.``, as ``Dumas III, Jr.`` does, where 2.0.0 through 2.8.0 gave given ``Dumas`` and no family; ``parse("Ortega y Gasset VI")`` gives family ``Gasset``, where 1.4.0 through 2.8.0 gave given ``Ortega y Gasset``. :issue:`893` - **Change a bare I or V ending the given part after a comma to report suffix-or-name.** ``parse("Doe, John V")`` still gives suffix ``V``, and now reports that the letter may be a middle initial, as ``parse("John Doe V")`` already did; 1.4.0 through 2.8.0 reported nothing. :issue:`877` - **Change a bracketed clause of credentials, generations or titles to be set apart from the name.** ``parse("John Smith (PhD) Withers")`` gives family ``Withers``, suffix ``PhD``, where 2.4.0 through 2.8.0 gave family ``Smith``, suffix ``PhD Withers``; ``John Smith (PhD, MBA)`` gives suffix ``PhD, MBA`` where 1.4.0 through 2.8.0 gave the nickname, and ``John Smith (Dr)`` title ``Dr`` where 2.2.0 through 2.8.0 gave the nickname. Initials inside a name read as the nickname (``Justin (JP) Stearns``, middle ``JP`` in 1.4.0 through 2.8.0), and a lone title of address (``(Sir)``) and a qualifier behind a credential (``FRCS (SN)``) stay the nickname. :issue:`792` diff --git a/nameparser/config/_compiled.py b/nameparser/config/_compiled.py index 7ccb51f62..5dd0b19a9 100644 --- a/nameparser/config/_compiled.py +++ b/nameparser/config/_compiled.py @@ -3173,15 +3173,23 @@ }) PARTICLE_GIVEN_NAMES: frozenset[str] = frozenset({ + 'amparo', 'angeles', 'caridad', 'carmen', + 'ceu', 'cisne', 'concepción', 'consuelo', + 'céu', + 'fatima', 'fátima', + 'gloria', + 'glória', 'jesús', + 'kássia', 'lourdes', + 'lurdes', 'mar', 'maria', 'maría', @@ -3190,10 +3198,12 @@ 'milagros', 'nieves', 'pilar', + 'ribamar', 'rocio', 'rocío', 'rosario', 'socorro', + 'tarso', 'valle', 'ángeles', }) diff --git a/nameparser/config/_index.py b/nameparser/config/_index.py index 9fa28606e..8a2f0a08b 100644 --- a/nameparser/config/_index.py +++ b/nameparser/config/_index.py @@ -105,6 +105,7 @@ ('amft', (('en', 'credential', ()),)), ('amieee', (('en', 'credential', ()),)), ('amn', (('en', 'military', ()),)), + ('amparo', (('pt', 'given', (('after_particle', True),)),)), ('ams', (('en', 'credential', ()), ('en', 'family', ()))), ('an-', (('ar', 'article', ()),)), ('analytics', (('en', 'modifier', ()),)), @@ -275,6 +276,7 @@ ('cas', (('en', 'credential', ()),)), ('casac', (('en', 'credential', ()),)), ('casp', (('en', 'credential', ()),)), + ('cassia', (('pt', 'given', (('after_particle', True), ('default', False))),)), ('catholicos', (('en', 'religious', ()),)), ('cb', (('en', 'credential', ()),)), ('cbe', (('en', 'credential', ()),)), @@ -333,6 +335,7 @@ ('cep', (('en', 'credential', ()),)), ('cera', (('en', 'credential', ()), ('en', 'given', ()), ('en', 'family', ()))), ('cet', (('en', 'credential', ()),)), + ('ceu', (('pt', 'given', (('after_particle', True),)),)), ('cf-slp', (('en', 'credential', ()),)), ('cfa', (('en', 'credential', ()),)), ('cfc', (('en', 'credential', ()),)), @@ -565,6 +568,8 @@ ('cyclist', (('en', 'occupation', ()),)), ('cyds', (('en', 'credential', ()),)), ('cysa', (('en', 'credential', ()),)), + ('cássia', (('pt', 'given', (('after_particle', True), ('default', False))),)), + ('céu', (('pt', 'given', (('after_particle', True),)),)), ("d'", (('fr', 'preposition', ()), ('it', 'preposition', ()))), ("d'affaires", (('en', 'office', ()), ('fr', 'office', ()))), ('da', (('it', 'preposition', ()), ('it', 'given', ()), ('pt', 'preposition', ()))), @@ -765,6 +770,7 @@ ('fasma', (('en', 'credential', ()),)), ('faspen', (('en', 'credential', ()),)), ('father', (('en', 'religious', (('addresses', 'given'),)),)), + ('fatima', (('es', 'given', (('after_particle', True),)),)), ('fca', (('en', 'credential', ()),)), ('fcas', (('en', 'credential', ()),)), ('fcela', (('en', 'credential', ()),)), @@ -855,6 +861,8 @@ ('ginoo', (('fil', 'salutation', (('after_given', True),)),)), ('gisp', (('en', 'credential', ()),)), ('git', (('en', 'credential', ()),)), + ('gloria', (('pt', 'given', (('after_particle', True),)),)), + ('glória', (('pt', 'given', (('after_particle', True),)),)), ('gm', (('en', 'credential', ()),)), ('gmb', (('en', 'credential', ()),)), ('gmr', (('en', 'credential', ()),)), @@ -867,12 +875,16 @@ ('governor', (('en', 'office', (('attaches', ('gen',)),)),)), ('gphr', (('en', 'credential', ()),)), ('graaf', (('nl', 'nobility', ()),)), + ('graca', (('pt', 'given', (('after_particle', True), ('default', False))),)), + ('gracas', (('pt', 'given', (('after_particle', True), ('default', False))),)), ('grace', (('en', 'style', ()), ('en', 'given', ()), ('en', 'family', ()))), ('graf', (('de', 'nobility', ()), ('de', 'rank', ()), ('de', 'given', ()))), ('grafas', (('lt', 'nobility', ()),)), ('grafi', (('ka', 'nobility', ()),)), ('gral', (('es', 'military', ()),)), ('grand', (('en', 'modifier', ()),)), + ('graça', (('pt', 'given', (('after_particle', True), ('default', False))),)), + ('graças', (('pt', 'given', (('after_particle', True), ('default', False))),)), ('greifi', (('is', 'nobility', ()),)), ('greve', (('da', 'nobility', ()), ('nb', 'nobility', ()), ('sv', 'nobility', ()))), ('gri', (('en', 'credential', ()),)), @@ -976,6 +988,7 @@ ('justice', (('en', 'office', ()), ('en', 'given', ()), ('en', 'family', ()))), ('júnior', (('pt', 'generation', ()), ('pt', 'given', ()), ('pt', 'family', ()))), ('kalbatoni', (('ka', 'salutation', (('after_given', True),)),)), + ('kassia', (('pt', 'given', (('after_particle', True), ('default', False))),)), ('kaur', (('mul', 'family', (('shared', True),)),)), ('kbe', (('en', 'credential', ()),)), ('kc', (('en', 'credential', (('acronym', False), ('cased', 'KC'), ('title_case_is_name', True))), ('en', 'given', ()), ('en', 'family', ()))), @@ -999,6 +1012,7 @@ ('krahv', (('et', 'nobility', ()),)), ('kreivi', (('fi', 'nobility', ()),)), ('kt', (('en', 'credential', (('cased', 'Kt'),)),)), + ('kássia', (('pt', 'given', (('after_particle', True),)),)), ("l'", (('fr', 'article', ()), ('it', 'article', ()))), ('la', (('ca', 'article', ()), ('es', 'article', ()), ('es', 'given', ()), ('fil', 'article', ()), ('fr', 'article', ()), ('it', 'article', ()))), ('lac', (('en', 'credential', ()), ('en', 'family', ()))), @@ -1080,6 +1094,7 @@ ('ltg', (('en', 'military', ()),)), ('ltgen', (('en', 'military', ()),)), ('ltjg', (('en', 'military', ()),)), + ('lurdes', (('pt', 'given', (('after_particle', True),)),)), ('lvn', (('en', 'credential', ()),)), ('lvo', (('en', 'credential', ()),)), ('lvt', (('en', 'credential', ()),)), @@ -1417,6 +1432,7 @@ ('physicist', (('en', 'occupation', ()),)), ('pi', (('en', 'family', ()),)), ('pianist', (('en', 'occupation', ()),)), + ('piedade', (('pt', 'given', (('after_particle', True), ('default', False))),)), ('pilar', (('es', 'given', (('after_particle', True),)),)), ('pilot', (('en', 'occupation', ()),)), ('pioneer', (('en', 'occupation', ()),)), @@ -1541,6 +1557,7 @@ ('reverend', (('en', 'religious', ()),)), ('rfp', (('en', 'credential', ()),)), ('rhca', (('en', 'credential', ()),)), + ('ribamar', (('pt', 'given', (('after_particle', True),)),)), ('richd', (('en', 'given', (('same_as', 'richard'),)),)), ('rid', (('en', 'credential', ()),)), ('right', (('en', 'modifier', ()),)), @@ -1710,6 +1727,7 @@ ('swami', (('bn', 'religious', (('addresses', 'given'),)), ('hi', 'religious', (('addresses', 'given'),)))), ('swordbearer', (('en', 'office', ()),)), ('sysselmann', (('nb', 'office', ()),)), + ('tarso', (('pt', 'given', (('after_particle', True),)),)), ('tax', (('en', 'modifier', ()),)), ('tbr-ct', (('en', 'credential', ()),)), ('td', (('en', 'credential', ()),)), diff --git a/nameparser/vocab/README.md b/nameparser/vocab/README.md index f0f2366e0..71382dfd1 100644 --- a/nameparser/vocab/README.md +++ b/nameparser/vocab/README.md @@ -183,7 +183,7 @@ A property is **behavioral** when it compiles into a set and so changes how name | `title_case_is_name` | behavioral | `suffix` | false | `TITLE_CASE_NAME_WORDS`, on a `generation` or a `credential` written as a word (`acronym = false`) only: says the word's Title case is its writing as a name, so where a name reading gates the word ([rules/S11](../../docs/design/rules/S11-suffix-word-that-is-also-a-name.md), a suffix word that is also a name), its writing leans: Title case is the name, lower case in a name of mixed case the suffix, capitals nothing ([rules/S6](../../docs/design/rules/S6-case-signal-at-the-suffix-slot.md), the case signal at the suffix slot), as in `Marie Fils` and `Alexandre Dumas fils`. Unwritten, a gated word's writing says nothing and the count decides, as for `Junior`, whose Title case is how the generation is written too; an agnomen's writing never speaks (#1039) | | `attached` | behavioral | `suffix` | absent | `GLUED_HONORIFICS`, on a word reading only: `{ side = "after" }` splits the word off the end of a name token, as in `田中さん` ([rules/W2](../../docs/design/rules/W2-glued-honorific-peel.md), the glued-honorific peel) | | `shared` | behavioral | `name` | false | `SHARED_NAMES`: a name a community gives every member, written after the personal name and before a family name where there is one (Sikh `kaur` and `singh`), no personal or family name of the bearer's own: two-family-name handling ([rules/O6](../../docs/design/rules/O6-two-family-names.md)) and two-given-name handling ([rules/O7](../../docs/design/rules/O7-two-given-names.md)) neither take it nor complete one with it, so `Harpreet Kaur Dhillon` keeps middle `Kaur` under either, and a given name a particle completes ([rules/P10](../../docs/design/rules/P10-given-name-a-particle-completes.md)) is never one, so `Kaur del Carmen García` keeps family `del Carmen García` by default (#704) | -| `after_particle` | behavioral | `name` | false | `PARTICLE_GIVEN_NAMES`, on a `given` reading only: a given name written behind a particle as the second of a compound given name (`carmen` in `María del Carmen`): behind a one-word given name the particle run and the word are a middle name, though the chain would run on into the family, as in `María del Carmen García` ([rules/P10](../../docs/design/rules/P10-given-name-a-particle-completes.md), a given name a particle completes, which states the conditions), which two-given-name handling ([rules/O7](../../docs/design/rules/O7-two-given-names.md)) then takes into the given name and two-family-name handling ([rules/O6](../../docs/design/rules/O6-two-family-names.md)) leaves where it stands. A word below the 90% bar decisions/O7 sets is left out (`luz`) (#704) | +| `after_particle` | behavioral | `name` | false | `PARTICLE_GIVEN_NAMES`, on a `given` reading only: a given name written behind a particle as the second of a compound given name (`carmen` in `María del Carmen`): behind a one-word given name the particle run and the word are a middle name, though the chain would run on into the family, as in `María del Carmen García` ([rules/P10](../../docs/design/rules/P10-given-name-a-particle-completes.md), a given name a particle completes, which states the conditions), which two-given-name handling ([rules/O7](../../docs/design/rules/O7-two-given-names.md)) then takes into the given name and two-family-name handling ([rules/O6](../../docs/design/rules/O6-two-family-names.md)) leaves where it stands. A word below the 90% bar decisions/O7 sets is left out of the default (`luz`) (#704), and one filers in its language's countries write in the given box from 75% of the time up to the bar, with ten given records or more, is written `default = false` in that language's file, for a choice of the language to turn on (`graças` in `pt.toml`, decisions/P10, #1118) | | `completed_by` | behavioral | `bound_given` | none | `BOUND_PHRASES`: the words that complete the bound given name into one name unit past the first name word, a list (`completed_by = ["الله", "الرحمن"]` on `عبد`), each one word written folded: `سعيد عبد الله` reads family `عبد الله` ([rules/P12](../../docs/design/rules/P12-bound-given-name-and-the-word-that-completes-it.md), a bound given name and the word that completes it), while a word left off the list after it is a name of its own, as in `نوال عبد هاشم` (#694) | | `may_be_given` | stated | `particle` | none: required where the word has no `given` reading | `false` is required where the word has no `given` reading in any file, and that absence, not the property, is what puts the word in `NON_GIVEN_NAME_PARTICLES`: a particle opening the name starts the family name ([rules/P1](../../docs/design/rules/P1-lone-particle-fold.md), the lone particle fold). `true` is refused, naming the `given` reading to write instead (check 11) | | `may_be_initial` | behavioral | `conjunction` | false | `CONJUNCTIONS_AMBIGUOUS`, on a cased single letter only: read as an initial in a name written in one case, as in `JOHN E SMITH` ([rules/P8](../../docs/design/rules/P8-single-letter-connective-against-the-initial-it-spells.md), a single-letter connective against the initial it spells) | diff --git a/nameparser/vocab/es.toml b/nameparser/vocab/es.toml index 2b15f96b9..379517c4c 100644 --- a/nameparser/vocab/es.toml +++ b/nameparser/vocab/es.toml @@ -225,6 +225,10 @@ given = { after_particle = true, note = "the second of a compound given name wri text = "consuelo" given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Consuelo): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 11 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } +[[word]] +text = "fatima" +given = { after_particle = true, note = "Fátima written without its accent (de Fatima): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 91 times and open a family box of more words with it behind a particle 0, over the whole extract, as fátima is marked on its Spanish counts; in Portugal, Brazil and Portuguese-speaking Africa 185 and 45 (2026-11-06, #1118)" } + [[word]] text = "fco" given = { same_as = "francisco", note = "Francisco written short: ORCID given boxes 137 (Spain 75, Mexico 7), family boxes 3 (2026-11-06); written with its period at the head of a name (Fco. Javier Barbas González), it read as a title by the leading-abbreviation shape (#1115)" } diff --git a/nameparser/vocab/pt.toml b/nameparser/vocab/pt.toml index 1c88ab68d..8e1ee52d8 100644 --- a/nameparser/vocab/pt.toml +++ b/nameparser/vocab/pt.toml @@ -126,3 +126,71 @@ text = "e" may_be_initial = true issue = [383, 479] note = "a bare E initial is common (Edward, Elizabeth) and an 'e' between two surnames is rare outside couple listings, so a name written wholly in one case reads it as an initial and reports the fork (decisions/P8)" + +[[word]] +text = "amparo" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (do Amparo): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 11 times and open a family box of more words with it behind a particle 1, over the whole extract (2026-11-06, #1118)" } + +[[word]] +text = "cassia" +given = { default = false, after_particle = true, note = "Cássia written without its accent, as registries write it (de Cassia): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 170 times and open a family box of more words with it behind a particle 31, 85%, below the 90% a default reading needs and well above a surname's (silva 32% on the same count), so a Portuguese choice turns it on (2026-11-06, #1118)" } + +[[word]] +text = "ceu" +given = { after_particle = true, note = "Céu written without its accent, as registries write it (do Ceu): 2 given boxes and 0 family boxes in ORCID's Portuguese-speaking countries, too few to decide alone, so it takes céu's reading (2026-11-06, #1118)" } + +[[word]] +text = "cássia" +given = { default = false, after_particle = true, note = "the second of a compound given name written behind a particle (Rita de Cássia): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 462 times and open a family box of more words with it behind a particle 64, 88%, below the 90% a default reading needs and well above a surname's (silva 32% on the same count), so a Portuguese choice turns it on (2026-11-06, #1118)" } + +[[word]] +text = "céu" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (Maria do Céu): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 31 times and open a family box of more words with it behind a particle 1, over the whole extract (2026-11-06, #1118)" } + +[[word]] +text = "gloria" +given = { after_particle = true, note = "Glória written without its accent (da Gloria): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 25 times and open a family box of more words with it behind a particle 1, over the whole extract (2026-11-06, #1118)" } + +[[word]] +text = "glória" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (Maria da Glória): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 38 times and open a family box of more words with it behind a particle 4, over the whole extract (2026-11-06, #1118)" } + +[[word]] +text = "graca" +given = { default = false, after_particle = true, note = "Graça written without its cedilla, as registries write it (da Graca): 3 given boxes and 0 family boxes in ORCID's Portuguese-speaking countries, too few to decide alone, so it takes graça's reading (2026-11-06, #1118)" } + +[[word]] +text = "gracas" +given = { default = false, after_particle = true, note = "Graças written without its cedilla, as registries write it (das Gracas): 4 given boxes and 0 family boxes in ORCID's Portuguese-speaking countries, too few to decide alone, so it takes graças's reading (2026-11-06, #1118)" } + +[[word]] +text = "graça" +given = { default = false, after_particle = true, note = "the second of a compound given name written behind a particle (Maria da Graça): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 87 times and open a family box of more words with it behind a particle 20, 81%, below the 90% a default reading needs and well above a surname's (silva 32% on the same count), so a Portuguese choice turns it on (2026-11-06, #1118)" } + +[[word]] +text = "graças" +given = { default = false, after_particle = true, note = "the second of a compound given name written behind a particle (Maria das Graças): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 138 times and open a family box of more words with it behind a particle 37, 79%, below the 90% a default reading needs and well above a surname's (silva 32% on the same count), so a Portuguese choice turns it on (2026-11-06, #1118)" } + +[[word]] +text = "kassia" +given = { default = false, after_particle = true, note = "Kássia written without its accent (de Kassia): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 19 times and open a family box of more words with it behind a particle 5, 79%, below the 90% a default reading needs, so a Portuguese choice turns it on; measured on its own, it parts from kássia, on by default (15 to 1), and no Spanish count weighs it (0 to 0) (2026-11-06, #1118)" } + +[[word]] +text = "kássia" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (Rita de Kássia): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 15 times and open a family box of more words with it behind a particle 1, over the whole extract (2026-11-06, #1118)" } + +[[word]] +text = "lurdes" +given = { after_particle = true, note = "the Portuguese spelling of Lourdes, the second of a compound given name written behind a particle (Maria de Lurdes): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 42 times and open a family box of more words with it behind a particle 1, over the whole extract (2026-11-06, #1118)" } + +[[word]] +text = "piedade" +given = { default = false, after_particle = true, note = "the second of a compound given name written behind a particle (Maria da Piedade): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 13 times and open a family box of more words with it behind a particle 4, 76%, below the 90% a default reading needs, so a Portuguese choice turns it on (2026-11-06, #1118)" } + +[[word]] +text = "ribamar" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (José de Ribamar, after the devotion of São Luís do Maranhão): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 17 times and open a family box of more words with it behind a particle 1, over the whole extract (2026-11-06, #1118)" } + +[[word]] +text = "tarso" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (Paulo de Tarso, Paul of Tarsus): ORCID holders in Portugal, Brazil and Portuguese-speaking Africa end the given box with it behind a particle 53 times and open a family box of more words with it behind a particle 3, over the whole extract (2026-11-06, #1118)" } diff --git a/tests/v2/rules_budget.json b/tests/v2/rules_budget.json index 27e71a139..fd92459a8 100644 --- a/tests/v2/rules_budget.json +++ b/tests/v2/rules_budget.json @@ -50,7 +50,7 @@ "P7": {"words": 138, "then_words": 64, "because_sentences": 3, "except": 0, "precedence": 0, "accepted": 1, "accepted_words": 28}, "P8": {"words": 496, "then_words": 431, "because_sentences": 2, "except": 0, "precedence": 0, "accepted": 2, "accepted_words": 309}, "P9": {"words": 420, "then_words": 280, "because_sentences": 1, "except": 2, "precedence": 0, "accepted": 0, "accepted_words": 0}, - "P10": {"words": 425, "then_words": 82, "because_sentences": 3, "except": 5, "precedence": 2, "accepted": 3, "accepted_words": 101}, + "P10": {"words": 478, "then_words": 135, "because_sentences": 3, "except": 5, "precedence": 2, "accepted": 3, "accepted_words": 101}, "P11": {"words": 229, "then_words": 80, "because_sentences": 3, "except": 0, "precedence": 1, "accepted": 2, "accepted_words": 77}, "P12": {"words": 296, "then_words": 87, "because_sentences": 3, "except": 2, "precedence": 0, "accepted": 3, "accepted_words": 116}, "P13": {"words": 426, "then_words": 128, "because_sentences": 2, "except": 4, "precedence": 2, "accepted": 1, "accepted_words": 42}, diff --git a/tests/v2/test_locales.py b/tests/v2/test_locales.py index 84ff56999..59eeb0209 100644 --- a/tests/v2/test_locales.py +++ b/tests/v2/test_locales.py @@ -234,7 +234,11 @@ def test_es_and_pt_pack_contents() -> None: # the choice changes only the words a collision keeps off assert changed_words(selected(locales.ES.languages)) == { "doña", "lady", "maestra"} - assert changed_words(selected(locales.PT.languages)) == {"do", "e"} + # the given names rules/P10 completes below the default's bar (#1118) + assert changed_words(selected(locales.PT.languages)) == { + "do", "e", + "cassia", "cássia", "graca", "gracas", "graça", "graças", + "kassia", "piedade"} # sr keeps es's and pt's generation beside the salutation, so the # choice leaves it a dual position decides, as the default reads it for packed in (_require_packed("es"), _require_packed("pt"), @@ -1630,7 +1634,20 @@ def test_a_choice_word_rotator_is_moved_by_the_choice_alone(code: str) -> None: *(n for n in _ROTATORS["es"] if n != "Jan\u200bPavel -"), "Ana\u200bGarcía -", # rules/O6 over a zero-width split and a lone mark ] +# rules/P10's given names below the default's bar, which pt's choice turns +# on, so the phrase reads as the middle name (#1118) +_PT_PARTICLE_GIVEN = [ + "Maria das Graças Silva", # graças + "MARIA DAS GRACAS SILVA", # gracas + "Maria da Graça Pereira", # graça + "MARIA DA GRACA PEREIRA", # graca + "Rita de Cássia Souza", # cássia + "RITA DE CASSIA SOUZA", # cassia + "Rita de Kassia Lima", # kassia + "Maria da Piedade Costa", # piedade +] _ROTATORS["pt"] = [ + *_PT_PARTICLE_GIVEN, "John Smith DO", # do, pt's particle, not DO "ANA SILVA E SOUZA", # e, pt's connective ] @@ -1638,6 +1655,7 @@ def test_a_choice_word_rotator_is_moved_by_the_choice_alone(code: str) -> None: "JOSE CARLOS SILVA SB", # sb "Jose Carlos Silva Sb.", # sb, dotted "JOSE SILVA SB", # sb, one word to spare + *_PT_PARTICLE_GIVEN, # pt's given names P10 completes "John Smith DO", # do, pt's particle, not DO "ANA SILVA E SOUZA", # e, pt's connective ] diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index adb476c6a..a2a43c5d2 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -895,6 +895,9 @@ "Maria Luisa y de la Cruz" "Maria López née García Pérez" "Maria da Silva Santos" +"Maria das Graças Silva" +"Maria de Fatima Costa" +"Maria de Lurdes Costa" "Maria do Carmo Silva" "Maria do Carmo da Silva" "Maria do Socorro Silva" @@ -982,6 +985,7 @@ "PEDRO DA SILVA FILHO" "PEDRO FILHO" "PEDRO JNR FILHO" +"Paulo de Tarso Ribeiro" "Pavlović, Antonio mlađi" "Pedro Alves Filho" "Pedro Alves Junior Filho" @@ -1043,6 +1047,7 @@ "RA DDS" "RA PhD Jr., John" "RAM BAHADUR B.K." +"RITA DE CASSIA SOUZA" "ROB ROY MACGREGOR" "ROBERT (RA) ALLEN" "RONALD MACDONALD" diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 6d23a90ec..4ad768517 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -2705,6 +2705,9 @@ {"name": "Maria Silva", "given": "Maria", "family": "Silva", "initials": "M. S."} {"name": "Maria Wijaya", "given": "Maria", "family": "Wijaya", "initials": "M. W."} {"name": "Maria da Silva Santos", "given": "Maria", "family": "da Silva Santos", "initials": "M. S. S."} +{"name": "Maria das Graças Silva", "given": "Maria", "family": "das Graças Silva", "initials": "M. G. S."} +{"name": "Maria de Fatima Costa", "given": "Maria", "middle": "de Fatima", "family": "Costa", "initials": "M. F. C."} +{"name": "Maria de Lurdes Costa", "given": "Maria", "middle": "de Lurdes", "family": "Costa", "initials": "M. L. C."} {"name": "Maria do Carmo Silva", "given": "Maria", "family": "do Carmo Silva", "initials": "M. C. S."} {"name": "Maria do Carmo da Silva", "given": "Maria", "middle": "do Carmo", "family": "da Silva", "initials": "M. C. S."} {"name": "Maria do Socorro Silva", "given": "Maria", "middle": "do Socorro", "family": "Silva", "initials": "M. S. S."} @@ -3095,6 +3098,7 @@ {"name": "Paul Phélypeaux de Pontchartrain", "given": "Paul", "middle": "Phélypeaux", "family": "de Pontchartrain", "initials": "P. P. P."} {"name": "Paul d'Ivoi", "given": "Paul", "family": "d'Ivoi", "initials": "P. d.", "capitalized_force": "Paul D'Ivoi"} {"name": "Paul de Maleingreau", "given": "Paul", "family": "de Maleingreau", "initials": "P. M."} +{"name": "Paulo de Tarso Ribeiro", "given": "Paulo", "middle": "de Tarso", "family": "Ribeiro", "initials": "P. T. R."} {"name": "Pavlović, Antonio mlađi", "given": "Antonio", "family": "Pavlović", "suffix": "mlađi", "initials": "A. P.", "capitalized_force": "Antonio Pavlović Mlađi", "generation": "mlađi"} {"name": "Pedro", "given": "Pedro", "ambiguities": ["GIVEN_OR_FAMILY"], "alternatives": ["GIVEN_OR_FAMILY: name as family (position)"], "initials": "P."} {"name": "Pedro", "order": "FAMILY_FIRST", "family": "Pedro", "ambiguities": ["GIVEN_OR_FAMILY"], "alternatives": ["GIVEN_OR_FAMILY: name as given (position)"], "initials": "P."} @@ -3261,6 +3265,7 @@ {"name": "RA DDS", "title": "RA", "family": "DDS", "initials": "D.", "capitalized": "Ra Dds"} {"name": "RA PhD Jr., John", "given": "John", "family": "RA", "suffix": "PhD Jr.", "initials": "J. R.", "capitalized_force": "John Ra PhD Jr.", "generation": "Jr.", "credentials": "PhD"} {"name": "RAM BAHADUR B.K.", "given": "RAM", "middle": "BAHADUR", "family": "B.K.", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: credential as suffix (shape)"], "initials": "R. B. B.", "capitalized": "Ram Bahadur B.K."} +{"name": "RITA DE CASSIA SOUZA", "given": "RITA", "family": "DE CASSIA SOUZA", "initials": "R. C. S.", "capitalized": "Rita de Cassia Souza"} {"name": "ROB ROY MACGREGOR", "given": "ROB", "middle": "ROY", "family": "MACGREGOR", "initials": "R. R. M.", "capitalized": "Rob Roy MacGregor"} {"name": "ROBERT (RA) ALLEN", "given": "ROBERT", "family": "ALLEN", "suffix": "RA", "initials": "R. A.", "capitalized": "Robert Allen RA"} {"name": "RONALD MACDONALD", "given": "RONALD", "family": "MACDONALD", "initials": "R. M.", "capitalized": "Ronald MacDonald"}