Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,7 @@ The compiled sets, read by `Lexicon` (mostly as the `_sets` field of the same na
- particles — `PARTICLES` (family-name particles, e.g. "de", "van") and `NON_GIVEN_NAME_PARTICLES`, the curated subset that is *never* a standalone given name (the particles whose word has no `given` reading, each stating `may_be_given = false`; a `given` reading on the word is what makes a particle possibly given, the phase 1 flag `may_be_given = true` refused since 2026-10-25). What the subset licenses is rules/P1's fold in `post_rules` (the "rule 1b" of older issue comments — decisions/legacy-rule-numbers maps the old numbers), and the fold has ONE site: the piece that OPENS the name. A lone never-given particle there, with at least one more name word behind it, starts the family. Titles do not move the opening position (rules/P4), so `Sir de Mesnil` reads family `de Mesnil` under every `name_order`, as `de Mesnil` does. How far the fold reaches depends on the order the name was read under: by default it takes the rest of the name (`de Mesnil Jean` → family `de Mesnil Jean`); under either family-first order it takes the particle run plus ONE name unit and lays the rest out by the order (`de Mesnil Jean` → family `de Mesnil`, given `Jean`); after a family comma it takes the rest of that part whatever the order (`Smith, de Mesnil Jean` → family `Smith de Mesnil Jean`). The leading particle stays a lone piece for the fold to find — `group` does not chain it forward, and the fold walks the particles after it itself — so `de la Vega` reads family `de la Vega` under all three orders. Behind a title-particle (`St`, `Freiherr`, P4's Except) it does chain, and under a family-first order the fold cuts that chain as it cuts a run (`St de Mesnil Jean` → family `de Mesnil`, given `Jean`, #828). The GIVEN position is not a site. It was a second one until #467, which removed it because under a family-first order that slot holds what the caller DECLARED to be the given name: `Mesnil de` under `FAMILY_FIRST` or `FAMILY_FIRST_GIVEN_LAST` reads given `de`, family `Mesnil`. Nor is a particle inside the name: `Juan de la Vega` under a family-first order reads family `Juan`, given `de la Vega`, with no ambiguity recorded — what the vocabulary forbids is the bare particle as a given name, not a name part that begins with one. A bare `de` has nothing to fold into and is placed by position like any one-word name: given `de` by default, family `de` under both family-first orders. A leading particle OUTSIDE the set is left to position — `Van Gogh` is given `Van`, family `Gogh` by default and family `Van`, given `Gogh` under both family-first orders — and records a `PARTICLE_OR_GIVEN` ambiguity under all three, save one written in lower case in a mixed-case name under a family-first order, which the fold takes and which records none (`van Gogh` → family `van Gogh`, rules/P11, #670); a never-given particle opening the name records none (`de Mesnil`), and neither does an ambiguous one inside it (`Vincent van Gogh`). The internal `Lexicon._sets.particles_ambiguous`, the particles holding a `given` reading, is its complement within `PARTICLES`, so the two mark OPPOSITE sets — see the flip warning in `docs/migrate.rst` before translating either
- bound given names — `BOUND_GIVEN_NAMES` (`bound_given` readings, e.g. "abdul", "abu"; one that is also a particle needs a `given` reading, a build check); a group-stage rule joins the first non-title piece to its following piece before roles are assigned, reserving a family word unless a family comma has already fixed it or a given-name title stands ahead (#369, rules/P5) (ported from v1's `_join_bound_first_name`), and `BOUND_PHRASES` (#694, rules/P12, the two-word given names a bound word's `completed_by` lists, Arabic `عبد` before the names of God), each pair joined past the first name word, where P5's join does not reach
- given names written short — `GIVEN_NAME_ABBREVIATIONS` (#789, a `given` reading carrying `same_as`, the full name: "chas", "wm", Filipino "ma"), the given names written short that rules/H2's shape yields to; nothing else reads it
- names a given name's second half reads — `PARTICLE_GIVEN_NAMES` (#704, a `given` reading written `after_particle`: "carmen", "ángeles", the Spanish words a compound given name writes behind a particle, which rules/P10 reads as a middle name in `María del Carmen García`, by default), `SAINT_PARTICLES` (the `saint` kind, `san`, `santa`, `st`, `ste`, `sta`, `sto`, whose run P10 refuses, `Santa María` being a surname) and `SHARED_NAMES` (#704, a name reading written `shared`: Sikh "kaur", "singh", which rules/O6 and O7 neither take nor complete and P10 does not complete, `Kaur del Carmen García`), private; each is read through classify's tags or the lexicon in `_post_rules`
- names a given name's second half reads — `PARTICLE_GIVEN_NAMES` (#704, a `given` reading written `after_particle`: "carmen", "ángeles", "lurdes", the Spanish and Portuguese words a compound given name writes behind a particle, which rules/P10 reads as a middle name in `María del Carmen García`, by default, and a few Portuguese ones, "graças", only under a Portuguese choice, #1118), `SAINT_PARTICLES` (the `saint` kind, `san`, `santa`, `st`, `ste`, `sta`, `sto`, whose run P10 refuses, `Santa María` being a surname) and `SHARED_NAMES` (#704, a name reading written `shared`: Sikh "kaur", "singh", which rules/O6 and O7 neither take nor complete and P10 does not complete, `Kaur del Carmen García`), private; each is read through classify's tags or the lexicon in `_post_rules`
- conjunctions — `CONJUNCTIONS` (e.g. "and", "of") used to chain multi-word titles and to join name parts (rules/P3), plus `CONJUNCTIONS_AMBIGUOUS` (#383/#479, rules/P8, `may_be_initial`, refused by the build on anything but a cased single letter), the subset of single cased letters that read as an INITIAL in a name written wholly in one case — a marker set
- maiden markers — `MAIDEN_MARKERS` (`birth_marker` readings, e.g. "née", "geb.") routing the following name to `maiden`; with an `addresses = "given"` title, the one kind a phrase may be (`z domu`)
- `config/_ko_surnames.py`, private since 3.0 (`config/surnames.py` through 2.x, 2.8.2 warning of the move) and read as `Lexicon.default().surnames` — `KOREAN_SURNAMES`, the census list the 2.0 API splits unspaced hangul on (#271), still written in Python with an import-time `assert_normalized`: a segmentation list, outside the per-word model, like the locale packs' lists.
Expand Down
15 changes: 9 additions & 6 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1187,12 +1187,15 @@ particle's words would otherwise run on into the family, the parser
reads them as a middle name, with or without the switch, if the
vocabulary marks the word behind the particle as a given name written
there (``after_particle``), and the switch then takes them into the
given name. The Spanish ones are marked, those that ORCID's researchers
from Spain and Spanish-speaking America write in the given box at least
nine times for each ten they also open a family box with it (``del
Carmen``, ``de los Ángeles``, ``del Mar``, ``del Pilar``: 23 words in
all on 2026-11-03). A phrase too often a surname to mark, as Portuguese
``do Carmo`` is, stays the family, the same shape as ``"João da Silva
given name. The words marked are those that ORCID's researchers write
in the given box at least nine times for each ten they also open a
family box with it: from Spain and Spanish-speaking America (``del
Carmen``, ``de los Ángeles``, ``del Mar``, ``del Pilar``), and from
Portugal and Brazil (``de Lurdes``, ``do Céu``, ``da Glória``, ``de
Tarso``). Portuguese words written as given names less surely
(``das Graças``, ``de Cássia``) are read only under ``locales.PT`` and
``locales.PT_BR`` (see :ref:`language-packs`). A phrase too often a
surname to mark, as Portuguese ``do Carmo`` is, stays the family, the same shape as ``"João da Silva
Santos"``, and so does one opening with a saint (``"Domingo Santa María
González"``):

Expand Down
2 changes: 1 addition & 1 deletion docs/design/decisions/O7-two-given-names.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
- **Open:** which particle phrases complete a given name, and whether that is a reading on the word or a list the rule kind holds (the particle-chain bullet above), #704 (decided 2026-11-03, below), and whether this rule reads O6's given-name list for it (`María del Carmen` against `João da Silva`), #959 (declined 2026-11-03, below); deriving `DEVIATES` from the rules (decisions/vocabulary-model's 2026-10-29 4f entry, item 6), which this rule leaves as it found it, having no surface in the name to declare, #754.
- 2026-10-30 #959 — THE WORD O6'S GIVEN-NAME LIST LEAVES IS THIS RULE'S. rules/O6 now leaves a listed given name where it stands (decisions/O6's entry of this date), so with both switches on the middle it leaves is this rule's to take where it is the unit beside the given name: `José Luis Pérez` reads given `José Luis`, family `Pérez`, where with the list empty O6 takes `Luis` first and the name keeps one given name. Where the given part holds two units before it, this rule takes the first and the listed word stays a middle (`Ana María Luis Pérez`: given `Ana María`, middle `Luis`), so a name of four words reads two and two only where O6 took its third (found by the #959 design-docs review). Nothing in this rule moved; its precedence clause states the reading with an example. Whether this rule reads the list itself is Open above.
- 2026-10-30 #753, phase 3g, decided in chat — NEITHER `es` NOR `pt` TURNS THIS SWITCH ON. ORCID cannot score it: its given box holds the middle names too, so the switch moves no record's agreement (0 fixed, 0 broken in every Spanish- and Portuguese-speaking country measured). PSC, whose form has a middle-name box, scores it against the filer's own split, and filers put the second given name in that box: under a `es` choice the switch alone takes Spanish filers from 31.8% to 15.7% (1 fixed, 980 broken), and beside `locales.ES`'s `two_family_names` from 82.0% to 54.7%; Portuguese filers fall from 61.1% to 28.5%. CLDR's `es` renderings read it right (the 2026-10-30 #704 entry, 6 → 8 of 8). The two sources disagree about a convention, not about a misreading, and the pack follows the boxes callers fill: a caller whose data writes the compound given name whole turns the switch on beside the pack. Recompute: decisions/O6's phase 3g recipe with `PolicyPatch(two_given_names=True)` added.
- 2026-11-03 #704, decided in chat — (Superseded in part by decisions/P10's 2026-11-04 #704 entry: the particle phrase is read by default, as a middle name, and this rule takes it as any middle unit; so the properties no longer move nothing with the switches off, `after_particle` reading by default and `shared`, like a patronymic, keeping a given name P10 would complete.) A PARTICLE PHRASE COMPLETES THE GIVEN NAME WHERE THE VOCABULARY MARKS ITS WORD, AND KAUR AND SINGH ARE NO GIVEN NAME. Settles the 2026-10-30 entry's Open. Two name-reading properties, each read by this rule and rules/O6 and by no other, so a parse with both switches off moves nowhere (the snapshot gains the rule examples' new texts and moves no reading; a grid of 128 texts under every order and switch pairing, token readings and `Reading.languages` included, moved no default-policy parse against master `a1b3e186`).
- 2026-11-03 #704, decided in chat — (Superseded in part by decisions/P10's 2026-11-04 #704 entry: the particle phrase is read by default, as a middle name, and this rule takes it as any middle unit; so the properties no longer move nothing with the switches off, `after_particle` reading by default and `shared`, like a patronymic, keeping a given name P10 would complete. Its Portuguese clause, that no Portuguese-speaking word reaches the bar, is superseded by decisions/P10's 2026-11-06 #1118 entry: over the whole ORCID extract eight words do, and six more are marked for a Portuguese choice alone; and the unaccented `fatima`, which this entry did not weigh, passes on its Spanish counts.) A PARTICLE PHRASE COMPLETES THE GIVEN NAME WHERE THE VOCABULARY MARKS ITS WORD, AND KAUR AND SINGH ARE NO GIVEN NAME. Settles the 2026-10-30 entry's Open. Two name-reading properties, each read by this rule and rules/O6 and by no other, so a parse with both switches off moves nowhere (the snapshot gains the rule examples' new texts and moves no reading; a grid of 128 texts under every order and switch pairing, token readings and `Reading.languages` included, moved no default-policy parse against master `a1b3e186`).
- **The shape: a property on the word behind the particle.** The particle cannot carry it (`del` opens `del Carmen` and `del Río` alike), and the question is a vocabulary one (the 2026-10-30 entry's particle-chain bullet). Weighed: (A) a phrase reading (`text = "del carmen"`), most precise, telling Spanish `de la Luz` from Portuguese `da Luz`, but a third kind of phrase reading beside a birth marker and a given-addressing title; (C) a list the rule holds as a `Policy` field or `[rules]` key, vocabulary in `Policy` that no choice of languages or strict parser reaches; (D) O6's given-name list (`Lexicon.given_names`), no new data, but it would split `de la Cruz` (175 family boxes in ORCID against 38 given), `de León` (151 against 5) and `de la Rosa`, and is empty by default and has no Portuguese list. B was chosen: `after_particle = true` on a `given` reading, one word, compiled into the internal `PARTICLE_GIVEN_NAMES`. Its cost against A is a word whose phrase reads both ways by particle, which is left out (`luz`).
- **The reach.** Given-first, no family comma (a comma, and both family-first orders, already read `Silva, Maria do Carmo` given `Maria do Carmo` through the middle path), the given name one word with no middle between it and the chain, the chain opening with particles and then the marked word, and a name word left after them for the family, one with a letter that is no connective standing alone (a family of particles alone never reaches the path: assign leaves a trailing particle the family by itself, the phrase a middle, found by the second #1057 review), and no Turkic marker behind the phrase under that rule, as for the middle path (both found by the #1057 review: `María del Carmen y` and `Ali del Carmen oglu` had moved): `María del Carmen` alone keeps family `del Carmen`, as `Juan Carlos` keeps family `Carlos`. A given name of two words (`María-José del Carmen García`) keeps what it has, and a middle between leaves the chain to the family, this rule taking that middle instead (`Ana María del Carmen García` reads given `Ana María`). With rules/O6 on too, O6 leaves the phrase where it stands (decisions/O6, this date), so `María del Carmen de la Vega` reads given `María del Carmen` with both on, where the 2026-11-03 #959 entry in decisions/O6 recorded family `del Carmen de la Vega`; and the family the split leaves keeps its chained words as one unit, so `María del Carmen García López` gives no `surname2` (accepted in the rule).
- **The words** (es.toml; each records its counts): `carmen`, `ángeles` and `angeles`, `pilar`, `rosario`, `valle`, `caridad`, `socorro`, `mar`, `rocío` and `rocio`, `lourdes`, `mercedes`, `maría` and `maria`, `milagro`, `milagros`, `cisne`, `fátima`, `concepción`, `nieves`, `consuelo`, `jesús` — 23. Measured 2026-11-03 on ORCID's `raw.jsonl` of the 2026-10-27 fetch (20,000 records a country at most): for each word, the records whose given box ends with a particle run (`Lexicon.default()`'s particles, NFC lower case) and the word, against those whose family box opens with a particle run and the word and holds more words after them, the shape that competes; kept where the given side is at least 90% with ten given records or more, counted over Spain and Spanish-speaking America (ES MX AR CO CL PE VE EC UY BO PY CR GT CU DO HN SV NI PA PR AD) and, separately, Portugal, Brazil and Portuguese-speaking Africa (BR PT AO MZ CV GW ST TL). `carmen` 1,261 to 5, `valle` 235 to 26 (exactly 90.0%), `rosario` 260 to 27. Left out: no Portuguese-speaking word reaches the bar (`carmo` 21 to 6, `rosário` 21 to 8, `conceição` 25 to 25, `fátima` 32 to 4, which the Spanish counts carry, 27 to 1); unaccented `jesus`, 203 to 16 in Spanish-speaking countries but 67 to 41 in Portuguese-speaking ones; `luz` (12 to 6, and Portuguese `da Luz` 4 to 5), `dios`, `paz`, `cruz`; and two that pass by count but are no compound given name, `c.` (an initial) and `piero` (Italian `del Piero`, 10 to 0). The Catalan phrases (`dels Àngels`, `de les Neus`, `de la Mercè`) barely occur, so no Catalan word is marked. The PSC register agrees where it can, its filers putting `del Carmen` in the middle-name box 21 times and the surname box 2, `del Mar` 11 and 0, `del Pilar` 10 and 1. `dels` stays out of `ca.toml` (the reason void since decisions/P10's 2026-11-04 entry, which holds the question Open): the property moves nothing with the switch off, so a `dels` particle would still move `Maria dels Àngels Puig` from middle `dels Àngels` to family `dels Àngels Puig` by default (the #704 comment of 2026-10-31).
Expand Down
Loading
Loading