Skip to content

Add locales.VI, a Vietnamese pack (#1108) - #1112

Merged
derek73 merged 9 commits into
masterfrom
claude/feat-vi-locale-pack-4787d7
Nov 6, 2026
Merged

derek73 merged 9 commits into
masterfrom
claude/feat-vi-locale-pack-4787d7

Conversation

@derek73

@derek73 derek73 commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

Closes #1108.

Adds locales.VI, a Vietnamese pack:

from nameparser import parser_for, locales
anh = parser_for(locales.VI).parse("Nguyen Van Anh")
anh.family, anh.middle, anh.given   # ('Nguyen', 'Van', 'Anh')

1.4.0 through 2.8.0, and the default, read Nguyen Van Anh as given Nguyen, family Van Anh.

What the pack is

  • A plain choice of vi, with no strict mode. Its order comes from vi.toml's new [rules] table (name_order = "family-first-given-last").

  • Titles vi.toml turns on, each default = false: the academic titles written before a name (ts, ths, gs, pgs, which also read the stacked PGS.TS. and GS.TS.) and the office đbqh with its ASCII spelling dbqh. bs is excluded, because preferred under the choice it evicted en.toml's credential BS.

  • A lexicon fragment that removes the readings other files give Vietnamese syllables:

    • particles: van le do du di la tho da san
    • connectives: the y
    • credentials: do ba chi

    This is the one pack fragment that removes. Should parser_for(locales.VI) read Nguyen Van Anh as family Nguyen, middle Van, given Anh, and Le Van Anh as family Le? #1108 assumed a given reading in vi.toml would win over the Dutch van, but choose prefers only by a vocabulary reading, and a name reading never settles a clash (decision 15). The pack therefore bends decisions 6 and 15 for this one pack: its order already commits a caller to Vietnamese names. mechanisms.md#LOCALE-PACKS-PURE-DATA's contract names the exception.

  • The default parse does not move. _compiled.py and snapshot.jsonl are unchanged.

Measured

On the local corpora #1110 added, scoring each record's parts derived from its order. Columns are default / the order alone / VI / strict VI.

Population Default Order alone VI Strict VI
UIT-ViNames, 21,090 names 13 21,065 21,085 21,089
UIT-ViNames, diacritics stripped 89 19,221 21,082 21,089
National Assembly deputies, 499 0 483 499 499

Accepted costs, each pinned by a test row:

  • A lone word reads as the family. Anh → family Anh, under rules/O5 (the lone name word's convention). Should a lone name word read as the given name under a family-first pack, so parser_for(locales.VI).parse("Anh") gives given Anh? #1111 asks for a Policy setting for this.
  • The credentials written for English-language readers. DO or BA after a Vietnamese name reads as a name word, with a comma or without (Nguyen Van Anh, DO → given DO), and CHI without a comma. That is 4 DO and 1 BA among ctgov and FEC names with a Vietnamese surname, each written given-first, which VI misreads anyway.
  • The removals act after a family comma. Beethoven, Ludwig van → middle van.
  • A trailing stacked title reads as the given name (Nguyen Van Anh PGS.TS.). No corpus writes one; all 16 found are leading.

Kept: after a family comma, the given part reads given-first. FEC's Vietnamese-surname donors write the given part Western-style 69 to 12 in native order.

CLDR's per-pack view: vi reads 16 of 24 renderings by default and 10 of 24 with the pack.

Also in this PR

  • config_diff.py diff --spelling stripped,upper,lower rewrites every name before it is read, so the romanized and one-case views recompute from the repository.
  • Tests that used vi as the stock file with no readings now use hu.
  • _surface_deviates (tests/v2/test_locales.py) matches a title phrase of two adjacent words. VI's office makes the first phrases a choice changes (chief đbqh).
  • tests/v2/rules_budget.json is re-recorded: rules/O5 gains one Accepted clause reading O5's own signal (one name word).
  • nameparser/vocab/AGENTS.md records the lesson: a reading taken out leaves the readings it overrode to act.

Reviews

Round Review Outcome
1 code-reviewer README available() doctest missing vi (a CI failure); NGUYEN VAN DO reading suffix DO under the pack, which led to the credential removals after measurement; the PGS.TS.BS. claim; the recipe not in the repository; stale --locale help, AGENTS.md and RFC text. All fixed.
1 pr-test-analyzer Eight of eleven removals were pinned only by a set assertion. A per-removal negative-control test was added, plus rows for each syllable, the comma shapes with titles, and the head titles.
1 comment-analyzer Rotator comments claimed witnesses they lacked; vetting notes overstated their sources. Fixed.
1 design-docs-reviewer Decision 6 and two Standing lines contradicted the exception (head notes added); the CLDR lone-word cost was 2, not 6; the recipe. Fixed.
1 /simplify Cached parsers in tests, one normalization per piece, phrases folded into held. Applied.
2 code-reviewer The credential cost was wider than stated; --spelling stripped broke Hangul and kana. Both measured and fixed (only marks on Latin letters are dropped). ma was measured at 0 gained and 1 cost, and is left out.
2 pr-test-analyzer Positive control added; credential cost shapes pinned; lowercase đ and Hangul rows added.
2 comment-analyzer The comma-name scope and the Y note corrected; the stacked-title comment now quotes H5.
2 design-docs-reviewer The ground for the exception was false for comma names, now recorded as an accepted cost; exact per-corpus credential counts with the comparator; a recipe that runs; decision 15's note names the credentials.
2 /simplify dataclasses.replace for the control pack applied. The _unmarked reuse was superseded by the Latin-only fix.
3 code-reviewer, on the last fix commit CHI after a comma stays a suffix: the capitals shape (rules/C6) takes any three-letter word in capitals there, whatever the vocabulary. The prose now says CHI reads as a name word only without a comma, and a test row pins the boundary. Everything else verified, including every row and the recipe's figures.

Checks: uv run pytest, -n0 -m serial, Python 3.15 in its own environment, mypy, ruff check, tools/vocab/build.py --check, sphinx-build -b doctest docs, python -m doctest README.rst.

🤖 Generated with Claude Code

derek73 and others added 8 commits November 5, 2026 20:26
Route A: a plain vi choice, vi.toml's [rules] family-first-given-last
order, and a lexicon fragment removing the particle and connective
readings of Vietnamese syllables. Titles provisional; decisions
figures pending the Vietnamese corpora.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…le-pack-4787d7

# Conflicts:
#	tests/v2/rules_budget.json
…ons (#1108)

- the fragment also removes the credential readings of do, ba, chi
  and ma, which took the given name of a name in one case
  (NGUYEN VAN DO, NGUYEN LINH CHI)
- a test puts each removal back in turn and checks its own row moves;
  rows for each syllable between name words, the one-case credentials,
  the comma shapes with titles, the head titles and the trailing
  stacked title
- config_diff.py diff --spelling (stripped, upper, lower), so the
  romanized and one-case views recompute from the repository
- decisions: head notes on decision 6 and the Standing lines; the CLDR
  lone-word cost is 2 renderings, not 6; PGS.TS.BS. reads whole; the
  credential measurement and its recipe
- stale text: README's available(), the --locale help, AGENTS.md's
  [rules] list, the vocabulary RFC
- nameparser/vocab/AGENTS.md: a reading taken out leaves the word's
  next reading to act

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…tin marks (#1108)

- ma keeps its credential reading (gained no corpus name, cost one);
  TRAN THI MA pinned as the known gap
- the credential and comma-name costs recorded and pinned: DO, BA, CHI
  after a Vietnamese name with or without a comma, dotted, bracketed,
  behind another credential; Beethoven, Ludwig van and Smith, John Do
  after a family comma, where the order reads nothing differently
- the per-removal test gains a positive control and builds its packs
  with dataclasses.replace; a mixed-case credential syllable is held
  to the report the removal drops
- config_diff.py --spelling stripped drops only the marks on Latin
  letters and composes again (Hangul, kana); README notes the option
- decisions: exact credential counts per corpus with the comparator,
  the declined positional rule, the trailing PGS.TS. count, a recipe
  that runs; decision 15's note names the credentials
- locales.rst, the release log, vi.py and test comments corrected

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@derek73 derek73 added this to the 3.0 milestone Nov 6, 2026
@derek73 derek73 added enhancement vocabulary Per-word vocabulary model and language packs labels Nov 6, 2026
@derek73 derek73 self-assigned this Nov 6, 2026
After a comma the capitals shape (rules/C6) keeps a three-letter word
in capitals a suffix whatever its readings, so Nguyen Van Anh, CHI
keeps suffix 'CHI'. The prose said otherwise; the boundary is pinned.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@derek73
derek73 merged commit 0c019fd into master Nov 6, 2026
13 checks passed
@derek73
derek73 deleted the claude/feat-vi-locale-pack-4787d7 branch November 6, 2026 05:38
derek73 added a commit that referenced this pull request Nov 6, 2026
The VI pack (#1112) reads Nguyen Van Anh family-first and removes the
Dutch particle reading, so the hand-built strict vi Locale goes. The
strict choice keeps an example: ES strict drops the English title
phrase Lady Justice.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
derek73 added a commit that referenced this pull request Nov 6, 2026
…1106)

* README: show 3.0's Spanish, Russian and output-view readings

The lead example now parses a Spanish name both ways: the default
reading and parser_for(locales.ES)'s two family names, with the
surname1/surname2 and generation views and the two_given_names switch.
More shapes: a title run, a listing form with generation and
credentials split, an ambiguity report, Reading.languages, the
honorifics and patronymic views, and the RU pack's patronymic beside
the default reading. The announcement links "by reading" to
customize.html#configuring-by-reading.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* README: say what the van and Arabic examples show, per review

Name what Reading.languages holds, show the Arabic family keeps the
patronymic's words, and describe the default as positional rather
than English.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* README: add a Dutch particles-last listing and a title-phrase title

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* README: show a language choice preferring its reading, and a strict choice

Lady reads as a title by default and a given name under ES; a strict
vi choice keeps Dutch van out of Nguyen Van Anh.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* README: say a choice turns Lady's given reading on, and what strict keeps

Per review: es holds a default-off given reading of lady rather than
dropping the title, and a strict choice keeps non-Latin-script words.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* README: add a measured accuracy section after Learn more

Figures from the 2026-11-05 agreement measurement against filed name
boxes (PatentsView, UK PSC, Czech candidates, ORCID per country).

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* README: show an Arabic name of God completing عبد (rules/P12)

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* README: show how to adjust readings with a Lexicon and a Policy

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* README: use locales.VI for the Vietnamese example, strict on ES

The VI pack (#1112) reads Nguyen Van Anh family-first and removes the
Dutch particle reading, so the hand-built strict vi Locale goes. The
strict choice keeps an example: ES strict drops the English title
phrase Lady Justice.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* README: name the VI parser vietnamese, as the other packs' parsers are named

Co-Authored-By: Claude Opus 5.5 <[email protected]>

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement vocabulary Per-word vocabulary model and language packs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Should parser_for(locales.VI) read Nguyen Van Anh as family Nguyen, middle Van, given Anh, and Le Van Anh as family Le?

1 participant