aegean.greek

greek

Greek NLP pipeline — composable, individually-callable stages.

The dependency-free core covers normalize (NFC/NFD + Beta Code ↔ Unicode, with a lenient OCR-repair mode), tokenize (word/sentence), syllabify, accent analysis (accentuation), prosody/meter scansion, phonology (IPA), a seed+rule lemmatize, baseline pos, and a rule-based morphology analyzer (analyze). pipeline runs the whole stack over a text in one call.

GreekPipeline instances isolate backend and profile configuration; pipeline_tokens analyzes typed corpus tokens without discarding editorial forms. Neural calls expose explicit long-input policies, task support, exact receipts, and manifest-validated runtime configuration. Lossless CoNLL-U document I/O retains comments, multiword ranges, empty nodes, enhanced dependencies, and MISC alongside the word-only predictive projection.

Opt-in backends layer on richer data and models:

• use_neural_pipeline (the [neural] extra) loads the joint neural model: one pass serving UPOS, full morphology (UD FEATS), UD dependency trees, and lemmas, with measured UD Ancient Greek (Perseus) results and comparisons in docs/benchmarks.md. Once active, pos_tag/pos_tags, lemmatize, parse, and pipeline all use it.
• use_treebank (Perseus AGDT) supplies attested, correctly-accented lemmas and full features for known forms.
• use_lsj (Perseus Liddell-Scott-Jones) provides glossing (gloss/lookup).
• use_parser (parse; arc-eager + averaged perceptron, trained on the AGDT) is a projective dependency parser (~0.63 UAS / 0.53 LAS on projective text).
• use_tagger is an averaged-perceptron POS tagger (~84% on unseen forms).
• use_lemmatizer is an edit-tree lemmatizer (~40% on unseen forms).
• use_neural_lemmatizer (the [neural] extra) is a GreTa T5 seq2seq model served as int8 ONNX without torch; it pairs a gold lookup with seq2seq decoding and reaches 76.3% on unseen forms. lemmatize cascades neural pipeline -> treebank -> neural -> edit-tree -> seed table -> generalizing ending rules.

Every stage is a plain function so it can be used standalone::

from aegean import greek
greek.betacode_to_unicode("mh=nin")      # 'μῆνιν'
greek.syllabify("ἄνθρωπος")              # ['ἄν', 'θρω', 'πος']
greek.accentuation("λόγος").classification  # 'paroxytone'
greek.pipeline("ἐν ἀρχῇ ἦν ὁ λόγος.")    # per-token records, one call

LabelMapping dataclass

LabelMapping(field: str, source_profile: str, target_profile: str, pairs: tuple[tuple[str, str], ...], reversible: bool, unmapped: tuple[str, ...] = (), context: str | None = None, losses: tuple[str, ...] = (), schema_version: int = 1)

A directional source-label mapping.

pairs always point from source_profile to target_profile. An inverse is available only for an explicitly validated bijection; a mapping that collapses labels must carry a loss disclosure and refuses inversion.

mapping property

mapping: Mapping[str, str]

Read-only source-to-target mapping view.

ProfileError

Bases: ValueError

Base error for invalid profile values or registry lookups.

ProfileMappingError

Bases: ProfileError

Raised when a mapping is lossy, ambiguous, or missing source context.

ProfileSchemaError

Bases: ProfileError

Raised when a profile value has an invalid or unknown schema field.

GreekPipeline

GreekPipeline()

An isolated Greek analysis pipeline with explicit backend ownership.

Constructing without arguments creates the zero-dependency baseline. Use GreekPipeline.neural() to load an isolated neural instance without changing the module-level default selected by use_neural_pipeline().

config property

config: GreekPipelineConfig

The immutable, serializable configuration identity.

neural_active property

neural_active: bool

Whether this instance owns a neural backend.

neural classmethod

neural(*, variant: str = 'default', force: bool = False, expected_receipt: AnalysisReceipt | None = None) -> GreekPipeline

Load one explicit neural variant without changing the default facade.

from_config classmethod

from_config(config: GreekPipelineConfig, *, force: bool = False) -> GreekPipeline

Recreate a pipeline and require its live configuration to match exactly.

analyze

analyze(text: str, *, parse: bool = False, with_confidence: bool = False, confidence_domain: str | None = None, confidence_policy: AbstentionPolicy | None = None, long_input: Literal['strict', 'partial', 'windowed'] = 'strict', document_id: str = 'input', sentence_policy: str = 'default', segmenter: SegmenterLike | None = None) -> list[TokenRecord]

Analyze text with this instance's backend and exact source alignment.

document_id scopes the deterministic sentence and source-token IDs. sentence_policy controls document splitting independently of the model configuration's pretokenized-input segmentation contract. A caller-supplied segmenter is validated before its boundaries reach the backend.

analyze_tokens

analyze_tokens(tokens: Sequence[Token], *, parse: bool = False, with_confidence: bool = False, confidence_domain: str | None = None, confidence_policy: AbstentionPolicy | None = None, long_input: Literal['strict', 'partial', 'windowed'] = 'strict', document_id: str = 'input', sentence_policy: str = 'default', segmenter: SegmenterLike | None = None) -> list[TokenRecord]

Analyze typed tokens while retaining alignment and editorial form state.

Complete, contiguous alignment sentence IDs are authoritative. Otherwise, sentence_policy or the validated external segmenter groups tokens.

analyze_sentence

analyze_sentence(words: list[str], *, with_probs: bool = False, long_input: LongInputMode = 'strict', confidence_domain: str | None = None, confidence_policy: AbstentionPolicy | None = None) -> SentenceAnalysis

Analyze one pre-tokenized sentence with this instance's neural backend.

analyze_sentences

analyze_sentences(sentences: Iterable[Iterable[str]], *, batch_size: int | None = None, with_probs: bool = False, long_input: LongInputMode = 'strict', confidence_domain: str | None = None, confidence_policy: AbstentionPolicy | None = None) -> list[SentenceAnalysis]

Analyze pre-tokenized sentences with this instance's neural backend.

iter_analyze_sentences

iter_analyze_sentences(sentences: Iterable[Iterable[str]], *, batch_size: int | None = None, with_probs: bool = False, long_input: LongInputMode = 'strict', confidence_domain: str | None = None, confidence_policy: AbstentionPolicy | None = None) -> Iterator[SentenceAnalysis]

Yield analyses lazily from this instance's captured neural backend.

GreekPipelineConfig dataclass

GreekPipelineConfig(schema_version: int, backend: Literal['baseline', 'neural'], model_id: str | None, dataset: str | None, runtime_variant: NeuralVariantLabel | None, variant_registry_sha256: str | None, variant_award_sha256: str | None, qualification_sha256: str | None, bundle_manifest_sha256: str | None, tokenizer_revision: str | None, annotation_profile: str, normalization: str, segmentation: str, preprocessing_version: str, execution_providers: tuple[str, ...])

Stable configuration identity for a GreekPipeline instance.

to_dict

to_dict() -> dict[str, Any]

Return a stable JSON-compatible representation.

to_json

to_json() -> str

Serialize canonically for storage or comparison.

from_dict classmethod

from_dict(value: Mapping[str, Any]) -> GreekPipelineConfig

Validate and deserialize configuration schema 1.

from_json classmethod

from_json(value: str) -> GreekPipelineConfig

Deserialize a configuration JSON object.

AccentInfo dataclass

AccentInfo(syllables: tuple[str, ...], accent_type: str | None, position_from_end: int | None, classification: str | None)

The accent analysis of one word.

AccentPlacement dataclass

AccentPlacement(form: str, accent_type: str, position_from_end: int, classification: str, certain: bool, note: str = '')

A predicted accent placement.

ResolvedForm dataclass

ResolvedForm(surface: str, words: tuple[str, ...], kind: str | None = None, uncertain: bool = False, note: str = '', alternatives: tuple[str, ...] = tuple())

The sandhi analysis of one token.

words is the underlying word sequence the surface form stands for; for a token with no sandhi (or one left unexpanded) it is just the input. kind names the phenomenon ("crasis" / "elision" / "movable-nu" / None). uncertain is set when a contraction is detected but cannot be expanded from standard forms, in which case words keeps the surface form unchanged. note is a short human-readable provenance string.

resolved property

resolved: bool

True if the surface form was expanded/normalised to something new.

Inflector

Inflector(index: dict[str, list[tuple[dict[str, str], str]]])

An inverse-lemmatization index: lemma -> attested (features, form) cells.

from_lexicon classmethod

from_lexicon(lexicon: dict[str, list[dict[str, str]]]) -> Inflector

Invert a treebank form->analyses lexicon into lemma->(features, form) cells.

Forms keep the lexicon's frequency order (most-attested first); duplicate (features, form) cells per lemma are collapsed.

inflect

inflect(lemma: str, **features: str | None) -> tuple[str, ...]

Attested form(s) of lemma matching features (a partial set of the keys in _FEATURES), most-attested first. Empty if nothing matches.

paradigm

paradigm(lemma: str) -> tuple[tuple[dict[str, str], str], ...]

Every attested (features, form) cell of lemma (empty if unattested).

InflectorNotLoadedError

Bases: RuntimeError

Raised when an inflection call is made before use_inflector.

LemmaSource

Bases: str, Enum

Where a lemma came from — the evidence class for one token's lemma.

A str Enum (not StrEnum: the floor is Python 3.10), so members are plain strings under json.dumps and comparisons; emit .value where a bare string is wanted. Resolution and verification are separate, so this is not a confidence order:

• ATTESTED — a treebank-lexicon hit (an attested, correctly accented lemma).
• NEURAL_LOOKUP — the joint model selected a training-form lookup lemma.
• NEURAL_EDIT — the joint model produced a non-identity edit-script lemma.
• NEURAL — a neural backend produced a lemma but does not expose its internal decision path (the older seq2seq and edit-tree backends).
• RULE — the ending-stripping rule layer recovered a regular citation form.
• SEED — the bundled seed table / a closed-class function word.
• PARADIGM — a hit in the opt-in UniMorph paradigm table (use_paradigms): a curated inflection lookup that recovers an irregular / third-declension form the ending rules cannot (γυναικός → γυνή), correctly accented and grounded.
• IDENTITY — a backend/model was consulted but returned the surface form unchanged (no real analysis), so the "lemma" is just the input.
• UNRESOLVED — the baseline cascade was exhausted; the normalized form is returned.
• PUNCT — a non-word token (punctuation / numeral): trivially its own lemma.
• USER — a human correction imported through the review workflow.

IDENTITY and UNRESOLVED are the classes a human should verify (see needs_review); the rest are grounded analyses.

Analysis dataclass

Analysis(lemma: str, pos: str, case: str | None = None, number: str | None = None, gender: str | None = None, tense: str | None = None, voice: str | None = None, mood: str | None = None, person: str | None = None, degree: str | None = None, lemma_certain: bool = True)

One candidate morphological reading of a form.

features

features() -> dict[str, str]

The non-empty morphological features, in a stable order.

RarityResult dataclass

RarityResult(overall: float, words: tuple[WordRarity, ...], corpus_lemmas: int, corpus_tokens: int)

A text's terminology rarity against a reference corpus.

hardest

hardest(n: int = 5) -> tuple[WordRarity, ...]

The n rarest words, most rare first.

WordRarity dataclass

WordRarity(word: str, lemma: str, count: int, rarity: float, label: str)

One word's rarity against the reference corpus.

TreebankLexicon

TreebankLexicon(data: dict[str, list[dict[str, str]]])

An attested form→analyses lexicon built from the AGDT treebank.

load classmethod

load(path: Path | str | None = None) -> 'TreebankLexicon'

Load a built lexicon JSON (defaults to the cached one).

analyze

analyze(form: str) -> tuple[Analysis, ...]

Attested analyses for a form (frequency-ordered), or () if unknown.

lemmatize

lemmatize(form: str) -> str | None

The most-attested lemma for a form, or None if unknown.

pos

pos(form: str) -> str | None

The most-attested part-of-speech tag for a form, or None if unknown.

ParadigmLexicon

ParadigmLexicon(data: dict[str, list[dict[str, str]]], *, resource_id: str = _PREBUILT, resource_sha256: str | None = None)

A form→analyses paradigm lexicon (UniMorph grc), served like TreebankLexicon.

resource_id property

resource_id: str

Stable dataset identifier recorded by documentary analysis receipts.

resource_sha256 property

resource_sha256: str

SHA-256 of the loaded resource bytes (or canonical fixture data).

load classmethod

load(path: Path | str | None = None) -> 'ParadigmLexicon'

Load a built paradigm index (defaults to the cached grc-paradigms.json.gz).

analyze

analyze(form: str) -> tuple[Analysis, ...]

Paradigm analyses for a form, or () if unknown — same record shape as TreebankLexicon.analyze, so the two backends are interchangeable.

lemmatize

lemmatize(form: str) -> str | None

The first (paradigm) lemma for a form, or None if unknown.

lemma_options

lemma_options(form: str) -> frozenset[str]

The DISTINCT lemmas (NFC-compared) a form maps to; frozenset() if unknown.

More than one element means the table is internally ambiguous for the form: φωτός is the genitive of both φώς 'man' and φῶς 'light', βασιλεία collides with βασίλεια. A caller can treat that as no confident pick rather than the arbitrary first entry. Several cells of a SINGLE lemma (γυναικός) collapse to one element and stay grounded.

pos

pos(form: str) -> str | None

The first part-of-speech tag for a form, or None if unknown.

LSJEntry dataclass

LSJEntry(headword: str, raw_key: str, lead: str, senses: tuple[Sense, ...], short: str)

A Liddell-Scott-Jones entry.

LSJLexicon

LSJLexicon(data: dict[str, dict[str, Any]])

A lemma→entry view of the Perseus LSJ, with lemmatize-on-miss lookup.

lookup

lookup(word: str) -> LSJEntry | None

The full LSJ entry for a word (form or lemma), or None if unknown.

gloss

gloss(word: str) -> str | None

A concise gloss — headword: <first sense> — or None if unknown.

LexiconNotLoadedError

Bases: RuntimeError

Raised when gloss/lookup is called before use_lsj.

UsageInfo dataclass

UsageInfo(dialects: tuple[str, ...], registers: tuple[str, ...])

Dialect and register tags recorded for a lemma in LSJ.

DepToken dataclass

DepToken(id: int, form: str, lemma: str, upos: str, head: int, relation: str, postag: str = '')

One token in a dependency tree (1-based id; head=0 is the root).

DepTree dataclass

DepTree(tokens: tuple[DepToken, ...])

A dependency tree over a sentence's tokens (AGDT/Prague relation labels).

root

root() -> DepToken | None

The token whose head is the artificial root (0).

is_projective

is_projective() -> bool

Whether the tree has no crossing arcs (arc-eager can only build these).

ParserNotLoadedError

Bases: RuntimeError

Raised when parse is called before use_parser.

TaggerNotLoadedError

Bases: RuntimeError

Raised when tag_pos is used before use_tagger.

LemmatizerNotLoadedError

Bases: RuntimeError

Raised when the trained lemmatizer is used before use_lemmatizer.

NeuralLemmatizerNotLoadedError

Bases: RuntimeError

Raised when the neural lemmatizer is used before use_neural_lemmatizer, or when the [neural] extra (onnxruntime/tokenizers) is not installed.

NeuralRuntimeVariant dataclass

NeuralRuntimeVariant(label: NeuralVariantLabel, availability: VariantAvailability, model_id: str | None, dataset: str | None, asset_sha256: str | None, bundle_manifest_sha256: str | None, award_sha256: str | None, qualification_sha256: str | None)

One immutable neural runtime selection and its evidence identities.

available property

available: bool

Whether this label currently resolves to an artifact.

to_dict

to_dict() -> dict[str, Any]

Return the exact JSON-compatible registry record.

from_dict classmethod

from_dict(value: Mapping[str, Any]) -> NeuralRuntimeVariant

Validate one exact registry record.

NeuralVariantError

Bases: ValueError

Raised when the bundled runtime-variant registry is invalid.

NeuralVariantUnavailableError

Bases: NeuralVariantError

Raised before download when a valid reserved variant has no artifact.

AnalysisReceipt dataclass

AnalysisReceipt(schema_version: int, source_schema_version: int, model_id: str, dataset: str, asset_sha256: str | None, asset_sha256_enforced: bool, bundle_manifest_sha256: str | None, bundle_schema_version: int | None, tokenizer_revision: str | None, package_version: str, python_version: str, runtime_versions: tuple[tuple[str, str], ...], execution_providers: tuple[str, ...], annotation_profile: str, normalization: str, segmentation: str, preprocessing_version: str, special_token_policy: str | None, max_subwords: int | None, input_tokens: int, analyzed_tokens: int, truncated: bool, windowed: bool, calibration_sha256: str | None = None, confidence_policy_sha256: str | None = None, output_profile_id: str | None = None, output_profile_sha256: str | None = None, postprocessing: tuple[str, ...] = (), runtime_variant: str | None = None, variant_registry_sha256: str | None = None, variant_award_sha256: str | None = None, qualification_sha256: str | None = None)

Stable, content-addressed provenance for one joint sentence analysis.

sha256 property

sha256: str

SHA-256 of the canonical receipt JSON.

create classmethod

create(manifest: ModelBundleManifest, *, execution_providers: tuple[str, ...], input_tokens: int, analyzed_tokens: int, truncated: bool, windowed: bool = False, calibration_sha256: str | None = None, confidence_policy_sha256: str | None = None, runtime_variant: str | None = None, variant_registry_sha256: str | None = None, variant_award_sha256: str | None = None, qualification_sha256: str | None = None) -> 'AnalysisReceipt'

Build a receipt from a validated bundle and the live execution session.

to_dict

to_dict() -> dict[str, Any]

Return the receipt as JSON-compatible values in a stable schema.

to_json

to_json() -> str

Serialize canonically for storage, comparison, or hashing.

from_dict classmethod

from_dict(value: Mapping[str, Any]) -> 'AnalysisReceipt'

Read supported receipt schemas or the pre-receipt backend-info shape.

The pre-receipt shape preserves only model and active providers. Unknown fields are represented honestly as empty or None rather than guessed.

from_json classmethod

from_json(value: str) -> 'AnalysisReceipt'

Deserialize a receipt from JSON.

assert_same_runtime

assert_same_runtime(actual: 'AnalysisReceipt') -> None

Require the artifact and software identity needed to reproduce this result.

with_analysis_profile

with_analysis_profile(profile: AnalysisProfile) -> 'AnalysisReceipt'

Bind an output profile, preserving schema-4 runtime-variant identity.

The profile module is imported lazily to keep contract import order cycle-safe. A validated profile contributes its stable id/hash and the ordered step ids; all runtime and confidence provenance stays unchanged.

assert_same_analysis_profile

assert_same_analysis_profile(actual: 'AnalysisReceipt') -> None

Require runtime identity and the complete output analysis profile to match.

ModelBundleError

Bases: RuntimeError

Raised when a neural model bundle is incomplete, corrupt, or incompatible.

ModelBundleManifest dataclass

ModelBundleManifest(schema_version: int, source_schema_version: int, model_id: str, dataset: str, asset_sha256: str | None, asset_sha256_enforced: bool, manifest_sha256: str, tokenizer_revision: str, max_subwords: int, annotation_profile: str, normalization: str, segmentation: str, preprocessing_version: str, special_token_policy: str, output_heads: tuple[str, ...], label_heads: tuple[str, ...], files: tuple[tuple[str, int, str], ...])

Validated, runtime-authoritative description of one neural bundle.

load classmethod

load(model_dir: str | Path, *, asset_sha256: str | None = None, asset_sha256_enforced: bool = False) -> 'ModelBundleManifest'

Load and fully validate a model directory before runtime activation.

Schema-1 bundles provide the runtime fields directly. The immutable legacy v3 bundle is migrated only after its exact published file table is recognized.

to_dict

to_dict() -> dict[str, Any]

Return the validated contract as JSON-compatible values.

NeuralInputTooLongError

NeuralInputTooLongError(*, input_tokens: int, analyzed_tokens: int, max_subwords: int)

Bases: ValueError

Raised when strict neural analysis cannot cover every input token.

NeuralPipelineNotLoadedError

Bases: RuntimeError

Raised when the neural pipeline is used before use_neural_pipeline, or when the [neural] extra (onnxruntime/tokenizers/numpy) is not installed.

NeuralWindowingError

Bases: ValueError

Raised when safe overlapping-window analysis cannot be supported.

Windowed inference is deliberately conservative: pathological inputs are refused before tokenizer/dense allocation, and every window must contain complete words with a useful overlap and at least one observed arc for each dependent.

ReceiptMismatchError

Bases: RuntimeError

Raised when a requested analysis receipt identity does not match.

SentenceAnalysis dataclass

SentenceAnalysis(tokens: tuple[str, ...], upos: tuple[str, ...], xpos: tuple[str, ...], feats: tuple[str, ...], head: tuple[int, ...], deprel: tuple[str, ...], lemma: tuple[str, ...], lemma_resolved: tuple[bool, ...] = (), upos_prob: tuple[float | None, ...] = (), lemma_script_prob: tuple[float | None, ...] = (), lemma_source_override: tuple[str, ...] = (), lemma_source: tuple[LemmaSource, ...] = (), lemma_verified: tuple[bool, ...] = (), lemma_source_path: tuple[str, ...] = (), token_confidences: tuple[TokenConfidence, ...] = (), sentence_confidence: SentenceConfidence | None = None, analyzed: tuple[bool, ...] = (), complete: bool = True, truncated: bool = False, warnings: tuple[str, ...] = (), receipt: AnalysisReceipt | None = None)

The joint model's full analysis, coverage status, provenance, and receipt.

incomplete property

incomplete: bool

Whether any input token lacks a neural analysis.

AbstentionPolicy dataclass

AbstentionPolicy(thresholds: Mapping[str, Any] | Iterable[tuple[str, Any]], *, name: str = '', schema_version: int = 1)

Immutable, explicitly configured accept/review policy.

thresholds is required from the caller and has no bundled defaults. A task without a threshold, or a confidence of None, produces unavailable rather than accepting, reviewing by an invented threshold, or converting None to 0. The canonical payload and SHA-256 travel with every decision.

to_json

to_json() -> str

Return the canonical JSON payload including its content hash.

Calibration dataclass

Calibration(temperature: dict[str, float], fitted_on: str = '', date: str = '', ece_before: dict[str, float] = dict(), ece_after: dict[str, float] = dict(), n: dict[str, int] = dict(), notes: str = '')

A fitted temperature calibration for the joint model's heads.

temperature maps each head name ("upos", "lemma") to its fitted scalar T (strictly positive). fitted_on describes the fold and model it was measured on, date when, and ece_before / ece_after / n record the measured Expected Calibration Error (before and after scaling) and the token count per head — the honesty evidence that travels with the number. Round-trips to JSON via to_dict / from_dict (and save / load).

to_dict

to_dict() -> dict[str, Any]

A JSON-ready dict of this calibration (the shape from_dict reads).

from_dict classmethod

from_dict(d: dict[str, Any]) -> 'Calibration'

Rebuild a Calibration from a to_dict mapping (validated on construction).

save

save(path: str | Path) -> None

Write this calibration to path as JSON.

load classmethod

load(path: str | Path) -> 'Calibration'

Read a calibration written by save (or an equivalent JSON file).

CalibrationEntry dataclass

CalibrationEntry(model: str, task: str, source: str | None = None, domain: str | None = None, fallback: bool = False, temperature: float | None = None, n: int | None = None, ece: float | None = None, brier: float | None = None, schema_version: int = 2, notes: str = '', calibrator: str = 'temperature', parameters: tuple[tuple[str, float], ...] = ())

One task/model/source/domain calibration record.

source and domain set to None mean a broader entry only when fallback=True is explicitly set. They are not wildcards when resolving an entry with a missing query scope; a caller must ask for an unscoped value explicitly. temperature is required for the legacy temperature calibrator; a logit-affine calibrator instead carries explicit finite slope and intercept parameters. The record still validates all evidence fields.

decision_path property

decision_path: str | None

Alias for source used by lemma calibration callers.

calibration_id property

calibration_id: str

Stable identity of this entry's canonical payload.

sha256 property

sha256: str

Alias for :attr:calibration_id suitable for receipts.

canonical property

canonical: str

Canonical parameter payload used to derive :attr:calibration_id.

to_dict

to_dict(*, include_id: bool = True) -> dict[str, Any]

Return a JSON-ready, deterministic mapping for this entry.

calibrate

calibrate(raw: Any) -> Any

Apply this entry's explicitly fitted monotone calibrator.

Temperature scaling accepts one class-logit vector and delegates to the existing lazy NumPy implementation. Logit-affine calibration accepts one raw probability, transforms its logit, and returns a probability. The distinct input contracts prevent a scalar probability from being mistaken for a raw logit; exact 0 and 1 are handled without taking log(0).

CalibrationRegistry dataclass

CalibrationRegistry(entries: tuple[CalibrationEntry, ...] = (), schema_version: int = 2)

Versioned collection of calibration entries with scope-aware resolution.

from_legacy classmethod

from_legacy(value: Mapping[str, Any]) -> 'CalibrationRegistry'

Read schema-1 Calibration.to_dict as unscoped aggregate evidence.

The legacy JSON has no model/source/domain fields. Only the known published grc-joint-v3 identity is attached when it is present in fitted_on; source and domain intentionally remain None so the record cannot be mistaken for source- or domain-specific evidence.

resolve

resolve(model: str, task: str, source: str | None = None, domain: str | None = None, *, decision_path: str | None = None) -> CalibrationResolution

Resolve exact scope first, then an explicitly broader fallback.

Rank order is exact source+domain, exact source with unscoped domain, unscoped source with exact domain, then fully unscoped. A missing query scope never borrows a narrower entry. Duplicate scopes are rejected at construction, so the result is deterministic regardless of JSON entry order.

CalibrationResolution dataclass

CalibrationResolution(entry: CalibrationEntry | None, model: str, task: str, source: str | None, domain: str | None, scope: str, fallback: bool = False, reason: str | None = None)

Result of deterministic calibration lookup.

entry is None on failure; reason is then always a stable explicit reason string. fallback is true only when a broader (None source/domain) entry was selected instead of an exact scope match.

ConfidenceResult dataclass

ConfidenceResult(task: str, value: float | None, reason: str | None = None, calibration_id: str | None = None, scope: str | None = None, model: str | None = None, source: str | None = None, domain: str | None = None, n: int | None = None, ece: float | None = None, brier: float | None = None)

A confidence value or an explicit unavailable reason.

CoverageRiskPoint dataclass

CoverageRiskPoint(threshold: float, coverage: float, risk: float | None, n_accepted: int, n_total: int)

One point in an explicit threshold coverage-risk curve.

PolicyDecision dataclass

PolicyDecision(task: str, action: str, confidence: float | None, threshold: float | None, reason: str | None, policy_sha256: str)

Result of applying an :class:AbstentionPolicy to one value.

SentenceConfidence dataclass

SentenceConfidence(result: ConfidenceResult, components: tuple[str, ...] = (), policy: 'PolicyDecision | None' = None)

Immutable sentence-level confidence and optional review decision.

TokenConfidence dataclass

TokenConfidence(index: int, upos: ConfidenceResult | None = None, xpos: ConfidenceResult | None = None, feats: ConfidenceResult | None = None, lemma: ConfidenceResult | None = None, head: ConfidenceResult | None = None, relation: ConfidenceResult | None = None, policy: tuple['PolicyDecision', ...] = ())

Immutable per-token confidence values and optional review decisions.

UncalibratedConfidenceError

Bases: RuntimeError

Raised when calibrated confidence is requested but none is loaded.

pyaegean never exposes a raw (uncalibrated) softmax probability: an uncalibrated number invites false confidence, which breaks the measured-claims-only rule. A temperature-scaled, ECE-measured calibration must be loaded (use_calibration) or fitted (fit_temperature) before any confidence is surfaced.

MissingForm dataclass

MissingForm(form: str, count: int, example_doc_id: str, example_position: int)

One unresolved Greek word form and where to find it.

• form — the surface word exactly as it appears in the corpus.
• count — how many WORD tokens with this exact surface the lemmatizer left unresolved (needs review).
• example_doc_id — the document id of the first occurrence, in corpus order.
• example_position — that token's position: its Token.position, or its index within the document's token stream when position is unset.

TokenExplanation dataclass

TokenExplanation(token: str, upos: str, lemma: str, lemma_source: LemmaSource, needs_review: bool, morphology: str | None, note: str)

One token's analysis with its evidence class spelled out in plain language.

lemma_source is the lemma's evidence class (see LemmaSource) and is the entire trust claim: there is deliberately no confidence field, and a calibrated number reaches the note only under with_confidence=True. needs_review is True for the two ungrounded classes (an identity fall-through or an unresolved miss). morphology is the UD FEATS string when the neural pipeline produced one, else None. note says in one line what the evidence class means for this token.

TextProfile dataclass

TextProfile(char_count: int, token_count: int, letter_count: int, script: str, greek_ratio: float, latin_ratio: float, is_polytonic: bool, has_accent: bool, has_breathing: bool, polytonic_ratio: float, majuscule_ratio: float, looks_like_betacode: bool, has_editorial_brackets: bool, editorial_mark_count: int, digit_or_numeral_ratio: float)

Observable, descriptive features of one raw text string.

Every field is a direct measurement of the input, not a prediction. There is deliberately no genre, register, or out-of-domain label: this profiles what the characters are, not what the text is about. script names the writing system most letters fall in by codepoint block (greek / latin / mixed / other); greek_ratio and latin_ratio expose the underlying evidence so a caller need not trust the label alone.

ErrorAnalysis dataclass

ErrorAnalysis(pos_confusions: tuple[tuple[str, str, int], ...], lemma_mismatches: tuple[tuple[str, str, str], ...], lemma_confusions: tuple[tuple[str, str, int], ...], per_pos: tuple[PosStat, ...], freq_bands: tuple[tuple[str, int, int, int], ...], pos_scored: int, pos_errors: int, lemma_errors: int, n_seen: int, n_unseen: int, pos_ok_seen: int, pos_ok_unseen: int, lemma_ok_seen: int, lemma_ok_unseen: int)

A breakdown of a tagger's errors against a gold set.

pos_confusions / lemma_confusions are (gold, predicted, count) triples, most-frequent first; lemma_mismatches samples (form, gold lemma, predicted lemma) for reading. per_pos is per-gold-POS accuracy (most-frequent POS first). The seen/unseen counts are substantive only for the AGDT held-out split, where forms carry a real seen flag; for out-of-domain sets (PROIEL, NT, UD) every token is unseen by construction, so the unseen figures equal the overall ones.

top_share property

top_share: float

Fraction of POS errors in the single most common confusion pair (higher = more systematic/convention-like, lower = more scattered).

summary

summary(*, top: int = 8) -> str

A short, readable breakdown: overall accuracy, top POS confusions, the weakest parts of speech, and (when meaningful) the unseen-form accuracy.

as_dict

as_dict() -> dict[str, Any]

A JSON-serializable view (for --json and receipts).

PosStat dataclass

PosStat(pos: str, n: int, pos_correct: int, lemma_correct: int)

Per-gold-POS accuracy over the scored tokens.

ConventionReport dataclass

ConventionReport(n_words: int, ufeats_correct: int, n_scheme_blocked_words: int, n_shared_only_words: int, shared_only_correct: int, feature_stats: tuple[FeatureConventionStat, ...], uas_correct: int, las_correct: int, label_only_errors: int, deprel_confusions: tuple[DeprelConfusion, ...])

Where the PROIEL UD-fold UFeats and LAS gaps come from, told apart into annotation-convention divergence (the AGDT-trained model structurally cannot close it) and real disagreement on the shared scheme.

Measurement only — it reproduces the official UFeats/UAS/LAS from the model's own outputs and partitions them; it does not replace any published number and nothing is fitted to the fold. Every count is over the fold's scored words (the evaluator scores every aligned syntactic word for these metrics; unlike the POS/lemma drift view it does not skip PUNCT/NUM).

ufeats property

ufeats: float

Per-word UFeats accuracy (reproduces the official UFeats F1 under gold tokenization, where precision = recall = accuracy).

gap_scheme_absent property

gap_scheme_absent: float

The share of ALL words lost purely to scheme-absent features (words carrying a universal feature the model can never emit — an unavoidable UFeats miss). One of the two additive parts of the UFeats gap.

gap_shared_disagreement property

gap_shared_disagreement: float

The share of ALL words lost to disagreement WITHIN the shared scheme (the other additive part; gap_scheme_absent + gap_shared_disagreement == ufeats_gap).

shared_subset_ufeats property

shared_subset_ufeats: float

UFeats accuracy on the subset of words whose gold features are all scheme-shared — the model's morphology quality with the convention gap removed.

scheme_absent_features property

scheme_absent_features: tuple[FeatureConventionStat, ...]

The universal feature types PROIEL uses that the AGDT scheme never emits, by count.

label_only_share property

label_only_share: float

The share of ALL words that are attachment-correct but label-wrong — the part of the LAS gap that is pure relabelling (uas - las == label_only_share).

deprel_top_share property

deprel_top_share: float

Fraction of the label-only-error mass in the single most common relation confusion (higher → more systematic/convention-like, lower → more scattered).

deprel_concentration

deprel_concentration(top: int = 5) -> float

Fraction of the label-only-error mass in the top-top relation confusions.

summary

summary(*, top: int = 8) -> str

A short, readable account of both decompositions.

as_dict

as_dict() -> dict[str, Any]

A JSON-serializable view (for --json and receipts).

DeprelConfusion dataclass

DeprelConfusion(gold: str, predicted: str, count: int)

A (gold relation → predicted relation) pair among the attachment-correct/label-wrong tokens — the label-only errors that separate UAS from LAS. Subtypes are stripped (the evaluator's LAS convention).

DriftReport dataclass

DriftReport(pos_confusions: tuple[tuple[str, str, int], ...], lemma_mismatches: tuple[tuple[str, str, str], ...], pos_scored: int, pos_errors: int, lemma_errors: int)

Where the PROIEL gap comes from — so systematic annotation-convention divergence can be told apart from scattered real error.

The shipped model is trained on the AGDT convention; scoring it on the differently-annotated PROIEL conflates real mistakes with convention differences. pos_confusions lists the (gold POS → predicted POS) disagreements most-frequent first: a few pairs carrying most of the POS errors points to a convention difference, a long flat tail to genuine error. lemma_mismatches samples the lemma disagreements (often homograph or normalization convention). POS here is already reconciled (PROPN→NOUN, SCONJ→CCONJ, AUX→VERB), so what remains is other convention drift plus real error.

pos_accuracy property

pos_accuracy: float

POS accuracy over the scored tokens (the same number evaluate_on_proiel reports).

lemma_accuracy property

lemma_accuracy: float

Lemma accuracy over the scored tokens.

top_share property

top_share: float

Fraction of POS errors in the single most common confusion pair — a rough systematic-vs-scattered signal (higher → more convention-like).

summary

summary(*, top: int = 8) -> str

A short, readable breakdown of the top POS confusions.

FeatureConventionStat dataclass

FeatureConventionStat(feature: str, gold_count: int, emitted_by_model_scheme: bool, shared_agree: int)

One UD feature type's contribution to the UFeats gap.

gold_count is how many scored gold words carry this (universal) feature. emitted_by_model_scheme is whether the AGDT→UD renderer can produce it at all — when False the feature is scheme-absent and every gold word carrying it is an unavoidable UFeats miss. shared_agree counts, among those gold words, how many the model labels with the SAME value (0 for a scheme-absent feature).

agreement_on_shared property

agreement_on_shared: float

Fraction of gold words carrying this feature that the model labels with the same value. 0.0 for a scheme-absent feature (the model never emits it).

UDComment dataclass

UDComment(text: str)

A preserved comment item in the original sentence order.

UDDependency dataclass

UDDependency(head: UDNodeID, relation: str)

One parsed DEPS arc; raw head text remains available on the row.

UDDocument dataclass

UDDocument(sentences: tuple[UDSentence, ...], leading_comments: tuple[str, ...] = (), trailing_comments: tuple[str, ...] = ())

A complete CoNLL-U document, including sentence order and document comments.

UDEmptyNode dataclass

UDEmptyNode(major: int, minor: int, raw_columns: tuple[str, ...] = (), deps: tuple[UDDependency, ...] = (), deps_raw: str = '_', misc: tuple[UDMiscEntry, ...] = (), misc_raw: str = '_', line_number: int | None = None)

One CoNLL-U empty node such as 5.1 (including valid 0.1).

UDMiscEntry dataclass

UDMiscEntry(key: str, value: str | None = None)

One ordered MISC item; a bare key has value=None.

UDMultiwordToken dataclass

UDMultiwordToken(start: int, end: int, form: str, raw_columns: tuple[str, ...] = (), deps: tuple[UDDependency, ...] = (), deps_raw: str = '_', misc: tuple[UDMiscEntry, ...] = (), misc_raw: str = '_', line_number: int | None = None)

One CoNLL-U range row such as 4-5.

UDNodeID dataclass

UDNodeID(raw: str, kind: Literal['word', 'root', 'range', 'empty', 'unknown'], major: int | None = None, minor: int | None = None, end: int | None = None)

A structurally typed CoNLL-U identifier, retained without float coercion.

kind is word, root, range, empty, or unknown. The unknown form is used only by lenient parsing so malformed input can be inspected and re-exported; strict parsing rejects it with a line number.

UDOpaqueRow dataclass

UDOpaqueRow(raw_columns: tuple[str, ...], line_number: int | None = None)

A malformed row retained by lenient parsing for forensic re-export.

UDProjection dataclass

UDProjection(ordinal_to_id: tuple[tuple[int, int], ...], omitted_ranges: tuple[str, ...] = (), omitted_empty_nodes: tuple[str, ...] = (), enhanced_dependencies_present: bool = False)

Explicit mapping from model ordinals to original word IDs.

UDSentence dataclass

UDSentence(sent_id: str, text: str, tokens: tuple[UDToken, ...], rows: tuple[UDRow, ...] = (), comments: tuple[str, ...] = (), items: tuple[UDItem, ...] = ())

One sentence with a legacy word projection and optional full row stream.

surface_projection property

surface_projection: UDProjection

Alias naming the model-facing word projection explicitly.

UDToken dataclass

UDToken(id: int, form: str, lemma: str, upos: str, xpos: str, feats: str, head: int | None, deprel: str, deps: tuple[UDDependency, ...] = (), deps_raw: str = '_', misc: tuple[UDMiscEntry, ...] = (), misc_raw: str = '_', raw_columns: tuple[str, ...] = (), line_number: int | None = None, form_state: TokenFormState | None = None)

One syntactic word, retaining additive enhanced/raw annotations.

The first eight fields and their positional/equality behavior are the legacy projection. Additive raw/typed fields are excluded from equality so old callers comparing parsed words remain compatible.

UnsupportedUDStructureError

Bases: ValueError

Raised when a complete CoNLL-U prediction is requested for unsupported rows.

PapyGreekConventionReport dataclass

PapyGreekConventionReport(n_words: int, upos_correct: int, upos_coordinator_errors: int, upos_other_errors: int, upos_confusions: tuple[tuple[str, str, int], ...], xpos_correct: int, xpos_coordinator_poscode: int, xpos_common_gender: int, xpos_underscore_encoding: int, xpos_residual_real: int, n_gold_xpos_underscore: int, xpos_position_errors: tuple[int, ...])

Where the PapyGreek UPOS and XPOS gaps come from — annotation/encoding convention (the AGDT-trained model structurally cannot close it on this fold) told apart from real error.

Measurement only: it reproduces the official UPOS/XPOS from the model's own outputs and partitions them; it does not replace any published number and nothing is fitted to the fold. Every count is over the fold's scored words (the evaluator scores every aligned syntactic word for UPOS/XPOS under gold tokenization).

upos property

upos: float

Per-word UPOS accuracy (reproduces the official UPOS F1 under gold tokenization).

coordinator_share property

coordinator_share: float

Fraction of ALL UPOS errors that fall on the coordinator class (gold CCONJ) — the single-phenomenon concentration signal.

upos_coordinator_pts property

upos_coordinator_pts: float

Share of ALL words lost to coordinator-class UPOS errors (one additive part of the UPOS gap; upos_coordinator_pts + upos_other_pts == upos_gap).

xpos property

xpos: float

Per-word XPOS (9-position exact) accuracy (reproduces the official XPOS F1).

xpos_convention_pts property

xpos_convention_pts: float

The three convention/encoding parts of the XPOS gap, together (coordinator + common-gender + _-encoding). xpos_convention_pts + xpos_residual_pts == xpos_gap.

xpos_forgiving_convention property

xpos_forgiving_convention: float

XPOS accuracy if the three convention/encoding buckets are forgiven — the model's morphology quality with the convention cap removed.

summary

summary(*, top: int = 8) -> str

A short, readable account of both decompositions.

as_dict

as_dict() -> dict[str, Any]

A JSON-serializable view (for --json / --drift and receipts).

EvalReceipt dataclass

EvalReceipt(id: str, package_version: str, manifest: dict[str, Any], model_id: str | None, treebank: str, split: str, protocol: str, scores: dict[str, float], extra: dict[str, Any] | None)

A tamper-evident record tying one evaluation result to its inputs.

Frozen and content-addressed: id is the short sha256 of the canonical JSON of every other field. Construct via eval_receipt. scores/manifest/extra are stored as-is; treat them as read-only (the dataclass is frozen, but their contents are plain dicts).

recompute_id

recompute_id(*, full: bool = False) -> str

Re-derive the id from the stored fields (full=True for the 64-char sha256).

verify

verify(other: 'EvalReceipt | None' = None) -> bool

Tamper check. With no argument, re-hash this receipt's fields and confirm they still produce self.id. With other, confirm both receipts describe the byte-identical evaluation (same content-addressed id).

as_json

as_json(*, indent: int | None = None) -> str

Serialize the full receipt (id + every field) to JSON.

With indent=None (the default) this is the canonical form whose bytes the id hashes, minus the id itself; as_dict() includes the id for storage.

as_dict

as_dict() -> dict[str, Any]

A plain dict of every field, including id (round-trips via from_dict).

from_dict classmethod

from_dict(data: dict[str, Any]) -> 'EvalReceipt'

Reconstruct a receipt from as_dict output (does not re-verify; call verify).

NormalizationWarning

Bases: UserWarning

Emitted by normalize(..., lenient=True) for each class of repair.

DodsonEntry dataclass

DodsonEntry(strongs: str, lemma: str, gloss: str, definition: str)

One Dodson lexicon entry.

DodsonNotLoadedError

Bases: RuntimeError

Raised when a Dodson glossing call is made before use_dodson().

LexEntry dataclass

LexEntry(headword: str, gloss: str, body: str, lexicon: str)

A dictionary entry, uniform across lexica.

LexiconInfo dataclass

LexiconInfo(id: str, name: str, scope: str, license: str, source: str, hosted: bool)

Metadata for one registered lexicon.

TokenRecord dataclass

TokenRecord(sentence: int, index: int, text: str, upos: str, lemma: str, lemma_source: LemmaSource, head: int | None = None, relation: str | None = None, xpos: str | None = None, feats: str | None = None, upos_confidence: float | None = None, lemma_confidence: float | None = None, neural_analyzed: bool | None = None, analysis_complete: bool = True, analysis_warning: str | None = None, analysis_receipt: AnalysisReceipt | None = None, alignment: SourceAlignment | None = None, form_state: TokenFormState | None = None, boundary_policy: str | None = None, boundary_policy_id: str | None = None, boundary_provenance: str | None = None, boundary_confidence: float | None = None, boundary_start_char: int | None = None, boundary_end_char: int | None = None, lemma_source_path: str | None = None, token_confidence: TokenConfidence | None = None, sentence_confidence: SentenceConfidence | None = None)

One token's full analysis from pipeline.

head refers to the index of another record in the same sentence (0 = sentence root, None = no parse). xpos/feats are filled only by the neural pipeline. lemma_source is the lemma's evidence class (see LemmaSource, one of attested / neural_lookup / neural_edit / neural / rule / seed / paradigm / identity / unresolved / punct / user): whether it was attested in a treebank, predicted by the neural model, recovered by a rule, from the seed table, from a paradigm-table lookup (the opt-in UniMorph inflection tables, use_paradigms()), an identity fall-through, unresolved, or punctuation. lemma_resolved, lemma_verified, and review_recommended keep resolution, human verification, and review triage separate. lemma_known is a deprecated compatibility alias for lemma_resolved.

upos_confidence / lemma_confidence are calibrated confidences (temperature-scaled on the UD Perseus dev fold; see aegean.greek.calibrate): the estimated probability the prediction is correct. They are None unless the neural pipeline is active, a calibration is loaded, AND the call set with_confidence=True — the project never surfaces a raw (uncalibrated) softmax. A calibrated number is model-only: lemmas resolved by an offline lexicon backend carry no model confidence (both fields stay None throughout the offline cascade, where the evidence class speaks). Within the neural pipeline the calibrated lemma_confidence covers the model's full lemma composition, including its internal training-form lookup, by design (the calibration target is composed-lemma correctness); it is None only for a token the model does not itself lemmatize (an identity fall-through or punctuation). The number is fitted on literary prose, so it carries that genre caveat. alignment maps the record to the exact input token, including original and normalized text, source IDs, half-open Unicode code-point offsets, whitespace, and normalization operations. On the final record in each sentence, the boundary_* fields identify the document segmentation policy, stable policy ID, provenance, optional plugin confidence, and exact source span when one is provable. Rules and explicit user IDs carry no invented confidence. lemma_source_path is the exact internal neural composition path when typed confidence requested it; it is None for an offline/punctuation replacement and on the legacy default path.

lemma_resolved property

lemma_resolved: bool

Whether the lemma is an actual decision rather than a surface fallback.

lemma_verified property

lemma_verified: bool

Whether a human reviewer explicitly verified or corrected the lemma.

review_recommended property

review_recommended: bool

Whether this lemma should be routed to human review.

lemma_known property

lemma_known: bool

Deprecated alias for lemma_resolved; use the explicit epistemic fields.

GitHubRateLimitError

Bases: DataNotAvailableError

The unauthenticated GitHub contents API (~60 requests/hour) is exhausted.

Raised distinctly from a generic fetch failure so a bulk run can stop cleanly instead of burning the next request. Set PYAEGEAN_GITHUB_TOKEN or GITHUB_TOKEN to raise the limit to 5,000/hour.

WorkFetchResult dataclass

WorkFetchResult(id: str, author: str, title: str, status: str, error: str | None = None)

One work's outcome in a bulk fetch: status is "fetched" (downloaded now), "cached" (already on disk, no network), or "failed" (error holds why). A dataclass so the CLI --json path serialises it directly.

Foot dataclass

Foot(name: str, syllables: tuple[str, ...], quantities: tuple[str, ...])

One metrical foot: its name and the syllables/quantities it spans.

LineScansion dataclass

LineScansion(line: str, meter: str, feet: tuple[Foot, ...], syllables: tuple[str, ...], quantities: tuple[str, ...], caesura: str | None, caesura_index: int | None, ambiguous: bool)

The scansion of one verse line.

pattern property

pattern: str

The classic glyph pattern, feet separated by |.

ScansionError

Bases: ValueError

Raised when a line cannot be fit to the requested meter.

RuleBasedSentenceSegmenter dataclass

RuleBasedSentenceSegmenter(policy: SegmentationPolicy = 'default', *, abbreviations: Sequence[str] | None = None)

Conservative built-in segmenter with a named domain policy.

SegmentationResult dataclass

SegmentationResult(source: str, boundaries: tuple[SentenceBoundary, ...], policy: str = 'default', provenance: BoundaryProvenance = 'rule', policy_id: str | None = None)

Immutable, validated result returned by every sentence segmenter.

segments property

segments: tuple[SentenceBoundary, ...]

Alias for callers that use segment terminology.

sentences property

sentences: tuple[str, ...]

The projected sentence strings, one per boundary and in boundary order: sentences[i] is the text of boundaries[i].

to_dict

to_dict() -> dict[str, Any]

A JSON-ready result whose boundaries and sentences are index-aligned: entry i of each describes the same sentence, the span and its projected text. Both lists are built in one pass over the boundaries, so a reader can zip them.

from_json classmethod

from_json(value: str) -> 'SegmentationResult'

Strictly decode a schema-1 JSON result, rejecting duplicate keys.

from_dict classmethod

from_dict(value: Mapping[str, Any]) -> 'SegmentationResult'

Decode and validate a JSON-ready result.

SentenceBoundary dataclass

SentenceBoundary(start: int, end: int, policy: str = 'default', provenance: BoundaryProvenance = 'rule', confidence: float | None = None, policy_id: str | None = None)

One half-open source span and the evidence that ended it.

start and end are Python code-point offsets into the exact source string. Whitespace between spans is intentionally retained in the source, but is not assigned to either sentence. Rule output has no confidence: callers must not mistake a deterministic rule marker for a calibrated score.

start_char property

start_char: int

Alias used by source-alignment consumers.

end_char property

end_char: int

Alias used by source-alignment consumers.

text

text(source: str) -> str

Return this exact source slice after validating the source type/range.

to_dict

to_dict(source: str | None = None) -> dict[str, Any]

Return JSON-safe boundary metadata, optionally including its text.

from_dict classmethod

from_dict(value: Mapping[str, Any]) -> 'SentenceBoundary'

Decode one JSON boundary and apply the same constructor validation.

SentenceSegmenter

Bases: Protocol

Protocol for caller-supplied deterministic or learned segmenters.

segment

segment(text: str) -> SegmentationResult | Sequence[Any]

Return a rich result or boundary-like sequence for text.

canonical_analysis_profile

canonical_analysis_profile() -> AnalysisProfile

Return the canonical no-postprocessing composed analysis profile.

get_annotation_profile

get_annotation_profile(profile_id: str) -> AnnotationProfile

Look up a built-in annotation profile by its exact stable ID.

get_domain_profile

get_domain_profile(profile_id: str) -> DomainProfile

Look up a built-in domain profile by its exact stable ID.

list_annotation_profiles

list_annotation_profiles() -> tuple[AnnotationProfile, ...]

Return the built-in annotation profiles in stable ID order.

The returned tuple is immutable; lookup by ID is available through :func:get_annotation_profile.

list_domain_profiles

list_domain_profiles() -> tuple[DomainProfile, ...]

Return the built-in domain profiles in stable ID order.

default_pipeline

default_pipeline() -> GreekPipeline

The immutable instance used by the historical module-level facade.

accentuation

accentuation(word: str) -> AccentInfo

Analyse the accent of a single Greek word.

place_accent

place_accent(word: str, *, recessive: bool, lemma: str | None = None, ultima_length: str | None = None, penult_length: str | None = None) -> AccentPlacement

Place an accent: recessive=True for finite verbs, else persistent (lemma required).

persistent_accent

persistent_accent(form: str, lemma: str, *, ultima_length: str | None = None, penult_length: str | None = None) -> AccentPlacement

Place a persistent accent (nominals): keep the lemma's syllable unless a lengthened ultima forces it one syllable toward the end. lemma supplies the home syllable via its own written accent.

recessive_accent

recessive_accent(word: str, *, ultima_length: str | None = None, penult_length: str | None = None) -> AccentPlacement

Place a recessive accent (finite verbs): as far toward the antepenult as the laws allow.

resolve_sandhi

resolve_sandhi(token: str) -> ResolvedForm

Resolve crasis, elision, and movable-ν / οὐκ-type sandhi in one token.

Returns a ResolvedForm. A token with no sandhi passes through unchanged (kind=None). Crasis is expanded only from the curated _CRASIS lexicon; an unlisted coronis form is flagged uncertain and left intact. Elision is restored only from the curated lexica (a listed proclitic/particle, or a listed full form such as ταῦτ'); any other clipped form is kept as it stands and flagged uncertain. The οὐκ/οὐχ/οὐ rule is purely contextual; movable-ν is asserted only for validated hosts (the -ουσι(ν) ending or the curated _MOVABLE_NU_HOSTS list), never for the look-alike i-stem accusatives (γνῶσιν, πίστιν), which pass through unclaimed.

Conservative throughout: the resolver never guesses an expansion. When the surface form is ambiguous it is returned unchanged with uncertain=True and an explanatory note.

resolve_sentence

resolve_sentence(text: str) -> list[ResolvedForm]

Run resolve_sandhi over every word of a sentence (punctuation dropped).

Returns one ResolvedForm per surface word, in order. Use [w for r in result for w in r.words] to get the flat expanded word stream the pipeline should index against.

disable_inflector

disable_inflector() -> None

Deactivate inflection synthesis.

paradigm

paradigm(lemma: str) -> tuple[tuple[dict[str, str], str], ...]

The full attested paradigm of lemma as (features, form) cells; requires use_inflector.

use_inflector

use_inflector(*, build: bool = True, force: bool = False) -> Inflector

Activate inflection synthesis for this session.

Builds (and caches) the AGDT lexicon on first use (build=True; downloads the AGDT files if needed), inverts it into the form index, and makes inflect / paradigm resolve against it. build=False loads an already-built lexicon.

lemma_resolved

lemma_resolved(source: LemmaSource) -> bool

Whether source represents an actual lemma decision rather than a fallback.

lemma_verified

lemma_verified(source: LemmaSource) -> bool

Whether a human reviewer explicitly verified or corrected the lemma.

lemmatize_sourced

lemmatize_sourced(word: str) -> tuple[str, LemmaSource]

Return (lemma, source): the lemma plus the evidence class it came from (see LemmaSource). This is the authoritative cascade; lemmatize and lemmatize_verbose are expressed on top of it, so the lemma and its known/verbose flag can never drift from the source.

Tier order: the neural joint pipeline (use_neural_pipeline) → the AGDT treebank (use_treebank, an attested lemma) → the GreTa seq2seq backend (use_neural_lemmatizer) → the trained edit-tree lemmatizer (use_lemmatizer) → the bundled seed table → the generalizing ending-stripping rule layer → the opt-in UniMorph paradigm table (use_paradigms, a curated PARADIGM-class lookup). A backend that returns the surface form unchanged is reported IDENTITY (not a real analysis); an exhausted baseline is UNRESOLVED.

The paradigm table is consulted after the ending rules, only when the guarded rules do not recover a citation form: the rules' recover-set (regular second-declension obliques and thematic verb endings) is exactly where the purely-nominal table shadows the dominant verb reading (the noun dative ἔχει = ἔχις would otherwise displace the verb ἔχει → ἔχω the rules resolve correctly), so the rules win there and the table supplies only the irregular / third-declension forms they cannot touch (γυναικός → γυνή). A paradigm hit is further gated: the closed-class and indeclinable guard, a capitalized surface (a proper name; the table is lowercase common vocabulary), and any form the table itself maps to more than one distinct lemma (φωτός = genitive of both φώς and φῶς) are all left as an honest miss rather than an arbitrary pick.

The joint pipeline's IDENTITY is decided by which branch composed the lemma (joint._compose_lemma), not by a surface-string compare, so a nominative singular whose lemma equals the form is correctly NEURAL. The auxiliary seq2seq/edit-tree backends have no such signal, so there a lemma equal to the surface reads IDENTITY (preserving their historical known semantics).

lemmatize_verbose

lemmatize_verbose(word: str) -> tuple[str, bool]

Return (lemma, known). known is False when the lemma is not a real analysis: an identity fall-through from a backend, or an exhausted baseline that returns the (normalized) input unchanged. Delegates to lemmatize_sourced (the tier order is documented there) so the flag tracks the evidence class exactly.

needs_review

needs_review(source: LemmaSource) -> bool

Whether a lemma with this source should be verified by a human: an IDENTITY fall-through or an UNRESOLVED baseline miss. ATTESTED/NEURAL/RULE/ SEED/PARADIGM/PUNCT are grounded and do not need review.

analyze

analyze(word: str) -> tuple[Analysis, ...]

All candidate morphological analyses of word (possibly several, given Greek's ambiguity; empty only for unanalysable tokens).

Closed-class words (article, prepositions, conjunctions, particles, pronouns, the copula) resolve to a single high-confidence analysis; open-class words yield the readings their ending permits. When the AGDT treebank backend is active (see aegean.greek.use_treebank), an attested form's analyses — correctly accented and covering irregular forms the rule engine can't — are returned instead; failing that, the opt-in UniMorph paradigm table (aegean.greek.use_paradigms) supplies irregular/third-declension paradigm cells, with the rule engine as the fallback for forms neither backend knows.

best_pos

best_pos(word: str) -> str | None

A single best part-of-speech guess from morphology, or None when the form yields no analysis. Returns the most likely reading's tag (verbal and closed-class readings, which are listed first, take precedence over the nominal default), or ADJ when a degree is marked.

lemmas

lemmas(word: str) -> list[str]

The distinct lemma candidates for a form (closed-class or rule-derived).

terminology_rarity

terminology_rarity(text: str, corpus: object) -> RarityResult

Score the terminology rarity of text against a reference corpus.

corpus is any aegean.Corpus, the results of a Corpus.query, or a list of documents; its word tokens define the frequency basis, so the score is relative to that material's register and a queried subset scores against the subset. Returns the overall score plus a per-word breakdown (use .hardest() to surface the rare terms).

disable_treebank

disable_treebank() -> None

Deactivate the treebank lexicon; restore the default rule/seed behaviour.

use_treebank

use_treebank(*, build: bool = True, force: bool = False) -> TreebankLexicon

Activate the AGDT lexicon for this session.

Downloads + builds it on first use (build=True); pass force=True to rebuild. Once active, aegean.greek.lemmatize / analyze prefer its attested analyses and fall back to the rule/seed engines on a miss.

disable_paradigms

disable_paradigms() -> None

Deactivate the paradigm lexicon; restore the default seed/rule behaviour.

use_paradigms

use_paradigms(*, build: bool = True, force: bool = False, path: Path | str | None = None) -> ParadigmLexicon

Activate the UniMorph paradigm lexicon for this session.

Fetches the prebuilt index on first use (build=True); pass path to load a local build/fixture offline, or force=True to re-fetch. Once active, aegean.greek.lemmatize / analyze consult it for irregular/third-declension forms, after the generalizing ending rules and only when those rules do not recover, subject to the cascade guards (closed-class/indeclinable, capitalized surface, intra-table ambiguity). A paradigm hit is reported under its own PARADIGM evidence class (a grounded, curated lookup).

disable_documentary_lemma_rescue

disable_documentary_lemma_rescue() -> None

Deactivate Lever B; remove the wrapper if no lever remains active.

disable_documentary_reconciliation

disable_documentary_reconciliation() -> None

Deactivate Lever A; remove the wrapper if no lever remains active.

documentary_lemma_rescue_active

documentary_lemma_rescue_active() -> bool

Whether Lever B (lemma OOV rescue) is currently active.

documentary_reconciliation_active

documentary_reconciliation_active() -> bool

Whether Lever A (coordinator reconciliation) is currently active.

rescue_lemma

rescue_lemma(form: str) -> tuple[str, LemmaSource] | None

The guarded offline lemma rescue for one form: (lemma, source) or None.

Consults the two CURATED offline sources: the bundled seed table (SEED), then the opt-in UniMorph paradigm table when use_paradigms is active (PARADIGM, gated by the same closed-class / indeclinable / capitalized-surface / intra-table-ambiguity guards). Returns None when neither recovers a citation form (the form stays an honest miss). Never consults the neural model, so a rescue is always an offline, grounded analysis under its own evidence class — never NEURAL.

The generalizing ending-stripping rules are deliberately NOT consulted here: on the OOV residue the neural model already left unresolved, the rules fabricate about as often as they fix (measured on the documentary dev fold: roughly break-even there, and net-negative on the literary dev fold, where they were the sole source of regressions), so the rescue keeps only the curated, correctly-accented tiers.

use_documentary_lemma_rescue

use_documentary_lemma_rescue() -> None

Activate Lever B: offline lemma OOV rescue over the neural output.

Requires the neural pipeline to be active (raises NeuralPipelineNotLoadedError otherwise). When the model leaves a lemma unresolved, the guarded offline cascade (seed → the opt-in paradigm table) is consulted for a rescue; a rescue never overrides a resolved neural lemma and carries its own offline evidence class (see rescue_lemma). Default-off and byte-identical to the plain pipeline until called; disable_* restores that.

use_documentary_reconciliation

use_documentary_reconciliation(*, aggressive: bool = False) -> None

Activate Lever A: closed-class coordinator reconciliation over the neural output.

Requires the neural pipeline to be active (raises NeuralPipelineNotLoadedError otherwise). aggressive=False (the default, recommended) corrects only the X / b drift, which is always wrong for a coordinator and so cannot mislabel literary text. aggressive=True additionally folds in the ADV / d drift, which clobbers the legitimate adverbial reading (measured to regress the literary dev fold heavily). Default-off and byte-identical to the plain pipeline until called; disable_* restores that.

disable_lsj

disable_lsj() -> None

Deactivate the LSJ lexicon.

lookup

lookup(word: str) -> LSJEntry | None

Full LSJ entry for a word; requires use_lsj. None if unknown.

use_lsj

use_lsj(*, build: bool = True, force: bool = False) -> LSJLexicon

Activate the LSJ lexicon for this session.

Downloads (~270 MB) + builds the index on first use (build=True); pass force=True to rebuild. Then gloss / lookup resolve words against it.

disable_parser

disable_parser() -> None

Deactivate the dependency parser.

evaluate_parser

evaluate_parser(*, source_dir: Path | str | None = None, holdout: float = 0.1, epochs: int = 5) -> dict[str, Any]

Train on a split and score the held-out trees → {"uas","las","tokens","sentences"} (gold POS/lemma; measures parsing in isolation). Exposed as greek.evaluate_parser.

parse

parse(sentence: str | list[str]) -> DepTree

Parse a Greek sentence (a string or a list of tokens) into a DepTree.

Uses the neural pipeline when it is active (aegean.greek.use_neural_pipeline) — relations are then UD (nsubj, obj, advcl, …) and postag carries the predicted 9-char tag. Otherwise requires use_parser (the arc-eager baseline, AGDT/Prague relations), with POS/lemma from the (treebank-aware) pipeline.

use_parser

use_parser(*, train: bool = True, force: bool = False) -> None

Activate the dependency parser for this session — training the model on first use (train=True; from the cached AGDT, a few minutes) or loading the cache.

disable_tagger

disable_tagger() -> None

Deactivate the POS tagger; restore the lookup/rule behaviour.

evaluate_tagger

evaluate_tagger(*, source_dir: str | None = None, holdout: float = 0.1, epochs: int = 8) -> dict[str, float]

Train on the train split and score POS on the held-out split (overall + unseen), via aegean.greek.heldout — the honest generalization number. Returns pos_all/pos_unseen plus the token counts (this tagger predicts POS only, so the lemma metrics are omitted).

use_tagger

use_tagger(*, train: bool = True, force: bool = False) -> None

Activate the generalizing POS tagger. With train=True (default) it trains on first use — from the cached AGDT, a few minutes — then caches the model; later calls load the cache. train=False loads an existing cached model without training (raises TaggerNotLoadedError if none exists). force=True retrains even if cached.

disable_lemmatizer

disable_lemmatizer() -> None

Deactivate the lemmatizer; restore the lookup/seed/identity behaviour.

evaluate_lemmatizer

evaluate_lemmatizer(*, source_dir: str | None = None, holdout: float = 0.1, epochs: int = 8) -> dict[str, float]

Train on the train split and score lemma accuracy on the held-out split (overall + unseen), via aegean.greek.heldout — the honest generalization number. A POS tagger is trained on the same split so the dev set is scored with predicted POS (the realistic pipeline), not gold. Returns lemma_all/lemma_unseen plus token counts (POS metrics are omitted).

use_lemmatizer

use_lemmatizer(*, train: bool = True, force: bool = False) -> None

Activate the generalizing lemmatizer. With train=True (default) it trains on first use — from the cached AGDT, a few minutes — then caches the model; later calls load the cache. train=False loads an existing cached model (raises LemmatizerNotLoadedError if none exists). force=True retrains even if cached.

disable_neural_lemmatizer

disable_neural_lemmatizer() -> None

Deactivate the neural lemmatizer; the cascade falls back to the edit-tree/seed/identity.

use_neural_lemmatizer

use_neural_lemmatizer(*, force: bool = False) -> None

Activate the neural (GreTa seq2seq) lemmatizer.

Fetches the model bundle (ONNX encoder/decoder + tokenizer + gold lookup) to the cache on first use — never bundled in the wheel — then loads it via onnxruntime. Requires the [neural] extra (pip install 'pyaegean[neural]'). Best paired with aegean.greek.use_treebank, whose attested lemmas take precedence for seen forms.

Raises NeuralLemmatizerNotLoadedError if the optional dependencies are missing (checked before any download), and aegean.data.DataNotAvailableError if the model URL is not yet pinned (set PYAEGEAN_GRC_LEMMA_NEURAL_URL to fetch from your own mirror) or the download fails.

neural_variant

neural_variant(label: str = 'default') -> NeuralRuntimeVariant

Inspect one label without fetching or activating its artifact.

neural_variants

neural_variants(*, available_only: bool = False) -> tuple[NeuralRuntimeVariant, ...]

List the stable labels, including unavailable reservations by default.

variant_registry_sha256

variant_registry_sha256() -> str

SHA-256 identity of the complete bundled selection registry.

analyze_sentence

analyze_sentence(words: list[str], *, with_probs: bool = False, long_input: LongInputMode = 'strict', domain: str | None = None, policy: AbstentionPolicy | None = None) -> SentenceAnalysis

The full joint analysis of one pre-tokenized sentence (raises if not active).

with_probs=True additionally fills the calibrated confidence fields and requires a loaded calibration. long_input is strict by default; partial mode returns explicit coverage status and warnings, while windowed mode reconciles complete-word overlap windows with one global observed-arc tree (see _JointModel.analyze).

analyze_sentences

analyze_sentences(sentences: Iterable[Iterable[str]], *, batch_size: int | None = None, with_probs: bool = False, long_input: LongInputMode = 'strict', domain: str | None = None, policy: AbstentionPolicy | None = None) -> list[SentenceAnalysis]

Collect iter_analyze_sentences into a list for compatibility.

Use iter_analyze_sentences when output memory must remain bounded or results should become visible incrementally. This collector still returns the historical list, but it no longer materializes a second complete copy of the input before analysis starts.

Every argument keeps its meaning there, batch_size included: the default of one encoder pass per sentence is the sequential path the published numbers are measured on, and a positive int trades prediction-identity for throughput.

iter_analyze_sentences

iter_analyze_sentences(sentences: Iterable[Iterable[str]], *, batch_size: int | None = None, with_probs: bool = False, long_input: LongInputMode = 'strict', domain: str | None = None, policy: AbstentionPolicy | None = None) -> Iterator[SentenceAnalysis]

Yield joint analyses lazily from a pre-tokenized sentence iterable.

The active backend and its opt-in wrapper state are captured when this function is called, before the source is touched. batch_size=None pulls and yields one sentence at a time. A positive integer pulls at most that many sentences, analyzes one transactional chunk, then yields it in source order. Pausing or closing the returned iterator never pulls another sentence. Once iteration begins, it takes ownership of that source iterator for cleanup: closing the result also calls a source close() when one is provided.

Memory therefore does not grow with the number of source sentences: it is bounded by one result/chunk plus the largest individual sentence. Results from completed chunks remain valid if a later source or backend operation fails. A failed chunk yields nothing, is not retried, and propagates the original backend/source exception. Every yielded SentenceAnalysis keeps its own unmodified receipt.

batch_size=None (the default) analyzes each sentence with its own encoder pass — identical to calling analyze_sentence in a loop, and the code path the published benchmark numbers are measured on (plain CPU, CPUExecutionProvider). A positive int runs padded chunks of that many sentences through the encoder (one ONNX call per chunk). That is a throughput convenience, not a free one: each chunk is padded to its longest sentence, which reorders float reductions in the batched matmuls, so a few tokens in a large run can be tagged, lemmatized, or attached differently than the sequential path tags them. The result is therefore not prediction-identical and is never used for the recorded protocol; a batched score should not be compared with a sequential one. with_probs behaves as in analyze_sentence (calibration required). Strict and partial modes apply independently per sentence; windowed mode is always sequential inside the captured backend, so batch chunking cannot change owner or global-tree reconciliation.

This is sentence-level streaming only. Raw-text analysis, corpus annotation, and CoNLL-U serialization retain their collecting contracts.

disable_neural_pipeline

disable_neural_pipeline() -> None

Deactivate the neural pipeline; every function falls back to its prior cascade.

neural_backend_info

neural_backend_info() -> dict[str, Any]

Which ONNX Runtime execution providers the neural pipeline can and does use.

Returns provider facts plus the stable available/default/active runtime labels. model is the joint model's dataset name (the pinned grc-joint release asset); available_providers is what the installed onnxruntime offers (None when the [neural] extra is not installed); active_providers is what the live joint session actually runs on (session.get_providers()), or None when the pipeline is not active. Never fetches, never raises for a missing extra.

The published benchmark numbers are measured on CPUExecutionProvider; GPU execution (PYAEGEAN_ORT_PROVIDERS, or auto-detected CUDA/DirectML) is a throughput convenience, and the int8-quantized models may partition only partially onto a GPU provider.

use_neural_pipeline

use_neural_pipeline(*, variant: str = 'default', force: bool = False, expected_receipt: AnalysisReceipt | None = None) -> None

Activate the default neural pipeline facade (tags, morphology, trees, lemmas).

Fetches the model bundle to the cache on first use — never bundled in the wheel — then loads it via onnxruntime. Requires the [neural] extra (pip install 'pyaegean[neural]'). Once active, aegean.greek.pos_tags / pos_tag, aegean.greek.parse (UD relations), and aegean.greek.lemmatize all use it; analyze_sentence returns the full joint analysis in one call.

variant selects an evidence-bound runtime label. The default remains the release-selected artifact; a reserved label fails before dependency checks or network access rather than silently falling back. Pass expected_receipt to require the exact model, artifact, package/runtime versions, provider, profile, and preprocessing identity from a prior analysis. A mismatch raises ReceiptMismatchError before the candidate becomes active.

Raises NeuralPipelineNotLoadedError if the optional dependencies are missing (checked before any download), and aegean.data.DataNotAvailableError if the download fails (set PYAEGEAN_GRC_JOINT_URL to fetch from your own mirror).

annotate_corpus

annotate_corpus(corpus: 'Corpus', *, tag_sentence: TagSentence | None = None, with_evidence: bool = True, long_input: Literal['strict', 'partial', 'windowed'] = 'strict', progress: Callable[[int, int], None] | None = None) -> 'Corpus'

Return a copy of corpus with each word token's lemma / upos annotations filled by the pipeline (plus explicit lemma provenance/review fields when with_evidence).

tag_sentence overrides the tagger (a forms -> [(lemma, upos)] callable); it carries no evidence class, so lemma_source is written only by the built-in paths. The default is the active pipeline: the neural joint model if loaded, else the offline lemmatize + POS baseline. Existing annotations on a token are preserved except for the keys written here. Neural input is strict by default. Pass long_input="partial" only when explicitly marked placeholder annotations are useful, or long_input="windowed" to reconcile supported long lines from overlapping whole-word windows; inspect analysis_complete and the analysis receipt in either non-default mode. progress is called as progress(done, total) once per document (a large corpus under the neural pipeline is a long run).

active_registry

active_registry() -> 'CalibrationRegistry | None'

Return the active v2 registry, or None when no calibration is active.

brier_score

brier_score(probs: Iterable[Any], correct: Iterable[Any]) -> float

Return the binary Brier score (mean squared probability error).

Empty vectors return 0.0 for consistency with :func:calibrate.ece; callers that need an empirical support count should retain it separately.

canonical_json

canonical_json(value: Any) -> str

Return the canonical JSON encoding used for confidence hashes.

Canonical encodings are UTF-8, compact, key-sorted, and reject non-finite JSON numbers. Rejecting NaN/Infinity here matters: the standard json encoder otherwise emits non-standard tokens that could hash differently across readers.

canonical_sha256

canonical_sha256(value: Any) -> str

Return the SHA-256 of :func:canonical_json encoded as UTF-8.

coverage_risk_curve

coverage_risk_curve(probs: Iterable[Any], correct: Iterable[Any], thresholds: Sequence[Any]) -> tuple[CoverageRiskPoint, ...]

Return deterministic coverage-risk points for caller-supplied thresholds.

Thresholds are deduplicated and sorted ascending; acceptance uses p >= t. No thresholds are invented by this function. Risk is None (not zero) when a threshold accepts no item, preserving the distinction between no evidence and no observed errors.

disable_calibration

disable_calibration() -> None

Unload the active calibration; the joint model then refuses to surface confidence.

ece

ece(probs: Any, correct: Any, *, n_bins: int = 15, np: '_np_t | None' = None) -> float

Expected Calibration Error via equal-width binning (Guo et al., 2017).

probs are per-item top-1 confidences in [0, 1]; correct are the matching 0/1 outcomes (whether the surfaced prediction was right). Each item is placed in one of n_bins equal-width bins over [0, 1] (a confidence of exactly 1.0 falls in the last bin); the ECE is the sample-weighted mean absolute gap between each bin's mean confidence and its empirical accuracy. 0 means perfectly calibrated. An empty input is defined as 0.0.

fit_logit_affine

fit_logit_affine(probs: Any, correct: Any, *, slope_bracket: tuple[float, float] = (0.001, 1000.0), intercept_bracket: tuple[float, float] = (-30.0, 30.0), np: '_np_t | None' = None) -> tuple[float, float]

Fit a monotone logit-affine calibrator to raw confidence/correctness pairs.

The returned (slope, intercept) parameters define sigmoid(slope * logit(p) + intercept). A strictly positive slope preserves confidence ordering, so calibration cannot make a less-confident prediction rank above a more-confident one. The fit minimizes binary negative log-likelihood by a deterministic nested grid/golden-section search and needs only NumPy.

probs must be a non-constant one-dimensional sequence in [0, 1] and correct must contain both binary outcomes. Exact endpoint probabilities are clipped only while fitting so their logits remain finite; applying the fitted :class:CalibrationEntry still maps exact 0 and 1 to themselves.

fit_temperature

fit_temperature(logits: Any, correct: Any, *, bracket: tuple[float, float] = (0.05, 20.0), np: '_np_t | None' = None) -> float

Fit the scalar temperature T that best calibrates a head's top-1 confidence.

logits is a 2-D [n_items, n_classes] array of a head's raw logits; correct is a length-n_items 0/1 array of whether the surfaced (argmax) prediction was right. T is chosen to minimize the binary negative log-likelihood of the top-1 confidence softmax(z / T).max() against correct — i.e. it directly calibrates the one number pyaegean surfaces. Overconfident heads (the usual case) fit T > 1 (flattening the distribution lowers confidence toward the observed accuracy); a well-calibrated head fits T ≈ 1.

The optimization is a coarse geometric grid over bracket (robust to a non-unimodal objective) followed by a golden-section refine around the grid minimum — no scipy. Returns a strictly-positive float.

Raises ValueError on an empty fold, a shape mismatch, non-finite logits, or an invalid bracket.

temperature_softmax

temperature_softmax(logits: Any, temperature: float, *, np: '_np_t | None' = None) -> Any

Numerically-stable softmax over the last axis after dividing by temperature.

temperature must be strictly positive; it only rescales the distribution's sharpness and never moves the argmax. logits may be 1-D (a single row) or 2-D [n_items, n_classes]; the returned probabilities have the same shape and sum to 1 along the last axis. Pass np to reuse an already-imported numpy module (the joint model does this in its decode loop).

top1_confidence

top1_confidence(logits: Any, temperature: float, *, np: '_np_t | None' = None) -> Any

The top-1 (max) softmax probability per row, temperature-scaled.

This is the single number pyaegean surfaces as a prediction's confidence: once the temperature is calibrated, it estimates the probability the argmax prediction is correct. 1-D logits return a scalar; 2-D [n_items, n_classes] return a length- n_items array. The argmax is unchanged by temperature (see temperature_softmax), so this only rescales the confidence, never the label.

use_calibration

use_calibration(source: 'str | Path | Calibration | dict[str, Any] | None' = None) -> Calibration

Load a calibration and make it the active one, so the joint model may surface calibrated confidence.

source is a Calibration, a path to a JSON file (save's format), a to_dict mapping, or None for the bundled default calibration (shipped in the wheel). The no-arg form raises UncalibratedConfidenceError only when that file cannot be loaded (a missing or corrupt install), never a raw softmax. Returns the loaded Calibration.

use_calibration_registry

use_calibration_registry(source: 'str | Path | CalibrationRegistry | dict[str, Any]') -> 'CalibrationRegistry'

Load and activate a version-2 task/source/domain calibration registry.

missing_forms

missing_forms(corpus: 'Corpus', *, limit: int = 0) -> list[MissingForm]

The distinct Greek word forms the active lemmatizer cannot resolve, most frequent first.

Walks every lexical (WORD) token of corpus, lemmatizes each with aegean.greek.lemmatize_sourced, and keeps the forms whose evidence class needs review (aegean.greek.needs_review: UNRESOLVED or IDENTITY). Returns one MissingForm per distinct surface form with its total count and its first attestation, sorted by descending count then by form so the order is stable. limit caps the number of rows returned; 0 (the default) returns all of them.

Read-only (the corpus is never mutated) and zero-dependency. Each distinct form is lemmatized once and the verdict cached, so the cost is one lemmatize per distinct word. The result reflects whichever lemmatizer is active: the offline baseline by default, or a loaded neural / treebank backend.

explain_pipeline

explain_pipeline(text: str, *, with_confidence: bool = False) -> list[TokenExplanation]

Analyze text with pipeline and explain each token's record.

Returns one TokenExplanation per token, in pipeline order: the surface form, UPOS, lemma, the lemma's evidence class with a plain-language note, whether it needs review, and the morphology (FEATS) when the neural pipeline filled it. Derived entirely from the TokenRecord fields pipeline returns, so it reflects exactly the backends that were active for that call (activate them first with the use_* functions). Empty or whitespace-only input yields an empty list.

The evidence class is the honesty surface. with_confidence=True (opt-in) additionally appends each token's calibrated confidence to its note, and a raw softmax is never shown, so the flag adds a number only where a calibrated one exists. Under the offline cascade there is none: the call succeeds and the notes read exactly as they do by default. Under the neural pipeline it needs a loaded calibration (aegean.greek.use_calibration) and raises UncalibratedConfidenceError without one. With a calibration loaded, every token's note carries a UPOS number, and a lemma number joins it wherever the joint model produced the lemma (neural_lookup / neural_edit) but not for an identity fall-through or punctuation.

render_explanations

render_explanations(explanations: Sequence[TokenExplanation]) -> str

Render explanations as an aligned plain-text table for terminal display.

Columns: token, upos, lemma, source, review, morphology, note. The review cell reads review for a token to verify and stays blank for a grounded one; morphology is blank when the record carried none. Returns "(no tokens)" for an empty list.

profile_text

profile_text(text: str) -> TextProfile

Compute the observable feature profile of text.

Reports measured features only (script blocks, polytonic vs bare, Beta Code look, majuscule share, editorial markers, numeral density) and never predicts a genre or an out-of-domain label. An empty string yields an all-zero profile with script="other".

analyze_errors

analyze_errors(tag_sentence: TagSentence, sentences: Sequence[Sequence[HeldoutToken]], *, samples: int = 40, freq: Callable[[str], int] | None = None) -> ErrorAnalysis

Run tag_sentence over gold sentences and tabulate the errors.

Only tokens flagged scored count (PUNCT/NUM excluded, as everywhere). The predicted lemma is cleaned (_clean_lemma) before comparison, matching the aggregate scorers, and the gold lemma is already cleaned by the gold builder. tag_sentence and the gold are expected to already agree on POS convention (the adapters reconcile both sides). freq, when given, maps a form to a corpus frequency for per-band accuracy.

heldout_error_analysis

heldout_error_analysis(tag_sentence: TagSentence | None = None, *, holdout: float = 0.1, source_dir: str | None = None, samples: int = 40) -> ErrorAnalysis

Error analysis on the leakage-free AGDT held-out split — the one source with a real seen/unseen contrast (so the seen/unseen figures are substantive here). Defaults to the active pipeline; disable the treebank backend first to avoid leakage (see aegean.greek.heldout).

nt_error_analysis

nt_error_analysis(tag_sentence: TagSentence | None = None, *, corpus: Any = None, book: str | None = None, samples: int = 40) -> ErrorAnalysis

Error analysis on the Nestle-1904 NT gold (out-of-domain Koine). Defaults to the neural joint pipeline, exactly like evaluate_on_nt.

proiel_error_analysis

proiel_error_analysis(tag_sentence: TagSentence | None = None, *, source_dir: Any = None, files: tuple[str, ...] | None = None, samples: int = 40) -> ErrorAnalysis

Error analysis on the PROIEL Greek treebank (out-of-AGDT). Same tagger/reconciliation as evaluate_on_proiel.

ud_error_analysis

ud_error_analysis(tag_sentence: TagSentence | None = None, *, treebank: str = 'perseus', split: str = 'test', source: Any = None, samples: int = 40) -> ErrorAnalysis

Error analysis on a UD Ancient Greek fold (default UD-Perseus test). Unlike evaluate_on_ud (which scores raw UPOS with the official evaluator), this reconciles POS on both sides so per-POS reads as real disagreement; treat it as a diagnostic, not the headline number. Defaults to the active pipeline (neural if loaded, else baseline).

evaluate_on_proiel

evaluate_on_proiel(tag_sentence: TagSentence | None = None, *, source_dir: Path | str | None = None, files: tuple[str, ...] = _GREEK_FILES, progress: Callable[[int, int], None] | None = None) -> dict[str, float]

Score a tagger on PROIEL gold — the neutral, out-of-AGDT generalization number.

tag_sentence maps a sentence's forms to (lemma, pos) per token; it defaults to pyaegean's current pipeline (lemmatize + pos_tag, honouring whichever backends are active — enable use_treebank/use_neural_lemmatizer first to measure them). progress (optional) is called as progress(done, total) per scored sentence. Returns {"lemma", "pos", "n"}: lemma and POS accuracy over the scored tokens. Lemma is the clean metric; POS is compared under a reconciled tagset (PROPN→NOUN, SCONJ→CCONJ). See proiel_drift for where the gap comes from. The PROIEL files are fetched on first use unless source_dir points at local XML.

load_proiel_gold

load_proiel_gold(*, source_dir: Path | str | None = None, files: tuple[str, ...] = _GREEK_FILES) -> tuple[tuple[HeldoutToken, ...], ...]

Parse the PROIEL Greek treebank into gold sentences of (form, lemma, POS) tokens.

Fetches the pinned PROIEL files into the cache unless source_dir is given (tests pass a local fixture for an offline run). Empty tokens are dropped, lemmas cleaned (#N homograph suffix removed), and POS mapped to pyaegean's tagset convention. Every token is flagged seen=False — PROIEL is wholly outside pyaegean's training.

proiel_convention_report

proiel_convention_report(*, split: str = 'test', source: Path | str | None = None, batch_size: int | None = 32, progress: Callable[[int, int], None] | None = None, predictions: Sequence[Sequence[_ConvToken]] | None = None) -> ConventionReport

Decompose the PROIEL UD-fold UFeats and LAS gaps into annotation-convention divergence versus real disagreement, on the neural pipeline's own outputs.

Runs the active neural pipeline (aegean.greek.use_neural_pipeline — the model behind the published UD-PROIEL numbers) over the fold's gold tokens and compares its FEATS/HEAD/DEPREL to gold. Returns a ConventionReport whose ufeats/uas/las reproduce the official metrics from the model's outputs, split into: the UFeats gap's scheme-absent vs shared-disagreement parts (with a per-feature-type table), and the LAS gap's attachment-correct/label-wrong mass (with the gold→predicted relation confusions). This is a measurement DECOMPOSITION: it changes no published number and fits nothing to the fold.

source overrides the UD-PROIEL fold path (tests pass a local CoNLL-U fixture); batch_size batches the encoder passes (a throughput convenience — the diagnostic does not feed the recorded sequential protocol); progress is called progress(done, total) per sentence. predictions injects the system outputs directly (one (feats, head, deprel) per gold token, sentence-aligned) so the decomposition can be exercised without the model; with it, no pipeline is required.

proiel_dir

proiel_dir(*, download: bool = True, files: tuple[str, ...] = _GREEK_FILES) -> Path

The cache directory of PROIEL Greek XML files, fetching any missing on first use. The data is CC BY-NC-SA 3.0 — kept in the cache for evaluation only, never bundled.

proiel_drift

proiel_drift(tag_sentence: TagSentence | None = None, *, source_dir: Path | str | None = None, files: tuple[str, ...] = _GREEK_FILES, samples: int = 40) -> DriftReport

Quantify where the PROIEL gap comes from, so systematic annotation-convention divergence can be separated from scattered real error.

Re-tags the PROIEL gold with the same (reconciled) tagger evaluate_on_proiel uses and returns a DriftReport: the gold→predicted POS confusion matrix (most-frequent first), a sample of lemma mismatches, and the scored counts. A few confusion pairs carrying most of the POS errors (high top_share) suggests a convention difference rather than real error. tag_sentence and source_dir are as for evaluate_on_proiel.

This is now a thin PROIEL view of the shared aegean.greek.erroranalysis engine (which also serves UD-Perseus, the NT, and the AGDT held-out split, and carries richer per-POS / seen-unseen breakdowns): see erroranalysis.proiel_error_analysis for the full report.

agdt_ud_overlap

agdt_ud_overlap(*, splits: tuple[str, ...] = ('dev', 'test'), source: Path | str | None = None, agdt_source: Path | str | None = None, verify: bool = True, write: bool = True) -> dict[str, Any]

Build the AGDT ↔ UD-Perseus leakage-exclusion manifest.

UD Perseus sentence ids are <agdt-file>@<sentence-id> — direct references into the AGDT source pyaegean trains on. This collects every AGDT sentence appearing in the given UD splits (default: dev + test, the folds that must stay unseen), verifies the reference by comparing NFC form sequences against the actual AGDT files, caches the manifest as JSON, and returns it. Every Stage A+ training split must exclude these sentences — see docs/benchmarks.md.

source overrides the UD fold path(s) and agdt_source the AGDT directory (used by offline tests); with defaults, both fetch to the cache on first use.

bootstrap_ud

bootstrap_ud(treebank: str = 'perseus', split: str = 'test', *, metrics: Sequence[str] = ('upos', 'xpos', 'ufeats', 'lemma', 'uas', 'las'), n_resamples: int = 999, level: float = 0.95, seed: int = 0, source: Path | str | None = None, parse: bool | None = None) -> dict[str, BootstrapCI]

Percentile bootstrap CIs for :func:evaluate_on_ud's metrics, over the fold's sentences.

The active pipeline runs once over the fold; each of n_resamples draws re-scores a sentence resample (with replacement) with the official evaluator. Sentences are the resampling unit — tokens within a sentence are not independent. Activate the same backends you would for :func:evaluate_on_ud; with no parser active, uas/las are dropped. The band is sampling variability given this fold — read the module docstring's leakage caveat before quoting the Perseus fold for an AGDT-trained model.

dump_conllu

dump_conllu(sentences: Iterable[UDSentence] | UDDocument, *, canonical: bool = False) -> str

Serialize sentences, preserving unchanged parsed blocks or using canonical rows.

dumps_conllu

dumps_conllu(sentences: Iterable[UDSentence] | UDDocument, *, canonical: bool = False) -> str

Alias for :func:dump_conllu for string-oriented callers.

evaluate_by_genre

evaluate_by_genre(treebank: str = 'perseus', split: str = 'test', *, metrics: Sequence[str] = ('upos', 'lemma', 'uas', 'las'), bootstrap: bool = True, n_resamples: int = 999, level: float = 0.95, seed: int = 0, source: Path | str | None = None, parse: bool | None = None, min_sentences: int = 20, progress: Callable[[int, int], None] | None = None, batch_size: int | None = None) -> dict[str, dict[str, Any]]

Score the active pipeline on a UD fold, sliced by literary genre.

Each sentence is bucketed by its sent_id author (a TLG id, mapped through _AUTHOR_GENRE to epic / tragedy / comedy / prose / other). The pipeline runs once over the whole fold; each genre is then scored with the official evaluator (and, when bootstrap, given a percentile CI). Returns {genre: {"n_sentences", "n_words", "authors", "thin" (True undermin_sentences), <metric>: value or BootstrapCI}} plus an "_unmapped" list of author ids not in the table (the built-in discovery step: run this before pinning any numbers, and extend _AUTHOR_GENRE from it).

This is meaningful only for the leakage-clean neural model on Perseus: the offline baseline has seen the Perseus test sentences (see the module leakage caveat), so do not publish genre slices for it. uas/las are dropped when no parser is active. progress and batch_size thread through to pipeline_conllu; the recorded protocol stays the sequential default.

evaluate_on_ud

evaluate_on_ud(treebank: str = 'perseus', split: str = 'test', *, source: Path | str | None = None, parse: bool | None = None, progress: Callable[[int, int], None] | None = None, batch_size: int | None = None) -> dict[str, Any]

Score the active pipeline on a UD Ancient Greek fold with the official evaluator.

Runs over the fold's gold tokens (gold-tokenization protocol), emits CoNLL-U, and scores it against the gold file with conll18_ud_eval. Activate the backends you want measured first (use_treebank, use_tagger, use_lemmatizer, use_neural_lemmatizer, use_parser). parse defaults to whether the parser is active; with parse=False UAS/LAS are returned as None. progress (optional) is called as progress(done, total) per analyzed sentence. batch_size (optional) batches the neural pipeline's encoder passes (see pipeline_conllu) — a throughput convenience; the recorded protocol behind every published number is the sequential default.

Returns {"upos", "lemma", "uas", "las", "n_words", "n_sentences", "treebank", "split", "parsed"} — accuracies in [0, 1]. Read the module docstring's leakage caveat before quoting the Perseus fold for an AGDT-trained model.

load_conllu

load_conllu(source: Path | str, *, strict: bool = False) -> list[UDSentence]

Load CoNLL-U from a path or raw string.

The default is compatibility-lenient: valid words, ranges, empty nodes, comments, and all ten raw columns are retained, while malformed rows become opaque rows. strict=True turns malformed columns, IDs, structural ranges, DEPS, MISC, and references into line-aware ValueError exceptions.

load_conllu_document

load_conllu_document(source: Path | str, *, strict: bool = False, allow_partial_dependencies: bool = False) -> UDDocument

Load a complete CoNLL-U document wrapper, retaining raw source text.

strict behaves as in :func:load_conllu. allow_partial_dependencies applies only under strict=True, and accepts one documented shape a released treebank never has: a sentence in which only some words carry a basic HEAD. That is what an analysis covering part of a sentence projects into CoNLL-U, so a caller holding the complete typed state alongside the text (the interoperability envelope) validates everything else without rejecting its own projection.

loads_conllu

loads_conllu(text: str, *, strict: bool = False) -> list[UDSentence]

Load CoNLL-U directly from a string.

write_conllu

write_conllu(sentences: Iterable[UDSentence] | UDDocument, path: Path | str, *, canonical: bool = False) -> None

Write CoNLL-U text atomically to path.

evaluate_on_papygreek

evaluate_on_papygreek(*, layer: str = 'reg', source: Path | str | None = None, parse: bool | None = None, progress: Callable[[int, int], None] | None = None, batch_size: int | None = None) -> dict[str, Any]

Score the active pipeline on the PapyGreek documentary-Koine fold (official evaluator).

Reuses aegean.greek.ud's machinery wholesale — ud.load_conllu, ud.pipeline_conllu (gold tokenization, the neural encoder pass, optional batching), the fetched official conll18_ud_eval (ud._eval_module), and its scorer (ud._score_conllu_text) — so this fold is measured byte-for-byte the same way as UD-Perseus/PROIEL; only the gold data and the "treebank" label differ. Activate the backends you want measured first (use_neural_pipeline for the shipped model). parse defaults to whether a parser/joint model is active; with parse=False UAS/LAS/CLAS are None. progress is called as progress(done, total) per analyzed sentence; batch_size batches the neural encoder's passes (a throughput convenience — the recorded protocol is the sequential default). Complete-word overlapping windows cover any sentence beyond the model's single-pass subword budget; partial placeholder tails are never scored. source overrides the fold path (tests pass a local CoNLL-U).

layer selects which fold is fetched when source is not given: "reg" (the default, the editorially regularized reading behind the published PapyGreek numbers) or "orig" (the diplomatic-surface variant: the same sentences and gold, the raw documentary orthography as the FORM; see papygreek_orig_path). The orig fold measures the same model against a harder input and is directly comparable to the reg row.

Returns {"treebank", "split", "layer", "parsed", "upos", "xpos", "ufeats", "lemma", "uas", "las", "clas", "n_words", "n_sentences"} — accuracies in [0, 1]. The fold is work- and sentence-form-disjoint from its training data (see the module docstring).

evaluate_on_papygreek_dev

evaluate_on_papygreek_dev(track: str = 'tagging', *, source: Path | str | None = None, parse: bool | None = None, progress: Callable[[int, int], None] | None = None, batch_size: int | None = None) -> dict[str, Any]

Score the active pipeline on a PapyGreek documentary-Koine DEV track (official evaluator).

The dev fold is work-disjoint from model training and document-disjoint from the pinned papygreek test fold. It ranks levers and catches regressions without touching the test fold; it yields no published number and nothing is fitted against the test fold. Reuses the exact _score_fold machinery evaluate_on_papygreek uses; only the gold data (a dev track) differs.

track is "tagging": annotated surface tokens of the non-fold artificial/partial sentences, scored for UPOS/XPOS/UFeats/lemma (parse forced False; its trees are placeholders, UAS/LAS meaningless), or "parse": the reattached single-artificial-node sentences, scored for tagging and UAS/LAS (parse defaults to whether a parser/joint model is active; the track is thin, treat its parse numbers as directional). source overrides the track path (tests pass a local CoNLL-U for an offline run). progress and batch_size are as for evaluate_on_papygreek.

Returns the same key set as evaluate_on_papygreek except "layer" (the dev fold has no orig variant), with "split" set to the track name.

papygreek_convention_report

papygreek_convention_report(*, source: Path | str | None = None, batch_size: int | None = None, progress: Callable[[int, int], None] | None = None, predictions: Sequence[Sequence[_PapyToken]] | None = None) -> PapyGreekConventionReport

Decompose the PapyGreek UPOS and XPOS gaps into annotation/encoding convention versus real error, on the neural pipeline's own outputs.

Runs the active neural pipeline (aegean.greek.use_neural_pipeline — the model behind the published PapyGreek numbers) over the fold's gold tokens and compares its UPOS/XPOS to gold. Returns a PapyGreekConventionReport whose upos/xpos reproduce the official metrics from the model's outputs, split into the coordinator / common-gender / _-encoding convention parts and the residual real error. This is a measurement DECOMPOSITION: it changes no published number and fits nothing to the fold.

batch_size defaults to None (sequential, unlike proiel_convention_report): the published PapyGreek numbers are the sequential run and batch-32 is not prediction-identical on this fold, so a sequential pass is needed to reproduce them exactly. source overrides the fold path (tests pass a local CoNLL-U fixture); progress is called progress(done, total) per sentence. predictions injects the system outputs directly (one (upos, xpos) per gold token, sentence-aligned) so the decomposition can be exercised without the model; with it, no pipeline is required.

papygreek_dev_path

papygreek_dev_path(track: str = 'tagging', *, download: bool = True) -> Path

The cached CoNLL-U path of a PapyGreek DEV track, fetched + decompressed on first use.

track is "tagging" (UPOS/XPOS/UFeats/lemma over annotated surface tokens) or "parse" (UPOS/XPOS/UFeats/lemma and UAS/LAS over the reattached artificial-node sentences). The dev fold is work-disjoint from model training and document-disjoint from the pinned test fold. It is for experiment/lever ranking only, yields no published number, and is never fitted against the test fold. CC BY-SA 4.0, cached for evaluation only, never bundled.

papygreek_orig_path

papygreek_orig_path(*, download: bool = True) -> Path

The cached CoNLL-U path of the PapyGreek ORIG (diplomatic) test fold, fetched + decompressed on first use.

The diplomatic-surface variant of papygreek_path: the same 1,551 sentences and the same gold columns (UPOS/XPOS/UFeats/lemma/head/deprel), with the emitted FORM swapped to the raw documentary orthography (itacism, phonetic spelling, non-standard breathing) that the orig layer preserves. The two folds are token-aligned line-for-line and differ only in the surface form (1,453 token forms differ), so the orig row isolates the effect of the harder orthography. Built by scripts/build_papygreek_fold.py --layer orig. See _fetch_conllu for the fetch/decompress/stamp mechanics. CC BY-SA 4.0 — cached for evaluation only, never bundled.

papygreek_path

papygreek_path(*, download: bool = True) -> Path

The cached CoNLL-U path of the PapyGreek test fold, fetched + decompressed on first use.

The release asset is a gzipped CoNLL-U file (papygreek-fold). See _fetch_conllu for the fetch/decompress/stamp mechanics. CC BY-SA 4.0 — cached for evaluation only, never bundled.

dbbe_path

dbbe_path(*, download: bool = True) -> Path

The cached CoNLL-U path of the DBBE Byzantine book-epigram tagging fold, fetched + decompressed on first use.

The release asset is a gzipped CoNLL-U file (dbbe-lingann-fold). Fetched via the shared aegean.data.fetch_text (capped decompress, atomic write, and the .sha256 stamp sidecar that makes a re-pinned asset re-extract instead of serving a stale copy). expect_gzip=True so a non-gzip body (a corrupt or swapped download) refuses rather than materializing as the fold. CC BY 4.0 — cached for evaluation only, never bundled.

evaluate_on_dbbe

evaluate_on_dbbe(*, source: Path | str | None = None, progress: Callable[[int, int], None] | None = None, batch_size: int | None = None) -> dict[str, Any]

Score the active pipeline's tagging on the DBBE Byzantine book-epigram fold (official evaluator).

Reuses aegean.greek.papygreek._score_fold wholesale — which in turn reuses aegean.greek.ud's machinery (ud.load_conllu, ud.pipeline_conllu, the fetched official conll18_ud_eval and its scorer) — so this fold is measured byte-for-byte the same way as UD-Perseus/PROIEL and PapyGreek; only the gold data and the "treebank" label differ. Activate the backends you want measured first (use_neural_pipeline for the shipped model).

Tagging only. The DBBE gold standard carries no dependency trees, so parse is forced False: the result reports UPOS/XPOS/UFeats/lemma and UAS/LAS/CLAS are None. progress is called as progress(done, total) per analyzed sentence; batch_size batches the neural encoder's passes (a throughput convenience — the recorded protocol is the sequential default). source overrides the fold path (tests pass a local CoNLL-U).

The score reads low by construction (see the module docstring: Attic-lemma normalization, the mapped tagset, copular εἰμί without tree context), and it is a small, single-register datapoint — a Byzantine-verse row, never a headline number.

Returns {"treebank", "split", "parsed", "upos", "xpos", "ufeats", "lemma", "uas", "las", "clas", "n_words", "n_sentences"} — the tagging accuracies in [0, 1], the parse metrics None.

evaluate_on_verse

evaluate_on_verse(track: str | None = None, *, source: Path | str | None = None, parse: bool | None = None, progress: Callable[[int, int], None] | None = None, batch_size: int | None = None) -> dict[str, Any]

Score the active pipeline on the Ancient Greek verse fold (official evaluator).

Reuses aegean.greek.ud's machinery wholesale — ud.load_conllu, ud.pipeline_conllu (gold tokenization, the neural encoder pass, optional batching), the fetched official conll18_ud_eval (ud._eval_module), and its scorer (ud._score_conllu_text) — so this fold is measured byte-for-byte the same way as UD-Perseus/PROIEL and PapyGreek; only the gold data and the labels differ. Activate the backends you want measured first (use_neural_pipeline for the shipped model).

track selects "tragedy" (Euripides, Bacchae 1-169) or "all"/None for the whole fetched fold (no filter). The fold is tragedy-only, so the former "hexameter" filter value is rejected with guidance (see the module docstring). parse defaults to whether a parser/joint model is active; with parse=False UAS/LAS/CLAS are None. progress is called as progress(done, total) per analyzed sentence; batch_size batches the neural encoder's passes (a throughput convenience — the recorded protocol is the sequential default). source overrides the fold path (tests pass a local CoNLL-U).

Returns {"treebank", "track", "split", "parsed", "upos", "xpos", "ufeats", "lemma", "uas", "las", "clas", "n_words", "n_sentences"} — accuracies in [0, 1].

This is a SMALL-SAMPLE genre-conditioned datapoint with wide bootstrap confidence intervals (tens of sentences); the tragedy fold is a leakage-clean tragedy evaluation (no prior one is known to us). It is never a headline number — report it with the sample size and CI, not on its own.

verse_path

verse_path(*, download: bool = True) -> Path

The cached CoNLL-U path of the verse fold, fetched + decompressed on first use.

The release asset is a gzipped CoNLL-U file (verse-fold) holding both tracks (verse:tragedy:... + verse:hexameter:...). See _fetch_conllu for the fetch/ decompress/stamp mechanics. This fold is a small-sample genre-conditioned datapoint with wide bootstrap CIs, never a headline number — CC BY-SA 4.0, cached for evaluation only, never bundled.

betacode_to_unicode

betacode_to_unicode(text: str) -> str

Convert a Beta Code string to precomposed (NFC) polytonic Greek.

A backtick escapes the character after it (unicode_to_betacode uses this to protect literal reserved markup), so `( emits a literal ( rather than a smooth breathing.

strip_diacritics

strip_diacritics(text: str) -> str

Remove all combining diacritics (accents, breathings, subscripts), keeping the base letters. Returns NFC.

unicode_to_betacode

unicode_to_betacode(text: str) -> str

Convert polytonic Greek to Beta Code (capitals as *; final sigma as s). Round-trips with betacode_to_unicode: any literal ASCII that Beta Code reserves as markup (( ) / \ = + | * and the s1/s2/s3 sigma digits, plus the backtick escape itself) is backtick-escaped, so Greek text with embedded parentheses, arithmetic, or other punctuation survives the trip unchanged. (ASCII letters are Beta Code's own alphabet, so plain Latin words are read back as Greek; this maps Greek, not mixed prose.)

Lunate sigma (ϲ U+03F2 / Ϲ U+03F9) is a display variant of sigma and is normalized to a standard sigma (s) here, so it converts cleanly but does not round-trip back to the lunate glyph. The combining length marks (macron U+0304, breve U+0306, the lexicon vowel-quantity notation) have no Beta Code representation: text carrying them does not round-trip and, next to an accent, can perturb the accent's placement, so keep them out of Beta Code round-trips (the analysis functions, syllabify/scansion, handle them directly).

gloss_nt

gloss_nt(word: str) -> str | None

Brief Koine gloss for a word (lemmatized on a miss); requires use_dodson.

gloss_strongs

gloss_strongs(strongs: str | int) -> str | None

Brief Koine gloss for a Strong's number (e.g. 3056 -> 'a word, speech, …'); requires use_dodson. The NT corpus tokens carry these numbers in Token.annotations['strongs'].

lookup_nt

lookup_nt(word: str) -> DodsonEntry | None

Full Dodson entry for a word; requires use_dodson. None if unknown.

use_dodson

use_dodson(*, force: bool = False) -> DodsonLexicon

Activate Dodson Koine glossing for this session (loads the bundled lexicon).

No download — the lexicon is bundled (CC0). gloss_nt / lookup_nt / gloss_strongs resolve against it afterwards.

active_lexica

active_lexica() -> list[str]

Ids of the lexica currently active.

disable_lexicon

disable_lexicon(dictionary: str) -> None

Deactivate a lexicon.

entry

entry(word: str, *, dictionary: str | None = None) -> LexEntry | None

The full LexEntry for word from dictionary (or the first active lexicon that has it).

gloss

gloss(word: str, *, dictionary: str | None = None) -> str | None

A concise gloss for word from dictionary (or the first active lexicon that has it). With no active lexicon, raises LexiconNotLoadedError.

lexica

lexica() -> list[LexiconInfo]

Every registered lexicon's metadata (hosted and deep-link), id-sorted.

lexicon_link

lexicon_link(word: str, *, service: str = 'logeion', lemmatize: bool = True) -> str

A deep-link to word in an online dictionary aggregator (Logeion by default).

Covers the lexica pyaegean does not host (Autenrieth, Slater, Montanari, DGE, Bailly, ...): Logeion aggregates them. lemmatize (default) links the lemma.

use_lexicon

use_lexicon(dictionary: str, **kwargs: object) -> Lexicon

Activate a hosted lexicon by id, fetching/building its index on first use.

evaluate_on_nt

evaluate_on_nt(tag_sentence: TagSentence | None = None, *, corpus: Corpus | None = None, book: str | None = None, progress: Callable[[int, int], None] | None = None, batch_size: int | None = None) -> dict[str, float]

Score a tagger on the Nestle1904 Greek NT gold — lemma + reconciled UPOS accuracy.

tag_sentence maps a sentence's forms to (lemma, upos) per token; it defaults to the neural joint pipeline (enable greek.use_neural_pipeline() first), so the number reflects the shipped model. corpus supplies the gold (defaults to greek.load_nt(book) — the whole NT, or one book). progress (optional) is called as progress(done, total) per scored verse — the whole-NT run is ~1 h on plain CPU, so this is how the CLI shows it moving. batch_size (optional) runs the default neural tagger's encoder over that many verses at a time (one ONNX call per chunk) — a throughput convenience; the recorded protocol (the published numbers) is the sequential default, and with a caller-supplied tag_sentence the value has no effect. Returns {"lemma", "upos", "n"}: accuracy over the scored tokens. Lemma is the clean metric; UPOS is compared under a reconciled tagset, mirroring evaluate_on_proiel.

pipeline_tokens

pipeline_tokens(tokens: Sequence[Token], *, parse: bool = False, with_confidence: bool = False, confidence_domain: str | None = None, confidence_policy: AbstentionPolicy | None = None, long_input: Literal['strict', 'partial', 'windowed'] = 'strict', document_id: str = 'input', sentence_policy: str = 'default', segmenter: SegmenterLike | None = None) -> list[TokenRecord]

Analyze already-tokenized core Token values without re-tokenizing them.

A token's typed form_state chooses the analyzer input in the deterministic order model input, regularized, normalized, diplomatic, then legacy Token.text. The returned record retains the token's alignment and form state; one input token always produces one output record. Complete, contiguous alignment sentence_id runs take precedence over sentence_policy and segmenter. Without them, the named policy or external segmenter groups the atomic typed tokens.

Editions frequently attach the sentence mark to the word it follows rather than tokenizing it separately, so the named policy reads a sentence-final mark carried inside a word token as well as a standalone punctuation token. The policy still decides which marks count: inscription and papyrus commit only to the strong period/!/? marks, and abbreviation dots, dotted citations and numbers, and editorially uncertain tokens never end a sentence. When a policy commits to no mark across a very long run of tokens, that run stays one sentence and the call warns rather than returning it silently.

catalog

catalog(query: str | None = None, *, author: str | None = None, title: str | None = None, source: str | None = None) -> list[dict[str, str]]

Search the full bundled index of Greek works loadable with :func:load_work.

Unlike :func:popular_works (25 curated highlights), this covers every work with a Greek (-grc) edition in Perseus canonical-greekLit + First1KGreek — ~1,800 works. Each entry is {'id', 'author', 'title', 'greek_title', 'source'}; pass any id straight to load_work. Pure bundled metadata — no network, no download.

All filters are case-insensitive substring matches and combine with AND:

• query — matches across id, author, English title, and Greek title (the catch-all)
• author — e.g. "plato"
• title — matches the English or Greek title
• source — "perseus" or "first1k"

Returns a list of dicts; pure bundled metadata, so it works offline and is instant.

fetch_works

fetch_works(author: str | None = None, *, works: list[dict[str, str]] | None = None, source: str | None = None, force: bool = False, limit: int | None = None, on_progress: Callable[[int, int, dict[str, str]], None] | None = None, abort: Callable[[], bool] | None = None) -> Iterator[WorkFetchResult]

Fetch every catalogue work matching author into the cache, yielding a :class:WorkFetchResult per work as it completes.

Shared by the CLI (greek work all) and the TUI works screen. works overrides the catalogue query (pass a pre-filtered list). Already-cached works are yielded "cached" with no network, so re-running resumes idempotently. on_progress(i, total, work) fires BEFORE each work (a UI "downloading…" cue). limit caps NEW downloads (cached works do not count). abort() is polled between works.

Terminal conditions raise, so the caller learns why the batch stopped: :class:GitHubRateLimitError (API exhausted), :class:aegean.data.FetchAborted (aborted), or :class:aegean.data.DataNotAvailableError (stopped after too many consecutive failures).

list_fetched_works

list_fetched_works() -> list[dict[str, Any]]

Which Greek works are already downloaded to the cache — a pure local scan, no network.

Walks cache_dir()/greek-works/<source>/<commit>/*.xml, recovers each CTS id from the edition filename (tlg0012.tlg001.perseus-grc2.xml -> tlg0012.tlg001), and joins the bundled catalogue for author/title. Returns [{'id','author','title','source','path','bytes'}] sorted by id (one entry per work, even if present under several sources/commits); [] when nothing is cached. Ignores the listings/ cache and any .part/.lock files.

citation_scheme

citation_scheme(work: str, *, source: str = 'auto', edition: str | None = None, force: bool = False) -> list[str]

How a Greek work is addressed: its ordered citation levels, from the TEI edition.

Reads the work's declared CTS <refsDecl> and returns the citation levels shallow→deep, exactly as the edition names them (no author-specific guessing): the Iliad → ["book", "line"], a Plato dialogue → ["section"], Xenophon's Anabasis → ["book", "chapter", "section"], Aristotle's Poetics → ["chapter", "subchapter"]. So ["book", "line"] means a --ref looks like 1 (a whole book) or 1.1 / 1.1-1.50 (a line or line-range within a book); a single-level ["section"] means --ref 17 (one section).

Returns [] when the work declares no CTS refsDecl. Like load_work, the TEI file is fetched once to the cache (source/edition/force as there); this is metadata about the edition, not its text. It reports the CTS <div> levels the edition declares. Finer references some editions print in the margin (a Stephanus sub-page 17a, a Bekker line 1447a10) live in <milestone> markers outside the CTS scheme, so they are not part of the returned levels — but load_work's ref does resolve them directly (it extracts the span between the marker and the next).

load_work

load_work(work: str, *, ref: str | None = None, source: str = 'auto', edition: str | None = None, force: bool = False) -> 'Corpus'

Load one Greek work from Perseus canonical-greekLit / First1KGreek.

work is the CTS-style id ("tlg0012.tlg001" = the Iliad). source is "perseus", "first1k", or "auto" (try both, in that order); edition picks a specific edition file when a work has several. The TEI file is fetched once into the cache (network on first use only).

ref selects a sub-section instead of the whole work — a citation address matching the work's structure: a textpart number ("1" = Iliad book 1), a nested div path ("1.2" = book 1, chapter 2 of a prose work), a verse line-range ("1.1-1.50" = book 1, lines 1–50), a marginal <milestone> marker outside the CTS <div> scheme (a Stephanus sub-page "17a", a Bekker line "1447a10" — the span to the next marked line — or a whole Bekker page-column "1447a", the physical page being the comma list "1447a,1447b"), or a comma list of any of these ("1.1,1.5", "1,3", "17a,17b") giving one Document per entry. A hyphen range must stay within a single textpart: "1.1-2.50" (crossing from book 1 into book 2) raises ValueError; use a comma list, or load each book separately and Corpus.merge the results. Without ref, the corpus is one Document per top-level textpart. The corpus provenance's citation is the canonical scholarly citation of exactly what was selected ("Homer, Iliad 1.1-1.50"; see canonical_citation), so corpus.cite() echoes the selection. <note>/<bibl> ride along in Document.meta.notes. Raises aegean.data.DataNotAvailableError when the work can't be found/fetched, or ValueError when ref matches nothing — that message names the work's declared citation scheme (cited by book.line); citation_scheme returns it directly.

popular_works

popular_works() -> list[dict[str, str]]

A curated, verified catalog of well-known Greek works loadable with :func:load_work.

Each entry is {'id', 'author', 'title'} where id is the CTS id passed to load_work (e.g. 'tlg0012.tlg001' → the Iliad). This is a deliberately small starting point — for the full reachable canon use :func:catalog, or browse the Scaife Viewer (https://scaife.perseus.org). Pure metadata — no download.

remove_fetched_works

remove_fetched_works(ids: list[str] | None = None, *, author: str | None = None, remove_all: bool = False) -> list[str]

Delete downloaded Greek works from the cache, returning the ids actually removed (sorted).

Select the targets one of three ways: explicit ids; every fetched work by an author (case-insensitive substring of the catalogue author, the same match as greek catalog --author); or remove_all. A no-op returning [] when nothing matches or nothing is cached. Removes every cached edition file for each targeted work (across sources/commits) and prunes the now-empty source/commit directories. Never touches the listings/ cache.

load_nt

load_nt(book: str | None = None, *, ref: str | None = None, force: bool = False) -> Any

Load the Greek New Testament (Nestle 1904) as an annotated Corpus.

book selects one book by name or abbreviation ('John', 'Jn', '1Cor', 'Rev'); None returns the whole NT. ref selects within a book, mirroring load_work: '3' a chapter, '3.16' a verse, '3.16-3.18' / '3.16-18' a verse range, '3-5' a chapter range. One Document per chapter; every token carries a gold lemma, Robinson morph, Strong's number, reconciled UD upos, and the normalized form in Token.annotations. Token text, lemmas, and normalized forms are NFC-normalized at load time (the source edition mixes oxia and tonos precomposition), so gold strings compare byte-for-byte with the library's NFC output.

The full 27-book corpus is fetched to cache on first use (sha256-pinned CC0 asset, or PYAEGEAN_NT_CORPUS_URL). When that asset is unavailable the bundled sample (John 1 + Philemon) is used as an offline fallback (its provenance says so).

nt_books

nt_books() -> list[dict[str, Any]]

The 27 books of the Greek New Testament, in canonical order.

Each entry is {'name': 'John', 'aliases': ['john', 'jn', 'jhn']}. Any of the name or its aliases is accepted as the book argument to :func:load_nt; the name is what appears in document ids (e.g. 'John 1'). Pure metadata — no download — so it works offline and is the answer to "which books can I load?".

scan

scan(word: str) -> list[tuple[str, str]]

(syllable, quantity) pairs for a word.

syllable_quantities

syllable_quantities(word: str) -> list[str]

The metrical quantity of each syllable: "heavy" / "light" / "common" (in syllable order).

scan_aeolic

scan_aeolic(line: str, line_type: str = 'glyconic') -> LineScansion

Scan an aeolic lyric line against a fixed quantity template.

line_type is one of AEOLIC_LINES — "glyconic", "pherecratean", "sapphic_hendecasyllable", "adonean", "alcaic_hendecasyllable", "alcaic_enneasyllable", "alcaic_decasyllable". These are fixed patterns (the choriamb does not resolve), so the line either matches or it doesn't — ScansionError is raised on a mismatch (e.g. the wrong syllable count, or a line needing synizesis on a word not in the lexicon).

scan_hexameter

scan_hexameter(line: str) -> LineScansion

Scan a line of dactylic hexameter (six feet; feet 1–5 dactyl or spondee, foot 6 — ×), resolving quantities and the main caesura.

Raises ScansionError if the line does not fit (e.g. it needs synizesis, which is not inferred).

scan_line

scan_line(line: str, meter: str = 'hexameter') -> LineScansion

Scan line against meter ("hexameter" or "pentameter").

scan_pentameter

scan_pentameter(line: str) -> LineScansion

Scan a line of elegiac pentameter: two dactyls-or-spondees, a longum, the central diaeresis, then two obligatory dactyls and a final longum (— ⏑⏑ — ⏑⏑ — ‖ — ⏑⏑ — ⏑⏑ —).

Raises ScansionError if the line does not fit.

scan_trimeter

scan_trimeter(line: str) -> LineScansion

Scan a line of iambic trimeter — three metra of x – ⏑ – (the final element anceps), with resolution of long elements into two shorts.

Raises ScansionError if the line does not fit (e.g. it needs synizesis on a word not in the lexicon).

syllable_options

syllable_options(line: str) -> list[tuple[str, list[str]]]

(syllable, [possible quantities]) across the whole line — the raw, pre-metrical analysis, with cross-word position and correptio applied.

to_ipa

to_ipa(text: str, period: Period = 'attic') -> str

Transcribe Greek text to reconstructed IPA. Whitespace-separated words are transcribed independently and rejoined with spaces.

pos_tag

pos_tag(word: str) -> str

Tag a single token. Closed classes come from the lexicon; when the treebank backend is active (see aegean.greek.use_treebank), an attested form's gold tag is used next; otherwise open-class words get a suffix heuristic (a few verb endings, else NOUN). Non-letter tokens are NUM (numeric) or PUNCT.

pos_tags

pos_tags(text: str) -> list[tuple[str, str]]

(token, tag) pairs for a text, in order (punctuation tagged PUNCT). When the trained tagger is active it tags the whole sentence in context, with the closed-class lexicon and the treebank lookup still taking precedence per token.

segment_sentences

segment_sentences(text: str, *, policy: str = 'default', segmenter: SegmenterLike | None = None) -> SegmentationResult

Compatibility spelling for :func:segment_text.

segment_text

segment_text(text: str, *, policy: str = 'default', segmenter: SegmenterLike | None = None) -> SegmentationResult

Segment text with a built-in policy or caller-supplied plugin.

validate_segmentation_result

validate_segmentation_result(text: str, value: SegmentationResult | Sequence[Any] | Mapping[str, Any], *, policy: str = 'default', plugin_policy_id: str = PLUGIN_POLICY_ID) -> SegmentationResult

Normalize and adversarially validate a plugin's output before use.

sentences

sentences(text: str, *, sentence_policy: str = 'default', segmenter: SegmenterLike | None = None) -> list[str]

Project rich, policy-aware segmentation to the legacy list-of-strings shape.

tokenize_aligned

tokenize_aligned(text: str, *, document_id: str = 'input', sentence_policy: str = 'default', segmenter: SegmenterLike | None = None) -> list[Token]

Tokenize with lossless alignment under one named sentence policy.

tokenize_words

tokenize_words(text: str) -> list[str]

Just the word strings, in order (punctuation dropped).

Neural runtime variants

neural_variants() returns the four stable runtime labels and their exact artifact/evidence identities; pass available_only=True to hide reservations. neural_variant(label) inspects one record without fetching a model. default remains the current grc-joint-v3 artifact and makes no speed or size claim. fast, compact, and balanced are reserved until a qualified successor artifact earns the corresponding operational label.

Select explicitly with use_neural_pipeline(variant=...) or GreekPipeline.neural(variant=...). An unavailable label fails before dependency probing, network access, or cache mutation and never falls back. GreekPipelineConfig schema 2 and AnalysisReceipt schema 4 bind the selected label and registry digest; non-default available artifacts also bind their qualification and runtime-variant award digests. Different artifact variants use different dataset keys so their cached files coexist.

Streaming neural sentence analysis

iter_analyze_sentences(sentences, batch_size=None) is the bounded-memory API for an iterable of already tokenized Greek sentences. It captures the active neural backend, opt-in documentary state, and (when requested) confidence calibration when called, pulls and yields one sentence at a time by default, preserves source order and each result's AnalysisReceipt, and applies synchronous backpressure. A positive batch_size holds at most one transactional chunk; a failed chunk yields nothing, while earlier yielded results remain valid.

analyze_sentences(...) is the compatibility collector over that engine and still returns a complete list. GreekPipeline.iter_analyze_sentences(...) provides the same iterator against an isolated instance. These APIs do not make raw-text pipeline(), corpus annotation, or CoNLL-U serialization streaming. Batching is a throughput option; the published benchmark protocol remains sequential CPU inference.

Sentence-policy registries

The facade also exports two immutable mappings that document the supported rule contracts:

• POLICY_IDS maps each named policy to its stable identity. It also contains the explicit identity used when complete source sentence_id runs supply boundaries.
• POLICY_RULES maps the five inferred policies (default, prose, verse, inscription, and papyrus) to their plain-language rule descriptions.

Use these mappings for inspection and provenance; do not modify or interpret a rule identity as a measured confidence value.

Typed confidence and abstention

pipeline()/pipeline_tokens() and GreekPipeline.analyze() accept the additive confidence_domain= and confidence_policy= arguments when confidence is requested. Their TokenRecord results carry TokenConfidence/SentenceConfidence values with task, source-path, domain, calibration, and explicit unavailable-reason fields. AbstentionPolicy thresholds are caller-supplied and hashed into each decision; no default threshold or automatic OOD claim is provided. For the shipped canonical runtime, schema-4 AnalysisReceipt output records calibration/policy hashes together with the composed output and post-processing identity. Schemas 1 through 3 remain readable where current hosted evidence uses them, without fabricating runtime-label provenance.

For development data, fit_temperature(logits, correct) fits a top-1 temperature and fit_logit_affine(probs, correct) fits monotone (slope, intercept) parameters for a logit-affine CalibrationEntry. Both are parameter-fitting helpers only: they do not select domains, thresholds, or release evidence.

Annotation, domain, and output profiles

The Greek facade exposes an immutable registry for annotation conventions and descriptive domain scope:

from aegean import greek

greek.list_annotation_profiles()            # tuple[AnnotationProfile, ...]
greek.list_domain_profiles()                # tuple[DomainProfile, ...]
greek.get_annotation_profile("pyaegean-canonical-v1")
greek.get_domain_profile("...")
greek.canonical_analysis_profile()          # shipped inference/output convention

The typed values are AnnotationProfile, DomainProfile, LabelMapping, AmbiguityDisclosure, ProfileEvidence, PostprocessingStep, and AnalysisProfile. Each profile has a stable profile_id, canonical compact to_json() form, and SHA-256 sha256 identity. Registry values are read-only; these APIs inspect declared conventions and never infer a domain or alter model predictions.

This registry is distinct from TextProfile/profile_text, which describe observable input characters, and from the caller-supplied confidence domain, which scopes calibration evidence. The canonical profile is the supported inference/output convention. PROIEL and Perseus/AGDT differences are documented diagnostics rather than a general source-compatible conversion mode. The UD-PROIEL profile records POS, feature, token-row, and dependency differences that include non-invertible losses. Separately, evaluate_on_proiel()'s native-XML projection strips #N homograph suffixes, omits empty tokens, and receives presentation punctuation outside token rows; exact UD-fold scoring does not use that cleanup. PapyGreek's orig convention changes the diplomatic FORM surface while retaining the regularized-layer gold analyses and its documented fallbacks.

When a composed output profile or documentary post-processing chain is attached, receipt schema 4 binds its output/composed profile ID and SHA-256 plus the ordered post-processing identity and runtime-variant evidence. Receipt schemas 1 through 3 remain readable for current hosted evidence, and the grc-joint-v3 model identity is unchanged.