aegean.io¶
io ¶
I/O adapters — move the corpus model to and from interchange formats.
Import your own material with from_text / from_text_file / from_text_dir /
from_csv (plain text, a folder of texts, or a CSV → a Corpus), or from_epidoc
(pyaegean output or token-carrier EpiDoc TEI → a Corpus). Export as tabular
CSV/Parquet, semantic EpiDoc, RDF Turtle/JSON-LD, SQLite, review CSV, or the
intentionally lossy Workbench surface format. The
Linear B-specific EpiDoc reader (DAMOS-style files, text-derived Aegean token kinds) lives
in aegean.scripts.linearb and Corpus.load("linearb"). For pyaegean's own lossless
archive format, use Corpus.to_json / Corpus.from_json; use JSON or SQLite
when every corpus field must survive. Loss-aware NLP adapters for CoNLL-U, spaCy,
Stanza, and CLTK use InteropDocument plus an integrity-bound JSON sidecar so
unsupported target fields are disclosed rather than silently discarded.
InteropBundle
dataclass
¶
InteropBundle(target: str, target_version: str | None, native: Mapping[str, Any], sidecar: str, report: InteropReport, report_sha256: str | None = None, schema: str = BUNDLE_SCHEMA)
One validated, portable adapter bundle.
native is the documented JSON projection of the target object, not a
promise that pyaegean can reconstruct the target library's private object.
document recovers the complete pyaegean envelope from the bound sidecar.
document
property
¶
document: InteropDocument
Decode the complete pyaegean document after revalidating the binding.
InteropDependencyError ¶
Bases: InteropError
An optional target dependency is not installed.
InteropDocument
dataclass
¶
InteropDocument(ud_document: UDDocument, source_text: str | None = None, document_id: str | None = None, token_metadata: Mapping[tuple[str, int], InteropTokenMetadata] = dict(), sentence_metadata: Mapping[str, InteropSentenceMetadata] = dict(), annotation_profile: str | None = None, provenance: Provenance | None = None)
has_richer_metadata ¶
Whether this envelope carries anything the CoNLL-U columns cannot hold.
True when the document has source text, a document id, token or sentence
metadata, an annotation profile, or provenance. to_conllu uses it to decide
whether a sidecar is needed at all.
InteropError ¶
Bases: Exception
Base class for interoperability failures.
InteropLossError ¶
Bases: InteropError
A strict conversion would lose information.
InteropResult
dataclass
¶
Bases: Generic[T]
InteropSchemaError ¶
Bases: InteropError
Malformed, unsupported, or tampered interchange data.
MergedReview
dataclass
¶
MergedReview(rows: tuple[dict[str, str], ...], conflicts: tuple[ReviewConflict, ...], reviewers: tuple[str, ...], source_paths: tuple[str, ...] = tuple())
The result of merging several reviewers' corrected copies of one export.
rows is the clean, agreed subset as review-table rows (ready to write or to apply with
apply_merged); conflicts are the disagreements held back for a human; reviewers is
every reviewer whose corrections are in rows; source_paths records the merged
tables. Use to_csv to write the agreed subset back out as a review table.
to_csv ¶
Write the agreed (clean) subset as a review CSV (UTF-8 BOM, formula-guarded).
ReviewConflict
dataclass
¶
ReviewConflict(doc_id: str, position: int, token: str, field: str, options: tuple[ReviewerValue, ...])
One field of one token where reviewers proposed different corrections.
field is the human field name (lemma / pos / morph); options lists each
reviewer's proposed value (and note). A conflict is never resolved silently — it is surfaced
for a human to settle.
ReviewerValue
dataclass
¶
One reviewer's proposed value for a field (with their note, if any).
bundle_from_document ¶
bundle_from_document(document: InteropDocument, *, target: str) -> InteropBundle
Run one lazy adapter and return its validated portable bundle.
bundle_from_result ¶
bundle_from_result(result: InteropResult[Any]) -> InteropBundle
Create a portable bundle from a lossless adapter result.
dumps_interop_bundle ¶
dumps_interop_bundle(bundle: InteropBundle) -> str
Serialize a bundle as deterministic UTF-8 JSON text.
loads_interop_bundle ¶
loads_interop_bundle(value: str) -> InteropBundle
Parse and validate deterministic bundle JSON text.
read_interop_bundle ¶
read_interop_bundle(path: str | Path) -> InteropBundle
Read a bounded UTF-8 bundle from one local path.
write_interop_bundle ¶
write_interop_bundle(bundle: InteropBundle, path: str | Path) -> Path
Atomically write one validated bundle and return its path.
from_cltk ¶
from_cltk(doc: Any, *, sidecar: str | None = None, allow_lossy: bool = False) -> InteropResult[InteropDocument]
Import a real CLTK Doc and validate its complete native state.
make_cltk_process ¶
Create a network-free CLTK process around one explicitly owned pipeline.
to_cltk ¶
to_cltk(document: InteropDocument, *, allow_lossy: bool = False) -> InteropResult[Any]
Project one immutable canonical document into a real CLTK Doc.
from_spacy ¶
from_spacy(doc: Any, *, sidecar: str | None = None, allow_lossy: bool = False) -> InteropResult[InteropDocument]
Import a real spaCy Doc and validate its complete native state.
to_spacy ¶
to_spacy(document: InteropDocument, *, vocab: Any = None, allow_lossy: bool = False) -> InteropResult[Any]
Project one immutable canonical document into a real spaCy Doc.
from_stanza ¶
from_stanza(doc: Any, *, sidecar: str | None = None, allow_lossy: bool = False) -> InteropResult[InteropDocument]
Import a real Stanza document and validate its complete native state.
to_stanza ¶
to_stanza(document: InteropDocument, *, allow_lossy: bool = False) -> InteropResult[Any]
Project one immutable canonical document into a real Stanza Document.
from_epidoc ¶
from_epidoc(source: str | Path, *, script_id: str = 'greek') -> Corpus
Load token-carrier EpiDoc TEI into a Corpus.
The inverse of write_epidoc: round-trips the id, find-place, token/line stream,
token kinds, editorial certainty, alternate readings, and typed form state. Other input
must carry tokens in <w>/<num>/<g>/<pc>/<seg> elements; free-text
editions need a source-specific extractor. script_id labels the corpus (default
"greek"). pyaegean parses your files locally and never re-hosts them.
read_epidoc ¶
read_epidoc(source: str | Path, *, script_id: str = 'greek') -> list[Document]
Parse token-carrier EpiDoc TEI into Documents.
A file, or every *.xml file in a directory, must represent tokens with
<w>, <num>, <g>, <pc>, or <seg> carriers. This is the inverse of
:func:write_epidoc; arbitrary free-text TEI needs a source-specific extractor.
The carrier gives each token its aegean.TokenKind, and a <seg> takes its kind from
@type (separator, unknown, logogram, ...). An untyped <seg> reads as a
numeral when its text is all digits and a word otherwise.
script_id labels the result: EpiDoc's xml:lang can't disambiguate (say) Linear A
from Cypro-Minoan, so the caller names the script. Uses the stdlib XML parser only.
Raises FileNotFoundError if source does not exist (or a directory holds no
*.xml files), and ValueError if nothing in it is EpiDoc (no <div type="edition">
or <body> in the TEI namespace) — rather than silently returning an empty list. A
malformed file inside a directory raises an xml.etree.ElementTree.ParseError whose
message names the offending file, so a single bad inscription in a large corpus folder
is identifiable (a directory has no line/column of its own).
to_epidoc ¶
to_epidoc(document: Document) -> str
Serialize a single Document to an EpiDoc TEI XML string.
The transliteration lives in a TEI <div type="edition"> (EpiDoc's required edition
division), as <lb/>-delimited lines of tokens. Each token's aegean.TokenKind picks
its carrier: <w> word, <num> numeral, <g> logogram, <pc> punct. A separator,
an unknown sign, and a logogram that must hold apparatus markup (TEI's <g> cannot) go in
<seg> with the kind in @type, e.g. <seg type="separator">. A token whose
aegean.ReadingStatus is not CERTAIN is wrapped in the matching EpiDoc apparatus
element (<unclear> or <supplied>), so editorial certainty survives the round trip
through aegean.scripts.linearb.parse_epidoc. The output validates against the EpiDoc
RelaxNG schema (see tests/test_io.py).
Token text round-trips subject to standard XML text normalization: a carriage return becomes a line feed, leading/trailing whitespace on a token is trimmed, and a token (or alternate reading) whose text is only whitespace does not survive the parse. Real transliteration tokens are never whitespace-only, so this affects only synthetic input; the transliteration content itself is preserved.
Token.annotations (lemma, morphology, evidence class, review stamps) are NOT
serialized to EpiDoc: the format carries edition text and apparatus, not an
analysis layer. A typed state projects its diplomatic form, one selected editorial
form (regularized before normalized), and apparatus segments. model_input, its
operations, and a second normalized form are not edition markup and are not serialized.
Use Corpus.to_json, aegean.db.to_sqlite, or CoNLL-U for the full typed record.
write_epidoc ¶
Write EpiDoc TEI XML to disk.
A single Document is written to the file path; a
Corpus is written as one {id}.xml file per document into the
directory path (created if needed) — the layout
aegean.scripts.linearb.parse_epidoc reads back. Ids are sanitized for
the filesystem (anything outside [A-Za-z0-9-_.] becomes _), which
can conflate distinct ids: when two ids sanitize to the same filename, the
later ones (in id order) get a -2, -3, ... suffix and a warning
names the colliding ids, so no document silently overwrites another.
Token.annotations are not serialized (see to_epidoc).
decode_sidecar ¶
decode_sidecar(sidecar: str, *, target: str | None = None, native_signature: str | None = None) -> dict[str, Any]
Validate a sidecar string and return its envelope mapping.
Everything is checked before the value comes back: the size bound, strict JSON
(a duplicate key or a non-finite number is rejected, not silently resolved), the
schema and a non-empty target, the document and payload digests, and a typed
decode of the payload so a malformed record fails here rather than downstream. The
returned mapping is the envelope, with the document under payload.
target and native_signature, when given, are additional bindings: the
sidecar must name that target and must have been written for that exact native
projection. Any mismatch raises InteropSchemaError, which is also what a
tampered payload produces.
encode_sidecar ¶
encode_sidecar(document: InteropDocument, *, target: str, native_signature: str) -> str
Serialize an envelope as the one-line JSON sidecar payload.
Returns canonical JSON with six keys: schema, target, the document,
payload, and native SHA-256 digests, and payload. The digests bind three
things at once: the canonical CoNLL-U projection, the canonical payload JSON, and
the caller's native_signature for the exact projection this sidecar travels
with, so decode_sidecar can refuse a payload that has drifted from its document.
target names the format the sidecar accompanies ("conllu" for
to_conllu, otherwise an adapter's name). The result carries no literal U+2028,
U+2029, or U+0085, so it survives as a single line through a consumer that splits
on Unicode line boundaries; the escapes are JSON, so decoding restores the identical
payload and every digest still verifies. A sidecar larger than
MAX_SIDECAR_BYTES (8 MiB) raises InteropSchemaError rather than being emitted,
because the reader enforces the same bound.
from_conllu ¶
from_conllu(source: str | Path, *, strict: bool = True) -> InteropResult[InteropDocument]
Read CoNLL-U text or a CoNLL-U file into an :class:InteropDocument.
strict=True (the default) validates the CoNLL-U itself: malformed columns and
IDs, structural ranges, DEPS, MISC, and a missing final blank line are rejected with
the offending line number. An interop sidecar comment carries envelope metadata and
does not switch that validation off; it explains exactly one shape, a sentence whose
words do not all carry a HEAD, which is how a partly analyzed document projects into
CoNLL-U. strict=False keeps the lenient reading that preserves unparsable rows
for inspection and re-export.
# sent_id is optional in CoNLL-U; a sentence written without one is given the
stable positional identifier input:sentence:N, since the envelope keys its token
and sentence metadata by sentence ID. A leading UTF-8 byte-order mark is an encoding
marker and is discarded rather than being read as document content.
from_token_records ¶
from_token_records(records: Iterable[TokenRecord], *, source_text: str, document_id: str, provenance: Provenance | None = None, annotation_profile: str | None = None) -> InteropDocument
Build an :class:InteropDocument from the TokenRecord stream pipeline returns.
source_text is the exact analyzed text and document_id its stable identity;
both are required, because every record must carry a SourceAlignment that
validates against them. Sentences are grouped by each record's sentence ordinal and
named by its alignment's sentence id, and the record's own analysis fields become
one UD word each.
The input is validated rather than repaired. Records must arrive in sentence and
token order, sentence ordinals must run contiguously from zero, word indices from
one within each sentence, source token ids must be unique, and a record's text must
equal its aligned normalized_text; a violation raises InteropSchemaError. An
empty stream yields an empty document.
CoNLL-U has no way to write "no value", so a missing head, relation, XPOS, or FEATS
is projected to UD's placeholder in the row and the original None is kept in
:class:InteropTokenMetadata, which is what stops a round trip from fabricating a
root or a tag. Confidence, lemma provenance, form state, receipts, and sentence
boundaries travel the same way. annotation_profile is taken from the records'
analysis receipts when it is not given, and receipts that disagree raise.
from_ud_document ¶
from_ud_document(document: UDDocument, *, source_text: str | None = None, document_id: str | None = None, annotation_profile: str | None = None, provenance: Provenance | None = None) -> InteropDocument
Wrap an already-parsed UDDocument as an :class:InteropDocument.
Nothing is inferred beyond what the rows already carry: a word that holds a typed
form_state gets an :class:InteropTokenMetadata record, and every other token
gets none. The keyword arguments add the document-level context CoNLL-U has no
column for. A document wrapped with none of them has no richer metadata, so
to_conllu writes plain CoNLL-U; supplying source_text or document_id
(or a profile or provenance) is what makes a sidecar necessary.
Raises TypeError for anything that is not a UDDocument.
to_conllu ¶
to_conllu(document: InteropDocument, *, include_sidecar: bool = True, allow_lossy: bool = False) -> InteropResult[str]
Write an envelope as CoNLL-U text, with its metadata in a leading sidecar comment.
The result's value is the text and its report says which fields the CoNLL-U
columns carried natively and which the sidecar had to. When the document holds
nothing beyond those columns the output is plain CoNLL-U and sidecar is
None; otherwise the text opens with one # aegean.interop = line and the
native document follows, so an ordinary CoNLL-U reader sees a comment and
from_conllu sees the whole envelope.
include_sidecar=False on a document with richer metadata would silently drop it,
so it raises InteropLossError unless allow_lossy=True makes the loss explicit:
the text is then plain CoNLL-U and the report lists the dropped fields in
lost_fields with lossless False.
to_rdf ¶
to_rdf(corpus: Corpus, path: str | Path, *, fmt: str = 'turtle', base_uri: str | None = None) -> None
Write corpus to disk as Linked Open Data.
fmt is "turtle" (aliases "ttl") or "jsonld" (alias "json-ld"). Each
document becomes a subject with a stable URI minted from its authoritative identifiers
(a papyri.info DDbDP document URI, else Trismegistos / I.Sicily / a base_uri fragment;
see the module docstring for the priority order and the DDbDP map), typed
dctype:Text, carrying its title, identifiers, the corpus license (dcterms:license,
NonCommercial included), source, place / date, and its reading text as an rdf:value
literal (language-tagged grc for the Greek corpora). base_uri defaults to the
non-resolvable urn:aegean: namespace.
The write is atomic (temp file + os.replace), so a failed or interrupted write never
truncates a prior export. Raises ValueError for an unknown fmt, or for a base_uri
that cannot appear literally in an IRI (a space, a control character, or an IRIREF-forbidden
character) since that would make the Turtle and JSON-LD subjects disagree.
RDF is an export only: there is no reader and no round-trip guarantee (use
aegean.core.corpus.Corpus.to_json / aegean.db.to_sqlite for lossless persistence).
apply_merged ¶
apply_merged(merged: MergedReview, corpus: 'Corpus') -> 'Corpus'
Land a MergedReview's agreed corrections onto corpus, returning a NEW corpus.
Runs the agreed (clean) subset through the same apply core as from_review_table, so every
guard still fires (each row's token text is verified against the corpus; a wrong-corpus
mismatch or an orphaned row raises). Each corrected field keeps the machine value under
<field>__pred; every contributing reviewer is stamped on the token (reviewed_by) and
listed in the review: provenance note, which records that it came from a merge.
from_review_table ¶
Read reviewer corrections from path back onto corpus, returning a NEW corpus.
Rows prefer stable source-token identity and fall back to doc_id + position for
old tables. Source alignment and exported token text are verified: a mismatch (the corpus changed
between export and apply, or the wrong corpus was passed) raises ValueError naming the
rows rather than silently landing a correction on the wrong word. Duplicate rows for one
token with conflicting corrections, corrections whose row matches no token, and a
malformed CSV also raise ValueError.
For each row whose correct_* differs from the machine value the reviewer saw (the
row's own pred_* cell, falling back to the token's current annotation), the token's
annotation for that field is set to the corrected value, the machine value is preserved
under <field>__pred, and the token is stamped reviewed_by /
review_status="corrected" (plus review_note when the reviewer left one). A
morphology correction lands on the same key that supplied the displayed prediction
(morph or UD feats). Rows left blank change nothing. The stamped reviewer is
reviewer when given, else each row's own reviewer column. A review: provenance
note records how many tokens were corrected. A file that is not a review table (no
doc_id / position / correct_* columns, or no header at all) raises rather than
reading as zero corrections. The input corpus is not mutated. Use apply_review_table
for the same result plus the counts.
merge_review_tables ¶
merge_review_tables(paths: Iterable[str | Path], corpus: 'Corpus', *, on_conflict: str = 'error') -> MergedReview
Merge several reviewers' corrected copies of the SAME review export.
Each table in paths is a corrected copy of one to_review_table export. Corrections the
reviewers agree on (or that only one reviewer made) are combined into MergedReview.rows;
where two reviewers give different values for the same field of the same token, the
disagreement is surfaced as a ReviewConflict and never silently resolved. With
on_conflict="error" any conflict raises ValueError listing them all; with
on_conflict="report" the conflicts are returned in MergedReview.conflicts and the
agreed subset stays applicable (apply_merged).
Reviewer identity comes from each table's reviewer column (or, when blank, the file
name). The tables must be copies of one export: a table whose token text disagrees at a
shared (doc_id, position) raises (a wrong-corpus mix-up), as does a reviewer name that
appears in more than one table (identities must be distinct to attribute a conflict).
corpus is used to verify the export's shape; the corrections are landed by
apply_merged.
needs_review_flag ¶
Whether a token's annotation should be verified by a human.
Prefer the explicit review_recommended annotation. For older corpora, infer it
from a low-confidence evidence class (identity / unresolved), then from the
legacy lemma_known=false signal. A token that carries none of these (for example
a gold-annotated corpus) is not flagged.
to_review_table ¶
to_review_table(corpus: 'Corpus', path: str | Path, *, source_key: str = 'lemma_source', only_needs_review: bool = False, reviewer: str = '') -> int
Write one reviewable row per WORD token of corpus to path (CSV, UTF-8 BOM).
Each row carries the token's identity (stable source-token ID plus
doc_id/position/line_no/ref), exact source span and normalization,
machine pred_lemma/pred_pos/pred_morph from its annotations, the
evidence_class and a needs_review flag, the corpus citation, and blank
correct_* / reviewer_note columns for the reviewer. With only_needs_review only
the flagged rows are written. Pass reviewer to pre-stamp every row's reviewer
column (hand a named copy to each reviewer when the corrected copies will be merged with
merge_review_tables); it is left blank by default. Returns the number of rows written.
A token without a position is not exported: the apply join key is
doc_id + position, so a correction on such a row could never be applied. Cells
that would open as a live formula in a spreadsheet are neutralized with a leading
apostrophe (stripped again by from_review_table).
to_csv ¶
to_csv(corpus: Corpus, path: str | Path, *, level: str = 'document', progress: Callable[[int, int], None] | None = None) -> None
Write the corpus's level DataFrame ("document"/"token"/"word") to CSV.
progress, when given, is called progress(done, total) once per document as the
rows are generated (total is the document count) so a very large export is not
silent; the default (None) is the byte-identical original path.
to_parquet ¶
to_parquet(corpus: Corpus, path: str | Path, *, level: str = 'document', progress: Callable[[int, int], None] | None = None) -> None
Write the corpus's level DataFrame to Parquet (needs a parquet engine).
progress (progress(done, total) per document, total = document count) covers
the row-generation phase only: Parquet buffers the whole DataFrame before its single
write call, so the final progress call lands at (total, total) and the write follows.
The default (None) is the byte-identical original path.
from_csv ¶
from_csv(path: str | Path, *, text_col: str = 'text', id_col: str | None = None, script_id: str = 'greek', meta_cols: Sequence[str] = (), encoding: str = 'utf-8-sig') -> 'Corpus'
Build a Corpus from a CSV file. text_col holds each row's text; id_col
(optional) holds its document id (otherwise ids are <stem>:<row>). meta_cols
names columns to carry into document metadata (recognized: site/period/scribe/support/
findspot/name). Raises ValueError if text_col is absent.
A row whose text_col is empty or blank yields no document, the way a blank line
yields no document on the text-file path: a zero-token document would sit in the corpus
contributing nothing to any count, search, or export. The provenance records how many
rows were skipped, and a CSV in which every row is blank raises ValueError rather
than returning a corpus of empty documents. Text that is present but yields no tokens
under script_id is a different case and keeps its document, as the text path does.
The default utf-8-sig encoding transparently strips a leading UTF-8 BOM (Excel
writes one), so the first column name is not silently prefixed with it; it reads a
BOM-less UTF-8 file identically.
from_text ¶
from_text(text: str, *, script_id: str = 'greek', doc_id: str = 'text', split: str = 'whole', meta: dict[str, str] | None = None) -> 'Corpus'
Build a Corpus from a raw string.
split controls how the text becomes documents: "whole" (default, one document),
"paragraph" (one per blank-line-separated block), or "line" (one per line).
Line breaks are preserved as physical lines. script_id picks the tokenizer ("greek"
by default). A leading UTF-8 BOM (text read from a file with encoding="utf-8") is
dropped rather than glued onto the first word. Raises ValueError if the text has no
content.
from_text_dir ¶
from_text_dir(path: str | Path, *, script_id: str = 'greek', glob: str = '*.txt', split: str = 'whole', encoding: str = 'utf-8-sig') -> 'Corpus'
Build one Corpus from a folder of text files (one or more documents per file,
per split). Document ids come from each file's stem (de-duplicated with a #n
suffix on collision). Raises NotADirectoryError / FileNotFoundError as appropriate.
Files are read as utf-8-sig by default, so a BOM one of them carries does not
become part of its first word.
from_text_file ¶
from_text_file(path: str | Path, *, script_id: str = 'greek', split: str = 'whole', doc_id: str | None = None, encoding: str = 'utf-8-sig', meta: dict[str, str] | None = None) -> 'Corpus'
Build a Corpus from a plain-text file. The document id defaults to the file's
stem. See :func:from_text for split. Raises FileNotFoundError if missing.
The default utf-8-sig encoding drops a leading UTF-8 BOM (an editor or a Windows
tool that writes one) instead of leaving it attached to the first word, and reads a
BOM-less UTF-8 file identically.
from_workbench_export ¶
from_workbench_export(source: str | Path | dict[str, Any] | list[Any], *, script_id: str = 'lineara') -> Corpus
Load a workbench corpus export into a Corpus.
source is a path to a JSON file, a JSON string, or already-parsed
JSON. Both forms the workbench produces are accepted: the schema-v1
export object (records under "inscriptions", provenance under
"_meta", per-record "derived" analyses — ignored here) and a
plain array of inscription records. script_id defaults to lineara
(the workbench's own corpus); pass the real script when re-importing an
export of some other corpus, so the documents are not rebranded.
The workbench schema carries token text only: any editorial
ReadingStatus or Token.annotations the original corpus had were not
in the export, so every re-imported token is CERTAIN and unannotated
(see to_workbench).
Token kinds are inferred the Corpus.from_records way (numerals by
parseability, everything else a word); glyphs, transcription, and image
references are carried onto the documents. The export's own metadata
(app version, generation time, scope) lands in the corpus provenance.
Both field spellings the workbench has used are read: the schema-v1
export writes the dating period as period and nests imagery under an
images object (facsimile/photograph/rights/rightsUrl),
while the plain-array shape (and to_workbench) uses context and the
flat facsimileImages/images lists.
A string that opens with { or [ is read as JSON, and otherwise as a
filename. Those two forms overlap — "[export].json" is a legal relative
path — so a string that opens like JSON but does not parse is read as a path
when a file of that name exists; a payload that is merely malformed keeps
reporting its own decode error, since no such file exists.
Files are read as utf-8-sig: an export saved with a leading UTF-8 BOM loads
like any other UTF-8 file, and a BOM on a JSON string is stripped before the
string is recognized as JSON rather than mistaken for a file path.
to_workbench ¶
to_workbench(corpus: Corpus, path: str | Path | None = None) -> list[dict[str, Any]]
Emit workbench-shaped inscription records (optionally writing JSON).
Each document becomes one record with the fields the workbench renders:
id/site/support/scribe/findspot/context (its name
for the dating period)/name, the flat words list, per-line
lines, translations, glyphs, transcription, and image
references. Image files are never embedded — the workbench treats the
references as paths under its own mirror, so corpora without one simply
show no imagery.
Content the format does NOT preserve: the workbench schema carries token
text only, so per-token editorial ReadingStatus (UNCLEAR/RESTORED/LOST)
and Token.annotations (lemma, morphology, evidence class, review stamps)
are not written, and a re-import reads every token as CERTAIN and
unannotated. For a lossless round-trip use Corpus.to_json or
aegean.db.to_sqlite.
With path, the records are also written as UTF-8 JSON — the file the
app loads via ?corpus=<url> or its corpus file picker.