gec_metrics.metrics.pt_errant module

class gec_metrics.metrics.pt_errant.PTERRANT(config: Config = None)[source]

Bases: ERRANT

class Config(beta: float = 0.5, language: str = 'en', weight_model_name: str = 'bertscore', weight_model_config: Config = None)[source]

Bases: Config

Configuration of PTERRANT

  • weight_model_name (str): Model to compute edit-level weights.

    Currently only “bertscore” is available.

  • weight_model_config (MetricBaseForSourceFree.Config):

    The config instance of the weight model. If not specified, it uses the default one.

Also, you can use the same configurations as ERRANT.

weight_model_config: Config = None
weight_model_name: str = 'bertscore'
calc_edit_weights(src: str, ref: str, edits: list[Edit]) → list[float][source]

Calculate a weight for each edit.

Parameters:

  • src (str) – Source sentence.

  • src – Reference sentence.

  • edits (list[errant.edit.Edit]) – Edits.

Returns:

The weight of each edit.

Return type:

list[float]

score_base(sources: list[str], hypotheses: list[str], references: list[list[str]]) → list[list[dict[str, Score]]][source]
Calculate scores while retaining sentence and reference boundaries.
The results can be aggregated according to the purpose,

e.g., at sentence-level or corpus-level.

Parameters:

  • sources (list[str]) – Source sentence.

  • hypothesis (list[str]) – Corrected sentences.

  • references (list[list[str]]) – Reference sentences. The shape is (the number of references, the number of sentences).

Returns:

The verbose scores.

  • The list shape is (num_sents, num_refs)

  • The dict contains error type-wise scores.

Return type:

list[list[dict[str, “Score”]]]