gec_metrics.metrics.green module

class gec_metrics.metrics.green.GREEN(config: Config = None)

Bases: MetricBaseForReferenceBased

class Config(n: int = 4, beta: float = 2.0, unit: str = 'word')

Bases: Config

GREEN configuration - n (int): Maxmimun n for n-gram. - beta (int): The beta for F-beta score. - unit (str): Word-level or character-level. Can be ‘word’ or ‘char’.

beta: float = 2.0

n: int = 4

unit: str = 'word'

aggregate_score(scores: list[Score]) → float

Aggregate n-gram scores to an overall score by the geometric mean.

Parameters:

scores (list[Score]) – The scores keeping n-gram boundary. The shape is (n, )

Returns:

The aggregated score.

Return type:

float

cached_get_all_ngrams(sentence: str) → dict[str, int]

Get frequency of n-gram for all n (1 <= n <= config.n)

score_base(sources: list[str], hypotheses: list[str], references: list[list[str]]) → list[list[list[Score]]]

Calculate scores while retaining sentence and reference boundaries.

The results can be aggregated according to the purpose,

e.g., at sentence-level or corpus-level.

Parameters:

• sources (list[str]) – Source sentence.

• hypothesis (list[str]) – Corrected sentences.

• references (list[list[str]]) – Reference sentences. The shape is (the number of references, the number of sentences).

Returns:

The verbose scores.

The shape is (num_iterations, num_sents, max_ngram).

Return type:

list[list[list[“Score”]]]

score_corpus(sources: list[str], hypotheses: list[str], references: list[list[str]]) → float

Calculate a corpus-level score. This accumulates n-gram count for TP, FP, FN

and calculates f-beta score.

Parameters:

• sources (list[str]) – Source sentence. The shape is (num_sentences, )

• hypotheses (list[str]) – Corrected sentences. The shape is (num_sentences, )

• references (list[list[str]]) – Reference sentences. The shape is (num_references, num_sentences).

Returns:

The corpus-level score.

Return type:

float

score_sentence(sources: list[str], hypotheses: list[str], references: list[list[str]]) → list[float]

Calculate sentence-level scores.

Parameters:

• sources (list[str]) – Source sentence. The shape is (num_sentences, )

• hypotheses (list[str]) – Corrected sentences. The shape is (num_sentences, )

• references (list[list[str]]) – Reference sentences. The shape is (num_references, num_sentences).

Returns:

The sentence-level scores.

Return type:

list[float]