Expectation Suite#

ExpectationSuite #

Metadata object representing a feature validation expectation in the Feature Store.

data_asset_type `property` `writable` #

data_asset_type: str | None

Data asset type of the expectation suite, not used by backend.

expectation_suite_name `property` `writable` #

expectation_suite_name: str

Name of the expectation suite.

expectations `property` `writable` #

expectations: list[GeExpectation]

List of expectations to run at validation.

ge_cloud_id `property` #

ge_cloud_id: int | None

ge_cloud_id of the expectation suite, not used by backend.

id `property` `writable` #

id: int | None

Id of the expectation suite, set by backend.

meta `property` `writable` #

meta: dict[str, Any]

Meta field of the expectation suite to store additional information.

run_validation `property` `writable` #

run_validation: bool

Boolean to determine whether or not the expectation suite shoudl run on ingestion.

validation_ingestion_policy `property` `writable` #

validation_ingestion_policy: Literal['always', 'strict']

Whether to ingest a df based on the validation result.

"strict": ingest df only if all expectations succeed, or
"always": always ingest df, even if one or more expectations fail.

add_expectation #

add_expectation(
    expectation: GeExpectation
    | great_expectations.core.ExpectationConfiguration,
    ge_type: bool = HAS_GREAT_EXPECTATIONS,
) -> (
    GeExpectation
    | great_expectations.core.ExpectationConfiguration
)

Append an expectation to the local suite or in the backend if attached to a Feature Group.

Example

# check if the minimum value of specific column is within a range of 0 and 1
expectation_suite.add_expectation(
    ge.core.ExpectationConfiguration(
        expectation_type="expect_column_min_to_be_between",
        kwargs={
            "column": "foo_id",
            "min_value": 0,
            "max_value": 1
        }
    )
)

# check if the length of specific column value is within a range of 3 and 10
expectation_suite.add_expectation(
    ge.core.ExpectationConfiguration(
        expectation_type="expect_column_value_lengths_to_be_between",
        kwargs={
            "column": "bar_name",
            "min_value": 3,
            "max_value": 10
        }
    )
)

PARAMETER	DESCRIPTION
`expectation`	The new expectation object. TYPE: `GeExpectation \| great_expectations.core.ExpectationConfiguration`
`ge_type`	Whether to return native Great Expectations object or Hopsworks abstraction. TYPE: `bool` DEFAULT: `HAS_GREAT_EXPECTATIONS`

RETURNS	DESCRIPTION
`GeExpectation \| great_expectations.core.ExpectationConfiguration`	The new expectation attached to the Feature Group.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.
`hopsworks.client.exceptions.FeatureStoreException`	If the expectation suite is not registered yet.

from_ge_type `classmethod` #

from_ge_type(
    ge_expectation_suite: great_expectations.core.ExpectationSuite,
    run_validation: bool = True,
    validation_ingestion_policy: Literal[
        "ALWAYS", "STRICT"
    ] = "ALWAYS",
    id: int | None = None,
    feature_store_id: int | None = None,
    feature_group_id: int | None = None,
) -> ExpectationSuite

Used to create a Hopsworks Expectation Suite instance from a great_expectations instance.

PARAMETER	DESCRIPTION
`ge_expectation_suite`	The great_expectations ExpectationSuite instance to convert to a Hopsworks ExpectationSuite. TYPE: `great_expectations.core.ExpectationSuite`
`run_validation`	Whether to run validation on inserts when the expectation suite is attached. TYPE: `bool` DEFAULT: `True`
`validation_ingestion_policy`	The validation ingestion policy to use when the expectation suite is attached. TYPE: `Literal['ALWAYS', 'STRICT']` DEFAULT: `'ALWAYS'`
`id`	The id of the expectation suite in Hopsworks. If not provided, a new expectation suite will be created. TYPE: `int \| None` DEFAULT: `None`
`feature_store_id`	The id of the feature store of the feature group to which the expectation suite belongs. TYPE: `int \| None` DEFAULT: `None`
`feature_group_id`	The id of the feature group to which the expectation suite belongs. TYPE: `int \| None` DEFAULT: `None`

RETURNS	DESCRIPTION
`ExpectationSuite`	Hopsworks Expectation Suite instance.

get_expectation #

get_expectation(
    expectation_id: int,
    ge_type: bool = HAS_GREAT_EXPECTATIONS,
) -> (
    GeExpectation
    | great_expectations.core.ExpectationConfiguration
)

Fetch expectation with expectation_id from the backend.

Example

# connect to the Feature Store
fs = ...

# get the Feature Group instance
fg = fs.get_or_create_feature_group(...)

expectation_suite = fg.get_expectation_suite()
selected_expectation = expectation_suite.get_expectation(expectation_id=123)

PARAMETER	DESCRIPTION
`expectation_id`	ID of the expectation to fetch from the backend. TYPE: `int`
`ge_type`	Whether to return native Great Expectations object or Hopsworks abstraction. TYPE: `bool` DEFAULT: `HAS_GREAT_EXPECTATIONS`

RETURNS	DESCRIPTION
`GeExpectation \| great_expectations.core.ExpectationConfiguration`	The expectation with `expectation_id` registered in the backend.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.
`hopsworks.client.exceptions.FeatureStoreException`	If the expectation suite is not registered yet.

remove_expectation #

remove_expectation(
    expectation_id: int | None = None,
) -> None

Remove an expectation from the suite locally and from the backend if attached to a Feature Group.

Example

expectation_suite.remove_expectation(expectation_id=123)

PARAMETER	DESCRIPTION
`expectation_id`	ID of the expectation to remove. The expectation will be deleted both locally and from the backend. TYPE: `int \| None` DEFAULT: `None`

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.
`hopsworks.client.exceptions.FeatureStoreException`	If the expectation suite is not registered yet.

replace_expectation #

replace_expectation(
    expectation: GeExpectation
    | great_expectations.core.ExpectationConfiguration,
    ge_type: bool = HAS_GREAT_EXPECTATIONS,
) -> (
    GeExpectation
    | great_expectations.core.ExpectationConfiguration
)

Update an expectation from the suite locally or from the backend if attached to a Feature Group.

Example

updated_expectation = expectation_suite.replace_expectation(new_expectation_object)

PARAMETER	DESCRIPTION
`expectation`	The updated expectation object. The meta field should contain an expectationId field. TYPE: `GeExpectation \| great_expectations.core.ExpectationConfiguration`
`ge_type`	Whether to return native Great Expectations object or Hopsworks abstraction. TYPE: `bool` DEFAULT: `HAS_GREAT_EXPECTATIONS`

RETURNS	DESCRIPTION
`GeExpectation \| great_expectations.core.ExpectationConfiguration`	The updated expectation attached to the Feature Group.

RAISES	DESCRIPTION
`hopsworks.client.exceptions.RestAPIError`	If the backend encounters an error when handling the request.
`hopsworks.client.exceptions.FeatureStoreException`	If the expectation suite is not registered yet.

to_ge_type #

to_ge_type() -> great_expectations.core.ExpectationSuite

Convert to Great Expectations ExpectationSuite type.

Creation with Great Expectations#

import great_expectations as ge

expectation_suite = ge.core.ExpectationSuite(
    "new_expectation_suite",
    expectations=[
        ge.core.ExpectationConfiguration(
            expectation_type="expect_column_max_to_be_between",
            kwargs={
                "column": "feature",
                "min_value": -1,
                "max_value": 1
            }
        )
    ]
)

Attach to Feature Group#

To attach an ExpectationSuite to a FeatureGroup, call FeatureGroup.save_expectation_suite.

Expectation Suite#

ExpectationSuite #

data_asset_type property writable #

expectation_suite_name property writable #

expectations property writable #

ge_cloud_id property #

id property writable #

meta property writable #

run_validation property writable #

validation_ingestion_policy property writable #

add_expectation #

from_ge_type classmethod #

get_expectation #

remove_expectation #

replace_expectation #

to_ge_type #

Creation with Great Expectations#

Attach to Feature Group#

data_asset_type `property` `writable` #

expectation_suite_name `property` `writable` #

expectations `property` `writable` #

ge_cloud_id `property` #

id `property` `writable` #

meta `property` `writable` #

run_validation `property` `writable` #

validation_ingestion_policy `property` `writable` #

from_ge_type `classmethod` #