Online Ingestion#

OnlineIngestion #

Metadata object used to provide Online Ingestion information for a feature group.

This class encapsulates the state and results of an online ingestion operation, including progress tracking and log retrieval.

feature_group `property` #

feature_group: fg_mod.FeatureGroup

Get the feature group associated with this ingestion.

RETURNS	DESCRIPTION
`FeatureGroup`	The associated feature group. TYPE: `fg_mod.FeatureGroup`

id `property` #

id: int | None

Get the unique identifier for the ingestion operation.

RETURNS	DESCRIPTION
`int \| None`	Optional[int]: The ingestion ID.

num_entries `property` `writable` #

num_entries: int | None

Get the total number of entries to ingest.

RETURNS	DESCRIPTION
`int \| None`	Optional[int]: The number of entries.

results `property` #

results: list[online_ingestion_result.OnlineIngestionResult]

Get the list of ingestion results.

RETURNS	DESCRIPTION
`list[online_ingestion_result.OnlineIngestionResult]`	List[OnlineIngestionResult]: List of ingestion result objects.

init #

__init__(
    id: int | None = None,
    num_entries: int | None = None,
    results: list[
        online_ingestion_result.OnlineIngestionResult
    ]
    | list[dict[str, Any]] = None,
    feature_group: fg_mod.FeatureGroup = None,
    **kwargs,
)

Initialize an OnlineIngestion object.

PARAMETER	DESCRIPTION
`id`	The unique identifier for the ingestion operation. TYPE: `Optional[int]` DEFAULT: `None`
`num_entries`	The total number of entries to ingest. TYPE: `Optional[int]` DEFAULT: `None`
`results`	List of ingestion results or their JSON representations. TYPE: `Union[List[OnlineIngestionResult], List[Dict[str, Any]]]` DEFAULT: `None`
`feature_group`	The feature group associated with this ingestion. TYPE: `FeatureGroup` DEFAULT: `None`

from_response_json `classmethod` #

from_response_json(
    json_dict: dict[str, Any],
    feature_group: fg_mod.FeatureGroup = None,
) -> OnlineIngestion

Create an OnlineIngestion object from a JSON response.

PARAMETER	DESCRIPTION
`json_dict`	The JSON dictionary from the API response. TYPE: `Dict[str, Any]`
`feature_group`	The feature group associated with this ingestion. TYPE: `FeatureGroup` DEFAULT: `None`

RETURNS	DESCRIPTION
`OnlineIngestion`	The created OnlineIngestion object, or a list of them if multiple items are present. TYPE: `OnlineIngestion`

json #

json()

Serialize the OnlineIngestion object to a JSON string.

RETURNS	DESCRIPTION
`str`	JSON string representation of the object.

print_logs #

print_logs(priority: str = 'error', size: int = 20)

Print logs related to the online ingestion operation from OpenSearch.

PARAMETER	DESCRIPTION
`priority`	Log priority to filter by (default: "error"). TYPE: `str` DEFAULT: `'error'`
`size`	Number of log entries to retrieve (default: 20). TYPE: `int` DEFAULT: `20`

refresh #

refresh()

Refresh the state of this OnlineIngestion object from the backend.

to_dict #

to_dict()

Convert the OnlineIngestion object to a dictionary.

RETURNS	DESCRIPTION
`dict`	Dictionary representation of the object.

wait_for_completion #

wait_for_completion(options: dict[str, Any] = None)

Wait for the online ingestion operation to complete, displaying a progress bar.

PARAMETER	DESCRIPTION
`options`	Options for waiting. - "timeout" (int): Maximum time to wait in seconds (default: 60). - "period" (int): Polling period in seconds (default: 1). TYPE: `Dict[str, Any]` DEFAULT: `None`

RAISES	DESCRIPTION
`Warning`	If the timeout is exceeded before completion.

Online Ingestion#