Base experiment version

class previsionio.experiment_version.BaseExperimentVersion(**experiment_version_info)

Bases: previsionio.api_resource.ApiResource

Base parent class for all experiments objects.

best_model

Get the model with the best predictive performance over all models (including Blend models), where the best performance corresponds to a minimal loss.

Returns:Model with the best performance in the experiment, or None if no model matched the search filter.
Return type:(Model, None)
delete()

Delete an experiment version from the actual [client] workspace.

Raises:
  • PrevisionException – If the experiment version does not exist
  • requests.exceptions.ConnectionError – Error processing the request
delete_prediction(prediction_id: str)

Delete a prediction in the list for the current experiment from the actual [client] workspace.

Parameters:prediction_id (str) – Unique id of the prediction to delete
Returns:Deletion process results
Return type:dict
delete_predictions()

Delete all predictions in the list for the current experiment from the actual [client] workspace.

Returns:Deletion process results
Return type:dict
done

Get a flag indicating whether or not the experiment is currently done.

Returns:done status
Return type:bool
fastest_model

Returns the model that predicts with the lowest response time

Returns:Model object – corresponding to the fastest model
get_holdout_predictions(full: bool = False)

Retrieves the list of holdout predictions for the current experiment from client workspace (with the full predictions object if necessary) :param full: If true, return full holdout prediction objects (else only metadata) :type full: boolean

get_predictions(full: bool = False)

Retrieves the list of predictions for the current experiment from client workspace (with the full predictions object if necessary) :param full: If true, return full prediction objects (else only metadata) :type full: boolean

models

Get the list of models generated for the current experiment version. Only the models that are done training are retrieved.

Returns:List of models found by the platform for the experiment
Return type:list(Model)
print_info()

Print all info on the experiment.

running

Get a flag indicating whether or not the experiment is currently running.

Returns:Running status
Return type:bool
schema

Get the data schema of the experiment.

Returns:Experiment schema
Return type:dict
score

Get the current score of the experiment (i.e. the score of the model that is currently considered the best performance-wise for this experiment).

Returns:Experiment score (or infinity if not available).
Return type:float
status

Get a flag indicating whether or not the experiment is currently running.

Returns:Running status
Return type:bool
stop()

Stop an experiment (stopping all nodes currently in progress).

train_dataset

Get the Dataset object corresponding to the training dataset of the experiment.

Returns:Associated training dataset
Return type:Dataset
update_status()

Get an update on the status of a resource.

Parameters:specific_url (str, optional) – Specific (already parametrized) url to fetch the resource from (otherwise the url is built from the resource type and unique _id)
Returns:Updated status info
Return type:dict
wait_until(condition, raise_on_error: bool = True, timeout: float = 3600.0)

Wait until condition is fulfilled, then break.

Parameters:
  • (func (condition) – (BaseExperimentVersion) -> bool.): Function to use to check the break condition
  • raise_on_error (bool, optional) – If true then the function will stop on error, otherwise it will continue waiting (default: True)
  • timeout (float, optional) – Maximal amount of time to wait before forcing exit

Example:

experiment.wait_until(lambda experimentv: len(experimentv.models) > 3)
Raises:PrevisionException – If the resource could not be fetched or there was a timeout.
class previsionio.experiment_version.ClassicExperimentVersion(**experiment_version_info)

Bases: previsionio.experiment_version.BaseExperimentVersion

advanced_models_list

Get the list of selected advanced models in the experiment.

Returns:Names of the normal models selected for the experiment
Return type:list(AdvancedModel)
correlation_matrix

Get the correlation matrix of the features (those constitute the dataset on which the experiment was trained).

Returns:Correlation matrix as a pandas dataframe
Return type:pd.DataFrame
dataset

Get the Dataset object corresponding to the training dataset of this experiment version.

Returns:Associated training dataset
Return type:Dataset
drop_list

Get the list of drop columns in the experiment.

Returns:Names of the columns dropped from the dataset
Return type:list(str)
dropped_features

Get dropped features

Returns:Dropped features
Return type:dict
feature_list

Get the list of selected feature engineering modules in the experiment.

Returns:Names of the feature engineering modules selected for the experiment
Return type:list(str)
features
  • feature types distribution
  • feature information list
  • list of dropped features
Returns:General features information
Return type:dict
Type:Get the general description of the experiment’s features, such as
features_stats
  • feature types distribution
  • feature information list
  • list of dropped features
Returns:General features information
Return type:dict
Type:Get the general description of the experiment’s features, such as
get_cv() → pandas.core.frame.DataFrame

Get the cross validation dataset from the best model of the experiment.

Returns:Cross validation dataset
Return type:pd.DataFrame
get_feature_info(feature_name: str) → Dict

Return some information about the given feature, such as:

  • name: the name of the feature as it was given in the feature_name parameter

  • type: linear, categorical, ordinal…

  • stats: some basic statistics such as number of missing values, (non missing) values count, plus additional information depending on the feature type:

    • for a linear feature: min, max, mean, std and median
    • for a categorical/textual feature: modalities/words frequencies, list of the most frequent tokens
  • role: whether or not the feature is a target/fold/weight or id feature (and for time series experiments, whether or not it is a group/apriori feature - check the Prevision.io’s timeseries documentation)

  • importance_value: scores reflecting the importance of the given feature

Parameters:
  • feature_name (str) – Name of the feature to get informations about
  • warning:: (.) – The feature_name is case-sensitive, so “age” and “Age” are different features!
Returns:

Dictionary containing the feature information

Return type:

dict

Raises:

PrevisionException – If the given feature name does not match any feaure

holdout_dataset

Get the Dataset object corresponding to the holdout dataset of this experiment version.

Returns:Associated holdout dataset
Return type:Dataset
normal_models_list

Get the list of selected normal models in the experiment.

Returns:Names of the normal models selected for the experiment
Return type:list(NormalModel)
predict(df, confidence=False, prediction_dataset_name=None) → pandas.core.frame.DataFrame

Get the predictions for a dataset stored in the current active [client] workspace using the best model of the experiment with a Scikit-learn style blocking prediction mode.

Warning

For large dataframes and complex (blend) models, this can be slow (up to 1-2 hours). Prefer using this for simple models and small dataframes, or use option use_best_single = True.

Parameters:
  • df (pd.DataFrame) – pandas DataFrame containing the test data
  • confidence (bool, optional) – Whether to predict with confidence values (default: False)
Returns:

Prediction results dataframe

Return type:

pd.DataFrame

predict_from_dataset(dataset, confidence=False, dataset_folder=None) → pandas.core.frame.DataFrame

Get the predictions for a dataset stored in the current active [client] workspace using the best model of the experiment.

Parameters:
  • dataset (Dataset) – Reference to the dataset object to make predictions for
  • confidence (bool, optional) – Whether to predict with confidence values (default: False)
  • dataset_folder (Dataset) – Matching folder dataset for the predictions, if necessary
Returns:

The registered prediction object in the current workspace

Return type:

previsionio.prediction.ValidationPrediction

simple_models_list

Get the list of selected simple models in the experiment.

Returns:Names of the simple models selected for the experiment
Return type:list(SimpleModel)