# Asenion Client APIs ## INITIALIZATION Initialize an AI class using this line of code: ### ai = AI() This AI class will allow you to use the functions of the AI to log your important metrics to Fairly. ## AI INFORMATION FUNCTIONS ### ai.identify_ai() Read the AI data specified in the fairly.yaml file, and return the information of the AI from the server. ### ai.tag_list_check() Check if the AI has tags corresponding to the bundle that was selected for the AI synced to Fairly. It will return a legend consisting of the following for each tag in the bundle: Value not stored in Fairly Server ❌ Value stored but not within appropriate bound: ⚠ Value stored in Fairly server and within bounds: ✅ If it does find a value stored in Fairly, it will also tell you the number of tags with that name associated with the AI. If no bundle was selected on AI creation, the function will state that there are no requirements for the AI. ### ai.get_synced_tags() Return a list of all tags, as well as their names, model IDs and dataset IDs. ### ai.show_models_in_session() Return a table containing all the models currently in session. ### ai.show_datasets_in_session() Return a list of all the datasets currently in session. ### ai.performance_analysis(model_score, debiased_score, bound) Take two scores, one from the original model and one from the debiased model, and determine whether the difference between each of the scores is in a user-determined bound. Arguments: * model_score: an array that stores the model’s accuracy score and precision score. * debiased_score: an array that stores the debiased scores of accuracy and precision. * bound: the limited difference between the actual scores and debiased scores. ### ai.check_tag_existence(tagName=None) Return whether or not the tag exists for the AI. Arguments: * tagName(string): the name of the tag to be retrieved. ### ai.set_most_recently_used_model(modelID=None) Set a model ID as the most recently used model. If no model ID is specified it will prompt for one. ### ai.set_most_recently_used_dataset(datasetID=None) Set a model ID as the most recently used model. If no model ID is specified it will prompt for one. ## LOGGING FUNCTIONS ### ai.sync_model(model, model_name = ‘’, modelID = None, params = {}, justification = ‘’, modelType = ‘’) Sync the model to Fairly servers. Arguments: * model: the model that is to be synced to Fairly servers. * model_name: the name of the model to be synced. The name must not already be in use when syncing a new model. * modelID: the ID of the model that you want to sync. If modelID is set to new, a new model ID will be created if the model name does not already exist. * model_params: the parameters of the model to be synced. If none are specified it will show up as ‘N/A: N/A’. * justification: A justification for the model. * modelType: the type of model ### ai.sync_dataset(dataframe, name = ‘’, datasetID = None, justification = ‘’, datasetType = ‘’, protectedCategories = ‘’) Sync the dataset to Fairly servers. Arguments: * dataset: the dataset that is to be synced to Fairly servers, should be in the form of a Pandas DataFrame. * name: the name of the dataset to be synced. The name must not already be in use. * datasetID: the ID of the dataset that you want to sync. If you want a new dataset ID set this to ‘NEW’. If you want to version a dataset ID, specify the dataset ID that you want to version. * justification: the justification for the dataset * datasetType: the type of dataset that is being synced * protectedCategories: the dataset categories that need to be protected. ### ai.show_available_tags() Retrieve the list of available tags in the Fairly database. ### ai.sync_asset(asset, asset_tag, modelID=None, datasetID=None, source=None, message="") Send an asset associated with an AI to the server. Arguments: * asset: a dataframe, string (e.g. json), or image (actual data/value of the tag) * asset_tag(string): a key identifying the unique purpose of the asset (name of the tag ex: auc_score) * message(string): freeform text similar to a git commit message * source(string): freeform text indicating the original source of the asset * modelID(integer): id of the model * datasetID(integer): id of the dataset ## MLFLOW FUNCTIONS ### ai.set_tracking_uri(directory = None) Set the MLflow tracking URI to a desired directory. The default tracking URI is ../assets/mlflow. If the directory does not exist, it will be created. If no directory is specified the function will tell you where the tracking URI is set to. Arguments: * directory: the directory where the MLflow tracking URI is to be set. ### ai.start_run(version_name, run_id = None) Start an MLflow run. This must be called before logging anything to a run. If desired, a run can be resumed by passing the run ID in. Arguments: * version_name: MLflow experiment to log the run to. * run_id: Run ID to resume an MLflow run from an experiment. Leave blank to generate a new run. ### ai.end_run(status = ‘FINISHED’) End an MLflow run, with a custom status if desired. Arguments: * status: the status that you want to end the run with. Leave blank to end as ‘FINISHED’. ### ai.list_experiments_and_runs() List the experiments and runs found in the tracking URI. ### ai.log_metrics(dict_metrics) Log a dictionary of tags to an MLflow run. An MLflow run must be active to use this function. Arguments: * dict_metrics: a dictionary of metrics that you want to be logged to an MLflow run. * Forbidden metric names: parameters, tags, artifact, mlflow_artifact, mlflow_model, mlflow_tag ### ai.log_params(dict_params) Log a dictionary of parameters to an MLflow run. This function requires a run to be active. Arguments: * dict_params: a dictionary of parameters that you want to be logged to an MLflow run. ### ai.log_artifact(path_to_artifact) Log a file to the currently active MLflow run as an artifact. This function requires a run to be active. Arguments: * path_to_artifact: A path to the file you want to log as an artifact. ### ai.log_model(model, model_name, library) Log a model to an MLflow run as an artifact. This function requires a run to be active. Arguments: * model: The model to log as an artifact. * model_name: the name of the model to log. * library: the library where the model came from: * Supported libraries: sklearn, pytorch, catboost, fastai, gluon, h2o, keras, lightgbm, onnx, paddle, spacy, spark, statsmodels, xgboost ### ai.get_run_metrics(run_id) Return the metrics associated with a run ID. Arguments: * run_id: the MLflow run ID that you want metrics for. ### ai.get_run_params(run_id) Return the parameters associated with a run ID. Arguments: * run_id: the MLflow run ID that you want parameters for. ### ai.get_run_artifacts(run_id) Return the artifacts associated with a run ID. Arguments: * run_id: the MLflow run ID that you want artifacts for. ### ai.get_run_tags(run_id) Returns the MLflow tags associated with the run ID. Arguments: * run_id: the MLflow run ID that you want the MLflow tags for. ### ai.get_root_artifact_uri(run_id) Return the root artifact URI where MLflow artifacts are stored for a given run. Arguments: * run_id: the MLflow run ID that you want the root artifact URI for. ### ai.mlflow_sync_asset(run_id = None, what_to_sync = None, artifact_uri = None, modelID = None, datasetID = None, source = None) Send an asset to the server based on an MLflow run. Arguments: * run_id(str): the run ID where the data is stored. If none is specified it will attempt to use the currently active run. * what_to_sync(str): Either a metric name to be logged, 'parameters' for parameters, 'tags' for tags, 'artifact' for a file, or 'model' for a logged model. * artifact_uri(str): The location of the file/model to be synced relative to the artifact URI. * modelID(int): id of the model. If not specified, the most recently used model ID will be used. * datasetID(int): id of the dataset. If not specified, the most recently used dataset ID will be used. * source(string): freeform text indicating the original source of the asset.