Concepts

create vs. create_async

You’ll note that in some cases you have a choice between create() and create_async() when creating new entities on the Driverless AI server.

• create(): Waits for the job to completely finish before returning a server object for you to work with. In most cases this is what you want to use.
• create_async(): Immediately returns an object which allows you to track the job status while the entity is being created on the server. Use if you have a long running job and you don’t want to wait for completion. With experiments, this also gives you the ability to monitor the experiment progress and stop it early, if desired.

Example

Using create_async(), then monitoring then experiment and finishing it when the validation score goes over 0.9 from a Jupyter notebook:

# returns immediately
ex = dai.experiments.create_async(
    **ds_split,
    **settings
)


import time
from IPython.display import clear_output

while ex.is_running():
    time.sleep(1)
    # grab experiment status
    status = ex.status(verbose=2)
    # grab current metrics
    metrics = ex.metrics()
    # pretty print info
    clear_output(wait=True)
    print(status, " - Validation ", metrics['scorer'], ": ", sep='', end='')
    if metrics['val_score'] is not None:
        print(round(metrics['val_score'], 4), '+/-', round(metrics['val_score_sd'], 4))
        if metrics['val_score'] > 0.9:
            ex.finish()
    else:
        print()
    print()
    ex.log.tail(3)
    time.sleep(1)

print("\nTest ", ex.metrics()['scorer'], ": ", round(ex.metrics()['test_score'], 4), sep='')

server entity vs. server object

• A server entity exists on the Driverless AI server.
• A server object is a local representation of a server entity.