Experiments and Runs

Azure ML is a cloud-based machine-learning solution that makes it easier to execute your code. A Run is an abstraction layer that wraps around each such submission and is used to keep track of your findings and monitor the task in real time.

• Run: A run is a single instance of your code being executed. See Run for more details.
• Experiments: A light-weight container for an experiment is an experiment. Run. Use experiments to submit and track runs.

Create an experiment in your workspace ws.

from azureml.core import Experiment
exp = Experiment(ws, '<experiment-name>')

ScriptRunConfig

ScriptRunConfig, which bundles your source code (Script) and run settings, is a typical approach to run programmes on the cloud (RunConfig).

For your code, consider the arrangement below.

source_directory/
    script.py    # entry point to your code
    module1.py   # modules called by script.py     
    ...

To run script.py in the cloud via the ScriptRunConfig

config = ScriptRunConfig(
    source_directory='<path/to/source_directory>',
    script='script.py',
    compute_target=target,
    environment=env,
    arguments = [
        '--learning_rate', 0.001,
        '--momentum', 0.9,
    ]
)

where:

• source_directory='source_directory' : Local directory with your code.
• script='script.py' : Script to run. This does not need to be at the root of source_directory.

Submit this code to Azure with the following code

exp = Experiment(ws, '<exp-name>')
run = exp.submit(config)
print(run)
run.wait_for_completion(show_output=True)

This will provide you a URL to watch your run on the web (https://ml.azure.com) as well as logs streaming to your terminal.

Command Line Arguments

Use the arguments option in ScriptRunConfig to send command line parameters to your script. Arguments are given in the form of a list:

arguments = [first, second, third, ...]

which are then passed to the script as command-line arguments as follows:

python script.py first second third ...

This also supports using named arguments:

arguments = ['--first_arg', first_val, '--second_arg', second_val, ...]

Arguments can be of type int, float str and can also be used to reference data.

Example: sys.argv

In this example we pass two arguments to our script. If we were running this from the console:

python script.py 0.001 0.9

To use this command as a model argument in ScriptRunConfig: run.py

arguments = [0.001, 0.9]

config = ScriptRunConfig(
    source_directory='.',
    script='script.py',
    arguments=arguments,
)

which can be consumed as usual in our script: script.py

import sys
learning_rate = sys.argv[1]     # gets 0.001
momentum = sys.argv[2]          # gets 0.9

Example: argparse

In this example we pass two named arguments to our script. If we were running this from the console:

python script.py --learning_rate 0.001 --momentum 0.9

To use this command as a model in ScriptRunConfig: run.py

arguments = [
    '--learning_rate', 0.001, 
    '--momentum', 0.9,
    ]

config = ScriptRunConfig(
    source_directory='.',
    script='script.py',
    arguments=arguments,
)

which can be consumed as usual in our script: script.py

import argparse
parser = argparse.Argparser()
parser.add_argument('--learning_rate', type=float)
parser.add_argument('--momentum', type=float)
args = parser.parse_args()

learning_rate = args.learning_rate      # gets 0.001
momentum = args.momentum                # gets 0.9

Commands

It is possible to provide the explicit command to run : script.py

command = 'python script.py'.split()

config = ScriptRunConfig(
    source_directory='<path/to/code>',
    command=command,
    compute_target=compute_target,
    environment=environment,
)

Setting the argument is the same as this example. script='script.py' in place of the command argument.

This option provides a lot of flexibility. For example:

• Set environment variables: Some useful examples:

  command = 'export PYTHONPATH=$PWD && python script.py'.split()
  

  command = f'export RANK={rank} && python script.py'.split()
  

• Run setup script: Run a setup script e.g. to download data, set environment variables.

  command = 'python setup.py && python script.py'.split()
  

Using Datasets

via Arguments

Pass a dataset to your ScriptRunConfig as an argument

# create dataset
datastore = ws.get_default_datastore()
dataset = Dataset.File.from_files(path=(datastore, '<path/on/datastore>'))

arguments = ['--dataset', dataset.as_mount()]

config = ScriptRunConfig(
    source_directory='.',
    script='script.py',
    arguments=arguments,
)

This mounts the dataset to the run where it can be referenced by script.py.

Run

Interactive

In an interactive setting e.g. a Jupyter notebook

run = exp.start_logging()

Example: Jupyter notebook

A common use case for interactive logging is to train a model in a notebook.

from azureml.core import Workspace
from azureml.core import Experiment
ws = Workspace.from_config()
exp = Experiment(ws, 'example')

run = exp.start_logging()                   # start interactive run
print(run.get_portal_url())                 # get link to studio

# A simple example in place of e.g. model
# training or exploratory data analysis
import matplotlib.pyplot as plt
import numpy as np

# 100 linearly spaced numbers
x = np.linspace(-5,5,100)

# the function, which is y = x^2 here
y = x**2

# setting the axes at the centre
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
ax.spines['left'].set_position('center')
ax.spines['bottom'].set_position('zero')
ax.spines['right'].set_color('none')
ax.spines['top'].set_color('none')
ax.xaxis.set_ticks_position('bottom')
ax.yaxis.set_ticks_position('left')

# plot the function
plt.plot(x,y, 'r')

# show the plot
plt.show()
run.log_row('sine', x=x, y=y)               # log metrics
run.complete()                              # stop interactive run

Get Context

A Run is used to track the execution of code in Azure ML. The code that was provided has access to its own run.

from azureml.core import Run
run = Run.get_context()

Example: Logging metrics to current run context

A common use-case is logging metrics in a training script : train.py

from azureml.core import Run

run = Run.get_context()

# training code
for epoch in range(n_epochs):
    model.train()
    ...
    val = model.evaluate()
    run.log('validation', val)

This code will report metrics to its related run when uploaded to Azure ML (e.g. through ScriptRunConfig).