Top Level API Reference

The Space Concept

Space

class Space(*args, **kwargs)[source]

Bases: object

Search space object

Important

Parameters: kwargs (Any) – parameters in the search space

Space(a=1, b=1)  # static space
Space(a=1, b=Grid(1,2), c=Grid("a", "b"))  # grid search
Space(a=1, b=Grid(1,2), c=Rand(0, 1))  # grid search + level 2 search
Space(a=1, b=Grid(1,2), c=Rand(0, 1)).sample(10, sedd=0)  # grid + random search

# union
Space(a=1, b=Grid(2,3)) + Space(b=Rand(1,5)).sample(10)

# cross product
Space(a=1, b=Grid(2,3)) * Space(c=Rand(1,5), d=Grid("a","b"))

# combo (grid + random + level 2)
space1 = Space(a=1, b=Grid(2,4))
space2 = Space(b=RandInt(10, 20))
space3 = Space(c=Rand(0,1)).sample(10)
space = (space1 + space2) * space3

assert Space(a=1, b=Rand(0,1)).has_stochastic
assert not Space(a=1, b=Rand(0,1)).sample(10).has_stochastic
assert not Space(a=1, b=Grid(0,1)).has_stochastic
assert not Space(a=1, b=1).has_stochastic

# get all configurations
space = Space(a=Grid(2,4), b=Rand(0,1)).sample(100)
for conf in space:
    print(conf)
all_conf = list(space)

property has_stochastic: Whether the space contains any StochasticExpression

sample(n, seed=None)[source]

Draw random samples from the current space. Please read Space Tutorial.

Parameters

n (int) – number of samples to draw
seed (Optional[Any]) – random seed, defaults to None

Returns

a new Space containing all samples

Return type

tune.concepts.space.spaces.Space

Note

it only applies to StochasticExpression
if has_stochastic() is False, then it will return the original space
After sampling, no StochasticExpression will exist in the new space.

TuningParametersTemplate

class TuningParametersTemplate(raw)[source]

Bases: object

Parameter template to extract tuning parameter expressions from nested data structure

Parameters: raw (Dict[str, Any]) – the dictionary of input parameters.

Note

Please use to_template() to initialize this class.

# common cases
to_template(dict(a=1, b=1))
to_template(dict(a=Rand(0, 1), b=1))

# expressions may nest in dicts or arrays
template = to_template(
    dict(a=dict(x1=Rand(0, 1), x2=Rand(3,4)), b=[Grid("a", "b")]))

assert [Rand(0, 1), Rand(3, 4), Grid("a", "b")] == template.params
assert dict(
    p0=Rand(0, 1), p1=Rand(3, 4), p2=Grid("a", "b")
) == template.params_dict
assert dict(a=1, x2=3), b=["a"]) == template.fill([1, 3, "a"])
assert dict(a=1, x2=3), b=["a"]) == template.fill_dict(
    dict(p2="a", p1=3, p0=1)
)

concat(other)[source]

Concatenate with another template and generate a new template.

Note

The other template must not have any key existed in this template, otherwise ValueError will be raised

Returns: the merged template
Parameters: other (tune.concepts.space.parameters.TuningParametersTemplate) –
Return type: tune.concepts.space.parameters.TuningParametersTemplate

static decode(data)[source]

Retrieve the template from a base64 string

Parameters: data (str) –
Return type: tune.concepts.space.parameters.TuningParametersTemplate

property empty: bool: Whether the template contains any tuning expression

encode()[source]

Convert the template to a base64 string

Return type: str

fill(params)[source]

Fill the original data structure with values

Parameters

params (List[Any]) – the list of values to be filled into the original data structure, in depth-first order
copy – whether to return a deeply copied paramters, defaults to False

Returns

the original data structure filled with values

Return type

Dict[str, Any]

fill_dict(params)[source]

Fill the original data structure with dictionary of values

Parameters

params (Dict[str, Any]) – the dictionary of values to be filled into the original data structure, keys must be p0, p1, p2, …
copy – whether to return a deeply copied paramters, defaults to False

Returns

the original data structure filled with values

Return type

Dict[str, Any]

property has_grid: bool: Whether the template contains grid expressions

property has_stochastic: bool: Whether the template contains stochastic expressions

property params: List[tune.concepts.space.parameters.TuningParameterExpression]: Get all tuning parameter expressions in depth-first order

property params_dict: Dict[str, tune.concepts.space.parameters.TuningParameterExpression]: Get all tuning parameter expressions in depth-first order, with correspondent made-up new keys p0, p1, p2, …

product_grid()[source]

cross product all grid parameters

Yield: new templates with the grid paramters filled
Return type: Iterable[tune.concepts.space.parameters.TuningParametersTemplate]

assert [dict(a=1,b=Rand(0,1)), dict(a=2,b=Rand(0,1))] ==                 list(to_template(dict(a=Grid(1,2),b=Rand(0,1))).product_grid())

sample(n, seed=None)[source]

sample all stochastic parameters

Parameters

n (int) – number of samples, must be a positive integer
seed (Optional[Any]) – random seed defaulting to None. It will take effect if it is not None.

Yield

new templates with the grid paramters filled

Return type

Iterable[tune.concepts.space.parameters.TuningParametersTemplate]

assert [dict(a=1.1,b=Grid(0,1)), dict(a=1.5,b=Grid(0,1))] ==                 list(to_template(dict(a=Rand(1,2),b=Grid(0,1))).sample(2,0))

property simple_value: Dict[str, Any]: If the template contains no tuning expression, it’s simple and it will return parameters dictionary, otherwise, ValueError will be raised

property template: Dict[str, Any]: The template dictionary, all tuning expressions will be replaced by None

Grid

class Grid(*args)[source]

Bases: tune.concepts.space.parameters.TuningParameterExpression

Grid search, every value will be used. Please read Space Tutorial.

Parameters: args (Any) – values for the grid search

Choice

class Choice(*args)[source]

Bases: tune.concepts.space.parameters.StochasticExpression

A random choice of values. Please read Space Tutorial.

Parameters: args (Any) – values to choose from

generate(seed=None)[source]

Return a randomly chosen value.

Parameters: seed (Optional[Any]) – if set, it will be used to call seed() , defaults to None
Return type: Any

property jsondict: Dict[str, Any]: Dict representation of the expression that is json serializable

property values: List[Any]: values to choose from

TransitionChoice

class TransitionChoice(*args)[source]

Bases: tune.concepts.space.parameters.Choice

An ordered random choice of values. Please read Space Tutorial.

Parameters: args (Any) – values to choose from

property jsondict: Dict[str, Any]: Dict representation of the expression that is json serializable

Rand

class Rand(low, high, q=None, log=False, include_high=True)[source]

Bases: tune.concepts.space.parameters.RandBase

Continuous uniform random variables. Please read Space Tutorial.

Parameters

low (float) – range low bound (inclusive)
high (float) – range high bound (exclusive)
q (Optional[float]) – step between adjacent values, if set, the value will be rounded using q, defaults to None
log (bool) – whether to do uniform sampling in log space, defaults to False. If True, low must be positive and lower values get higher chance to be sampled
include_high (bool) –

generate(seed=None)[source]

Return a randomly chosen value.

Parameters: seed (Optional[Any]) – if set, it will be used to call seed() , defaults to None
Return type: float

property jsondict: Dict[str, Any]: Dict representation of the expression that is json serializable

RandInt

class RandInt(low, high, q=1, log=False, include_high=True)[source]

Bases: tune.concepts.space.parameters.RandBase

Uniform distributed random integer values. Please read Space Tutorial.

Parameters

low (int) – range low bound (inclusive)
high (int) – range high bound (exclusive)
log (bool) – whether to do uniform sampling in log space, defaults to False. If True, low must be >=1 and lower values get higher chance to be sampled
q (int) –
include_high (bool) –

generate(seed=None)[source]

Return a randomly chosen value.

Parameters: seed (Optional[Any]) – if set, it will be used to call seed() , defaults to None
Return type: float

property jsondict: Dict[str, Any]: Dict representation of the expression that is json serializable

General Non-Iterative Problems

suggest_for_noniterative_objective(objective, space, df=None, df_name='__tune__df_', temp_path='', partition_keys=None, top_n=1, local_optimizer=None, logger=None, monitor=None, stopper=None, stop_check_interval=None, distributed=None, shuffle_candidates=True, execution_engine=None, execution_engine_conf=None)[source]

Given non-iterative objective, space and (optional) dataframe, suggest the best parameter combinations.

Important

Please read Non-Iterative Tuning Guide

Parameters

objective (Any) – a simple python function or NonIterativeObjectiveFunc compatible object, please read Non-Iterative Objective Explained
space (tune.concepts.space.spaces.Space) – search space, please read Space Tutorial
df (Optional[Any]) – Pandas, Spark, Dask or any dataframe that can be converted to Fugue DataFrame, defaults to None
df_name (str) – dataframe name, defaults to the value of TUNE_DATASET_DF_DEFAULT_NAME
temp_path (str) – temp path for serialized dataframe partitions. It can be empty if you preset using TUNE_OBJECT_FACTORY.set_temp_path(). For details, read TuneDataset Tutorial, defaults to “”
partition_keys (Optional[List[str]]) – partition keys for df, defaults to None. For details, please read TuneDataset Tutorial
top_n (int) – number of best results to return, defaults to 1. If <=0 all results will be returned
local_optimizer (Optional[Any]) – an object that can be converted to NonIterativeObjectiveLocalOptimizer, please read Non-Iterative Optimizers, defaults to None
logger (Optional[Any]) – MetricLogger object or a function producing it, defaults to None
monitor (Optional[Any]) – realtime monitor, defaults to None. Read Monitoring Guide
stopper (Optional[Any]) – early stopper, defaults to None. Read Early Stopping Guide
stop_check_interval (Optional[Any]) – an object that can be converted to timedelta, defaults to None. For details, read to_timedelta()
distributed (Optional[bool]) – whether to use the exeuction engine to run different trials distributedly, defaults to None. If None, it’s equal to True.
shuffle_candidates (bool) – whether to shuffle the candidate configurations, defaults to True. This is no effect on final result.
execution_engine (Optional[Any]) – Fugue ExecutionEngine like object, defaults to None. If None, NativeExecutionEngine will be used, the task will be running on local machine.
execution_engine_conf (Optional[Any]) – Parameters like object, defaults to None

Returns

a list of best results

Return type

List[tune.concepts.flow.report.TrialReport]

optimize_noniterative(objective, dataset, optimizer=None, distributed=None, logger=None, monitor=None, stopper=None, stop_check_interval=None)[source]

Parameters

objective (Any) –
dataset (tune.concepts.dataset.TuneDataset) –
optimizer (Optional[Any]) –
distributed (Optional[bool]) –
logger (Optional[Any]) –
monitor (Optional[Any]) –
stopper (Optional[Any]) –
stop_check_interval (Optional[Any]) –

Return type

tune.concepts.dataset.StudyResult

Level 2 Optimizers

Hyperopt

class HyperoptLocalOptimizer(max_iter, seed=0, kwargs_func=None)[source]

Bases: tune.noniterative.objective.NonIterativeObjectiveLocalOptimizer

Parameters

max_iter (int) –
seed (int) –
kwargs_func (Optional[Callable[[tune.noniterative.objective.NonIterativeObjectiveFunc, tune.concepts.flow.trial.Trial], Dict[str, Any]]]) –

run(func, trial, logger)[source]

Parameters

func (tune.noniterative.objective.NonIterativeObjectiveFunc) –
trial (tune.concepts.flow.trial.Trial) –
logger (Any) –

Return type

tune.concepts.flow.report.TrialReport

Optuna

class OptunaLocalOptimizer(max_iter, create_study=None)[source]

Bases: tune.noniterative.objective.NonIterativeObjectiveLocalOptimizer

Parameters

max_iter (int) –
create_study (Optional[Callable[[], optuna.study.study.Study]]) –

run(func, trial, logger)[source]

Parameters

func (tune.noniterative.objective.NonIterativeObjectiveFunc) –
trial (tune.concepts.flow.trial.Trial) –
logger (Any) –

Return type

tune.concepts.flow.report.TrialReport

General Iterative Problems

Successive Halving

suggest_by_sha(objective, space, plan, train_df=None, temp_path='', partition_keys=None, top_n=1, monitor=None, distributed=None, execution_engine=None, execution_engine_conf=None)[source]

Parameters

objective (Any) –
space (tune.concepts.space.spaces.Space) –
plan (List[Tuple[float, int]]) –
train_df (Optional[Any]) –
temp_path (str) –
partition_keys (Optional[List[str]]) –
top_n (int) –
monitor (Optional[Any]) –
distributed (Optional[bool]) –
execution_engine (Optional[Any]) –
execution_engine_conf (Optional[Any]) –

Return type

List[tune.concepts.flow.report.TrialReport]

optimize_by_sha(objective, dataset, plan, checkpoint_path='', distributed=None, monitor=None)[source]

Parameters

objective (Any) –
dataset (tune.concepts.dataset.TuneDataset) –
plan (List[Tuple[float, int]]) –
checkpoint_path (str) –
distributed (Optional[bool]) –
monitor (Optional[Any]) –

Return type

tune.concepts.dataset.StudyResult

Hyperband

suggest_by_hyperband(objective, space, plans, train_df=None, temp_path='', partition_keys=None, top_n=1, monitor=None, distributed=None, execution_engine=None, execution_engine_conf=None)[source]

Parameters

objective (Any) –
space (tune.concepts.space.spaces.Space) –
plans (List[List[Tuple[float, int]]]) –
train_df (Optional[Any]) –
temp_path (str) –
partition_keys (Optional[List[str]]) –
top_n (int) –
monitor (Optional[Any]) –
distributed (Optional[bool]) –
execution_engine (Optional[Any]) –
execution_engine_conf (Optional[Any]) –

Return type

List[tune.concepts.flow.report.TrialReport]

optimize_by_hyperband(objective, dataset, plans, checkpoint_path='', distributed=None, monitor=None)[source]

Parameters

objective (Any) –
dataset (tune.concepts.dataset.TuneDataset) –
plans (List[List[Tuple[float, int]]]) –
checkpoint_path (str) –
distributed (Optional[bool]) –
monitor (Optional[Any]) –

Return type

tune.concepts.dataset.StudyResult

Continuous ASHA

suggest_by_continuous_asha(objective, space, plan, train_df=None, temp_path='', partition_keys=None, top_n=1, monitor=None, execution_engine=None, execution_engine_conf=None)[source]

Parameters

objective (Any) –
space (tune.concepts.space.spaces.Space) –
plan (List[Tuple[float, int]]) –
train_df (Optional[Any]) –
temp_path (str) –
partition_keys (Optional[List[str]]) –
top_n (int) –
monitor (Optional[Any]) –
execution_engine (Optional[Any]) –
execution_engine_conf (Optional[Any]) –

Return type

List[tune.concepts.flow.report.TrialReport]

optimize_by_continuous_asha(objective, dataset, plan, checkpoint_path='', always_checkpoint=False, study_early_stop=None, trial_early_stop=None, monitor=None)[source]

Parameters

objective (Any) –
dataset (tune.concepts.dataset.TuneDataset) –
plan (List[Tuple[float, int]]) –
checkpoint_path (str) –
always_checkpoint (bool) –
study_early_stop (Optional[Callable[[List[Any], List[tune.iterative.asha.RungHeap]], bool]]) –
trial_early_stop (Optional[Callable[[tune.concepts.flow.report.TrialReport, List[tune.concepts.flow.report.TrialReport], List[tune.iterative.asha.RungHeap]], bool]]) –
monitor (Optional[Any]) –

Return type

tune.concepts.dataset.StudyResult

For Scikit-Learn

sk_space(model, **params)[source]

Parameters

model (str) –
params (Dict[str, Any]) –

Return type

tune.concepts.space.spaces.Space

suggest_sk_models_by_cv(space, train_df, scoring, cv=5, temp_path='', feature_prefix='', label_col='label', save_model=False, partition_keys=None, top_n=1, local_optimizer=None, monitor=None, stopper=None, stop_check_interval=None, distributed=None, execution_engine=None, execution_engine_conf=None)[source]

Parameters

space (tune.concepts.space.spaces.Space) –
train_df (Any) –
scoring (str) –
cv (int) –
temp_path (str) –
feature_prefix (str) –
label_col (str) –
save_model (bool) –
partition_keys (Optional[List[str]]) –
top_n (int) –
local_optimizer (Optional[tune.noniterative.objective.NonIterativeObjectiveLocalOptimizer]) –
monitor (Optional[Any]) –
stopper (Optional[Any]) –
stop_check_interval (Optional[Any]) –
distributed (Optional[bool]) –
execution_engine (Optional[Any]) –
execution_engine_conf (Optional[Any]) –

Return type

List[tune.concepts.flow.report.TrialReport]

suggest_sk_models(space, train_df, test_df, scoring, temp_path='', feature_prefix='', label_col='label', save_model=False, partition_keys=None, top_n=1, local_optimizer=None, monitor=None, stopper=None, stop_check_interval=None, distributed=None, execution_engine=None, execution_engine_conf=None)[source]

Parameters

space (tune.concepts.space.spaces.Space) –
train_df (Any) –
test_df (Any) –
scoring (str) –
temp_path (str) –
feature_prefix (str) –
label_col (str) –
save_model (bool) –
partition_keys (Optional[List[str]]) –
top_n (int) –
local_optimizer (Optional[tune.noniterative.objective.NonIterativeObjectiveLocalOptimizer]) –
monitor (Optional[Any]) –
stopper (Optional[Any]) –
stop_check_interval (Optional[Any]) –
distributed (Optional[bool]) –
execution_engine (Optional[Any]) –
execution_engine_conf (Optional[Any]) –

Return type

List[tune.concepts.flow.report.TrialReport]

For Tensorflow Keras

class KerasTrainingSpec(params, dfs)[source]

Bases: object

Parameters

params (Any) –
dfs (Dict[str, Any]) –

compile_model(**add_kwargs)[source]

Parameters: add_kwargs (Any) –
Return type: keras.engine.training.Model

compute_sort_metric(**add_kwargs)[source]

Parameters: add_kwargs (Any) –
Return type: float

property dfs: Dict[str, Any]

finalize()[source]

Return type: None

fit(**add_kwargs)[source]

Parameters: add_kwargs (Any) –
Return type: keras.callbacks.History

generate_sort_metric(metric)[source]

Parameters: metric (float) –
Return type: float

get_compile_params()[source]

Return type: Dict[str, Any]

get_fit_metric(history)[source]

Parameters: history (keras.callbacks.History) –
Return type: float

get_fit_params()[source]

Return type: Tuple[List[Any], Dict[str, Any]]

get_model()[source]

Return type: keras.engine.training.Model

load_checkpoint(fs, model)[source]

Parameters

fs (fs.base.FS) –
model (keras.engine.training.Model) –

Return type

None

property params: tune.concepts.space.parameters.TuningParametersTemplate

save_checkpoint(fs, model)[source]

Parameters

fs (fs.base.FS) –
model (keras.engine.training.Model) –

Return type

None

keras_space(model, **params)[source]

Parameters

model (Any) –
params (Any) –

Return type

tune.concepts.space.spaces.Space

suggest_keras_models_by_continuous_asha(space, plan, train_df=None, temp_path='', partition_keys=None, top_n=1, monitor=None, execution_engine=None, execution_engine_conf=None)[source]

Parameters

space (tune.concepts.space.spaces.Space) –
plan (List[Tuple[float, int]]) –
train_df (Optional[Any]) –
temp_path (str) –
partition_keys (Optional[List[str]]) –
top_n (int) –
monitor (Optional[Any]) –
execution_engine (Optional[Any]) –
execution_engine_conf (Optional[Any]) –

Return type

List[tune.concepts.flow.report.TrialReport]

suggest_keras_models_by_hyperband(space, plans, train_df=None, temp_path='', partition_keys=None, top_n=1, monitor=None, distributed=None, execution_engine=None, execution_engine_conf=None)[source]

Parameters

space (tune.concepts.space.spaces.Space) –
plans (List[List[Tuple[float, int]]]) –
train_df (Optional[Any]) –
temp_path (str) –
partition_keys (Optional[List[str]]) –
top_n (int) –
monitor (Optional[Any]) –
distributed (Optional[bool]) –
execution_engine (Optional[Any]) –
execution_engine_conf (Optional[Any]) –

Return type

List[tune.concepts.flow.report.TrialReport]

suggest_keras_models_by_sha(space, plan, train_df=None, temp_path='', partition_keys=None, top_n=1, monitor=None, distributed=None, execution_engine=None, execution_engine_conf=None)[source]

Parameters

space (tune.concepts.space.spaces.Space) –
plan (List[Tuple[float, int]]) –
train_df (Optional[Any]) –
temp_path (str) –
partition_keys (Optional[List[str]]) –
top_n (int) –
monitor (Optional[Any]) –
distributed (Optional[bool]) –
execution_engine (Optional[Any]) –
execution_engine_conf (Optional[Any]) –

Return type

List[tune.concepts.flow.report.TrialReport]