Non-Iterative Objective

Non-Iterative Objective refers to the objective functions with single iteration. They do not report progress during the execution to get a pruning decision.

Interfaceless

The simplest way to construct a Tune compatible non-iterative objective is to wirte a native python function with type annotations.

from typing import Tuple, Dict, Any

def objective1(a, b) -> float:
    return a**2 + b**2

def objective2(a, b) -> Tuple[float, Dict[str, Any]]:
    return a**2 + b**2, {"metadata":"x"}

If you function as float or Tuple[float, Dict[str, Any]] as output annotation, they are valid non-iterative objectives for tune

Tuple[float, Dict[str, Any]] is to return both the metric and metadata.

The following code demos how it works on the backend to convert your simple functions to tune compatible objects. You normally don’t need to do that by yourself.

from tune import to_noniterative_objective, Trial

f1 = to_noniterative_objective(objective1)
f2 = to_noniterative_objective(objective2, min_better=False)

trial = Trial("id", params=dict(a=1,b=1))
report1 = f1.safe_run(trial)
report2 = f2.safe_run(trial)

print(type(f1))
print(report1.metric, report1.sort_metric, report1.metadata)
print(report2.metric, report2.sort_metric, report2.metadata)

<class 'tune.noniterative.convert._NonIterativeObjectiveFuncWrapper'>
2.0 2.0 {}
2.0 -2.0 {'metadata': 'x'}

Decorator Approach

It is equivalent to use decorator on top of the functions. But now your functions depend on tune package.

from tune import noniterative_objective

@noniterative_objective
def objective_3(a, b) -> float:
    return a**2 + b**2

@noniterative_objective(min_better=False)
def objective_4(a, b) -> Tuple[float, Dict[str, Any]]:
    return a**2 + b**2, {"metadata":"x"}

report3 = objective_3.safe_run(trial)
report4 = objective_4.safe_run(trial)

print(report3.metric, report3.sort_metric, report3.metadata)
print(report4.metric, report4.sort_metric, report4.metadata)

2.0 2.0 {}
2.0 -2.0 {'metadata': 'x'}

Interface Approach

With interface approach, you can access all properties of a trial. Also you can use more flexible logic to generate sort metric.

from tune import NonIterativeObjectiveFunc, TrialReport

class Objective(NonIterativeObjectiveFunc):
    def generate_sort_metric(self, value: float) -> float:
        return - value * 10

    def run(self, trial: Trial) -> TrialReport:
        params = trial.params.simple_value
        metric = params["a"]**2 + params["b"]**2
        return TrialReport(trial, metric, metadata=dict(m="x"))

report = Objective().safe_run(trial)
print(report.metric, report.sort_metric, report.metadata)

2.0 -20.0 {'m': 'x'}

Factory Method

Almost all higher level APIs of tune are using TUNE_OBJECT_FACTORY to convert various objects to NonIterativeObjectiveFunc.

from tune import TUNE_OBJECT_FACTORY

assert isinstance(TUNE_OBJECT_FACTORY.make_noniterative_objective(objective1), NonIterativeObjectiveFunc)
assert isinstance(TUNE_OBJECT_FACTORY.make_noniterative_objective(objective_4), NonIterativeObjectiveFunc)
assert isinstance(TUNE_OBJECT_FACTORY.make_noniterative_objective(Objective()), NonIterativeObjectiveFunc)

That is why in the higher level APIs, you can just pass in a very simple python function as objective but tune is still able to recognize.

Actually you can make it even more flexible by configuring the factory.

def to_obj(obj):
    if obj == "test":
        return to_noniterative_objective(objective1, min_better=False)
    if isinstance(obj, NonIterativeObjectiveFunc):
        return obj
    raise NotImplementedError

TUNE_OBJECT_FACTORY.set_noniterative_objective_converter(to_obj)  # user to_obj to replace the built-in default converter

assert isinstance(TUNE_OBJECT_FACTORY.make_noniterative_objective("test"), NonIterativeObjectiveFunc)

If you customize in this way, then you can pass in test to the higher level tuning APIs, and it will be recognized as a compatible objective.

This is a common approach in Fugue projects. It enables you to use mostly primitive data types to represent what you want to do. For advanced users, if you spend some time on such configuration (one time effort), you will find the code is even simpler and less dependent on fugue and tune.