You are viewing v0.0.12 version. Click here to see docs for the latest stable version.

Function

A Function is a portable code block that can be sent to remote hardware to run as a subroutine or service. It is comprised of the entrypoint, system (Cluster), and requirements necessary to run it.

Function Factory Method

runhouse.function(fn: Optional[Union[str, Callable]] = None, name: Optional[str] = None, system: Optional[Union[str, Cluster]] = None, env: Optional[Union[str, List[str], Env]] = None, resources: Optional[dict] = None, dryrun: bool = False, load_secrets: bool = False, serialize_notebook_fn: bool = False, reqs: Optional[List[str]] = None, setup_cmds: Optional[List[str]] = None)[source]

Builds an instance of Function.

Parameters:

fn (Optional[str or Callable]) – The function to execute on the remote system when the function is called.
name (Optional[str]) – Name of the Function to create or retrieve. This can be either from a local config or from the RNS.
system (Optional[str or Cluster]) – Hardware (cluster) on which to execute the Function. This can be either the string name of a Cluster object, or a Cluster object.
env (Optional[List[str] or Env or str]) – List of requirements to install on the remote cluster, or path to the requirements.txt file, or Env object or string name of an Env object.
resources (Optional[dict]) – Optional number (int) of resources needed to run the Function on the Cluster. Keys must be num_cpus and num_gpus.
dryrun (bool) – Whether to create the Function if it doesn’t exist, or load the Function object as a dryrun. (Default: False)
load_secrets (bool) – Whether or not to send secrets; only applicable if dryrun is set to False. (Default: False)
serialize_notebook_fn (bool) – If function is of a notebook setting, whether or not to serialized the function. (Default: False)

Returns:

The resulting Function object.

Return type:

Function

Example

>>> import runhouse as rh

>>> cluster = rh.ondemand_cluster(name="my_cluster")
>>> def sum(a, b):
>>>    return a + b

>>> summer = rh.function(fn=sum, name="my_func").to(cluster, env=['requirements.txt']).save()

>>> # using the function
>>> res = summer(5, 8)  # returns 13

>>> # Load function from above
>>> reloaded_function = rh.function(name="my_func")

Function Class

class runhouse.Function(fn_pointers: Optional[Tuple] = None, name: Optional[str] = None, system: Optional[Cluster] = None, env: Optional[Env] = None, dryrun: bool = False, access: Optional[str] = None, resources: Optional[dict] = None, **kwargs)[source]

__init__(fn_pointers: Optional[Tuple] = None, name: Optional[str] = None, system: Optional[Cluster] = None, env: Optional[Env] = None, dryrun: bool = False, access: Optional[str] = None, resources: Optional[dict] = None, **kwargs)[source]

Runhouse Function object. It is comprised of the entrypoint, system/cluster, and dependencies necessary to run the service.

Note

To create a Function, please use the factory method function().

classmethod from_config(config: dict, dryrun: bool = False)[source]: Create a Function object from a config dictionary.

get(run_key)[source]

Get the result of a Function call that was submitted as async using run.

Parameters:: run_key – A single or list of runhouse run_key strings returned by a Function.remote() call. The ObjectRefs must be from the cluster that this Function is running on.

Example

>>> remote_fn = rh.function(local_fn).to(gpu)
>>> remote_fn_run = remote_fn.run()
>>> remote_fn.get(remote_fn_run.name)

get_or_call(run_name: str, load=True, local=True, *args, **kwargs) → Any[source]

Check if object already exists on cluster or rns, and if so return the result. If not, run the function. Keep in mind this can be called with any of the usual method call modifiers - remote=True, run_async=True, stream_logs=False, etc.

Parameters:

run_name (Optional[str]) – Name of a particular run for this function. If not provided will use the function’s name.
load (bool) – Whether to load the name from the RNS if it exists.
*args – Arguments to pass to the function for the run (relevant if creating a new run).
**kwargs – Keyword arguments to pass to the function for the run (relevant if creating a new run).

Returns:

Result of the Run

Return type:

Any

Example

>>> # previously, remote_fn.run(arg1, arg2, run_name="my_async_run")
>>> remote_fn.get_or_call()

http_url(curl_command=False, *args, **kwargs) → str[source]: Return the endpoint needed to run the Function on the remote cluster, or provide the curl command if requested.

keep_warm(autostop_mins=None)[source]

Keep the system warm for autostop_mins. If autostop_mins is None or -1, keep warm indefinitely.

Example

>>> # keep gpu warm for 30 mins
>>> remote_fn = rh.function(local_fn).to(gpu)
>>> remote_fn.keep_warm(autostop_mins=30)

map(*args, **kwargs)[source]

Map a function over a list of arguments.

Example

>>> def local_sum(arg1, arg2, arg3):
>>>     return arg1 + arg2 + arg3
>>>
>>> remote_fn = rh.function(local_fn).to(gpu)
>>> remote_fn.map([1, 2], [1, 4], [2, 3])
>>> # output: [4, 9]

notebook(persist=False, sync_package_on_close=None, port_forward=8888)[source]: Tunnel into and launch notebook from the system.

remote(*args, local=True, **kwargs)[source]

Helper property to allow for access to remote properties, both public and private. Returning functions is not advised.

Example

>>> my_module.remote.my_property
>>> my_module.remote._my_private_property
>>> my_module.remote.size = 14

send_secrets(providers: Optional[List[str]] = None)[source]

Send secrets to the system.

Example

>>> remote_fn.send_secrets(providers=["aws", "lambda"])

starmap(args_lists, **kwargs)[source]

Like map() except that the elements of the iterable are expected to be iterables that are unpacked as arguments. An iterable of [(1,2), (3, 4)] results in [func(1,2), func(3,4)].

Example

>>> arg_list = [(1,2), (3, 4)]
>>> # runs the function twice, once with args (1, 2) and once with args (3, 4)
>>> remote_fn.starmap(arg_list)

to(system: Optional[Union[str, Cluster]] = None, env: Union[List[str], Env] = [], reqs: Optional[List[str]] = None, setup_cmds: Optional[List[str]] = [], force_install: bool = False)[source]

Set up a Function and Env on the given system.

See the args of the factory method function() for more information.

Example

>>> rh.function(fn=local_fn).to(gpu_cluster)
>>> rh.function(fn=local_fn).to(system=gpu_cluster, env=my_conda_env)

Previous
Resource

Next
Cluster