Module

Returns a Module object, which can be used to instantiate and interact with the class remotely.

The behavior of Modules (and subclasses thereof) is as follows:

• Any callable public method of the module is intercepted and executed remotely over rpc, with exception of certain functions Python doesn’t make interceptable (e.g. __call__, __init__), and methods of the Module class (e.g. to, fetch, etc.). Properties and private methods are not intercepted, and will be executed locally.

• Any method which executes remotely may be called normally, e.g. model.forward(x), or asynchronously, e.g. key = model.forward.run(x) (which returns a key to retrieve the result with cluster.get(key)), or with run_obj = model.train.remote(x), which runs synchronously but returns a remote object to avoid passing heavy results back over the network.

• Setting attributes, both public and private, will be executed remotely, with the new values only being set in the remote module and not the local one. This excludes any methods or attribtes of the Module class proper (e.g. system or name), which will be set locally.

• Attributes, private properties can be fetched with the remote property, and the full resource can be fetched using .fetch(), e.g. model.remote.weights, model.remote.__dict__, model.fetch().

• When a module is sent to a cluster, it’s public attribtes are serialized, sent over, and repopulated in the remote instance. This means that any changes to the module’s attributes will not be reflected in the remote

Parameters:

• cls – The class to instantiate.

• name (Optional[str]) – Name to give the module object, to be reused later on.

• system (Optional[str or Cluster]) – File system or cluster name. If providing a file system this must be one of: [file, github, sftp, ssh, s3, gs, azure]. We are working to add additional file system support. If providing a cluster, this must be a cluster object or name, and whether the data is saved to the object store or filesystem depends on whether a path is specified.

• env (Optional[str or Env]) – Environment in which the module should live on the cluster, if system is cluster.

• dryrun (bool) – Whether to create the Blob if it doesn’t exist, or load a Blob object as a dryrun. (Default: False)

Returns:

The resulting module.

Return type:

Module

Example - creating a module by defining an rh.Module subclass:

>>> import runhouse as rh
>>> import transformers
>>>
>>> # Sample rh.Module class
>>> class Model(rh.Module):
>>>    def __init__(self, model_id, device="cpu", system=None, env=None):
>>>        # Note that the code here will be run in your local environment prior to being sent to
>>>        # to a cluster. For loading large models/datasets that are only meant to be used remotely,
>>>        # we recommend using lazy initialization (see tokenizer and model attributes below).
>>>        super().__init__(system=system, env=env)
>>>        self.model_id = model_id
>>>        self.device = device
>>>
>>>    @property
>>>    def tokenizer(self):
>>>        # Lazily initialize the tokenizer remotely only when it is needed
>>>        if not hasattr(self, '_tokenizer'):
>>>            self._tokenizer = transformers.AutoTokenizer.from_pretrained(self.model_id)
>>>        return self._tokenizer
>>>
>>>    @property
>>>    def model(self):
>>>        if not hasattr(self, '_model'):
>>>            self._model = transformers.AutoModel.from_pretrained(self.model_id).to(self.device)
>>>        return self._model
>>>
>>>    def predict(self, x):
>>>        x = self.tokenizer(x, return_tensors="pt")
>>>        return self.model(x)

Copied

>>> # Creating rh.Module instance
>>> model = Model(model_id="bert-base-uncased", device="cuda", system="my_gpu", env="my_env")
>>> model.predict("Hello world!")   # Runs on system in env
>>> tok = model.remote.tokenizer    # Returns remote tokenizer
>>> id = model.local.model_id       # Returns local model_id, if any
>>> model_id = model.model_id       # Returns local model_id (not remote)
>>> model.fetch()                   # Returns full remote module, including model and tokenizer
>>>

Copied

Example - creating a Module from an existing class, via the rh.module() factory method:

>>> other_model = Model(model_id="bert-base-uncased", device="cuda").to("my_gpu", "my_env")
>>>
>>> # Another method: Create a module instance from an existing non-Module class using rh.module()
>>> RemoteModel = rh.module(cls=BERTModel, system="my_gpu", env="my_env")
>>> remote_model = RemoteModel(model_id="bert-base-uncased", device="cuda")
>>> remote_model.predict("Hello world!")  # Runs on system in env
>>>
>>> # You can also call remote class methods
>>> other_model = RemoteModel.get_model_size("bert-base-uncased")

Copied

>>> # Loading a module
>>> my_local_module = rh.module(name="~/my_module")
>>> my_s3_module = rh.module(name="@/my_module")

Copied