Den Quick Start

Installing Runhouse

To use Runhouse to launch on-demand clusters, run the following installation command.

!pip install "runhouse"

Account Creation & Login

You can create an account on the run.house website or by calling the login command in Python or CLI.

To login, call runhouse login --sync-secrets in CLI. This will ask you a series of questions on whether to sync local secrets to Runhouse - e.g. your AWS / GCP / Azure secrets; once synced, you can launch compute via Runhouse anywhere you are authenticated with Runhouse.

For more information on Secrets management, refer to the Secrets Tutorial. Secrets are always securely stored in Vault. If you have any questions about Runhouse’s information security policies, please reach out at hello@run.house.

Den Launcher

Instead of launching clusters directly from your local environment, the Den launcher allows you to launch clusters in your own cloud via the Runhouse control plane. We recommend this approach for a couple of reasons:

• Resource Management: Clusters launched through Runhouse are automatically persisted in the Den UI, making it easier to track and manage your entire collection of resources.

• Resource Sharing: Share clusters with your team members, enabling collaborative features like SSH access to shared clusters.

• Resource Monitoring: Monitor cluster activity and logs, and configure autostop settings to terminate clusters after periods of inactivity.

• Distributed Workflows / Production Pipelines: Launch clusters as part of distributed workflows or pipelines without needing to configure cloud credentials in your environment.

To enable the Den launcher, you can set launcher=den in the cluster factory, or update your local runhouse config with launcher: den to apply the setting globally across all subsequent clusters created.

Launching and Saving Clusters

Let’s start by constructing some Runhouse resources that we’d like to save.

import runhouse as rh
cluster = rh.ondemand_cluster(
    name="rh-cluster",
    cpus="4",
    provider="aws"
).up_if_not()

def get_platform(a = 0):
    import platform
    return platform.platform()

remote_get_platform = rh.function(get_platform).to(cluster)

INFO | 2024-05-16 03:51:58.483032 | Because this function is defined in a notebook, writing it out to /Users/donny/code/notebooks/docs/get_platform_fn.py to make it importable. Please make sure the function does not rely on any local variables, including imports (which should be moved inside the function body). This restriction does not apply to functions defined in normal Python files.
INFO | 2024-05-16 03:51:58.493093 | Port 32300 is already in use. Trying next port.
INFO | 2024-05-16 03:51:58.494347 | Forwarding port 32301 to port 32300 on localhost.
INFO | 2024-05-16 03:51:59.587613 | Server rh-cluster is up.
INFO | 2024-05-16 03:51:59.595752 | Copying package from file:///Users/donny/code/notebooks to: rh-cluster
INFO | 2024-05-16 03:52:01.252665 | Sending module get_platform of type <class 'runhouse.resources.functions.function.Function'> to rh-cluster

You can save the resources we created above to your Den account with the save method:

cluster.save()
remote_get_platform.save()

Reloading

Once saved, resources can be reloaded from any environment in which you are logged into. For instance, if you are running this in a Colab notebook, you can jump into your terminal, call runhouse login, and then reconstruct and run the function on the cluster with the following Python script:

import runhouse as rh

if __name__ == "__main__":
    reloaded_fn = rh.function(name="get_platform")
    print(reloaded_fn())

The name used to reload the function is the method name by default. You can customize a function name using the following syntax:

remote_get_platform = rh.function(fn=get_platform, name="my_function").to(cluster)

Sharing

You can also share your resource with collaborators, and choose which level of access to give. Once shared, they will be able to see the resource in their dashboard as well, and be able to load and use the shared resource. They’ll need to load the resource using its full name, which includes your username (/your_username/get_platform).

remote_get_platform.share(
    users=["teammate1@email.com"],
    access_level="write",
)

Web UI

After saving your resources, you can log in and see them on your Den dashboard, labeled as /<username>/rh-cluster and /<username>/get_platform.

Clicking into the resource provides information about your resource. You can view the resource metadata, previous versions, and activity, or add a description to the resource.

Dive Deeper

Check on more in-depth tutorials on:

• Resource Management

• Secrets Management