Folder

A Folder represents a specified location for organizing and storing other Runhouse primitives across various systems.

Folder Factory Method

Creates a Runhouse folder object, which can be used to interact with the folder at the given path.

Parameters:

name (Optional[str]) – Name to give the folder, to be re-used later on.
path (Optional[str or Path]) – Path (or path) that the folder is located at.
system (Optional[str or Cluster]) – File system or cluster name. If prpre-oviding a file system this must be one of: [file, s3, gs].
load_from_den (bool) – Whether to try loading the Folder resource from Den. (Default: True)
dryrun (bool) – Whether to create the Folder if it doesn’t exist, or load a Folder object as a dryrun. (Default: False)

Returns:

The resulting folder.

Return type:

Folder

Example

>>> import runhouse as rh
>>> rh.folder(name='training_imgs', path='remote_directory/images', system='s3').save()

>>> # Load folder from above
>>> reloaded_folder = rh.folder(name="training_imgs")

Folder Class

class runhouse.Folder(name: str | None = None, path: str | None = None, system: str | Cluster | None = None, dryrun: bool = False, **kwargs)[source]

__init__(name: str | None = None, path: str | None = None, system: str | Cluster | None = None, dryrun: bool = False, **kwargs)[source]

Runhouse Folder object.

Note

To build a folder, please use the factory method folder().

contains(name_or_path: str) → bool[source]

Whether path exists locally inside a folder.

Parameters:: name_or_path (str) – Name or path of folder to check if it exists inside the folder.

Example

>>> my_folder = rh.folder("local/folder/path")
>>> in_folder = my_folder.contains("filename")

exists_in_system()[source]

Whether the folder exists in the filesystem.

Example

>>> exists_on_system = my_folder.exists_in_system()

static from_config(config: Dict, dryrun: bool = False, _resolve_children: bool = True)[source]

Load or construct resource from config.

Parameters:

config (Dict) – Resource config.
dryrun (bool, optional) – Whether to construct resource or load as dryrun (Default: False)

property fsspec_url: Generate the FSSpec style URL using the file system and path of the folder

get(name, mode: str = 'rb', encoding: str | None = None)[source]

Returns the contents of a file as a string or bytes.

Parameters:

name (str) – Name of file to get contents of.
mode (str, optional) – Mode for opening the file. (Default: "rb")
encoding (str, optional) – Encoding for opening the file. (Default: None)

Example

>>> contents = my_folder.get(file_name)

is_local()[source]

Whether the folder is on the local filesystem.

Example

>>> is_local = my_folder.is_local()

locate(name_or_path) → Tuple[str, str][source]

Locate the local path of a Folder given an rns path.

Parameters:: name_or_path (str) – Name or path of folder to locate inside the folder.

Example

>>> my_folder = rh.folder("local/folder/path")
>>> local_path = my_folder.locate("file_name")

ls(full_paths: bool = True, sort: bool = False) → List[source]

List the contents of the folder.

Parameters:

full_paths (Optional[bool]) – Whether to list the full paths of the folder contents. Defaults to True.
sort (Optional[bool]) – Whether to sort the folder contents by time modified. Defaults to False.

mkdir()[source]: Create the folder in specified file system if it doesn’t already exist.

mv(system: str | Cluster, path: str | None, overwrite: bool = True) → None[source]

Move the folder to a new filesystem or cluster.

Parameters:

system (str or Cluster) – Filesystem or cluster to move the folder to.
path (str) – Path to move the folder to.
overwrite (bool, optional) – Whether to override if a file already exists at the destination path. (Default: True)

Example

>>> folder = rh.folder(path="local/path")
>>> folder.mv(my_cluster)
>>> folder.mv("s3", "s3_bucket/path")

open(name, mode: str = 'rb', encoding: str | None = None)[source]

Returns the specified file as a stream (botocore.response.StreamingBody), which must be used as a content manager to be opened.

Parameters:

name (str) – Name of file inside the folder to open.
mode (str, optional) – Mode for opening the file. (Default: "rb")
encoding (str, optional) – Encoding for opening the file. (Default: None)

Example

>>> with my_folder.open('obj_name') as my_file:
>>>        pickle.load(my_file)

put(contents: Dict[str, Any], overwrite: bool = False, mode: str = 'wb')[source]

Put given contents in folder.

Parameters:

contents (Dict[str, Any] or Resource or List[Resource]) – Contents to put in folder. Must be a dict with keys being the file names (without full paths) and values being the file-like objects to write, or a Resource object, or a list of Resources.
overwrite (bool, optional) – Whether to overwrite the existing file if it exists. (Default: False)
mode (str, optional) – Write mode to use. (Default: wb).

Example

>>> my_folder.put(contents={"filename.txt": data})

resources(full_paths: bool = False)[source]

List the resources in the folder.

Parameters:: full_paths (bool, optional) – Whether to list the full path or relative path. (Default: False)

Example

>>> resources = my_folder.resources()

rm(contents: List | None = None, recursive: bool = True)[source]

Delete a folder from the file system. Optionally provide a list of folder contents to delete.

Parameters:

contents (Optional[List]) – Specific contents to delete in the folder. If None, removes the entire folder. (Default: None)
recursive (bool, optional) – Delete the folder itself (including all its contents). (Default: True)

Example

>>> my_folder.rm()

property rns_address: Traverse up the filesystem until reaching one of the directories in rns_base_folders, then compute the relative path to that.

to(system: str | Cluster, path: str | Path | None = None)[source]

Copy the folder to a new filesystem or cluster. Currently supported: here, file, gs, s3, or a cluster.

Parameters:

system (str or Cluster) – Filesystem or cluster to move the folder to.
path (str, optional) – Path to move the folder to.

Example

>>> local_folder = rh.folder(path="/my/local/folder")
>>> s3_folder = local_folder.to("s3")

Previous
Module

Next
Secrets