spotty

Spotty drastically simplifies training of deep learning models on AWSand GCP:

• it makes training on GPU instances as simple as training on your local machine
• it automatically manages all necessary cloud resources including images, volumes, snapshots and SSH keys
• it makes your model trainable in the cloud by everyone with a couple of commands
• it uses tmux to easily detach remote processes from their terminals
• it saves you up to 70% of the costs by using AWS Spot Instancesand GCP Preemtible VMs

Documentation

• See the documentation page.
• Read thisarticle on Medium for a real-world example.

Installation

Requirements:

• Python >=3.6
• AWS CLI (see Installing the AWS Command Line Interface)if you're using AWS
• Google Cloud SDK (see Installing Google Cloud SDK)if you're using GCP

Use pip to install or upgrade Spotty:

$ pip install -U spotty

Get Started

1. Prepare a spotty.yaml file and put it to the root directory of your project:

   • See the file specification here.
   • Read thisarticle for a real-world example.

2. Start an instance:

   $ spotty start

   It will run a Spot Instance, restore snapshots if any, synchronize the project with the running instanceand start the Docker container with the environment.

3. Train a model or run notebooks.

   To connect to the running container via SSH, use the following command:

   $ spotty sh

   It runs a tmux session, so you can always detach this session usingCtrl + b, then d combination of keys. To be attached to that session later, just use thespotty sh command again.

   Also, you can run your custom scripts inside the Docker container using the spotty run <SCRIPT_NAME> command. Read moreabout custom scripts in the documentation:Configuration: "scripts" section.

Contributions

Any feedback or contributions are welcome! Please check out the guidelines.

License

MIT License