Building AI Applications in the Cloud

By Lukas Schneider —

Use SSH, RSYNC, TMUX, and VSCode to develop and deploy AI applications in the cloud

Training Machine Learning models requires expensive hardware that is not affordable to anyone. An NVIDIA RTX4090 costs around $1600 and comes with barely enough memory to run Llama 7B on degraded precision. However, with cloud computing, this hardware can be rented at a fraction of the cost, for however long you need it, and be upgraded at the click of a button. And you can get started right away without installing hardware, drivers, or software. This guide will teach you how to get started building AI applications in the cloud.

First, you need to rent a GPU, which takes a few clicks on gpudeploy.com. Once you have launched your GPU instance, you will usually get access via SSH (Secure Shell), which allows you to run commands over an encrypted connection. Alternatively, gpudeploy enables you to develop and run Python programs on a Jupyter Notebook server, accessible from within your browser.

Connect using Secure Shell

On Linux, you can connect to your GPU server using SSH with a simple command:

ssh <username>@<server-ip> -i <path/to/private-key>

SSH uses key pairs for authentication. The private key stays on your computer, while the public key is stored on the server (you upload it during instance creation). Make sure to never share your private key.

Pro Tip: set up an SSH configuration file to make connecting easier. Create a file at ~/.ssh/config and add:

Host <name>
    HostName <server-ip>
    User <username>
    IdentityFile <path/to/private-key>

Now you can connect by simply typing ssh gpu-server. Note, that whenever the IP address of the server changes, you will need to adapt the configuration file.

Synchronize files with RSYNC

RSYNC is a powerful tool for file synchronization. To copy files from your local machine to the server, use:

rsync -avz <local/folder/> <username>@<server-ip>:<remote/folder/>

Important flags:

  • -a: archive mode, preserves file permissions
  • -v: verbose, shows progress
  • -z: compresses data during transfer

Use --delete to remove files on the destination that don't exist in the source. Be careful, as this can lead to data loss if used incorrectly.

Keep applications running with TMUX

TMUX is a terminal multiplexer that allows you to keep processes running even after disconnecting from SSH. It's essential for long-running tasks like model training. But you should make a habit of running everything in TMUX as it will keep your processes alive when your SSH connection gets interrupted.

Start a new session: tmux new -s <session-name> or simply tmux
Detach: Press CTRL+b, then d
List sessions: tmux list-sessions
Reattach: tmux attach -t <session-name>

Use multiple windows and panes to organize your work efficiently.

Develop Remotely with VSCode Extensions

Visual Studio Code (VSCode) is the most popular code editor and it comes with endless extensions created by both companies and hobbyists. Developing on a remote GPU server with VSCode and its Remote - SSH extension allows you to open a remote folder and work with it just like it’s on your local machine.

  1. Open the VSCode extensions tab and install Remote - SSH.
  2. Open the Command Palette (Ctrl+Shift+P) and select Remote-SSH: Connect to Host....
  3. Enter the SSH configuration details, such as the hostname defined in your ~/.ssh/config.

Once connected, you can open and edit files, run terminal commands, and manage your project directly within VSCode, making remote development smooth and efficient.

Frequently Asked Questions

Do I need a GPU for Machine Learning?

A GPU is highly recommended for Machine Learning as it significantly reduces training times. A GPU can perform many of the mathematical operations needed for ML in parallel, while a CPU will execute them sequentially. An affordable and convenient alternative to buying a GPU is renting one through cloud services.

Which GPU is good for Deep Learning?

The most important factor when choosing a GPU is its memory capacity, as it determines the size and complexity of the models you can train. Popular options include the NVIDIA RTX 4090 (24GB), NVIDIA A100 (40GB), and NVIDIA H100 (80GB). These GPUs provide excellent performance, ample memory, and broad support across deep learning frameworks.

How to get started with AI?

To get started with AI, you should build a strong foundation in linear algebra and statistics. Then, you have to learn Machine Learning basics, understand Neural Networks and Large Language Models.