Selecting a GPU virtual machine vendor for Machine Learn¶
There are many competing cloud services providers, such as:
Really great comparison from GPU computation perspective has been published recently on Maximize your GPU dollars, which I strongly recommend for in depth analysis. The general conclusion is that for stand alone virtual machine both AWS and Google Cloud Platform (GCP for short) offer similar prices. For example CPU instance (2 vCPU, 7.5 GB RAM, 50 GB HDD) - the price is 5 USD for 1 month assuming 1h per day usage (0.16 USD per hour). On the other hand GPU instance with better specification (4 vCPU, 15 GB RAM, 50 GB HDD, GPU Tesla K80) - the price is 21 USD per month (0.66 USD per hour). But when higher computing power of GPU is necessary, the more powerful GPU can be selected such as Tesla P100, then the price rises to 50 USD per month (1.7 USD per hour). Unfortunately, although for the CPU and GPU you are being charged per hour, but storage (HDD or SDD and snapshots) are being charged regardless if the machine is working or stopped. More exact estimations can be found on GCP calculator.
Different model of cloud instance can be found on paperspace. Beside typical virtual machines offered as a Core product, papersapace also offers something called Gradient. Gradient offers only an access to instance over jupyter notebooks. The advantage is limited configuration (almost to zero, it is ready to work in just 3 minutes), and you are charged only for the time you actually use the instance. No additional costs when the instance is stopped. The prices are affordable, and CPU instance starts from 0.05 USD per hour (1 vCPU, 2 GB RAM, 250 SSD). The GPU enabled machines stars from 0.6 USD per hour (Tesla K80, 2 vCPU, 12 GB RAM, 250 GB SSD), and 1.7 USD per hour for better machine (Tesla P100, 4 vCPU, 24 GB RAM, 250 GB SSD), wich are similar to AWS and GCP. The disadvantage of such solution is limited possibilities of installing additional packages, other means of accessing than through jupyter Notebook, or changing instance type between CPU and GPU.
Following the conclusion from the mentioned article and my own experience with AWS, paperspace, and GCP, my choice was GCP service. There are few reasons of such decision, just to mention few:
- competitive prices
- simpler maintains than in AWS
- 300$ for new users to spent during 1 year!
- possibility of modifying instances type (between CPU and GPU)
I would like to comment the last point. My wish was to find a solution, where I would be able to create an inexpensive virtual machine for general purpose (low price, with simple interface, few vCPU cores, price below 0.1$ per hour). But the most important point is, that from time to time I would like to have GPU computing capabilities, especially for Deep Learning purposes. With GCP (as well as in AWS) I can create a cheap CPU instance, where I would make regular work, and code development and when GPU would be necessary, all I need to do is to stop instance, and then add GPU to its configuration. This way you can have a flexible solution with GPU, for which you will pay only on time slots when it is really needed.
VM instance creation on Google Cloud¶
Here I will assume that you already have a GCP account, if not please find any tutorial on web or GCP website.
First we need to open Console of GCP and then go to Compute Engine. Then open "VM instances" - where you can add, remove or edit your virtual machines.
- First click create virtual machine button:
- Then there are multiple choices you can make to your machine: name, number of CPU, adding GPU card (I do not advise adding GPU now, it can be added later), image from which VM should be created. For now I suggest the following choices: Region: us-central(Iowa), CPU: 4, RAM 15GB, Image Deep Learning Image: PyTorch 0.4.1..., increasing storage form 30GB into 50GB. The cost of such configuration is just $0.15 per hour.
Once you have all necessary parameters specified, next clicked on create button. My configuration looked as follows:
If creation of instance was successful, you should see newly created machine in your VM listing: Congratulations, you have your own Virtual Machine somewhere in the cloud.
Connection to VM over SSH with putty¶
SSH key generation¶
Beside convenient control of VM through command line provided in Internet browser (by clicking on SSH label by VM name), more capabilities are possible when accessing it through putty, when for example port forwarding can be enabled.
In order to make such connection possible, first we need to generate a private and public SSH-RSA keys. For that you can use "puttygen", which you can download (as well as putty) here.
In puttygen select key type RSA, and in the filed "Key comment" input your VM identity (you can check it over console with command whoami). It should look like in the image below:
Now you should save your private key (ppk file, you can additionally protect it with password for even better security) - which later on will be necessary in putty authentication configuration. Moreover, you have to save your public key - which will have to be provided in GCP in SSH keys pool.
Your public key should have the following format: ssh-rsa SOME_RANDOM_CHARACTERS== your_Id
As a note, saved public key with puttygen includes some line braking (end lines) which makes it hard to provide directly into GCP ssh-key vault. For that reason the most convenient way is transfer it by copy-paste method, so in the puttygen using mouse just select an output of Public key window (just remember to select all text, it does not fit into output window).
Now you have to paste your public key into GCP. Here it can be done over two means: either apply it to general setting, so all VM machines will utilize this key, or it can be done on each VM separately. For convenient use, I prefer the first method, which is described below.
First you need open Compute Engine/Metadata/SSH Keys, and click edit button, and there you will be able to add new public key. Now just click "add item" and in the new box paste a public key which was generated by "puttygen". Your SSH should look similar as image below: Now just remember to click save button, so public key will be stored permanently.
Now we need to configure our putty for accessing VM. First configure your private key, for that go to Connections/SSH/Auth, and Authentication parameters, then click Browse button and select your previously saved private key. Next go to Sessions and in Host Name provide a public IP of your VM. In case you don't have static IP of your VM, each time you start VM your public IP can be different - you can read IP in your VM console. Next provide some friendly name to your session, for example "my_Cloud_instance", and click save button.
Properly configured putty should look like that: Now just click Open button. In case of connection to new IP, you can get a prompt for accepting new key - just click ok.
New new putty window should appear, where you will be prompted: login as, here you need one more time provide your account name, same as used for generating private/public keys.
If everything worked correctly, you should get access to your VM command line over SSH with putty, congratulation:
In order to avoid each time checking your public IP you can assign a static IP to your instance. In this article you can read how to achieve that.