Set Up

Run on AWS

Create a runner on AWS

To run your models on AWS, you first need to get hold of an EC2 virtual machine (VM). If you already have one, check the requirements below to make sure your VM has sufficient resources.

The second step is to attach your VM runner to Dotscience

Get an AWS Ubuntu VM.

1. Go to the AWS management console. If you have an AWS account, sign in, or else create an account and then sign in.

2. When you are logged into AWS, click on Services on the top navigation. Under Compute, select EC2.

3. Click Launch instance to see a list of available EC2 instances.

4. Now choose an EC2 instance to launch.

(a) Select a machine image. In order to run Dotscience, we recommend chosing a recent release of Ubuntu Server as your operating system.

(b) Given your selected machine image, select an instance type. This decision will be based on your memory and CPU requirements, but must meet the following criteria:

  • The instance type should be at least t2.small (1 vCPU, 2GB of RAM) to meet the memory requirements for running Docker.
  • We recommend that you increase the capacity of the disc from the default 8GB to 30GB, so that there is enough storage for the Docker images you’ll use. To do this, click Configure instance details, go to the tab marked Add storage and edit the value in the Size column. Note that you do not want to add a volume here: just increase the root disc size.

VM requirements:

| Requirement           | Value                 |
|---------------------- |---------------------- |
| Operating System      | 64 bitUbuntu Server           |
| Instance type         | `t2.small` or larger  |
| Memory                | 30GB                  |

(d) Click Review and Launch to see a summary, then click Launch on that page. Follow the prompt to either create a key pair, or use an existing one. The key pair allows you to connect to your instance securely. If creating a new one, make sure you remember its name and location.

Once you’ve created or selected your key pair, click Launch Instances. Your instance will spend a few minutes initialising before launching.

(e) Go to Services, select EC2, and click on Running instances to see the status of your new instance.

B: Install Docker on your instance:

Next, SSH into your instance to initialise it for use as your Dotscience runner. You will need:

  • Your key pair, downloaded from the AWS console. It has the name you assigned to it on creation and the file extension .pem.
  • The default user name for the machine image that you used to launch your instance. For an Ubuntu image, the user name is ubuntu.
  • The public DNS name of your runner, which you can copy and paste from the AWS EC2 Console: click on Running Instances to find it, where is is listed in the column Public DNS (IPv4).

1. SSH into your instance from the terminal using the information listed above. Use the syntax:

$ ssh -i <path/to/my-key-pair.pem> <user-name>@<public-dns-name>`

2. If your instance isn’t new, you might want to uninstall any old Docker files that might be on there with the following command. If your VM is brand new, go to step 3.

  $ sudo apt-get remove docker docker-engine

3. Install Docker

Add the GNU Privacy Guard (GPG) key for the official Docker repository:

$ curl -fsSL | sudo apt-key add -

Then add the Docker repository to APT sources:

$ sudo add-apt-repository "deb [arch=amd64] $(lsb_release -cs) stable"

Update the packages on your instance:

$ sudo apt-get update

And now you can download Docker Community Edition (CE):

$ sudo apt-get install -y docker-ce

3. Add your user to the docker group so that you can execute Docker commands without using sudo

$ sudo usermod -a -G docker ec2-user

You need to log out and in again to the instance to activate the new user permissions. Do this by closing the terminal and opening a new one. Then ssh back into the instance with the command from step 1.

Check that Docker is up and running and doesn’t require sudo:

$ docker info

Next, see how to attach your runner to Dotscience.