Create a runner on AWS
To run your models on AWS, you first need to get hold of an EC2 virtual machine (VM). If you already have one, check the requirements below to make sure your VM has sufficient resources.
The second step is to attach your VM runner to Dotscience
Get an AWS Ubuntu VM.
1. Go to the AWS management console. If you have an AWS account, sign in, or else create an account and then sign in.
2. When you are logged into AWS, click on Services on the top navigation. Under Compute, select EC2.
3. Click Launch instance to see a list of available EC2 instances.
4. Now choose an EC2 instance to launch.
(a) Select a machine image. In order to run Dotscience, we recommend chosing a recent release of Ubuntu Server as your operating system.
(b) Given your selected machine image, select an instance type. This decision will be based on your memory and CPU requirements, but must meet the following criteria:
- The instance type should be at least
t2.small(1 vCPU, 2GB of RAM) to meet the memory requirements for running Docker.
- We recommend that you increase the capacity of the disc from the default 8GB to 30GB, so that there is enough storage for the Docker images you’ll use. To do this, click Configure instance details, go to the tab marked Add storage and edit the value in the Size column. Note that you do not want to add a volume here: just increase the root disc size.
| Requirement | Value | |---------------------- |---------------------- | | Operating System | 64 bitUbuntu Server | | Instance type | `t2.small` or larger | | Memory | 30GB |
(d) Click Review and Launch to see a summary, then click Launch on that page. Follow the prompt to either create a key pair, or use an existing one. The key pair allows you to connect to your instance securely. If creating a new one, make sure you remember its name and location.
Once you’ve created or selected your key pair, click Launch Instances. Your instance will spend a few minutes initialising before launching.
(e) Go to Services, select EC2, and click on Running instances to see the status of your new instance.
B: Install Docker on your instance:
Next, SSH into your instance to initialise it for use as your Dotscience runner. You will need:
- Your key pair, downloaded from the AWS console. It has the name you assigned to it on creation and the file extension
- The default user name for the machine image that you used to launch your instance. For an Ubuntu image, the user name is
- The public DNS name of your runner, which you can copy and paste from the AWS EC2 Console: click on Running Instances to find it, where is is listed in the column Public DNS (IPv4).
1. SSH into your instance from the terminal using the information listed above. Use the syntax:
$ ssh -i <path/to/my-key-pair.pem> <user-name>@<public-dns-name>`
2. If your instance isn’t new, you might want to uninstall any old Docker files that might be on there with the following command. If your VM is brand new, go to step 3.
$ sudo apt-get remove docker docker-engine docker.io
3. Install Docker
Add the GNU Privacy Guard (GPG) key for the official Docker repository:
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
Then add the Docker repository to APT sources:
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
Update the packages on your instance:
$ sudo apt-get update
And now you can download Docker Community Edition (CE):
$ sudo apt-get install -y docker-ce
3. Add your user to the
docker group so that you can execute Docker commands without using
$ sudo usermod -a -G docker ec2-user
You need to log out and in again to the instance to activate the new user permissions. Do this by closing the terminal and opening a new one. Then
ssh back into the instance with the command from step 1.
Check that Docker is up and running and doesn’t require
$ docker info
Next, see how to attach your runner to Dotscience.