Set Up

Run on Google Cloud Platform

Create a runner on Google Cloud

To run your models on Google Cloud, you first need to get hold of an Google Cloud virtual machine (VM). If you already have one, check the requirements below to make sure your VM has sufficient resources.

The second step is to attach your VM runner to Dotscience

Get an Ubuntu VM instance on Google Cloud.

1. Start by logging in to Google Cloud with your Google credentials. If you don’t have a Google account, click Sign in to create an account and then sign in.

2. When you are signed in, click Console on the top right.

3. On the left navigation, select Compute Engine and then VM instances.

4. At this point, if you haven’t already enabled Google Cloud billing, you need to do so. Click Enable billing and you will be guided through the process of setting up a payment method. You only pay for the compute you use, and there are typically free credits available to new users.

5. Select VM instances from the top of the left navigation. Then click Create to set up a new instance.

6. Make sure the operating system for your VM is Ubuntu, preferably the most recent release. You can change this from the default operating system(Debian GNU) in the section labeled Boot disk.

  • Increase the size of the boot disk from the default (10GB) to at least 30GB.

  • Select an instance with at least 2GB of RAM from the dropdown menu under Machine type.

VM requirements:

| Requirement       | Value                 |
|------------------ |---------------------- |
| Operating System  | 64 bit Ubuntu                 |
| Instance type     | Any with at least 2GB of RAM  |
| Memory            | 30GB +                    |

Optionally, configure further instance settings, such as the Region and Zone. Click Create when you’vee finished setting up, and the instance will boot up.

When your instance is online, it will appear on the VM instances page with a green check mark next to its name.

B: Install Docker on your instance:

1. On the VM instances page of the Google Cloud console,in the column labeled Connect, is a dropdown menu for your instance marked SSH. Select Open in browser window from this dropdown. This will launch a browser windown that resembles a terminal, through which you can issue controls to your VM.

2. If your instance isn’t new, you might want to uninstall any old Docker files that might be on there with the following command. If your VM is brand new, go to step 3.

  $ sudo apt-get remove docker docker-engine docker.io

3. Update the package index:

$ sudo apt-get update

4. Enable apt to use a Docker repository over HTTPS:

  $ sudo apt-get install \
      apt-transport-https \
      ca-certificates \
      curl \
      software-properties-common

5. Add Docker’s GNU Privacy Guard (GPG) key:

  $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

6. Set up the stable Docker repository:

  $ sudo add-apt-repository \
    "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
    $(lsb_release -cs) \
    stable"

7. Update the package index again:

  $ sudo apt-get update

8. Get the latest Docker version:

$ sudo apt-get install docker-ce

9. Now Docker is installed but no users have been added to its group. Add your user to the docker group so that you can execute Docker commands without using sudo. You can see your username in the VM by looking at the command prompt: your name is the part of the prompt before the @ symbol, and will typically have been generated from your Google username. Run the following, substituting in your username:

  $ sudo usermod -a -G docker <username>
  1. You need to log out and in again to the instance to activate the new user permissions. Do this by closing the SSH browser window, and opening a new one as before, by clicking the drop-down menu for your instance marked SSH and selecting Open in browser window .

  2. Check that Docker is up and running and doesn’t require sudo:

    $ docker info

C: Install Dotscience on your VM:

1. Go to the Runners page on Dotscience https://cloud.dotscience.com/runners/. Click Add a runner. Give your runner a name and, optionlly, a description to remind you which machine the runner is.

2. Copy the code snippet from the Runners page. Your snippet will have the following form, but note that your TOKEN will vary:

docker pull quay.io/dotmesh/dotscience-runner:latest && \
docker run --name dotscience-runner -d -e TOKEN=C2B3KKP5CTVUDTJW6FQJ6DIA5E3LXTIKVOSQPM4LEMFH2NFH6XYA==== \
    --restart always -v /var/run/docker.sock:/var/run/docker.sock \
    -v dotscience-task-spool:/spool \
    quay.io/dotmesh/dotscience-runner:latest

3. Verify that the Docker container that Dotscience needs is running. Run docker ps on your Google Cloud instance to see which containers are running. You should see that under CONTAINERS there is a container called dotscience-runner whose status is up.

4. Refresh the Dotscience Runners page. You should see your newly added runner is online.

Next, see how to attach your runner to Dotscience.