Deploy Your Machine Learning Model For $5/Month

Disclaimer: This post is not sponsored or supported in any way by Digital Ocean.

Run your Machine Learning model without paying a dime. Do this by receiving $100 for free by using our referral link. If you choose to spend $25 dollars besides the free credits, the referral program gives ML From Scratch an extra $25, which helps the website run smoothly.

Table Of Contents (Click To Scroll)

In four simple steps, you will see exactly how you can cheaply and simply deploy your model to the cloud within.

The Setup:
a. Installing Docker
b. Creating a $5 droplet (also known as virtual machine (VM))
c. Accessing your droplet
Training Our Model
Preparing Our Model For Deployment:
a. Making A Flask Application
b. Running The Flask Application In A Container
Deploying Our Model:
a. Accessing The Online Application
Final Things To Consider

The Setup

The setup is crucial to make this tutorial work. We need to install docker, setup your droplet and make sure you can access it through SSH.

Installing Docker

Note that you cannot use Docker with Windows Home – consider using a local Ubuntu VM by using Hyper-V on Windows instead.

Windows Pro/Enterprise/Education: Sign up and download Docker Desktop on Windows. Once logged in, you can download Docker. Make sure the program is running and that you are logged in locally.

MacOS: Sign up and download Docker Desktop on Mac. Once logged in, you can download Docker. Make sure the program is running and that you are logged in locally.

Linux: Use the three following command to download and start Docker:
1) apt install docker.io, 2) systemctl start docker, and 3) systemctl enable docker. You can login with docker login if you have a registry you want to login in to.

Creating A Droplet

The next step is creating your Digital Ocean account and setting billing up. Remember you get $100 for free if you use our referral link.

After logging in, you want to look in the top right corner, click Create and then Droplets.

For this guide, you only have to modify the parts showed in the screenshots – meaning you will be using Ubuntu 18.04.3 since that is the standard. Start by expanding the plans by clicking Show all plans.

Then choose the $5 plan.

Next, choose a region, preferably the one closest to you or your customers.

We turned on monitoring, so you can see how much RAM, CPU, Disk, Network etc. is being used throughout time. We also chose to use a password for accessing the server, because it was the easiest way, but we recommend spending time setting up a SSH keys since it is much more secure.

As a last option, you can enable snapshot backups that you can revert to if something every goes wrong.

Accessing Your Droplet

After creating your droplet, click on your project name in the Projects tab in the left side menu, then click on your droplet. Here, you can see the ip-address (ipv4) in the upper left corner. Note that the ip-address you see in this article is not online anymore.

Using your ip-address, open Terminal on Mac/Linux and use the following command. You can also use similar commands on Windows, look here for more information. Another good option for Windows is setting up PuTTy.

After having logged in, you will be prompted to enter your current password and then a new password for the root user.

When you are done, you want to run the two following commands to update all the necessary packages from apt and install git for later.

apt update
apt install git

Training Our Model

To deploy a model, we first need to train a model. In this example, we will use the Iris Dataset with a Random Forest model. We start by importing all the packages we need for training.

import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
import joblib

Then we load the iris dataset from scikit-learn into a Pandas dataframe and Pandas series.

data_loader = load_iris()

X_data = data_loader.data
X_columns = data_loader.feature_names
x = pd.DataFrame(X_data, columns=X_columns)

y_data = data_loader.target
y = pd.Series(y_data, name='target')

We always split our dataset into training and testing, so here we use a function from scikit-learn that splits our input x and output y into training and testing.

x_train, x_test, y_train, y_test = train_test_split(
    x, y, test_size=0.2, random_state=42)

Next up is training our model. We used the default parameters for the random forest classifier, and then we input the training dataset into the fit function.

rf = RandomForestClassifier()
rf.fit(x_train, y_train)

After having trained the model, we can make predictions and check how well we perform on the test set. After that, we save the random forest model as a pickle file.

pred = rf.predict(x_test)
score = accuracy_score(pred, y_test)

print(score)

joblib.dump(rf, 'trained_models/iris.pkl')

Note that this is a scikit-learn model, but you could use XGBoost or LightGBM at almost the same speed. Usually, one of those models will perform much better on a more complex and larger dataset.

The only limitation here, is that we are only using a CPU, which means the setup is not suited for running deep learning, since those applications require GPUs and are much more expensive.

Preparing Our Model For Deployment

We need to make a flask application to serve a webpage, that can make requests to our machine learning model on the server, so we can return predictions based on the input from the user on the webpage.

Making A Flask App

We create a Flask application that will run inside a container. There are two routes specified by the @app.route() decorator; one for serving a web page at the route /, and another that acts as an API that both the webpage /predict, or some third party can interact with on a request-reply basis.

The following code snippet is all there is to our main.py, which we in a moment’s time will show you how to containerize. The code makes a service on localhost using port 5000, and it can be run locally by python main.py.

from flask import Flask, request
import pandas as pd
import joblib

app = Flask(__name__, static_url_path='/static')
model = joblib.load('trained_models/iris.pkl')

@app.route('/')
def root():
    return app.send_static_file('index.html')

@app.route('/predict', methods=['POST'])
def predict():
    data = request.get_json(force=True)
    df = pd.DataFrame(data, index=[0])
    prediction = model.predict(df)
    return str(prediction[0])

if __name__ == '__main__':
  app.run(host='0.0.0.0', port=5000, debug=True)

We won’t go into the HTML, CSS or JavaScript files, but we did some basic work using Bootstrap and jQuery to build the webpage and send requests using Ajax.

Check the static folder on GitHub to find out more.

Running The Flask App In A Docker Container

In order for us to run our service in a container, we must make a Dockerfile to make an image. In a Dockerfile, you specify the base image (Python 3.7), copy or add all the necessary files, and then run the container.

FROM python:3.7

COPY ./requirements.txt /app/req.txt

WORKDIR /app

RUN pip install -r req.txt

ADD ./trained_models ./trained_models
ADD ./static ./static
ADD ./main.py main.py

EXPOSE 5000
CMD [ "gunicorn", "--bind", "0.0.0.0:5000", "main:app" ]

We use Gunicorn as a WSGI, since Flask is only for development and needs a stable server, hence the use of Gunicorn.

To build this image using Docker, you would run the following command:

docker build -t iris:1.0 .

Where the -t is for tagging the image with an image name iris and a version 1.0, and we want to build the image in the current working directory, specified by the dot at the end.

The part where it says Successfully built 735e1e847ba6 is where your attention should be, because that is the ID of the image, which you can also get through running a Docker command docker images.

For running the container locally, you can now use the following command. The flag -d means we are running it in detached mode, while the flag -p indicates the port numbers in the format to:from.

docker run -d -p 5000:5000 735e1e847ba6

Go to http://localhost:5000 and check out the application.

Note that we always need to check if we can access our web application when running it through a Docker container, before we get to the deployment. This is important, because then we can always eliminate this if any error occurs later.

Deploying Our Model

You installed git earlier – now, we need to use it on the VM. Start by cloning my repository using the following git command.

git clone https://github.com/casperbh96/iris-deployment.git

After this, you need to change to the directory of the repository you just cloned.

cd iris-deployment

The next thing is modifying the setup.sh file. Please note that you don’t need to modify all the variables, only the one’s where you deviated from this tutorial.

Modify the LISTEN variable on line 6 by inserting the IP address of your own server LISTEN=<ip_address>:80
Modify the LOCATIONS variable on line 10. This variable refers to how to access your web application through the URL, so you could use something like /webapp instead of the default /.
Modify the PROXIES variable on line 11 if your container runs on another port than port 5000.
Modify the ports variable if your container runs on another port than port 5000. The mapping is in the format to:from, which means the first port number is mapped to local host, while the second port number is mapped to the port number used in the container.
Modify the image_name variable to be the name of your image. This is referred to as repository if you run the command docker images.
Modify the image_version variable if needed.

#!/bin/bash
reinstall_docker_nginx=true
configure_nginx=true
kill_docker_images=false

LISTEN=64.227.72.137:80
SERVER_NAME=localhost

# Arrays
declare -a LOCATIONS=("/")
declare -a PROXIES=("http://localhost:5000/")
declare -a image_name=("iris")
declare -a image_version=("1.0")
declare -a ports=("5000:5000")

#-------------------------------------------------

####
#### Install all necessary packages
####

if [ "$reinstall_docker_nginx" = true ]
then
	# Remove previous docker
	apt update
	apt-get -y purge docker docker-engine docker.io

	# Install and enable docker
	apt-get -y install docker.io
	systemctl start docker
	systemctl enable docker

	# Remove previous nginx
	apt-get -y purge nginx nginx-common nginx-full

	# Install and enable nginx
	apt-get -y install nginx
	systemctl enable nginx
	systemctl start nginx
	ufw enable
fi

#### 
####Setup nginx config
####

# get length of an array
n_locations=${#LOCATIONS[@]}

# insert at
LINE_NUMBER=63

if [ "$configure_nginx" = true ]
then
	sed -i ''"${LINE_NUMBER}"'i
	server {
		listen '"${LISTEN}"'; 
		server_name '"${SERVER_NAME}"'; 
		
		location '"${LOCATIONS[0]}"' { 
			proxy_pass '"${PROXIES[0]}"'; 
		} 
	}' /etc/nginx/nginx.conf

	# insert at
	LINE_NUMBER=70

	# use for loop to read all values and indexes
	for (( i=1; i<${n_locations}; i++ ));
	do
		
		sed -i ''"${LINE_NUMBER}"'i
		location '"${LOCATIONS[i]}"' { 
			proxy_pass '"${PROXIES[i]}"'; 
		}' /etc/nginx/nginx.conf
		LINE_NUMBER=$(($LINE_NUMBER + 4))
	done
fi

####
#### Run all images
####

if [ "$kill_docker_images" = true ]
then
	docker kill $(docker ps -q)
fi

# get length of an array
n_images=${#image_name[@]}

# use for loop to read all values and indexes
for (( i=0; i<${n_images}; i++ ));
do
    docker build -t "${image_name[$i]}:${image_version[$i]}" .
    ID="$(docker images | grep ${image_name[$i]} | head -n 1 | awk '{print $3}')"
	docker run -d -p ${ports[$i]} ${ID}
done

service nginx restart

After you have went through and modified the variables correctly, you are ready to run the script. This is a one-and-done script, that sets up the whole server fast, and you can run it using the command bash setup.sh.

Executing the script will do the following in order:

Uninstall any existing Docker installation (if any), and install Docker. Also makes sure to always start Docker when the system is restarted.
Uninstall any existing nginx installation (if any), and install nginx. Also makes sure to always start nginx when the system is restarted.
Correctly modifies the nginx.conf file that you can find in the path /etc/nginx/nginx.conf. This step is crucial, as we insert some lines that points the external ip address of your server to the Docker container.
Builds the Docker image from the Dockerfile and runs a container with the image on the ports specified earlier.
Restarts nginx for all changes to apply.

I encourage you to go and check out the code in the nginx.conf file, because it is important that you understand the nuances of how to get it to work. Small things like a missed / at the end of a location can make nginx not work.

Accessing The Online Application

Now you should be ready to access your application through the IP address of the server. If you modified the LOCATIONS variable, you should remember to access the IP address and then /webapp or whatever you used.

Because we have this as part web application and part API, we can make a POST request to the public IP address by Postman.

Note that the output is encoded as a 0 for Iris Setosa, 1 for Iris Versicolour, and 2 for Iris Virginica.

We can also do the same thing by a Python script.

import json
import requests

data = {
    "sepal length (cm)": 6.1,
    "sepal width (cm)": 3.2,
    "petal length (cm)": 3.95,
    "petal width (cm)": 1.3
}

ip_address = 'http://64.227.72.137/predict'

r = requests.post(ip_address, json=data)

print(r.text)
print(r.status_code)

Note that r.text returns the output, which is a 0 for Iris Setosa, 1 for Iris Versicolour, and 2 for Iris Virginica.

Final Things To Consider

VM security: Creating non-root users and using SSH-keys to login to the server. Also think about security of the Flask app, e.g. XSS, X-XSS, CSRF, clickjacking (iframe) attacks.
Version control: Create a container registry for version control of your container.
Traffic: You can scale nginx and add more droplets to the upstream, so that the requests are distributed if you need to handle more traffic.
Repository size: Don’t put big files in your GitHub repository, because it will take a long time to download from the VM.

Source: https://mlfromscratch.com/cheap-deployment/