Cloud Tips

Dockerising python applications

October 23, 2023

Dockerize python application

Simple Dockerfile for Python script

Source code can be found

Dockerise simple python script with no dependencies

Lets take a basic example based on official Python Docker image

FROM python:3.9 
ADD main.py .
CMD ["python3", "./main.py"]

We simply copy our main.py script and run it with python executable This path will only work for one-off scripts, jobs and everything that executes after logic is complited

This is not suitable for cases, when we will need an API or a webserver or any standard Backend Implementation. For those use cases, we will need to use a web server/api code implementation plus make sure something keeps our API running

This is not the most optimal path, as this python:3.9 Docker image is very bulky (1 GB in size) We gonna change to Alpine based Docker image in FastAPI example

As well, there are some security features we can add.

Dockerise simple python script with dependencies

We will use "requests" package as example. When we have a very simple Dockerfile, it sometimes simpler define all inside our Dockerfile without adding extra requirments files.

Using requirments.txt file in general is a best practice as it allows to lock and use specific versions of dependencies Where pip install installs the latest available version. Which in some use cases can cause issues

FROM python:3.9 
RUN pip install requests
ADD main.py .
CMD ["python3", "./main.py"]

Our application code looks like this :

import requests

url = 'https://checkip.amazonaws.com'

response = requests.get(url)

if response.status_code == 200:
    print(f"Response received successfully from {url}")
    print(f"Response content:\n{response.text.strip()}")
else:
    print(f"Error: Unable to fetch data from {url}. Status code: {response.status_code}")

We gonna send a simple HTTP requests to show our Public ip

Response received successfully from https://checkip.amazonaws.com
Response content:
86.86.17.218

How to run in this script Docker code sample

  cd  python-docker-with-dependencies
  docker build -t python-dep:v1 ./
  docker run -it python-dep:v1

Dockerise simple python script with dependencies in requirments.txt file

This will require is having an extra file called requirments.txt It defines dependencies and their versions. You can pass this file to "pip install" command

code

FROM python:3.9 
ADD requirments.txt .
RUN pip install -r requirments.txt
ADD main.py .
CMD ["python3", "./main.py"]

We add the requirments.txtfile and perform dependency installation, prior to adding our code Because, we will oftenly update the code in main.py and rarely update dependencies. Sometimes, they might never change until upgrading to the latest version or because of vulnerabilities found If we do it another way around, we would be waiting every time for Docker to rebuild the "pip install -r requirments.txt" dependency layer

Dockerise python Fast API app

FastAPi is one of the simplest and widely addopted frameworkd for building APIs in Python

code

FROM python:3.9-alpine
WORKDIR /code
ADD requirments.txt .
RUN pip install -r requirments.txt
COPY ./app /code/app
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80"]

We are now using "python:3.9-alpine" Docker image as base to reduce size of Docker image

Before size was 1.01GB

python-requirments v1  8dd7eaf356d9   2 hours ago   1.01GB

Minamal Python Docker images based on Alpine becomes 64.5MB

fast-api  v1 6e3f45386205   3 minutes ago   64.5MB

This is a massive ~950MB difference Good to mention, that Alpine is not a silver bullet. Once you have lot of dependencies, and especially something more complex targetting AI/ML there would be a lot of corner use cases. In those, you might spend hours fighting with compatibility issues and tailoring your implementations For regular APIs/web servers, you will be able to find lots of examples

As well, in this FastAPI Docker file we introduce Uvicorn which is is an ASGI web server implementation for Python.

We instuct in our CMD command to do following :

  • under folder "apps" , launch main.py and executee an "app" function
  • Bind to all addresses/interfaces
  • Use port 80
  CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80"]

if we visit http://0.0.0.0:80 in the broswer, we would see {"Test": "Success"} returned from our FastAPI You can add more routes and can read about FastAPI

docker build -t fast-api:v1 ./
docker run -p 80:80 fast-api:v1


INFO:     Started server process [1]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:80 (Press CTRL+C to quit)

Dockerise python Flask Docker app

Lets try to Dockerise a simple Flask API example

FROM python:3.9-alpine
WORKDIR /code
ADD requirments.txt .
RUN pip install -r requirments.txt
COPY ./app /code/app

CMD  ["gunicorn", "-w 4", "app.main:app", "--bind", "0.0.0.0:8080"]

One thing different, is a different Web server implementation via Gunicorn

We create

  • 4 workers
  • under folder "apps" , launch main.py and executee an "app" function
  • run on all addresses/interfaces plus use port 8080

Our Simple Flask App looks like this :

from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    return {"Test": "Flask response"}

It will return a JSON response like {"Test": "Flask response"} if you open http://0.0.0.0:8080

cd flask-docker
docker build -t flask-alpine:v1 ./
docker run -p 8080:8080  flask-alpine:v1


[2023-10-23 13:57:13 +0000] [1] [INFO] Starting gunicorn 19.10.0
[2023-10-23 13:57:13 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2023-10-23 13:57:13 +0000] [1] [INFO] Using worker: sync
[2023-10-23 13:57:13 +0000] [8] [INFO] Booting worker with pid: 8
[2023-10-23 13:57:13 +0000] [9] [INFO] Booting worker with pid: 9
[2023-10-23 13:57:13 +0000] [10] [INFO] Booting worker with pid: 10
[2023-10-23 13:57:13 +0000] [11] [INFO] Booting worker with pid: 11

How to start a new Django Project

If you clone and use sample code, you don't need to create a new django project This was done to illustrate how we came up with a Codebase. Dockefile was added later

[ Optional ]Install django-admin

pip3 install django
django-admin --version
django-admin startproject django_project .
python3 manage.py runserver

Dockerise python Django Docker app

Example with Django is bit more complex as there are lots of different files controlling different components & interactions with Web Server, API , routes, settings Django is a framework with pre-buld components which can be used to buld fullstacks apps as it can serve both FrontEnd and Backend Framework that encourages rapid development and clean, pragmatic design

Django Docker file

FROM python:3.9-alpine
WORKDIR /code
ADD requirments.txt .
RUN pip install -r requirments.txt
ADD manage.py .
COPY ./django_project /code/django_project

RUN python3 manage.py migrate 
CMD ["gunicorn", "--bind", "0.0.0.0:8080", "--workers", "3", "django_project.wsgi:application"]

We will :

  • 3 workers
  • under folder "django_project" , launch wsgi.py and executee an "application" function
  • run on all addresses/interfaces plus use port 8080

Building and Running Django in Docker using reference code Django example :

cd django-docker
docker build -t django-docker:v1 ./
docker run -p 8080:8080 django-docker:v1


[2023-10-23 14:03:59 +0000] [1] [INFO] Starting gunicorn 19.10.0
[2023-10-23 14:03:59 +0000] [1] [INFO] Listening at: http://0.0.0.0:8080 (1)
[2023-10-23 14:03:59 +0000] [1] [INFO] Using worker: sync
[2023-10-23 14:03:59 +0000] [9] [INFO] Booting worker with pid: 9
[2023-10-23 14:03:59 +0000] [10] [INFO] Booting worker with pid: 10
[2023-10-23 14:03:59 +0000] [11] [INFO] Booting worker with pid: 11

Some important notes:

  • update your port in manage.py (default is using 8000) from django.core.management.commands.runserver import Command as runserver runserver.default_port = "8080"

  • add Allowed hosts in settings.py file in django_project folder ALLOWED_HOSTS = ['0.0.0.0']

  • Update the Secure Key in SECRET_KEY if re-using the code form this example