Goal#
The goal here is to setup a local development dockerized python project that utilises CI/CD to my linode VPS.
First Step#
First step is to get a simple local app running in a docker container.
I’ve chosen the streamlit example app. (This is an app that I’ve previously dockerized and tested on the server. I want at this point to minimise the potential problems.)
First Problem#
The first problem I ran into was that github wouldn’t recognise my push requests. It seems github had changed their ssh key. More information here https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/
Once I got that sorted, I was ready to move forward.
Test for local development#
However, my streamlit example was pulling directly from the streamlit example git repo. So, I cloned locally, and updated the Dockerfile to use a local file rather than pull from a git repo.
This did bring up some interesting use cases for the future. #TODO
Faster Build#
I used a split up COPY command to allow my docker build to cache the pip install requirements and then copy the streamlit-example.py file if changed.
This greatly sped up the build time.
My finished Dockerfile was:
# Dockerfile
FROM python:3.9-slim
WORKDIR /app
RUN apt-get update && apt-get install -y \
build-essential \
curl \
software-properties-common \
git \
&& rm -rf /var/lib/apt/lists/*
COPY /app/requirements.txt .
RUN pip3 install -r requirements.txt
COPY /app .
# EXPOSE 8501
# HEALTHCHECK CMD curl --fail http://localhost:8080/_stcore/health
ENTRYPOINT ["streamlit", "run", "streamlit_app.py"]
And my docker-compose.yaml was:
version: '3.9'
services:
streamlit_example:
build:
context: .
dockerfile: Dockerfile
container_name: streamlit_example
ports:
- '8501:8501'
networks:
- nginx_default_network
networks:
nginx_default_network:
external: true
Here the network is set to the pre-established nginx docker network on the target VPS. (I’m using portainer to manage my docker dockererised sites on the VPS)
I ran the docker image locally.
It built.
I made some slight changes to the streamlit app file to test and rebuilt the docker image:
docker compose up --build
The changes I made to streamlit_app.py were applied and the image did not need to reinstall the requirements as stated in the requirements.txt.
Success so far.
Makefile#
I then wrote a Makefile.
If you get a *** missing separator. Stop. Error then maybe you’re using spaces to indent. Makefile requires tab to indent. Details here https://stackoverflow.com/questions/920413/make-error-missing-separator
# Makefile
build:
docker compose up --build -d --remove-orphans
up:
docker compose up -d
down:
docker compose down
show_logs:
docker compose logs
I ran the command:
make build
The docker image built, updated with changes to the streamlit_app.py file.
Nice.
I then pushed it to github and manually cloned it to my VPS into a new folder named “streamlit”.
I needed to sudo apt install make to run the Makefile, but it built.
I then changed the repository to private on github to test if it would still perform as before. It did.
Github Actions#
Back in my local project directory, I ran the following commands to get the CI/CD workflow setup. See the reference below for a full blog post on this written by Yash Prakash.
mkdir -p .github/workflows
touch .github/workflows/main.yml
name: Streamlit Example CI-CD
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [main]
pull_request:
branches: [main]
# Run this workflow manually from the Actions tab on Repo homepage
workflow_dispatch:
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- name: Deploy to VPS
uses: appleboy/ssh-action@master
with:
# VPS IP
host: ${{ secrets.VPS_SSH_HOST }}
# VPS username
username: ${{ secrets.VPS_SSH_USERNAME }}
# SSH key (copy it from your local machine)
key: ${{ secrets.VPS_SSH_SECRET }}
# SSH port
port: ${{ secrets.VPS_SSH_PORT }}
# passphrase
passphrase: ${{ secrets.VPS_SSH_PASSPHRASE }}
script: |
cd ${{ secrets.PROJECT_PATH }}
git pull origin main
make down
make build
This lead me to setting up a limited scope ssh user on my VPS to allow this specific github repo to access the hosting server.
I put together some notes on this here.
Next Step#
The next step will now be to develop a more in-depth CI/CD to function with a complex project (Dockerized Django - ninja-django - API) with a full suite of tests.
note: On April 21st, I got my code portfolio public git submodule to deploy and build using the same setup. It’s still a simple static site but it was a good second test of this process.