In a Dockerfile I have a layer which installs
FROM python:2.7 RUN pip install -r requirements.txt
When I build the docker image it runs the whole process regardless of any changes made to this file.
How do I make sure Docker only runs
pip install -r requirements.txt if there has been a change to the file?
Removing intermediate container f98c845d0f05 Step 3 : RUN pip install -r requirements.txt ---> Running in 8ceb63abaef6 Collecting https://github.com/tomchristie/django-rest-framework/archive/master.zip (from -r requirements.txt (line 30)) Downloading https://github.com/tomchristie/django-rest-framework/archive/master.zip Collecting Django==1.8.7 (from -r requirements.txt (line 1))
Here is Solutions:
We have many solutions to this problem, But we recommend you to use the first solution because it is tested & true solution that will 100% work for you.
I’m assuming that at some point in your build process, you’re copying your entire application into the Docker image with
COPY . /opt/app WORKDIR /opt/app RUN pip install -r requirements.txt
The problem is that you’re invalidating the Docker build cache every time you’re copying the entire application into the image. This will also invalidate the cache for all subsequent build steps.
To prevent this, I’d suggest copying only the
requirements.txt file in a separate build step before adding the entire application into the image:
COPY requirements.txt /opt/app/requirements.txt WORKDIR /opt/app RUN pip install -r requirements.txt COPY . /opt/app # continue as before...
As the requirements file itself probably changes only rarely, you’ll be able to use the cached layers up until the point that you add your application code into the image.
This is directly mentioned in Docker’s own “Best practices for writing Dockerfiles“:
If you have multiple Dockerfile steps that use different files from
your context, COPY them individually, rather than all at once. This
will ensure that each step’s build cache is only invalidated (forcing
the step to be re-run) if the specifically required files change.
COPY requirements.txt /tmp/ RUN pip install --requirement /tmp/requirements.txt COPY . /tmp/
Results in fewer cache invalidations for the RUN step, than if you put
the COPY . /tmp/ before it.
Alternatively as a quicker means to run requirements.txt file without typing "yes" to confirm installation of libraries, you can re-write as:
COPY requirements.txt ./ RUN pip install -y -r requirements.txt COPY ./"dir"/* .
Note: Use and implement solution 1 because this method fully tested our system.
Thank you 🙂