Docker is imposing new rate limits on how many Container images free account users can pull. That is big news and will take effect on November 1st, 2020. Are you prepared for what that could mean for your automated build and CI/CD pipelines? Let’s cover what you need to know.
New Rate Limits for Docker | Docker Hub
The following new rate limits for Docker only apply to free user accounts. If you have a Docker Pro orTeam subscription then this doesn’t apply to you as you get unlimited pulls.
If your using a Docker Hub free (un-authenticated ) account then you will be limited to 100 container image pulls per six hours. Authenticated (but free) users will be limited to 200 pulls per six hours.
After you exceed those limits docker returns a 429 response code with the following body on either the Docker command line interface or your Docker Engine Logs.
|
1 |
You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits |
Why is Docker Introducing new Rate Limits?
Looks like Kubernetes is hurting Docker’s bottom line which means that they had to find a way to change their business model to stay afloat.
Storage may be cheapest resource on the cloud but it’s not free for Docker itself at the scale it needs to store all those free tier account Docker images. Did you know that there are over 13 billion Docker pull requests per month!
They also wanted to crack down on the users which are over exaggerating their usage. Interesting finding .. about 30 percent of all downloads on Docker Hub come from only 1 per cent of anonymous users! These are knows as “Download Whales”.
“Establishing upper limits on ‘docker pulls’ is the first step in our move toward usage-based pricing”. – Docker CEO Scott Johnston
Docker Hub Image Retention Policy | Delayed
Originally Docker was planning to also delete Docker Images for free account users after six months of inactivity. Oh boy! that is pretty bad for many of us. Speaking for myself, I put lots of time building Docker Images for tutorials and then I don’t touch them again – usually. Now, if I don’t update my images every 6 months, they’ll be deleted! What can we do?
Well, thankfully they delayed the image retention policy until mid 2021 so we have time to prepare at least. Docker received lots of push-back from the community and so this rove them to delay this.
Docker Hub Rate Limits | Getting Preparing
You have several options to mitigate the Docker Hub Pull limits and preparing for the soon to be retention policy. If you want to stay within the free account tier then make sure that you’re at least authenticated! You can double your pull request limit from 100 to 200 every 6 hours just by authenticating your pull requests.
Free Solutions
There are many ways to get authenticated based on whether your using Docker Desktop, The standalone Docker Engine, Kubernetes, Docker Swarm etc .. so refer to this link here to get you on the right track in setting up your authenticated pull request.
If you really need more than those free limit rates and still do not want to pay then think about configuring, running and maintaining your own private docker server registry, follow these instructions. Of course this is more work but at least you can maintain full control and remain in the free zone.
Incorporate Caching to avoid hitting the Docker Hub registry. Try setting up a caching proxy like JFrog’s Artifactory. This applies to developers all the way to your CI/CD pipelines. Some Cloud providers are increasing their caching size limits to mitigate against this as well.
As for preparing for the upcoming retention policy, in the short term you can at least ensure you have a separate backup of your Docker Images on some local and/or Cloud based storage.
Paid Solutions
The easiest solution is of course to upgrade your Docker free account to a paid plan! The Docker Pro account for individuals goes for $5/month and the Team subscriptions is for organizations at $7 user/month.
Of course you don’t have to use Docker Hub as your online registry. There are many great alternatives like …
Here is an exhaustive list on Docker Hub alternatives and competitors.
Summary
The Days of 100% Free Docker are over. Although these new rate limits are there to eliminate abuse they are really there to ensure the Docker ship doesn’t sink in financial waters.
The reality is that these limits are very reasonable and in the age of dirt cheap cloud storage, quite affordable. Personally, I am more concerned about the image retention policy … time to get prepared for that!