Using Gitlab CI Cache for Clojure Dependencies

Photo by Pankaj Patel on Unsplash

I want to share my hard-won lessons on how to setup the Gitlab CI for Clojure projects based on tools.deps. I think that the Gitlab CI is a wonderful tool for CI workloads. But when you’re going a bit sideways from the documented ways of doing things you have to do a bit of discovery for yourself.

Gitlab CI Cache Setup

Usually I want to cache dependencies between all build and all branches. To achieve this I hard-code the cache key at the root of the .gitlab-ci.yml file e.g.:

cache:
  key: one-key-to-rule-them-all

When it comes to caching Clojure dependencies we have to be aware that there different types of dependencies. Two most common ones are: Maven and gitlibs.

The Gitlab CI cache works only with directories inside the project directory. While local repositories (i.e. cache) for Clojure dependencies by default are stored outside the project directory (~/.m2 and ~/.gitlibs). Therefore, we have to provide parameters for our build tool to change the default directories for storing the dependencies.

To specify Maven local repository we can provide :mvn/local-repo parameter e.g.:

clojure -Sdeps '{:mvn/local-repo "./.m2/repository"}' -A:test

Having configured local maven repository in our gitlab-ci.yml we can specify:

cache:
  key: one-key-to-rule-them-all
  paths:
    - ./.m2/repository

When it comes to gitlibs there is no public API for changing the default directory in tools.deps. But the underlying tools.gitlibs uses an environment variable to set where to store the gitlibs conveniently named GITLIBS. E.g.

$ (export GITLIBS=".gitlibs/" && clojure -A:test)

Of course, we should not forget to configure the cache:

cache:
  key: one-key-to-rule-them-all
  paths:
    - ./.gitlibs

To use caching for both types of dependencies:

(export GITLIBS=".gitlibs/" && clojure -Sdeps '{:mvn/local-repo "./.m2/repository"}' -A:test)

And setup the cache:

cache:
  key: one-key-to-rule-them-all
  paths:
    - ./.m2/repository
    - ./.gitlibs

If you want to disable cache for a particular job (e.g. you’re linting with clj-kondo, which is delivered as a GraalVM compiled native image), just give an empty map for a job’s cache setup, e.g.:

lint:
  stage: test
  image: borkdude/clj-kondo
  cache: {}
  when: always
  script:
    - clj-kondo --lint src test

I’ve used the Gitlab CI cache while working on a streaming-text search library Beagle. A full .gitlab-ci.yml file example of the setup can be found here.

Hope this helps!