Mirror ubuntu repository using aptly and use it as a local repository.

This article is to show mainly how to work with aptly by mirroring an official Ubuntu mirror.

It may be obvious to some of you, but the very first thing to do when building an apt package mirror is to list the mirrors you will need in your offline environment. If you do not know where you want to go, how in the world could you get there?

To find out which mirrors you might want to replicate, you could look into the mirrors listed as package sources for your package manager. apt keeps a list of all package sources in /etc/apt, distributed into multiple files. The main file is sources.list, which contains all default package sources. If you add custom ones, they will either be appended to this file, or be added in an extra file in /etc/apt/sources.list.d.

Installing Aptly

In order to install aptly on ubuntu you can run below command.

apt update && apt install aptly

For any other distribution Installing it is as easy as downloading it, and moving it to your binaries.

# Download Aptly
wget https://github.com/aptly-dev/aptly/releases/download/v1.4.0/aptly_1.4.0_linux_amd64.tar.gz
tar -xzf aptly_1.4.0_linux_amd64.tar.gz

# Move Aptly into your PATH
sudo mv aptly_1.4.0_linux_amd64/aptly /usr/bin/aptly

Now you can either use aptly’s default config, or create a custom one. If you use a custom config file, you will be able to specify the location where the mirrored packages will be stored, the download speed limit, concurrency, and many other parameters. For the sake of simplicity, we will go with the default config in this article, which stores config and packages in your home directory, under .aptly/.

Adding a mirror

To create a mirror, you can use the aptly mirror create command. This will create a local mirror, that is linked to the public mirror directly, without downloading anything. In order to obtain a local copy of the mirror, you will need to update your local mirror first, using aptly mirror update.

# Create a local Ubuntu mirror
aptly mirror -architectures="amd64" create ubuntu-bionic  http://en.archive.ubuntu.com/ubuntu/ bionic main restricted universe multiverse

# Update your local mirror
aptly mirror update ubuntu-bionic

This will download all packages present in the public mirror to your local machine. Depending on the size of the mirror you plan on mirroring, this can end up taking hours. (In our example, using the Ubuntu bionic mirror, this will take several hours, as there are over 70Gb of packages to download.)

I recommend using a tool like nohup or screen to run this task in the background. Do not worry, in case of failure, running the update command again will figure out the deltas and finish downloading the missing packages.

Making a mirror public

The mirror we built is a useful tool to maintain and update your local mirror, but is not enough to be used as an effective mirror from which you could install packages. Since a mirror in aptly is an object that you can update, and change in time, it is not fit for being exposed reliably to users. Here comes the concept of snapshots.

Aptly allows you to take a snapshot of a mirror at a given moment in time. This is an immutable object, whose versioning you can manage more precisely, which we will use to install packages from. We will need to publish it and expose the packages it contains using an HTTP server.

Making a snapshot

The first step here is to make a snapshot of our local mirror. This can be easily done using aptly snapshot create.

aptly snapshot create my-ubuntu-snapshot from mirror ubuntu-bionic

Publishing the snapshot

Publishing a snapshot can be a bit trickier. In fact, the packages you publish need to be signed with a GPG key, which is the standard signing method for packages as of today. This is important, because it enables the recipient of the data to verify that no modifications occurred after the data was signed. If you have ever added a new package repository to your computer, you probably went through some steps that involve configuring your local package manager to trust a given GPG key.

So let’s start by creating our very own GPG key, that we will use to sign the packages in our local mirror. You will be prompted to fill in a couple fields while generating the GPG key, simply go through the process, and you should have an output looking like the one shown below. If you want to find out more about generating your own keys, check out the official docs!

# Generate a GPG key
gpg --full-generate-key

# You should get an output looking like this
-----------------------------------
sec   rsa3072 2020-06-25 [SC]
      305D2741842C85E88FAEFE57EDE7437687457710
uid          Rajesh k <rajesh@rsupernova.com>
ssb   rsa3072 2020-06-25 [E]

The field located above your uid is the ID of your key. We will need it in the next steps!

We can now publish our snapshot, signing packages with our very own GPG key!

aptly publish snapshot -distribution="bionic" -architectures="amd64" -gpg-key="0A6487455018B959390A45BCB68453F8C0557085" my-ubuntu-snapshot

This will create a public directory in ~/.aptly, whose contents are organized like this:

public
├── dists
│   └── bionic
└── pool
    └── main
	├── 0
	├── 2
	.
	.
	├── v
	├── w
	├── x
	├── y
	└── z

There we go, our snapshot is officially published! To make it public, all that’s left is to expose it.

Exposing the mirror

To expose your local packages as a Debian mirror, all you need is an HTTP server capable of serving static files.

You could also start a Nginx/Apache server, using a config file such as this one to expose it to a specific domain.

server {
  listen 80;
  server_name my.debian.mirror.domain.name;

  access_log /var/log/nginx/packages-error.log;
  error_log /var/log/nginx/packages-error.log;

  location / {
    root /home/pierre/.aptly/data/public;
    index index.html;
    autoindex on;
  }
}

To make it easier for anyone to use your mirror, you should place your public GPG key at the root of your mirror, to be accessible at my.debian.mirror.domain.name/gpg.

gpg --armor --output ~/.aptly/data/public/gpg --export 305D2741842C85E78FAEFE57EDE7437687457710

To use this mirror with your apt package manager, you have to trust its public key, and add it as a package source. Once that’s done, running apt update, you should see your package among the listed sources!

# Trust your mirror's public key
wget -O - -q http://my.debian.mirror.domain.name/gpg | sudo apt-key add -

# Add your Debian mirror to source list
echo 'deb http://my.debian.mirror.domain.name/ bionic main' | sudo tee -a /etc/apt/sources.list

# Update apt cache
sudo apt update

You’re 100% ready to work in offline mode now, installing packages directly from your custom package source! This process could be the first step in deploying your solution on-premise in a secure offline environment.