APT repository

Wikimedia infrastructure

Data centers

Networking

Global traffic routing

MediaWiki SRE

MediaWiki Engineering

Multimedia

Data Engineering

Search

SRE Data Persistence

SRE Infra Foundations

• CAS-SSO
• Conftool
  • Dbctl
• Cumin
• Debian packaging
  • APT repository
  • DebMonitor
  • Reprepro
• Ganeti
• Puppet-compiler
• Homer
• irc.wikimedia.org
• Netbox
• Network Error Logging
• Spicerack and cookbooks

SRE Observability

Wikimedia Performance

Event Platform

Release Engineering

[]

Browse the repository contents at https://apt-browser.toolforge.org/

Wikimedia maintains its own APT repository available at https://apt.wikimedia.org/wikimedia/ and currently hosted on apt1002 and apt2002.

This repository contains Debian and Ubuntu packages modified for use by Wikimedia as well as packages for Wikimedia-originated projects.

Repository Structure

We are using the following repository structure:

• main contains all the packages that we either create internally or packages that we backport or modify from Debian and which are useful fleet-wide (e.g., Icinga plugins, Cumin, etc.)
• thirdparty/hwraid contains binary-only drivers for RAID management used on baremetal servers only.
• All other components using the thirdparty/ prefix are synchronised from external repositories (e.g., thirdparty/cloudera, thirdparty/confluent, thirdparty/ci (Jenkins) and thirdparty/k8s (Docker)). Please do not add internally built packages to thirdparty.
• Some packages should not be available fleet-wide, but rather be used in more specific scenarios such as the following:
  • component/ci provides co-installable PHP packages that are only necessary for CI
  • "contain" specific packages that depend on outdated or backported libraries (e.g. crypto libraries)
  • ease migrations and upgrades for certain roles, while leaving the rest of the fleet untouched

You can find a complete list of repository components in our Puppet config

External Access

For Wikimedia servers and Cloud VPS instances, the repositories are automatically configured via Puppet.

To use this repository from an external host, the following lines need to be present in /etc/apt/sources.list or /etc/apt/sources.list.d/wikimedia.list:

## Wikimedia APT repository
deb http://apt.wikimedia.org/wikimedia stretch-wikimedia main
deb-src http://apt.wikimedia.org/wikimedia stretch-wikimedia main

Use the right distribution, depending on which Debian/Ubuntu version was installed (e.g. bullseye-wikimedia, buster-wikimedia or trusty-wikimedia).

Here is a complete list of all the available sources:

deb http://apt.wikimedia.org/wikimedia trusty-wikimedia main experimental backports
deb http://apt.wikimedia.org/wikimedia jessie-wikimedia main experimental backports
deb http://apt.wikimedia.org/wikimedia stretch-wikimedia main
deb http://apt.wikimedia.org/wikimedia buster-wikimedia main
deb http://apt.wikimedia.org/wikimedia bullseye-wikimedia main
deb-src http://apt.wikimedia.org/wikimedia jessie-wikimedia main backports
deb-src http://apt.wikimedia.org/wikimedia trusty-wikimedia main
deb-src http://apt.wikimedia.org/wikimedia stretch-wikimedia main
deb-src http://apt.wikimedia.org/wikimedia buster-wikimedia main
deb-src http://apt.wikimedia.org/wikimedia bullseye-wikimedia main

Additionally, to make sure that the system prefers packages from this repository, and not packages from the origin distributions even if they have a higher version number. For example, this APT source is "pinned" with a higher priority in /etc/apt/preferences.d/wikimedia.pref:

Package: *
Pin: release o=Wikimedia
Pin-Priority: 1001

You can confirm Wikimedia's APT repository is taking preference now by running `apt-cache policy <package>`:

$ apt-cache policy puppetmaster
puppetmaster:
  Installed: (none)
  Candidate: 4.8.2-5
  Version table:
     4.8.2-5 1001
        500 http://deb.debian.org/debian stretch/main amd64 Packages
       1001 http://apt.wikimedia.org/wikimedia stretch-wikimedia/main amd64 Packages

Security

The Wikimedia repository is signed with the Wikimedia Archive Automatic Signing Key <root@wikimedia.org>. This public key must be installed in APT's GPG keyring so it can verify packages successfully.

The key can be retrieved from /Stretch-Key (this key was introduced with stretch and also applies to later distros)

To install it, download the key to /etc/apt/trusted.gpg.d/wikimedia-apt-key.asc, or on older systems add it using `apt-key`:

# Debian buster or later
$ wget -O /etc/apt/trusted.gpg.d/wikimedia-apt-key.asc "https://wikitech.wikimedia.org/w/index.php?title=APT_repository/Stretch-Key&action=raw"
# Debian Stretch or earlier
$ wget -O wikimedia-apt-key "https://wikitech.wikimedia.org/w/index.php?title=APT_repository/Stretch-Key&action=raw"
$ apt-key add wikimedia-apt-key
OK

You can also retrieve the key from a Cloud VPS host: ssh <host> apt-key export root@wikimedia.org

See also

• pbuilder for building packages for different distribution versions and architectures
• reprepro for additional information on how to use this tool
• Wikimedia APT browser for a simple web package listing