apt-walkabout

An APT with random-access removable media HOWTO
by James Cameron, 2006-08-23

No .deb should ever have to be downloaded more than once!

Synopsis

Describes how to upgrade non-connected or low-speed linked Debian systems using APT with random-access removable media.

Package is a set of shell scripts.

References

"Using APT Offline" by Jason Gunthorpe describes some of the techniques used by apt-walkabout. Install the apt-doc package and then open in your web browser:

file:/usr/share/doc/apt-doc/offline.html/index.html by local file access,
http://localhost/doc/apt-doc/offline.html/ if you have a web server installed on your system,

google for "Using APT Offline".

Usage Cases

take a pen drive or other storage in to work or a cafe to download updates,
take it to friend's place to upgrade their systems without having to wait for downloads.

Resources

apt-walkabout.8 manual page,
package archive,
apt-walkabout_1.0-0_i386.deb,
apt-walkabout_1.0-0.tar.gz,
darcs repository (see below for instructions).

Using Darcs

Darcs is a distributed revision control system by David Roundy designed to replace centralised CVS. apt-walkabout is provided via darcs to encourage contributions.

install darcs : apt-get install darcs

checkout initial copy : darcs get http://quozl.linux.org.au/darcs/apt-walkabout
(this creates a directory apt-walkabout in your current directory)

propose a change : darcs record darcs send
(this sends a mail to me with your changes)

update : darcs pull
(this updates your directory with changes made by me and others)

install darcs :	`apt-get install darcs`
checkout initial copy :	`darcs get http://quozl.linux.org.au/darcs/apt-walkabout` (this creates a directory `apt-walkabout` in your current directory)
propose a change :	`darcs record darcs send` (this sends a mail to me with your changes)
update :	`darcs pull` (this updates your directory with changes made by me and others)

APT Normality

On a normal system APT is used to maintain a list of package sources (/etc/apt/sources.list), a set of downloaded package lists (/var/lib/apt/lists), and a cache of downloaded package archives (/var/cache/apt/archives).

The update phase downloads the package lists from the internet. The dist-upgrade phase downloads the packages and installs them.

apt-walkabout

With apt-walkabout, we operate on removable media instead of using the root filesystem. A status file for each target system is kept, and a package list and archives cache is maintained.

The package lists are downloaded from the internet, the upgrade is simulated to fetch the packages using the per-target status file, and when the media is mounted on the target system the changes can be applied.

A fetch can be run for each remote target system being administered; all the packages required are downloaded to the removable media.

Timing comparison, side by side, old method, vs apt-walkabout:

normal

visit the target, (or ssh to it),
download packages using target's internet connection,
go home.

interim

download some packages, or most of them, e.g. my apt cache, cut to a CD or take a laptop,
visit them,
install packages, download missing, have a drink while stressing over it,
go home.

walkabout

download all packages required,
visit them,
install packages, no download required,
have a drink without stress,
go home.

normal	visit the target, (or ssh to it), download packages using target's internet connection, go home.
interim	download some packages, or most of them, e.g. my apt cache, cut to a CD or take a laptop, visit them, install packages, download missing, have a drink while stressing over it, go home.
walkabout	download all packages required, visit them, install packages, no download required, have a drink without stress, go home.

Types of removable media

write-once sequential-access (CD-R) [partially supported]
write-many sequential-access (CD-RW) [partially supported]
write-many random-access (USB storage) [supported]

Types of random-access removable media

floppy disk (too small these days)
zip drive (too rare these days)
USB pen drive (/dev/uba)
USB hard disc drive
IDE removable drive

Relevant features of APT

uses dpkg's /var/lib/dpkg/status to choose upgrade packages
keeps package lists in /var/lib/apt/lists
keeps package archives in /var/cache/apt/archives
can be told to use any of these on removable media

Assumptions

removable media is mounted exec and writable
all but package installation can be non-root

Basic Procedure

obtain the status file from the target system, e.g. by e-mail, ssh, or as part of a prior visit,
maintain a package list on the removable media, updating it in the same way "apt-get update" works,
when well connected, download the packages required onto the removable media,
travel to the target system, run the apply script.

Phase 0, Structure

Aim: to build filesystem structure on removable media.

Why: to enable the following procedures to work.

insert media in target system,
mount filesystem,
unpack the .tar.gz,
run the tree script,
umount filesystem,
remove media.

Phase 1, Adoption

Aim: obtain the list of package versions installed.

Why: so that the fetch phase knows what to get.

insert media in target system,
mount filesystem,
run the adopt script, supplying the system name,
umount filesystem,
remove media.

Phase 2, Update

Aim: to update the package list on the media.

Why: so that the fetch phase knows what to get.

Why: so that the target system can be updated with the new list.

insert media in either system,
mount filesystem,
ensure internet connection is live,
run the update script,
umount filesystem,
remove media.

Phase 3, Fetch

Aim: to download the packages required by a target system.

Why: so that the packages can be carried on the media.

insert media in either system,
mount filesystem,
ensure internet connection is live,
run the fetch script, once for each target system, [TODO: describe each which runs through each known system]
umount filesystem,
remove media.

Phase 4, Apply

Aim: to update the target system's package list.

Aim: to upgrade the target system packages from the media.

insert media in target system,
mount filesystem,
run the apply script,
umount filesystem,
remove media.

Phase 5, Purge

Aim: to maintain free space on the media.

Why: to prevent a fetch from failing due to lack of space.

insert media in either system,
mount filesystem,
run the purge script, and optionally the cleanup scripts,
umount filesystem,
remove media.

Procedure Summary

adopt, gather system state from target ... get a copy of the target system's status file, and for interest, their sources.list file, can also be via email. [while on target system]
update, update package lists ... runs apt-get update on the pen drive from internet. [while on either system]
fetch, downloads packages required ... using the package lists on the host (not the pen drive, so that my apt-proxy is used), using the target system's status file on the pen drive, into the pen drive archives. [while on either system]
apply, upgrades the target ... copies package lists from pen drive to target system, runs a dist-upgrade with --no-download --fix-missing using the pen drive archives. [while on target system]
purge, maintains the pen drive cache ... does an "apt-get autoclean" to remove any packages no longer downloadable. [while on either system]

| quozl@us.netrek.org | up |