An APT with random-access removable media HOWTO<br>
by James Cameron, 2006-08-23<p>

<div align=right><i>No .deb should ever have to be downloaded more
than once!</i></div><p>

<h3>Synopsis</h3>

	Describes how to upgrade non-connected or low-speed linked
	Debian systems using APT with random-access removable media. <p>

	Package is a set of shell scripts. <p>

<h3>References</h3>

"Using APT Offline" by Jason Gunthorpe describes some of the
techniques used by apt-walkabout.  Install the <i>apt-doc</i> package
and then open in your web browser: <p>

<ul>
<li><a href="file:/usr/share/doc/apt-doc/offline.html/index.html">file:/usr/share/doc/apt-doc/offline.html/index.html</a> by local file access,
<li><a href="http://localhost/doc/apt-doc/offline.html/">http://localhost/doc/apt-doc/offline.html/</a> if you have a web server installed on your system,
</ul>
<ul>

Or find it online:<p>

<li><a href="http://www.google.com/search?q=%22Using+APT+Offline%22">google for "Using APT Offline"</a>.
</ul>

<h3>Usage Cases</h3>

<ul>
<li>take a pen drive or other storage in to work or a cafe to download
updates,
<li>take it to friend's place to upgrade their systems without having
to wait for downloads.
</ul>

<h3>Resources</h3>

<ul>
<li><a href="apt-walkabout.8.html">apt-walkabout.8</a> manual page,
<li><a href="../lenny/">package archive</a>,
<li><a href="../lenny/apt-walkabout_1.0-0_i386.deb">apt-walkabout_1.0-0_i386.deb</a>,
<li><a href="../lenny/apt-walkabout_1.0-0.tar.gz">apt-walkabout_1.0-0.tar.gz</a>,
<li><a href="http://quozl.linux.org.au/darcs/apt-walkabout/">darcs repository</a> (see below for instructions).
</ul>

<h3>Using Darcs</h3>

Darcs is a distributed revision control system by David Roundy
designed to replace centralised CVS.  apt-walkabout is provided via
darcs to encourage contributions.<p>

<table border=0>

<tr valign=top>
<th align=left>install darcs&nbsp;:</th>
<td><tt>apt-get install darcs</tt></td>
<td rowspan=4><img src=http://quozl.linux.org.au/darcs/logo.png width=73 height=78></td>
</tr>

<tr valign=top>
<th align=left>checkout initial copy&nbsp;:</th>
<td><tt>darcs get http://quozl.linux.org.au/darcs/apt-walkabout</tt><br>(this creates a directory <tt>apt-walkabout</tt> in your current directory)</td>
</tr>

<tr valign=top>
<th align=left>propose a change&nbsp;:</th>
<td><tt>darcs record<br>darcs send</tt><br>(this sends a mail to me with your changes)</td>
</tr>

<tr valign=top>
<th align=left>update&nbsp;:</th>
<td><tt>darcs pull</tt><br>(this updates your directory with changes made by me and others)</td>
</tr>

</table>

<ul>
<li><a href="http://en.wikipedia.org/wiki/Darcs">Darcs</a> on Wikipedia
<li><a href="http://darcs.net/">Darcs</a> homepage
<li><a href="http://darcs.net/manual/">Darcs manual</a>
<li><a href="http://psi-im.org/wiki/Darcs_Instructions">one of many examples</a>
</ul>

<h3>APT Normality</h3>

	On a normal system APT is used to maintain a list of package
	sources (<tt>/etc/apt/sources.list</tt>), a set of downloaded package
	lists (<tt>/var/lib/apt/lists</tt>), and a cache of downloaded package
	archives (<tt>/var/cache/apt/archives</tt>).<p>

<img src=diagram-normal.png><p>

	The update phase downloads the package lists from the
	internet.  The dist-upgrade phase downloads the packages and
	installs them.<p>

<img src=diagram-old.png><p>

<h3>apt-walkabout</h3>

	With apt-walkabout, we operate on removable media instead of
	using the root filesystem.  A status file for each target
	system is kept, and a package list and archives cache is
	maintained.<p>

<img src=diagram-directories.png><p>

	The package lists are downloaded from the internet, the
	upgrade is simulated to fetch the packages using the
	per-target status file, and when the media is mounted on the
	target system the changes can be applied.<p>

<img src=diagram-data-flow.png><p>

	A fetch can be run for each remote target system being
	administered; all the packages required are downloaded to the
	removable media.<p>

	Timing comparison, side by side, old method, vs apt-walkabout: <p>
<table>

<tr valign=top><th>normal</th>
<td>
<ul>
<li>visit the target, (or ssh to it),
<li>download packages using target's internet connection,
<li>go home.
</ul></td></tr>

<tr valign=top><th>interim</th><td>
<ul>
<li>download some packages, or most of them, e.g. my apt cache, cut to a CD or take a laptop,
<li>visit them,
<li>install packages, download missing, have a drink while stressing over it,
<li>go home.
</ul></td></tr>

<tr valign=top><th>walkabout</th><td>
<ul>
<li>download all packages required,
<li>visit them,
<li>install packages, no download required,
<li>have a drink without stress,
<li>go home.
</ul></td></tr>
</table>

<h3>Types of removable media</h3>
<ul>
	<li>write-once sequential-access (CD-R) [partially supported]
	<li>write-many sequential-access (CD-RW) [partially supported]
	<li>write-many random-access (USB storage) [supported]
</ul>

<h3>Types of random-access removable media</h3>
<ul>
	<li>floppy disk (too small these days)
	<li>zip drive (too rare these days)
	<li>USB pen drive (/dev/uba)
	<li>USB hard disc drive
	<li>IDE removable drive
</ul>

<h3>Relevant features of APT</h3>
<ul>
	<li>uses dpkg's /var/lib/dpkg/status to choose upgrade packages
	<li>keeps package lists in /var/lib/apt/lists
	<li>keeps package archives in /var/cache/apt/archives
	<li>can be told to use any of these on removable media
</ul>

<h3>Assumptions</h3>
<ul>
	<li>removable media is mounted exec and writable
	<li>all but package installation can be non-root
</ul>

<h3>Basic Procedure</h3>
<ul>
	<li>obtain the status file from the target system, e.g. by
	  e-mail, ssh, or as part of a prior visit,<p>

	<li>maintain a package list on the removable media, updating it
          in the same way "apt-get update" works, <p>

	<li>when well connected, download the packages required onto the
          removable media,<p>

	<li>travel to the target system, run the apply script.
</ul>

<h3>Phase 0, Structure</h3>

<b>Aim</b>: to build filesystem structure on removable media.<p>
<b>Why</b>: to enable the following procedures to work.<p>
<ul>
	<li>insert media in target system,
	<li>mount filesystem,
	<li>unpack the .tar.gz,
	<li>run the tree script, 
	<li>umount filesystem,
	<li>remove media.
</ul>
<h3>Phase 1, Adoption</h3>

	<b>Aim</b>: obtain the list of package versions installed.<p>
	<b>Why</b>: so that the fetch phase knows what to get.<p>
<ul>
	<li>insert media in target system,
	<li>mount filesystem,
	<li>run the adopt script, supplying the system name,
	<li>umount filesystem,
	<li>remove media.
</ul>
<h3>Phase 2, Update</h3>

	<b>Aim</b>: to update the package list on the media.<p>
	<b>Why</b>: so that the fetch phase knows what to get.<p>
	<b>Why</b>: so that the target system can be updated with the new list.<p>
<ul>
	<li>insert media in either system,
	<li>mount filesystem,
	<li>ensure internet connection is live,
	<li>run the update script,
	<li>umount filesystem,
	<li>remove media.
</ul>
<h3>Phase 3, Fetch</h3>

	<b>Aim</b>: to download the packages required by a target system.<p>
	<b>Why</b>: so that the packages can be carried on the media.<p>
<ul>
	<li>insert media in either system,
	<li>mount filesystem,
	<li>ensure internet connection is live,
	<li>run the fetch script, once for each target system,
	  [TODO: describe <i>each</i> which runs through each known system]
	<li>umount filesystem,
	<li>remove media.
</ul>
<h3>Phase 4, Apply</h3>

	<b>Aim</b>: to update the target system's package list.<p>
	<b>Aim</b>: to upgrade the target system packages from the media.<p>
<ul>
	<li>insert media in target system,
	<li>mount filesystem,
	<li>run the apply script,
	<li>umount filesystem,
	<li>remove media.
</ul>
<h3>Phase 5, Purge</h3>

	<b>Aim</b>: to maintain free space on the media.<p>
	<b>Why</b>: to prevent a fetch from failing due to lack of space.<p>
<ul>
	<li>insert media in either system,
	<li>mount filesystem,
	<li>run the purge script, and optionally the cleanup scripts,
	<li>umount filesystem,
	<li>remove media.
</ul>
<h3>Procedure Summary</h3>
<ul>
<li>	adopt, gather system state from target ... get a copy of the
	target system's status file, and for interest, their
	sources.list file, can also be via email.  [while on target
	system]

<li>	update, update package lists ... runs apt-get update on the pen
	drive from internet.  [while on either system]

<li>	fetch, downloads packages required ... using the package lists
	on the host (not the pen drive, so that my apt-proxy is used),
	using the target system's status file on the pen drive, into
	the pen drive archives.  [while on either system]

<li>	apply, upgrades the target ... copies package lists from pen
	drive to target system, runs a dist-upgrade with --no-download
	--fix-missing using the pen drive archives.  [while on target
	system]

<li>	purge, maintains the pen drive cache ... does an "apt-get
	autoclean" to remove any packages no longer downloadable.
	[while on either system]
</ul>