Saturday, May 06, 2006

How to build your own package repositories

For quite some time now, I've been working on a (personal) project to create an extras disk for Ubuntu that will contain most of the optional packages that are available in the ubuntu repositories. I worked out a few scripts that took most of the workload off my fingers and was quite happy until I tried to convert a windo$e fan to linux. I didn't want to scare him off with scripts so I went hunting for an easy point-n-click solution. It turns out that a lot has been published on this topic. I found it all rather confusing so I decided to compile my own How To for my much simpler mind. Let's get our hands dirty...

1) Create folders for the files you would like to keep in your repository. Here is an example. On the desktop create a new folder called “Repositories” in which you'd keep all your downloaded packages. If you plan to burn the files onto a CD, I suggest you create separate folders for each disk. The naming is up to you – Disk_1, Disk_2, etc. This is just to ensure that you use the next folder once your collection exceeds the size of a CD (usually 700mb). If you are using a local folder or a DVD, you obviously have higher limits to keep in mind. For those of you who do not plant to create a CD/DVD, you may simply dump all the files in the Repositories folder.

2) The next step is to copy all your xxx.deb files to the folders. Go to /var/cache/apt/archives and copy all the files to your folder Disk_1. If you have many files, it may take quite a while so be patient.

3) Fire up a terminal and cd into your repositories folder:

cd ~/Desktop/Repositories/Disk_1

Now do the following command to create the Packages.gz file that is needed to for Synaptic to “see” your repository.

sudo dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz

I suggest you copy and paste this to avoid any typos.

4) Please note that every time you add any more xxx.deb files to this folder, you have to create a new Packages.gz file using the above command before the file(s) will show up in Synaptic (or Aptitude).

5) There are several ways of using your newly created repository. First, you'll have to add a line to your apt sources list.

I) Keeping the files on your local hard disk...
If you are comfortable with the terminal, then add the following line to your /etc/apt/sources.list file:

## My local repository
deb file: /"folderpath"

"folderpath" here will be the path to the folder that contains your Packages.gz file *and* your packages; for instance:

deb file:/home/"username"/Desktop/Repositories/Disk_1

If you prefer to edit the repositories directly from Synaptic (point-n-click):
a) Fire up synaptic
b) Settings --> Repositories -->Add
c) On the Edit Repository dialog, click Custom and enter the line we worked out in step 5...

deb file:/home/"username"/Desktop/Repositories/Disk_1 /

d) Click yes to reload your packages list

Voila!

II) Using a CD/DVD as a repository:
a) Burn the folder containing your packages onto a CD
b)With the CD loaded in your drive, open Synaptic > Edit > Add CD Rom.
c) You will be asked to type in a description for the CD; type in anything, for instance : My Repository Disk 1.
d) You can now install the files from the CD/DVD!

III) Keeping the repository on your local web server
Put the folder with the .deb files and latest Package.gz file on your server (/var/www directory) and serve this directory over http and/or ftp by typing the following:

sudo ln -s /"path"/Disk_1 /var/www/

Assuming that the URL for the repository directory being served is
http://localreposerver/Disk_1 or ftp://localreposerver/pub/Disk_1

You now have to configure each remote machine that uses this server to see the repositories. Edit each machines /etc/apt/sources.list to contain the following line:
deb http://localreposerver/Disk_1 /
or
deb ftp://localreposerver/pub/Disk_1 /
or use the IP address
deb http://192.168.0.1/Disk_1 /

A suggestion is to always add these entries in the sources.list at the top just below the original Ubuntu or Kubuntu CD entry as the fetch from repositories is done from the top down: your repository, then the Internet repositories etc. You do not want to first download something you already have. If this causes problems, just go to sources.list and comment out the internet sites or even easier, use Synaptic and go to Repositories and uncheck the other sites and only check your repository.

I found the following tips for using Synaptic very useful:

Ensure that all repositories are enabled and use Synaptic to reload the lists. Now have a loom at the Synaptic settings (Settings --> Repositories --> Settings). I suggest you decide if you always want to get the highest/latest version or if you want to stick to the “breezy” version. Also ensure that you have checked the box that allows downloaded packages to be kept in the cache, otherwise you will just have to download them again if you uninstall them or need them again.

If you have done the above, and have the install CD’s as well as all your own programs available in a repository of your own, life will be sweet... very sweet.

Use the ‘search’ function in Synaptic to search for the program by typing the name or any part of the name in the search dialog. This will search the repositories in your updated list to see if such a program is available. When the package is found and has a green square beside it, that package is already installed on your system. If the square is blank with an Ubuntu icon next to it, it means it is ready for installation from the ubuntu repositories. These are normally stable packages that have been tested. If the square is blank, it means that the programs shown are available in your custom repository, scroll through the list and mark by right clicking and choosing “select for installation”.

Once you have marked all the files you want to download, click Apply on the Synaptic tool bar. You have the option to either donwload and install the package immediately afterwards or just download the package for later installation. In both cases the downloaded files are stored in the cache. You can use the download only option to download programs while you are connected to the Internet for installation later or just to fill up your own repository. If you click install, you can test the application and later go exactly the same way with Synaptic and just select remove. The files will stay in the cache and can be installed again later without going online again.

Finally make a copy of your /etc/apt/sources.list file and keep it in your repository as well. You can then just overwrite the old one on a new installation with this one.

Cheers,
Odzangba