With a lot of freely available and open source applications, you have a great amount of flexibility and control over what is installed on your computer. For instance, if you had the time, you could audit every single piece of open source software you install on your computer. Unfortunately, this is an unlikely thing for many of us who simply don't have the time or skills to audit every single line of code in every single application we encounter and use.
Problems may also arise when individuals pass around modified code of certain applications. For instance, someone could have hacked into a website, gotten access to the download area, uploaded a modified version of the source code, and it could be days or weeks before the change was noticed. This has happened in the past and likely will happen again. Thankfully, more software vendors are providing digital signatures with their downloads.
Why is this significant? Well, MD5 hashes have traditionally been used as a means to verify archive files. Unfortunately this is not very secure. Not only can a file be modified and have the end result match the MD5 sum of the original (not easily accomplished, but it can be done), but if someone breaks into a website and modifies the source files, how easy would it be for them to modify the pages or files that provide the MD5 hashes you are expected to compare against?
It doesn't even need to be this difficult. Someone could spoof the IP of a legitimate source for distribution a particular program or set of packages in an attempt to trick a particular individual or system into obtaining the package from their own site. No need to break into the real host when you can pretend to be that host yourself. And without any kind of real validation, you would never know if what you had installed was in fact what you meant to install.
With the use of a digital signature, such as a detached GnuPG or PGP signature, the chances of a package being modified and escaping notice for any length of time are much less likely. However, a few things should be observed when dealing with digitally-signed packages.
The first is not to download the GPG or PGP public key corresponding to the signature associated with the package from the website itself. A package could be trojaned, the key could have been faked to look similar to the legitimate key, and the package legitimately signed with the rogue key and the corresponding public key having been placed on the site to look legitimate. The best thing to do is to grab the public key associated with the private key used to sign the package from a third-party, such as one of the many keyservers available. By downloading the key in this way, you are less likely to obtain a rogue key (of course, the key used to sign a trojaned package could likewise have been uploaded to a keyserver so you must be careful even then, however this is far less likely).
The second is to attempt to verify the key in some manner; perhaps by contacting the author of the software directly, asking on a mailing list devoted to discussing the package, etc. More than likely, the key would have been in use for a while so you can verify with long-time users that the key you have just imported is identical to the one they have had for some time.
The third is to always download the detached signature and verify the package you have just downloaded. This is an absolute must; the one time you fail to do this may be the one time you unwittingly install a rootkit on your system.
To verify a detached signature with gpg, download the file you are interested in (ie. foo-1.0.tar.gz) and the detached signature for that file (ie. foo-1.0.tar.gz.sig or foo-1.0.tar.gz.asc). Place both files together in the same directory and execute:
$ gpg --verify foo-1.0.tar.gz.sig
This will look for the file associated with the signature file and verify that it has not been altered. As a quick test, make a copy of the file in a different directory as a backup, and type "echo " " >>foo-1.0.tar.gz" and then re-verify the signature. You will see that the signature fails with this slight modification.
Verifying MD5 hashes, while not as secure as a digital signature, is important as well. If nothing else, this will verify that the file you downloaded did not have errors during the download that could result in possible corruption. Download, or make note of the corresponding md5sum for the file you are interested in and then execute:
$ md5sum foo-1.0.tar.gz 2d93797afec7147386b7688cece4daf8 foo-1.0.tar.gz
Make sure that the md5sums match before you unpack or install the package. However, reliance on only the md5sum is discouraged, and using a digital signature will likewise verify that the file has not been "injured" during download. Whenever possible, use a detached GPG or PGP signature to verify a file's authenticity and validity over md5.
If you're dealing with packages from a Linux vendor that uses RPM or DEB packages, life is even easier. Most vendors sign their RPM packages with a GPG key that was distributed on your installation CDs. This allows you to import your keys from a source you can be fairly confident has not been tampered with. Before installing any RPM packages on your system, you should use the verify option in rpm, which will verify both the md5sum and GPG signature embedded in the package. For instance:
$ rpm -Kv foo-1.0-1mdk.i586.rpm
This will verify that the md5sum and GPG signature is intact. Adding the verbose option there will also print out what key was used to sign the package; a good idea to make sure that the key you expected to sign the key did in fact sign it, and not another key on your keyring. For instance, if you are installing a Mandrakelinux security update, you would expect the package to be signed with the key belonging to security_(at)_linux-mandrake.com, rather than a key that belongs to another organization like some third-party outfit that compiles packages for Mandrakelinux.
In the case of Mandrakelinux, if you use urpmi or the rpmdrake GUI tool, RPM signatures are verified for you and urpmi will prompt you if a signature fails or if the corresponding public key to the key used to sign a particular package is not available. Unfortunately, there is currently no way to map a key to a particular urpmi source. For instance, the security_(at)_linux-mandrake.com key should only show up in a security updates source; the official Mandrake keys on packages in Cooker or the release distribution, PLF keys in the third-party PLF repository, etc.
The last time I looked at apt4rpm from Conectiva, it looked like it had the proper facilities to map keys to sources. I'm unsure of other tools and distributions.
The main principle here is to be vigilant. And prompt the authors of software that you do use, if they do not do so already, to sign source packages they release with a GPG signature. Doing this will greatly reduce the possibilities of installing and running trojan software. While nothing is ever 100%, verifying that the software you have downloaded is what the author intended to be distributed will go a long way in the overall security of your system.