Courier-IMAP comes with some bundled scripts to create self-signed certificates. Depending on what operating system you installed Courier-IMAP on, and whether you compiled it yourself or if you installed a pre-compiled RPM package, the location for these scripts may be different. The scripts in question are the mkimapdcert and mkpop3dcert files, for IMAPS and POP3S respectively. You can use these scripts to create the certificate for your IMAPS (IMAP+SSL) and POP3S (POP3+SSL) services, or you can do it yourself. The scripts are merely wrappers for openssl. The scripts create a single file which have your private key and certificate together. If you only need a certificate for testing purposes, the scripts are fine. If you need to have your own CA or another CA sign the certificates, you can also use the scripts or create them yourself. In either case, you will have to send the resulting PEM file to your CA to sign.
To create the certificates manually, execute:
# openssl req -x509 -newkey rsa:1024 -keyout imapd.pem -out imapd.pem -days 9999 -nodes
This creates your SSL certificate in the current directory as imapd.pem. For instance, on a Mandrakelinux system, the scripts and resulting PEM files would be located in /usr/share/courier/. Send the resulting PEM file to your CA for signing. If you have your own CA (perhaps as described in SelfCA), you will need to resign the certificate with your CA certificate.
If you need to sign the SSL certificates you just generated with your own CA, it's very easy to do. The following command is used if you have your CA setup as described in SelfCA, using the "old fashioned" commandline method of managing certificates. Sign the key using:
$ openssl ca -ss_cert imapd.pem
This will create a new PEM file in your _certs/_ directory (ie. /Volumes/encrypted/ssl/certs/17.pem). This file needs to be copied back to the server on which the certificate will be used, but it must not overwrite the original PEM file!
At this point, you need to replace the certificate in your original imapd.pem file with the certificate in the new PEM file (ie. 17.pem). If you examine your current imapd.pem file you will see two or three sections: the "RSA PRIVATE KEY", "CERTIFICATE", and possibly a "DH PARAMETERS" section. In the original imapd.pem file, use your favourite text editor and remove the entire CERTIFICATE section (ie. the lines in between and including the "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----"). Replace this section of text with the same section of text in your newly signed PEM file. Only copy the CERTIFICATE section with the BEGIN and END lines intact.
Once you are done, save the file and you should have a new imapd.pem file that contains the original RSA PRIVATE KEY and the newly-signed CERTIFICATE. At this point, you can start the Courier-IMAP IMAPS service with the new certificate and test using your favourite IMAP mail client.
The instructions are identical if you are creating a certificate for POP3S.
This procedure is not limited to Courier-IMAP. Any service that uses a self-signed certificate (such as OpenLDAP, or Exim for TLS, etc.) can likely follow the same procedure (I have successfully used this for Courier-IMAP POP3S/IMAPS and Exim TLS).