I recently looked into which versions of software I was running and was a little concerned to see how backward Fedora Core 4 was in some areas. I did a little Googling around and found a website (http://remi.collet.free.fr/) which is predominately written in French (but has a few English translations) which provides a new repository for Fedora Core 4's Yum Installer Package which provides far more recent RPM's for the likes of PHP and MySQL... So I installed it!
The install is VERY easy... You simply visit the site and download the appropriate RPM using a program like wget and then you install it using the RPM program. Next you go to /etc/yum.repos.d/ and download the repo file into that folder (again using a program like wget or curl). Finally, you tell Yum to install a package or just update your system and leave it going, for example:
sudo yum --enablerepo=remi update php
You need to tell it to enable the repository as by default it is disabled. I assume this is so you can control which packages get updated and when they get updated (you might not always want your server to be upgraded to the latest version of PHP at 4am automatically, I know I'd prefer to be present when this kind of update happens!)
As of now, this site is being powered by PHP 5.2.4 (rather than 5.0.4 - at time of writing) and MySQL 5.0.45 rather than 4.1.20 (at time of writing).