With what you stated, I've altered my process and it seems to work. Here's what I did, it's not perfect but it works:
created the slipstreamed drivers pack/os installation and moved it to the given network location
copied the OEM folder and all associated files to CD
Started the network install and once the boot cd is no longer needed swap it out for the driver pack cd.
Completed the OS install to include the slipstreamed network drivers to allow the additional installation process to continue from the network.
I know that there's away to get this to work all from the single boot cd, the unattended wiki mentions using driver packs to install network drivers and chipset drivers in the installation process, but it's not very clear on how to set it up (a couple of dead links)
I'm going to keep plugging away at it and figure this out.