Yes, you do need the drivers added to your image at some point, but you don't need to integrate your original install of Windows to make it all work. Remember, I've been trying to find a good use for OfflineSysprep in my implementation (as it is, it was meant more for someone using WinPE, which I am not using), and the injection of HDC drivers makes it absolutely golden to me. Is it required for me to have my image work though? No, it's just relieving a BIG amount of work needed to maintain my images.
Before, I had to use a script to parse all the HDC INFs to get a list of paths and HWIDs. From this list, I would have to remove all the references to drivers that are for 2000 or 2003 and not for XP. Then I would have to remove all the duplicate HWIDs. Add on to that, some drivers cause problems with Sysprep (such as the V and V4 folders, and even more recently the SC folder containing the AMD64 driver which can easily be removed). All of this work is unneeded if you inject the drivers from OSP. This method also makes it so it doesn't install services that are unneeded and that can be installed by Windows later when it's required.
Just remember, install only what is needed for your VM to work ok, then update to your Standard IDE drivers, and then make a snapshot before executing commands like Sysprep and setting up your ROE. Typically, my snapshots are one from right after installing Windows, one from right after installing .NET (1.1 through 3.5 in my case, with reboots between each version) since that is kind of a critical point, and right before booting into OSP and injecting drivers. Then one from right before Sysprep (actually have command lines open with all the commands I'm about to run right there so all I have to do is hit Enter... leaves very little room for error that way and this is the most critical point of your whole image), and one after Sysprep shuts my VM down.
Don't worry, if you make the drivers available, either via SysprepMassStorage or via OSP injecting drivers, so long as you have Standard IDE set, you should be ok.