To make things clear, removing PrecompiledApp.config under 4.0 may result in other side effects (Global.asax events not being called, resulting in page caching functionality being disabled).
This is an undocumented flaw in 4.0. We are planning to solve it by offering separate installations for different versions of .NET framework. We did try to stick with a "single package fits all" strategy for as long as we could, but such annoying issues force us to change the approach. It will probably make your lives easier too, since everything will work out of the box, without moving config files around.