In Ubuntu it is assumed a kernel for generic x86 support should be about as fast as a kernel suited for your arch. So lets see how true this is.
Is it worth the effort for the average user? No absolutely not, If I would use Ubuntu tomorrow with the generic kernel I probably would not notice the difference.
If you like tweaking, go for it.