Voltage 'surging'

[bobber01]

My '80 CBX has a charging issue which is frustrating me. When I start the bike, it shows a little over 13 volts then, once rpm's increase, the voltage 'bounces' between 10 - 13 before finally settling at 10. Ride the bike for about 5 mins then the 'bouncing' returns with the voltage settling at 13. After that , no issue. Volt meter shows normal operation (12-13 at idle then 13+ once off idle... remaining steady). Can turn bike off, go for a Timmies, come back and voltage works way it should. However let bike sit for a couple off days and issue returns.
I didn't realize just how much of a parasite the alternator is until hearing the engine when the voltage 'bounces'. I've replaced the battery, brushes and alt plates. The alt spring seems strong and there's 2 washers behind it. I had put in Motul Double ester oil at almost $20 a litre and went back to syn oil thinking it may of been the oil. Any ideas???