I have a 64 cj5 and mines a little wobbly side to side too. I think they are basically set up the same. Center shaft that articulates the break and clutch. Probably just a bit worn out. You could grease the zirk fitting. Make sure the cotter pin is still there holding the master cylinder plate to the shaft assembly