I've only calibrated mine twice. When I first got it, using EESDR 2 and recently with the firmware update, it reset my calibration so I did it again.
Are you sure you are changing a significant enough digit in the calibration value? Both times I had to fish around a bit to figure out which value was closer to 1Hz or .5Hz and work from there. It has very fine adjustment range, which is great.
Once calibrated it is stable as I work a fair amount of digital mode, so I would notice it.