Moffsoft FreeCalc is a great replacement for Windows calculator. The main reason why I use it is the tape option. The free version is pretty basic, so if you want more functionality check out the paid version.
Some of the other features of the free version include:
Make the calculator as big or as small as you like.
Save, print, or clear the simulated paper history tape.