By default when I open an image it fits to the window size (scaled down). I have to hit the button Show the image at its normal size every time when I go to the Next/Previous image or open a new image.
I tried View images fit to width in Image Viewer, but it doesn't work.
Best Answer
I think that it is probably just impossible. Image Viewer is a pretty simple app generally.
What I would recommend is installing Gthumb, which has more functions, including the one you need.
There you can find this option in Preferences > Viewer tab I guess.
I know that this isn't the exact answer you're looking for, but I hope that it will help anyway.