On the krita.org Download page, the Krita developers direct people to the latest stable version. This is an easy path, and most people will always want this. The Krita developers also store a history of previous versions of Krita. This can be useful if there is a bug and you need to install an earlier version.
Find and download an older version of Krita.
From the Download page, there will be a link near the top that says http://files.kde.org/krita. You will be taken to another website where you can select your operating system. For Windows, the zip files are self-contained versions. These do not need to be installed. Once downloaded, you can open up the bin folder and click the executable file. This self-contained version makes the application more portable. The msi files are normal installer files that are typical with Windows applications.