Both are possible. the name of the MSE Database is stored in the app.config files of both the Runtime server and the Catalog server. If you change the database name, you'll need to make sure you reflect the new name in these two app.config files.
If you want to use the installer to install the database with a different name, you'll need to work around the fact that the installer only allows the MSI to install once per machine. When you do the installation on a clean machine, you can run a custom
installation and only select the repository feature to be installed. Provide the server and database name for the MSE repository. You'll then need to go to add/remove programs on the machine where you ran the installer from and uninstall the
MSE (this won't touch the database). Repeat the process providing a different database name.
Finally, you'll need to update the app.config files I mentioned above to make sure the two Windows services know where to go to access the repository database.