How to display matrices the "right" way

Execute the following once - it should persist even after quitting Mathematica:

SetOptions[$FrontEnd, CommonDefaultFormatTypes -> {"Output" -> TraditionalForm}]

Running the command

SetOptions[$FrontEnd, CommonDefaultFormatTypes -> {"Output" -> StandardForm}]

permanently switches back to the default.