One thing that I've found from various c++ vector/matrix libs, is that the most useful have flexible storage backends. In other words, all of them can allocate their own storage that they 'own', but the most useful can also wrap a matrix 'view' around existing storage. This allows interworking with other libraries.