First, realize that transformers have a size limit and a time limit. If the source file is too big or if it takes too long to transform, the transformation will not happen. These limits are configurable, but of course the bigger they are the more server resources you will consume.
If there is no support for your video type (MTS) out-of-the-box, you have two choices. If you are an Enterprise customer you can install Media Management (Installing Media Management | Alfresco Documentation). From there you may still have to define a transformer for MTS--I don't know if it is defined out-of-the-box with Media Management.
If you are not an Enterprise customer you will have to do it yourself. A good start would be to install ffmpeg (FFmpeg) on your server, then use it to define transformations that go from MTS to a format that is previewable (MPEG4?). You can search and find a lot of different discussion on installing and leveraging ffmpeg.
