mate it's probably more the size of the fish, some larger fish can be tougher than the smaller versions. also make sure you bleed the fish straight away and get it onto ice, or even better into an ice slurry. I iki jime any fish I'm keeping as they die straight away and don't bang around the esky (bruising the flesh).
if you get a larger fish and find it a bit tough, try cutting the other fillet into cubes, crumb it and fry it then add a sauce at the end, the sauce will help compensate for the dryness. otherwise try a curry or a laksa, laksa is great for any fish as it adds moisture.