These asymmetric formats makes driver / behavior difficult.
It is not HW limitation, but SW limitation. To makes code reading
easy, this patch removes asymmetric format support.
Signed-off-by: Kuninori Morimoto <kuninori.morimoto.gx@renesas.com> Signed-off-by: Mark Brown <broonie@kernel.org>