We present the results of the first Dalitz plot analysis of the decay D-0 -> K-pi(+)eta. The analysis is performed on a data set corresponding to an integrated luminosity of 953 fb(-1) collected by the Belle detector at the asymmetric-energy e(+)e(-) KEKB collider.
The Dalitz plot is well described by a combination of the six resonant decay channels (K) over bar*(892)(0)eta, K(-)a(0)(980)(+), K(-)a(2)(1320)+, K(-)a(2)(1410)(0)eta, K*(1680)(-)pi(+) and K-2*(1980)(-)pi(+), together with K pi and eta K eta S-wave components. The decays K* (1680)(-)-> K-eta and K-2* (1980)--> K-eta are observed for the first time.
We measure ratio of the branching fractions, B(D-0 -> K-pi(+)eta)/B(D-0 -> K-pi(+)) = 0.500 +/- 0.002(stat) +/- 0.020(syst) +/- 0.003(B-PDG). Using the Dalitz fit result, the ratio B(K*(1680)-> K eta)/B(K*(1680)-> K pi) is measured to be 0.11 +/- 0.002(stat)(-0.04)(+0.06)(syst) +/- 0.04(B-PDG); this is much lower than the theoretical expectations (approximate to 1) made under the assumption that K*(1680) is a pure 1(3)D(1) state.
The product branching fraction B(D-0 -> [K-2*(1980)- -> K-eta]pi(+)) = (2.2-1.9+1.7) x 10(-4) is determined. In addition, the pi eta' contribution to the a(0)(980)(+/-) resonance shape is confirmed with 10.1 sigma statistical significance using the three-channel Flatte model.
We also measure B(D-0 -> (K) over bar*(892)(0)eta) = (1.41(-0.12)(+0.13))%. This is consistant with, and more precise than, the current world average (1.02 +/- 0.30)%, deviates with a significance of more than 3 sigma from the theoretical predictions of (0.51-0.92)%.