Inflorescence branching is a key agronomic trait determining rice yield. The primary branch of the ancestral wild rice (Oryza rufipogon Griff.) bears few grains, due to minimal secondary branching. By contrast, Oryza sativa cultivars have been selected to produce large panicles with more secondary branches. Here we showed that the CONTROL OF SECONDARY BRANCH 1 (COS1) gene, which is identical to FRIZZY PANICLE (FZP), plays an important role in the key transition from few secondary branches in wild rice to more secondary branches in domesticated rice cultivars. A 4-bp tandem repeat deletion approximately 2.7 kb upstream of FZP may affect the binding activities of auxin response factors to the FZP promoter, decrease the expression level of FZP and significantly enhance the number of secondary branches and grain yield in cultivated rice. Functional analyses showed that NARROW LEAF 1 (NAL1), a trypsin-like serine and cysteine protease, interacted with FZP and promoted its degradation. Consistently, downregulating FZP expression or upregulating NAL1 expression in the commercial cultivar Zhonghua 17 increased the number of secondary branches per panicle, grain number per panicle and grain yield per plant. Our findings not only provide insights into the molecular mechanism of increasing grain number and yield during rice domestication, but also offer favorable genes for improving the grain yield of rice.