Grain size is an important yield trait in cereal crops and understanding the mechanisms that control grain size is essential for precise breeding and improving cereal crop yields. Epigenetics plays a crucial role in plant growth and development, but how it is involved in grain size regulation remains largely unclear. Here we report that the histone-binding protein LARGE3 associates with the histone deacetylase OsHDT1 (HD-tuins 1) to control grain length by influencing histone H4 acetylation in rice (Oryza sativa). The large3 mutants form large and heavy grains because of increased cell proliferation. LARGE3 physically interacts with OsHDT1 to negatively regulate grain length. LARGE3 and OsHDT1 repress the expression of an important grain size gene OsMKKK10 by decreasing its histone H4 acetylation level and increasing its nucleosome density. Importantly, genome editing of LARGE3 homolog in foxtail millet (Setaria italica) causes large grains and increased grain yield. These findings reveal a molecular mechanism for LARGE3-OsHDT1 mediated epigenetic modulation of grain length by regulating histone H4 acetylation of OsMKKK10, suggesting that this module has potential applications in improving grain size and yield in key cereal crops.