Higher plants have evolved multiple proteins in the RNase III family to produce and regulate different classes of small RNAs with specialized molecular functions. In rice, numerous genomic clusters are targeted by one of two miRNAs, miR2118 and miR2275, to produce secondary siRNAs of either 21- or 24-nucleotide in a phased manner. The biogenesis requirements or the functions of the phased small RNAs are completely unknown. Here we examine the rice Dicer-Like (DCL) family, including OsDCL1, 3a, 3b, and 4. By deep sequencing of small RNAs from different tissues of wild type and osdcl4-1, we revealed that the processing of 21-nucleotide siRNAs, including trans-acting siRNAs (tasiRNA) and over one thousand phased small RNA loci, was largely dependent on OsDCL4. Surprisingly, the processing of 24-nucleotide phased small RNA requires the DCL3 homolog OsDCL3b rather than OsDCL3a, suggesting functional divergence within DCL3 family. RNA ligase mediated-5′ rapid amplification of cDNA ends and parallel analysis of RNA ends (PARE)/degradome analysis confirmed that most of the 21- and 24-nucleotide phased small RNA clusters were initiated from the target sites of miR2118 and miR2275, respectively. Furthermore, the accumulation of the two triggering miRNAs requires OsDCL1 activity. Finally, we show that phased small RNAs are preferentially produced in the male reproductive organs and are likely to be conserved in monocots. Our results revealed significant roles of OsDCL4, OsDCL3b and OsDCL1 in the 21- and 24-nucleotide phased small RNA biogenesis pathway in rice.