For fast in-situ assessment of tiller phenotypes in rice breeding, we introduce the TillerPET model, an improved transformer-based deep learning solution that permits phenotyping the number and compactness of rice tillers in images of post-harvest rice stubble. A rice tiller phenotype dataset covering three years of field data and four experimental sites across China was constructed to train and validate the model. TillerPET reports an R2 of 0.941 for counting tiller number, demonstrating state-of-the-art performance on the proposed RTP dataset. Beyond its minimal errors in estimating tiller number, TillerPET also achieves an R2 of 0.978 for characterizing tiller compactness. The two phenotypic parameters exhibit a high degree of consistency with expert breeders, offering reliable phenotypic indicators to guide further breeding.