Hypercholesterolemia, typically due to excessive cholesterol uptake, is a major risk factor for cardiovascular disease, which is responsible for ~50% of all deaths in developed societies. Although it has been shown that intestinal cholesterol absorption is mediated by vesicular endocytosis of the Niemann-Pick C1–like 1 (NPC1L1) protein, the mechanism of sterol-stimulated NPC1L1 internalization is still mysterious. Here, we identified an endocytic peptide signal, YVNXXF (where X stands for any amino acid), in the cytoplasmic C-terminal tail of NPC1L1. Cholesterol binding on the N-terminal domain of NPC1L1 released the YVNXXF-containing region of NPC1L1 from association with the plasma membrane and enabled Numb binding. We also found that Numb, a clathrin adaptor, specifically recognized this motif and recruited clathrin for internalization. Disrupting the NPC1L1-Numb interaction decreased cholesterol uptake. Ablation of Numb in mouse intestine significantly reduced dietary cholesterol absorption and plasma cholesterol level. Together, these data show that Numb is a pivotal protein for intestinal cholesterol absorption and may provide a therapeutic target for hypercholesterolemia.