Story points abstract work estimates away from units of time.
People are bad at estimating timelines for specific groups of tasks, so story points attempt to create an abstraction between time and work. Story points originally referred to “ideal days”, but relatedly sometimes use a smallest task as the foundational unit.