This is where a good Product Owner/Manager is worthwhile.

They're not there to allocate resources or assign work or create schedules. They're supposed to be in that "higher perspective" headspace most of the time, so that by the time their decisions turn into actionable work...it's been through these mental filters.