Many of the largest early sources of electricity were, in fact, hydro power. We could've just kept building more of those until the mid 20th century when we would've pretty much maxed out the rivers (in the US).
And don't forget wind power. Could do that, too. The intermittentness handled in areas by hydro as well. Then nuclear in the mid 20th century.
If we talk about world, not just USA, then many places just don't have the luxury of hydro, or even wind. And often gas needs to be imported making it expensive. Coal was just cheap and effective to begin with.
And don't forget wind power. Could do that, too. The intermittentness handled in areas by hydro as well. Then nuclear in the mid 20th century.