There's no real consensus on why the punt is there, but it turns out to have a lot of practical uses. So no matter why glass blowers started putting punts in bottles, they've stuck around.
The first and maybe most plausible reason for the indentation at the base of the wine bottle is that it makes the bottle less tippy. Wine bottles are often tall and narrow, and when they were originally hand blown, the glass blowing process created a seam at the bottom.
Adding the punt pushed the seam up into the bottle, and the extra weight helped keep the bottom where it belongs on the table.
You might assume that the punt is there to catch wine sediment — or dregs — in the bottom of the bottle, which it kind of does. But when you're pouring wine, the sediment is actually caught by the shoulders of the bottle, where it curves before the neck.
Many bottles of wine are stored on their sides anyway, so the punt doesn't matter. And there's the fact that wines — like whites or rosé varieties — don't really have sediments but their bottles still have punts.
That leads us to a couple of sneakier reasons for that indentation.