If you count ice as solid ground, Antarctica has the highest average elevation of any continent. If you don't, then it's the lowest, since the weight of the two-mile thick icecap has depressed the underlying rock to below sea level. Most of the interior of the continent is a vast, high, flat plateau of ice, much of it over 9000'/2500m high, but without enough relief to be called mountainous. Dome Argus, the summit of the gargantuan East Antarctic Ice Cap, is, at 4030m/13,222', perhaps the highest point of non-mountainous terrain in the world.
There are, however, several significantly mountainous areas in Antarctica. The Trans-Antarctic Mountians, in some ways an extention of the Andes after their temporary submergence under the Drake Passage, stretch for 2,000 miles clear across the continent, reaching their highest point in Mount Kirkpatrick (4528m/14,856') in the Queen Alexandra sub-range. The early south pole explorers--Shackleton, Amundsen, and Scott--had to force a passage through these intimidating peaks on their way south in the early 20th century.
The highest point in all Antarctica is Vinson Massif (5140m/16,864'), one of the high peaks of West Antarctica's Sentinel Range. Other significant summits that are virtually unknown are scattered about the continent, many of them of a stature that would make them famous if they were located anywhere else.