The Sultanahmet district is the heart of historic Old Istanbul, what 19th-century travelers used to call "Stamboul."
This is where you'll find Topkapi Palace, Ayasofya (Hagia Sophia), the Blue Mosque (Sultan Ahmet Camii), the Byzantine Hippodrome, Yerebatan Saray (Sunken Palace Cistern), the Istanbul Archeological Museums, Great Palace Mosaic Museum, and several lesser sights (see map below).
It's your best base for sightseeing in Istanbul. For a rundown on things to see, go to Istanbul Sights. (Here's a good, inexpensive guided walking tour.)
Luckily for visitors, the district of Sultanahmet (sool-tahn-ah-MEHT, named for Sultan Ahmet I, builder of the Blue Mosque) also has a number of serviceable restaurants and dozens of good hotels in all price ranges.
Many of the hotels are literally a few minutes' stroll from the major sights.