Budapest is really two cities, Buda (castle side of the river) and Pest (downtown area). When you are looking at a map of Budapest, the conference venue is right beside the Metro stop Astoria in the centre of Pest.

The conference venue is called the Gólyavár (here is a Google Maps link). The address is

Gólyavár, H-1088 Budapest, Múzeum krt. 6-8

And their website:

http://golyavar.hu/english/megkozelites.html

To see where the Gólyavár is in relation to the Toastmaster hotels and Hilton (Gala Dinner location), check out the Google Map below (zoom in to see the centre of the city).

Click on [ ] in top right to open map in full page mode: