Choose the Izu-Kogen railway approach and the return first
Izu-Kogen (伊豆高原) is the useful rail anchor for the mountain-and-zoo version. JR East lists it on the Odoriko route towards Izukyu-Shimoda; the Shuzenji branch is a different journey. Choose the train that actually serves your destination and the Tokyo-area station you will use.
JR describes the E257 Odoriko as a reserved-seat limited express. Arrange the outward and return service for your date, rather than arriving with a sightseeing list and deciding the return at the end. A local-train connection may be part of the journey; its waiting time belongs in the day too.
For a day trip, travel light and choose the main area before leaving Tokyo. For an overnight stay, place your accommodation and luggage drop into the route. Staying near the part of Izu you will actually visit is more useful than treating every attractive place on the peninsula as a nearby stop.