The exploration-exploitation dilemma has been an intriguing and unsolved problem within the framework of reinforcement learning. ``Optimism in the face of uncertainty'' and model building play central roles in advanced exploration methods. Here, we integrate several concepts and obtain a fast and simple algorithm. We show that the proposed algorithm finds a near-optimal policy in polynomial time, and give experimental evidence that it is robust and efficient compared to its ascendants.