The emergence of online marketplaces has introduced important new dimensions to online decision-making. Classical algorithms developed to guarantee worst-case performance often focus strongly on the worst case; in typical inputs one can perform much better which makes these approaches not practical. Moreover, these marketplaces serve multiple agents who interact in complex ways; this adds important facets to designing online decision making approaches in these systems. This thesis aims to shed light on both of these issues. In particular, in the first theme of the thesis, we show how to utilize nice structures in the data to enhance classical worst-case guarantees without requiring that these structures are perfectly present. Instead the performance gracefully degrades as these structures become less present. We discuss how to exploit three such nice structures: existence of a really good alternative, well-behaved randomness, and predictability of future requests. The second theme of the thesis explores the multi-agent aspect of modern online decision-making which adds important constraints to the classical tasks. In this direction, we discuss pricing under the existence of network externalities (such as ones arising in ridesharing systems), outcomes in evolving game settings with multiple strategic learning agents, and tradeoffs between effective online decision-making and ethical goals regarding non-discrimination.