Frontier-based exploration can be used in a wide-range of robotics applications. For example, a common utilization of exploration algorithms is in Urban Search and Rescue (USAR). In order to define the next place a robot shall visit, cost or utility functions in frontier-based exploration are often employed. In this paper we compare five different cost or utility functions known in the scientific community which are used in frontierbased exploration algorithms. This paper seeks to address two main goals: (1) to provide and explain utility and cost functions which are used in the exploration problem along with steps of how to implement them; (2) to show the results of each algorithm using a simulated environment. Finally, a discussion about the peculiarities and notable differences between the exploration methods as well as the consequences of their differences is presented.