Customers of e-commerce web sites frequently use the full text search to find the desired products. The ranking of the search result page depends on various criteria such as the matching of search terms or popularity of the product. E-commerce vendors usually use additional ranking criteria and may want to increase conversion rates by varying the rankings of the search hits. This paper proposes a method to measure the impact of changing the ranking of the search result page. The method is applied to a b2b e-commerce shop with office products.