Why User Testing Your Own Product Is A Bad Idea – OLX App Review

by | Aug 31, 2017 | User Experience


The growth of mobile app development since the introduction of the iPhone and the App Store has started a revolution in the online space. Many businesses have not only started providing products and services via mobile apps but in many cases, the mobile app has become the product. This calls for a need to evaluate how we can ensure the apps we launch, whether revised or a new product launch, is easily adopted by the market and on the road to success.

The Value of User Testing

Before a traditional household product is released to market, it undergoes extensive testing to not only ensure its safety and reliability, but also its usability. Many developers have undervalued the testing of their applications before releasing to market and many of those who have tested, have done so internally – creating a confirmation bias. One of the main reasons for this mistake is when executives and developers become overconfident in their ability to predict user behaviour – how users will use their system, what features the user wants, etc.

The consequences of this mindset can be dire. On 15 June 2017, OLX released their new app. While OLX has been dominating the online used sales landscape in South Africa, and considered the experts in their field, the result of the app performance provides a good case study that illustrates a point: No matter how experienced or informed a company thinks it is, without conducting the appropriate user testing, an app can fail before its launch; and will fail in reaching its full potential.

While it’s too early to see statistics, OLX’s customer reviews on the app store tell the story:

OXL | InteractRDT | Insights and Experience Design Agency


People don’t like change – or do they?

One of the biggest complaints some power users have about Apple is their tendency to mostly incorporate minor changes for updates. There is something to Apple’s philosophy, users want something to “just work”, and work the same way it did yesterday. Most users don’t like major changes to familiar interfaces because it requires the user to relearn the interface. Apart from being easier on the users, it allows developers to stick to one interface solution and perfect it to a high standard.

Sometimes the primary reason developers & designers opt for a fresh new design is because they want a redesign. It’s not surprising since we can spend up to thousands of hours looking at our own designs. The design soon becomes tired and outdated to us. However, we forget that the average user only spends about 2 – 3 minutes on a website, and in OLX’s case, many users might only be using the app 5 minutes a month at most. Rethinking a design is not wrong of course, but fixing something that isn’t broken only for the sake of “staying fresh” is a bad strategy. When competitors start launching new, fresh looking apps into the market, companies often feel pressured to follow suit, even if it means launching before the user experience is ready. This is often a consequence of lifting our own wants and needs for our app above the users – or even more probable: Confusing the distinction between the users’ needs and wants, with our own. We end up crafting an experience ideal for a developer/power user such as ourselves instead of the average consumer.


Determining what went wrong


In this review, we will be conducting a small 5 minute user test to identify the OLX app’s biggest user experience problem, review the issue and quickly wireframe a redesign as a solution. In an ideal world, a larger sample size is recommended to conduct user testing, but for this example, we will be displaying the findings that even one test can provide.

In one of the first steps after our user opened the app for the first time, she was asked to “Select your exact location” and presented with a map. The location presented was far from the user’s location, there was no text input where she would be able to enter the address of her exact location. This left her frustrated and worried about how getting the first step wrong will impact the rest of her shopping experience. This is an important note, as we can deduct that the user is not sure if he/she can trust the information presented by the app, a noteworthy issue for an app where trust is key.



In fact, due to this uncertainty, when she searched for an item, she kept trying to check the item’s location; and the unclear map (displayed with listings) with no clear location information further caused uncertainty. While users can click on the map to open Google Maps, in our user test, she didn’t realise this, because the map appearance doesn’t communicate this.


5 Km | InteractRDT | Insights and Experience Design Agency


She was also confused by the ”5KM” filter link. The top bar displays a distance (5km), and she thought she would be able to change this distance by clicking on it, but this was not the case. The current design changes the distance based on the results you are looking at, but she never realised this and was confused on how she could change the value to see different results.

A minor change for major improvement

From this short test, we could deduct that there is a genuine issue with how location information is displayed, selected by users, and used by the app for displaying results. In an app designed for connecting users to make safe transactions, location is everything.

A simple change to the initial screen presented to the user could rather require the user to use text input to enter their location. Location services can still be used, but rather to aid the user instead of providing the primary means of determining their approximate location.  Many users might also like to search in locations they are not currently in. We also added a simple indicator to show users the current step in the process so they can see how many questions are left before they can start using the app.

Proposed Wireframe 1 | InteractRDT | Insights and Experience Design Agency


With the discovery and addressing of some of these issues, the OLX review scores would have been much better. When we understand our users and the issues they encounter, it becomes very easy to apply the necessary (often small) changes to improve the experience significantly.

User testing is becoming an increasingly important methodology as the app landscape becomes more competitive. Including user testing as an essential step in your UX process will ensure the avoidance of various UX problems that would not have been discovered otherwise. Unfortunately, neither us nor our IT colleague is a suitable fit to test on, nor are we a suitable fit to moderate an unbiased test with users. Hiring an external UX consultant will ensure results that are measurable, trustworthy and unbiased to ensure your next app reaches its full potential in the hands of your users.

Interact RDT is an independent UX consultant that conducts user testing, heuristic reviews, and other UX services to some of South Africa’s largest mobile app developers.