Back to Articles * Please note all UI-related content in this article was accurate at the time of writing and that one or both sites may have made changes since that time.
Contents
- Test details
- Test participants
- Fundamental design differences between the two sites
- Discovery #1: Decisions in regard to viewing search results
- Discovery #2: Missing the mark in regard to displaying shipping fees
- Discovery #3: Understanding and consideration of user actions in regard to confirming orders
- Summary
1. Testing details
In order to provide an example of the sort of research we perform at Tacchi, we conducted an independent study of both the Amazon and Rakuten e-commerce sites, said to be Japan’s two largest examples of this kind of site. During this study we conducted usability testing wherein we observed research participants actually interacting with the sites to determine areas for potential improvement.
2. Test participants
Tests were conducted with five participants in total, one participant being tested per day. The participants were gathered from online sources and were all individuals with no prior connection to Tacchi. Test participants included both men and women and were all individuals in their 20s and 30s. In consideration of the potential for the participants to be apprehensive or nervous about participating in this sort of testing, as it was the first such experience for all of them, we did not ask them to come directly to our office but instead the researcher met them at the nearest train station and guided them to our office while explaining testing procedures and engaging them in casual conversation. By doing so the researcher was able to learn the general details of their lifestyles, their knowledge of digital products, and their computer literacy, and testing was therefore conducted based on this understanding.
The five participants all had an average level of computer literacy but were not particularly well-versed in digital products and therefore could be categorized as within the bounds of what constitutes typical users. However, while conversing with them the researcher discovered some differences in their online shopping-related needs. For example, one participant preferred to shop online instead of physically visiting stores for time-saving purposes, while another participant preferred online shopping as prices are generally cheaper than at brick-and-mortar stores. However, in some cases these needs overlapped depending on the situation and the product to be purchased, and therefore these points were kept in mind when analyzing the test findings, which will be detailed later in this report.
Testing took approximately 1.5 hours per participant, including roughly 30 minutes spent per site as well as discussion afterward. As prior payment, each participant was given a gift card and the participants were directed to actually use these gift cards to shop online as they normally would with the researcher then analyzing their behavior doing so.
3. Fundamental differences in design between the two sites
A major premise of this research is that there is a large difference between Rakuten and Amazon’s sites. Namely, Amazon makes use of a uniform interface while for Rakuten each seller has their own web page for their product details and this web page is placed within the common Rakuten interface (which includes, for example, the button to purchase the product), which results in considerable differences in design depending on the seller. Therefore, this aspect was left out of this research and instead Rakuten’s common interface, regardless of seller, was what was focused on.
4. Discovery #1: Decisions in regard to viewing search results
As the test participants already knew what they wanted to purchase, at the start of the test they were asked to use related keywords to search for those products. The participants then scrolled down through the search results to find those products they were interested in, while occasionally scrolling back up. The researcher noticed that, compared to when using Amazon, several participants took longer to click through to the product detail pages when using Rakuten. When asked what they were thinking about during this time, the participants answered they weren’t sure which products to select. Per their explanations, it became clear one reason for their indecision was the small sizes of the product images.
As you can see in the above screenshots, Amazon’s site has three product images per row while Rakuten’s site has only one. From the actions of the participants, it was clear that screen layout had an extremely large influence as at this stage participants were more likely to roughly scan through all the products looking for those that stood out rather than closely examining each.
When using Amazon’s site, participants often either moved their eyes horizontally across the screen and then downward in a left-to-right then right-to-left motion or scanned each row from left to right before moving on to the next row. On the other hand, when using Rakuten’s site participants’ lines of sight moved directly down the page, making it easy to miss information. Combined with the aforementioned difficulty in viewing the small sizes of the product images, the researcher learned from this that the participants were only viewing the part of the above screenshot indicated by the red, transparent overlay when they were scrolling through the list of products. As the product images themselves weren’t providing enough information participants were examining the product titles as well; however, since titles are of course words and not images, what was causing participants to take more time in choosing a product for which to view the product details compared to Amazon was the difficulty in quickly processing information while scrolling.
Additionally, Amazon places on the search results page not only the number of reviews but also icons showing the overall ratings for products. Rakuten, on the other hand, only displays the number of reviews. Participants used this information as one basis for their decisions on whether or not to click through to the product detail pages. As it’s not possible on Rakuten’s search results page to determine whether the overall ratings for a product are positive or negative, it’s difficult for users to get an idea of a product’s popularity or quality and users may therefore not click through to the product detail pages. As Amazon provides users with an easier product selection experience thanks to its reduction in the number of required clicks and the need to click through to additional pages, Amazon can be said to have some superiority over Rakuten in regard to providing information to the user.
What’s more, on Amazon’s site hovering your mouse’s cursor over a product provides additional details which gives users another basis for making a judgment on whether or not to click through to the product details, an action which of course requires loading that page and therefore takes a non-zero amount of time.
Unlike Amazon, however, Rakuten’s site includes a product comparison feature. Users access this feature by placing a check in one of the boxes for making comparisons, as shown in the above screenshot, and then clicking the comparison button at the bottom of the screen. However, this feature is somewhat complicated and difficult to understand. Regardless of whether the participant had used Rakuten before, there were no participants who had used this feature. Please see the screenshot below for an example of this feature in use.
Lastly, during the research many participants mentioned Rakuten’s point system as being one reason they liked Rakuten. Rakuten Points can be used with a number of Rakuten services, including Rakuten Bank. As this point system can be considered one motivating factor in choosing Rakuten, it may be possible to create a comparison screen that takes this into account. For example, if it were possible to display during some early stage of the buying process the expected number of points to be received, this could stimulate those users who like points and want to feel they’re getting a deal with the desire to make a purchase, and this could provide those users with a shopping experience that leads to an increase in happiness. During the test, some participants even mentioned they’d like to be able to sort the search results by the products that would provide the most points.
5. Discovery #2: Missing the mark in regard to displaying shipping fees
Another problem arose once participants had mostly found the products they were looking for: Participants with a preference for low-cost products wanted to know the product cost, including shipping, at an early stage of the purchasing process. Their reason for this was the importance in considering whether purchasing from an online store would be cheaper than from a physical one. What’s more, the cheaper the product, the more important the price of shipping was in making a decision whether to purchase the product or not. On this point, the participants were satisfied with neither Amazon nor Rakuten. In the case of Amazon, it’s not possible to check the price of shipping unless a customer clicks through two additional pages from the search results. Although for Rakuten this differs depending on the seller’s store, in most cases the customer cannot check the total price including shipping until they click through two pages from the search results.
The Qoo10 shopping site shows approximate shipping costs on the search results page. As per the screenshot below, the minimum shipping fee is displayed for each product (see the blue rectangles). Based on this it’s possible for customers to get an idea of the final price they will be paying without having to click through to another page.
6. Discovery #3: Understanding and consideration of user actions in regard to confirming orders
Next I’d like to compare the order confirmation screens for each site’s purchase flow. When comparing the two screenshots below there doesn’t appear to be any large differences in informational content between the two sites. However, Amazon guides you to enter gift coupons and other codes by having an input box specifically for doing so (the area delineated with a red rectangle below), allowing preservation of the shopping flow without the need to return to an earlier page to input such information. Compared to this, the difficulty in finding the area on the Rakuten site to input gift cards and coupon codes as well as using said cards and codes on that same site became apparent during testing.
One other thing that must be mentioned here, however, is in regard to the bottom portion of Rakuten’s order confirmation screen. If a user scrolls down there are sections for registering for Rakuten’s mail magazine and for registering for shops you’re interested in, as well as a section for a questionnaire.
Note that all checkboxes have already been checked without the user having voluntarily done so themselves. Because of this, users who have only confirmed the bare minimum required information higher on the page and who proceed with the purchasing process without checking this section will unintentionally be signing up to receive these mail magazines. If this page were designed with the users in mind, the checkboxes should first initially be unchecked and the users should opt to check them themselves should they desire to do so. Email inboxes, particularly nowadays, are already “overflowing” with messages and companies should therefore be wary of damaging their image and the trust placed in their brand by this sort of practice targeting users who have only made a purchase on a site, not intentionally signed up for email magazines.
Additionally, during testing several participants mentioned the mechanical nature of the recommendation feature. Although a feature that recommends products is useful, participants were concerned that, for example, recommended products were not ones they were interested in. It’s even possible this may have the negative effect of making it more difficult to search for the products they are actually interested in. In that sense, by paying greater consideration to users when designing sites, it may be possible to provide actually useful features that leave users feeling positive about having used them.
7. Summary
As per the above findings, usability testing provides the opportunity to examine user actions and to understand their thought processes. Making use of that knowledge for site design will allow users to more easily find, compare, select, and purchase products, and can lead to a prevention in users leaving a site and an increase in conversions. Additionally, that ease of use can be one reason for users to select your site, and lead to repeat purchases and a continual increase in revenue.