First version of football SNS: evaluation

29Mar10

To evaluate the user interface and user experience of the first version of my social network site I conducted an evaluation experiment. 9 football fans participated in this evaluation which consisted of an evaluation by think aloud procedure and a questionnaire. These evaluation sessions, on average, were about 40 minutes long. In this post, I will discuss the problems that became apparent during these experiments. I’ll also give my thoughts on some of the issues/suggestions cause I have to decide on which to act.

I will be doing one more evaluation experiment on wednesday, with my thesis supervisor prof. Duval. I am mentioning this because it means some more issues could be found, and I could change my mind about which things to focus on based on his thoughts.

1 General concept

The purpose of the site is to enable football fans to get the football news that interests them all in one place and to connect with each other. Users submit news they come across on sport websites, newspaper websites, blogs, … and vote on what they like. Users can see what their friends are submitting and voting on. The news is tagged and this categorization is used for the navigation between news streams.

All the test users liked the general concept. No fundamental problems, problems with the general idea behind the site were signaled. The test persons provided great feedback to improve the site. Even though they considered the site to be user friendly in general, there are a lot of things that can be improved.

For each part of the site, I will list the information gathered during the evaluations. These are issues, suggestions, remarks,…. Some issues were reported by multiple test persons (or I noticed it by analyzing their behavior), others only by one. I will definitely act on the issues reported by several users. Some things that were reported by only one test user had a more preferential nature. It is not a good idea to get into all of those. I will look at them on an individual basis and judge whether they are things that should be fixed or added in case of suggestions.

For each part of the site, I will list the issues and suggestions. Ideas relating to issues or things that should be improved will be put under issues as well. Suggestions will be the kind of things that are more about new features then about current features or solutions to address issues.

2 General

2.1 Issues

• Two test persons found the order of items in the top menu to be illogical. They thought putting “About” in beween “Join” and “Login” was weird. This will be changed. You can see the problem in the image below.

• At first it isn’t very clear that news isn’t automatically fetched from news sources but its submitted by users. I believe things like this are important issues for every website. Its about communicating purpose, about writing copy. This is something that will require an ongoing effort, not something that can be fixed easily.

• Redirects on actions that require user login don’t go back to original page after user logs in. This should obviously be fixed.

• Theres too much blue in some screens, especially the profile / friends feed (pictured below). This will be addressed by doing something about the background colour of story titles.

• Having to login right after signing up.

2.2 Suggestions

• Add widgets with latest news items of sports sites: My opinion is that it could be nice to have, but its a threat to the focus of the site as well.

• Add a way to list fans of a certain club. This is a way to stimulate social discovery and I think it should be added.

• Have the algorithm that takes care of sorting the news stories take into account the number of friend votes. There are several things to consider here. It adds complexity to integrate this to the algorithm, it makes the site less straight forward. There are advantages to having a uniform ranking system. I think its a good idea to show how many friends voted on a story but at this point I don’t want to change the algorithm in this way.

• Integrate friends feed as a category rather then a separate page. I think I like this idea. It increases focus by eliminating a separate page. At the moment there aren’t many kinds of friend updates implemented yet, this is a factor in this decision as well. I believe the friend feed still requires a lot of polishing anyway.

3 Categorization

3.1 Issues

• There is room for improvement when it comes to the top menu (picture below). There is a preset menu for people that are not logged in but logged in users can customize it to fit their interests. The feedback I received is that it isn’t visible enough and that it would be a good idea to use different colours (for example different colours for clubs, players, leagues,…). Will be implementing this idea.

• There is a category “World Wide” which every story is part of (can be seen in the first screenshot of this blog post). Several users noted that this wasn’t a very good description (“World wide”). Definitely need to rename this.

• There currently is no way to search for categories or an overview page containing them. These things are lacking at the moment and will be added.

• There should be a way to organize ones favourite categories. At the moment it isn’t possible to arrange the favourite categories, this was something not included in the initial implementation but it should be possible (preferably enabling the user to do this by dragging them around in some kind of edit mode, similar to arranging apps on the iphone). It should also be easy to add/remove categories in this edit mode.

• The way in which related categories are shown isn’t good (too small being one factor):

3.2 Suggestions

• A recommended category containing all the news from all your favourite categories combined. This probably is a very good thing to add. I had it in original designs but didn’t implement it for this version.

• Use the source as a category as well. This way people can look at the most relevant news items from a certain source.

• Custom categories. This could be used to for example make a list of players you want to follow and put them under one category “favourite belgian players”. This is definitely something interesting, its a bit a more advanced feature though

• Change the category system so that the news overview shows the news from the currently active categories. The user would be able to activate/deactivate categories at the top. Doing this would mean a big change in the sites concept. Definitely not implementing this suggestion. It could be an alternative later on if the current system would not be effective (but then that problem should have emerged during the evaluations).

4 News items

4.1 Issues

• Some users thought that clicking the headline of a story would open the comments section rather then open the original article. I’m unsure what can be done about this. The users had problems finding how to go to the comments too, the icon at the top left corner isn’t visible or good enough. Need to think some more about this!

• There could be a bit of a learning curve for the sharing icons (e-mail, facebook, twitter), especially the e-mail one (that one isn’t very straight forward it seems.

• Maybe the description for the votes (ie. “154 fans” and “like it”) could be better. Only one test person brought this up, I don’t think its bad now but I should continue to look for something better (tried several things with the paper prototypes already).

• Not possible to go to source website when clicking on it. Don’t know if this is something I want to enable.

• Categories, submitter and source should be more visible.

• The popup that opens when sharing a story on twitter is a bit weird in dimensions. Should be fixed.

• Share using e-mail opens the users default e-mail program, maybe a form is better here.

4.2 Suggestions

• Show number of comments somewhere, and perhaps number of friend comments. I thinks this improves the social factor. I want to include this.

• make it possible to exclude certain sources from the news stream (so when a user doesn’t like a certain site, their stories won’t show up in his/her stream). This is a nice to have, not a priority.

• Show which friends liked a certain story. Like the first suggestion, it adds some more social behavior so I want to include this.

5 Discussion

5.1 Issues

• Nothing happens when a non-logged in user tries to moderate a comment (should ask the user to login or sign up).

• Moderating your own comments is possible and shouldn’t be.

• Long user names aren’t being displayed properly here:

• User names don’t link to the profiles, should be fixed.

5.2 Suggestions

• Highlight friend comments. I like this one.

• Display the best comments (as decided on by the users). Could be nice, not really a priority.

6 Profile

Profile (feed)

Profile (info)

6.1 Issues

• Feed/info links are not clear to several of the test persons. Definitely need to change the way to change between these two parts of a profile page.

• The link to enable edit mode on your profile is hard to find (arrow on 2nd screenshot)

• Need to put up a default picture when the user hasn’t submitted his/her picture yet.

• The form for editing profile information needs to improve. For example, the favourite clubs field should have autocomplete based on the tags related to clubs.

• Date of birth not showing.

• Change “Recent stories” into “my recent stories” or “Wouter’s recent stories”, to make clear that its about the user which profile is showing.

6.2 Suggestions

-

7 Update feeds

7.1 Issues

• Voting on stories doesn’t work from the feed pages (bug)

• It somehow says “wouter submitted” when another user submitted a story.

• Distinction between news pages and the friends/profile feed possibly isn’t clear enough for some users. Didn’t seem to be too much of a problem during the evaluations though.

• Blueness of the page

7.2 Suggestions

• There should be a way to easily see which items are new since last visit. Possibly by placing a dot next to them or so. Would be a good idea I believe.

8 News submission

8.1 Issues

• Adding a story should be a lot easier. Adding content to sites often is already a barrier high enough for a lot of users. Title should be fetched from the site after pasting url, maybe even tags and summary. There are several more ways to approach this to make it a lot easier (bookmarklet, buttons for websites to use) which I should implement.

• http:// is required, it shouldn’t be.

• Form should display how many characters are required for the fields.

• Certain tags are in the autocomplete twice.

8.2 Suggestions

• Preview of how the news item looks after submission. Think this is a very good idea, will implement it.

9 Fuboh?

I asked the users what they thought about the name. No one thought it was great, only one user thought it was outright bad. The rest was somewhere in the middle. Its hard to find a good name for something like this, especially a fairly short one. The obvious requirement for the name is its domain availability. Ideas are very welcome! If I can’t find a better alternative, I’ll stick to this one.

10 Are the test users interested in using the site?

This was one of the questions I asked. The responses were consistent: It depends on the content, the community and whether friends use it. Most users do not seem eager to submit news themselves, especially not if its as much of a chore as it is right now. It is extremely important to optimize this process.

11 Lesson learned from evaluation procedure

My mouse, which test persons used to do the evaluation, is a bit defective (I don’t notice it anymore, its a small problem). I don’t think this interfered with the evaluation but its definitely something to avoid.

There probably were too many additional questions. People participating in the experiment weren’t expected to fill in all of them, but it still seemed too much work (for some users I wrote the answers down, others filled it in themselves).

12 Conclusion

These evaluations have provided me with a lot of information to move forward with. I was happy to learn that the main concept of the site is good and the main components (categories, …) are too. There are some things that need to be improved a lot (news submission), there are quite some bugs to iron out, there definitely is room for improvement in the design (usability issues). There are some other things beside the improvements listed in this post that need to be done before I can make the site available publicly. I need to implement e-mail (for activating accounts for example), I need to make the site cross browser compatible (currently only webkit-compatible, no major issues with other browsers though), improve the sorting algorithm and I need to create the required tags in the system (using an API).

There is one thing I should definitely look out for, that is the simplicity and focus of the site. I think the site should be more simple then it currently is (even though there were no problems in that area reported by users). Implementing new features can be a threat to to this simplicity. I will have to be vigorous about it.

Advertisement

Filed under: Uncategorized   |  1 Comment