share
Meta Stack OverflowFastest Gun in the West Problem
[+763] [50] Omer van Kloeten
[2008-09-11 09:12:49]
[ discussion answers fastest-gun ]
[ http://meta.stackoverflow.com/questions/9731/fastest-gun-in-the-west-problem ] [DELETED]

I feel like there's a problem with Stack Overflow, as the number of people prowling it increases.

Each question's answers are sorted by descending score and then by descending time of posting. This means that if a person sits down and answers a question in a long, thorough way, going through every nook and cranny, once they post their answer, it will already be one of about seven different ones, some of which have already been upmodded. This wouldn't be a problem if those answers were as thorough as the one this guy's posting, but they usually aren't. Some of them are downright wrong, some aren't even answers to the question asked because their poster didn't bother to read the question all the way through.

This causes a problem I like to call Stack Overflow's Fastest Gun in the West Problem. I've come to a point where I'd rather just send a short, simple, correct explanation, than to go and do some proper research, write a whole blog post about it or even make sure the code I post even compiles, just so it will be noticed, as opposed to the incorrect ones.

I'm sure I'm not the only one doing this and that it despairs many people from even trying to answer questions.

I've long ago learned to try to always raise a problem with a solution in hand, rather than just say "This is a problem, handle it," so my question, after this long tirade, is:

How do you think this can be changed? What would you change in Stack Overflow to make this problem go away without hurting the site?

I promise to vote on answers I like, even if it takes you a long while to post them. :)

(71) create a quick answer, and then edit the detail in steps?? - Jonathan.
Please don't edit such old questions for such minor grammar problems, it bumps the post back on the front page. - Lasse V. Karlsen
Edit what old questions for what minor grammar problems? Please don't comment air. - Morten Bergfall
(4) @MortenBergfall he is talking about this edit - ajax333221
(2) @Jonathan. That's still too bad, we're always in a hurry. - Ramy Al Zuhouri
@RamyAlZuhouri, the time to click post and edit is insignificant compared to writing an answer that is long enough and thorough enough to need to click post and edit. (in other words if your in a hurry the answer your writing will be shorter and less thorough so you won't have to edit in the detail in steps anyway) - Jonathan.
@Jonathan. But the answers should be sorted considering also the edits then. - Ramy Al Zuhouri
[+297] [2010-05-13 18:41:33] BlueRaja

This problem was solved 80 years ago [1]. (See here [2] for XKCD-author Randall's explanation).

Of course, this answer is late, so it will never get upvoted :)

[Edit] Here is some code:

///<summary>
///Returns a rating for the given post.  Larger is better.
///Based on the equation found at http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
///</summary>
public double GetPostRating(int numPositiveVotes, int numNegativeVotes)
{
    int totalVotes = numPositiveVotes + numNegativeVotes;
    if(totalVotes == 0)
        return 0;

    const double z = 1.96; //Constant used for 95% confidence interval in a p-distribution
    double positiveRatio = ((double)numPositiveVotes)/totalVotes;

    //Crazy equation to find the "Lower bound of Wilson score confidence interval for a Bernoulli parameter"
    //Again, see the above webpage
    return (positiveRatio + z*z/(2*totalVotes) - z * Math.sqrt((positiveRatio*(1-positiveRatio)+z*z/(4*totalVotes))/totalVotes))/(1+z*z/totalVotes);
}

Here [3] is an example of the kind of ratings it returns.

IMO, downvotes should be weighted more than upvotes (controversial answers should be at bottom), and newer answers should be given preferential treatment. But even without those, this is a neat improvement.

[1] http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
[2] http://blog.reddit.com/2009/10/reddits-new-comment-sorting-system.html
[3] http://i.stack.imgur.com/4MXwV.png

(62) Holy crap! +1 +1 +1 - Phrogz
(1) +1 infinity, although the formula would have to be modified to take into account the pinned "correct answer" somehow. Even Randal of XKCD fame is behind on this one! - Gleno
(15) "newer answers should be given preferential treatment" - I should explain. When people are racing to answer a brand new question, the answers posted a few minutes later tend to be slightly longer but higher quality; but, by then the early answers have gained too much first-five-minutes-momentum. If you add a factor of say, someConstant/secondsSinceAnswerWasPosted to the above rating, this would give the later answers a precious minute or two at top to gather some votes of their own. And usually when someone an older question it's because none of the existing answers were satisfactory. - BlueRaja
(2) @BlueRaja: mostly right, but I've seen questions with accepted answers still getting answers a year later, often very poor answers from new users who don't yet know The Ways Of Our People. - sarnold
(2) @sarnold: In that case, the new answer will be conveniently at top for us to flag (or upvote). It will only be there a few minutes though, so it will likely be only the people who spend all day hitting F5 on the "active questions" page who see it at top, not the people who actually need to find an answer to that question. - BlueRaja
@BlueRaja: Ha! Good point. :) - sarnold
(2) Statistics FTW! However, this would require that the cost of downvoting is more or less the same as the cost of upvoting. Currently you get points for upvoting, and lose points for downvoting. That will seriously screw up the statistics (although there might be a way to adjust for it). - naught101
(1) @naught101: This is completely unrelated to a user's reputation, it's only for sorting answers on a page. - BlueRaja
(14) Yes, but the facts that a) you require more reputation points to downvote, and b) you lose reputation points for downvoting (and gain them for upvoting), means that down votes are much less likely, and are therefore more meaningful than upvotes (people probably put more consideration into their down votes than their upvotes - I know I do). The above equation doesn't take this into account at all, although it could, if you weighted the downvotes higher than the upvotes (eg. by total votes on all sites/down votes) - naught101
+1, even though I strongly disagree regarding up/downvote weights (I like them as they are). Note that the confidence score statistical method is completely independent of the process that generated the data (actual voting). - Jirka Hanika
(2) This would be a perfect solution, but unfortunately I feel people are put off downvoting due to the negative reputation it gives - especially new users with low reputation. - James Donnelly
@BlueRaja With my +1 (for posting the code), you are just 1 vote away from tie-ing with the most-upvoted answer. - Shiva
1
[+259] [2008-09-11 13:43:20] Adam Davis

I do NOT want to, in any way, discourage the quick and dirty answer.

I've asked questions that have received an immediate answer with enough information to get me past my block, but not served on a platter as you propose, with all the information I might need.

This is extraordinarily helpful - I know I can post something on here, day or night, and get an answer within minutes, often seconds, that will be better than searching through books, online, etc, even though it's a throw-away post that took someone 35 seconds, who may have only answered for the reputation. If it's at all helpful, I upvote it.

Later on I'll review my question again and select the answer that is what I consider the best of the bunch (I try and wait a day or so for long answers like you provide).

But I DO NOT want to 'adjust' the system to discourage short quick answers, or make long thoughtful answers that come a day late so highly valued that no one posts the short ones anymore.

Further:

Let's look at it from the perspective of what the site is supposed to be:

A long term repository of searchable, free, and freely editable information.

In the long run, it doesn't matter if the long, good answers don't get the initial rush of votes. They will ultimately go to the top because

  • There are generally few answers per question, so future searchers will read the entire post and may upvote the better answer. The special case of the question with tons of answers is just that - a special case, and it will be gamed and mined for rep a dozen ways from Tuesday, so it's not worth fixing - it's less than 1% of total content - don't design the system around the special case, or else all your cases will become special.

  • Once someone posts a new answer, the post gets a new chance to be on the front page, which will entice some more people to see it that haven't - they'll upvote the good, long answer at the bottom of the page if they feel it's better than what was initially offered.

  • The person who initially asked the question will often come back later and review it, and if your post is truly the better answer they'll select it as the new accepted answer.

A key point, though, is that if someone is looking for the information later, they'll read all the answers unless it's obvious that the first one is good enough for their purposes.

So that completely guts your argument that it's important that the longer, complete answers should get a boost to the top, somehow. The only reason left to complain about the tradeoff between short posting and long posting, then, is because you may not get as much of the initial rep as you would otherwise.


I respectfully disagree; I believe some of the statements of fact here are not true. Please see this response for evidence: beta.stackoverflow.com/questions/56103/… - Jason Cohen
(15) I disagree too. Even in beta there are enough new questions that old answered questions get rotated off the front page. Most of your argument assumes that the person who asked the question performed due diligence to maintain that question. I don't think that happens as much as it should do. - tonylo
(2) tonylo, the main problem being discussed here is that there's a lot of initial activity on a question, most of which makes spending time on a great answer less advantageous. So the issue is really only with questions that will stay on the front page and deciding whether to make a great post or not. - Adam Davis
(19) Taking your stance as valid, I think there is still an issue whereby most question-posters will not come back a day+ later to upvote/select the best answer. If there was some [more obvious / greater] incentive to do that then I could agree with you more completely. - Sparr
Well to combat this and other problems with too many users hitting threads, it could always be set up with invisible 'mirrors', but that would mean that not every person would be able to see every question, but it would have to stay consistent so people would always be in the same realm together. - Ryan
(3) The bounty system is working very well to give the detailed long answers a chance for great rep increases... - Adam Davis
(54) I disagree. As to your key point, I don't think people who come much later read through all the answers and upvote the good ones. A more likely scenario is that they go through the first couple of answers, because in the mind of that person, the first couple of answers probably a) are the best b) contain 80% of the information one would get by reading all of the answers, which might take considerable time. That person is not likely to review all the answers, even if the first couple do not contain all the details he cares about. Would you start from the middle or end of the list of answers?? - allyourcode
(21) In that case, the quick and dirty answers that start at the top tend stay there, and the carefully thought-out ones tend to stay near the bottom. You can't dismiss this by saying "when that happens, it means the quick and dirty answers are deserve to be on top". The problem is that quick and dirty answers benefit from a positive feedback loop: the higher an answer is in the list, the more likely someone is to view and upvote it. Long and thorough answers, on the other hand, have to fight an uphill battle in order to gain attention, a prerequisite for upvotes. - allyourcode
(6) @allyourcode - but if the quick and dirty answer is right and sufficient to resolve the issue, then who is to say that the longer, more ponderous message is righter or sufficienter than the quick and dirty answer? Why do people hold so tightly to the idea that a longer, perhaps even more complete answer, is better than the shorter answer which also resolves the problem, and has come earlier to boot? Keep in mind that we're not talking about wrong answers. Both are right. - Adam Davis
@allyourcode - further, my own experience shows that longer posts which come later still receive significant upvotes over time. Otherwise the necromancer badge would never be awarded. People still vote on posts I made 2 years ago. Your assumption that people don't go back and upvote posts, even shown way down on the page, is flat out wrong. But don't take my word for it - examine the data dump and draw your own conclusions. - Adam Davis
(1) It's not necessarily about "length". No one is suggesting that word or char count be used as the sort criterion. As for "this is sufficient, therefore it should win", you're assuming that every answer that reaches some threshold of completeness is just as good. I completely disagree. - allyourcode
(6) re Pollyanna's second point: I do not doubt that people look at all answers to old questions occasionally, and upvote the better ones, but I think that's rare. One of the points that I'm trying to make is that there's alot of room for improvement when the desired outcome is requires a whole series of rare occurrences. - allyourcode
A quick, but relatively complete answer: ok. A quick "this question has been answered" gamer is not ok (so either I keep typing and ignore it, or kill yet more time waiting to load the other questions to see if any are ok). I can all but name names of people who habitually answer with < 10 words on certain tags then come back and add enough to actually get an answer. At what point is enough rep enough rep? - Jim Leonardo
@Pollyanna: Do you mean "There are generally few answers per question", rather than vice versa? - Andrew Grimm
(4) Even though I encourage quick and dirty answers, better (and later) answers should triumph over quick and dirty ones...in the long run. But "in the long run" is not good enough for right now. The underlying problem is that questions with one up-voted post are considered "answered". Early upvoted posts build momentum by being shown first and allot the question as "answered", discouraging delayed higher-quality posts which would provide better information to all future viewers, but which can never build the same momentum as the first quick and dirty answer. - pate
George Patton said (in the military) that a pretty good plan executed today was worth more than a brilliant plan executed next week - Tom Au
2
[+182] [2008-09-12 02:11:28] Justin Standard

Grace period idea (currently editing...)

One solution would be to have a grace period where votes are hidden:

  • The grace period begins when the question is posted and lasts for some set length of time.
  • During the grace period the following is true:
    • Sorting by votes is disabled (questions are displayed in a random order if 'votes' is chosen)
    • Votes from other users are invisible meaning...
      • At first all answers appear to have a vote of 0
      • Any answer that you vote on will display your vote only (+1 or -1)
  • When the grace period ends, all votes which were hidden will be revealed.

Maybe the grace period could be the same as the grace period for editing the Question.

@Jeff Atwood

Sounds Complicated

No more Complicated than...

  • a grace period in which your edits don't count as real edits.
  • hiding the number of Offensive flags from some users and not others.

All you need is a simple flag that switches between two modes of the question: "Grace Period Mode" and "Regular Mode". You must already have this because of the grace period for counting edits on questions.

Will people understand what is happening?

Yes, if you explain it to them. Lets say we name this grace period "Secret Voting". All you have to do is put a banner of some sort above all the answers that says "The Voting is Currently Secret". When the grace period expires, the banner would be removed.

Also during the grace period, the Votes sort tab would not be present, but instead a Random sort tab would be present. Again, people will understand it, because the interface explains it to them and doesn't make them think.


(21) I really like this idea - partly because I only upvote when the answer seems lower than it should be. With votes not visible for the question's first appearance on hte homepage (until it falls off, maybe?) you'll get a much more interesting, and possibly truthier, voting. - Adam Davis
sounds really complicated -- will people understand what's happening? - Jeff Atwood
(1) Jeff, I think if you add some visual clues or simply descriptive text on question page, people will understand. IMO it's not that complicated. - aku
Agreed. See my updates to the post addressing Jeff's objections. - Justin Standard
(1) I like this idea - except. It doesn't yet address the problem where a good answer is added a long time after the question is no longer hot. - Richard Corden
(8) Disallow marking an answer as "correct" during the grace period as well. This increases the chance that the "correct" answer isn't just "the best answer to arrive in the first 10 minutes." I envision a secret-voting period of, say, an hour, but the editing grace period is 10 minutes -- too short. - Rob Kennedy
@Rob I like your idea. It encourages the questioner to return back to the site and accept an answer. But it also have a risk of questions with no accepted answers. - blntechie
(5) @blntechie we already have lots of questions with no accepted answers - jcolebrand
(3) I think this is a great idea. @Jeff any chance of this being considered? - Box9
(1) @box9 we already show answers randomly by vote, so the important part of this is implemented. The rest, well, it's been 2 years.. so.. :) - Jeff Atwood
(27) @Jeff, true, but I don't see the random sorting feature really getting to the meat of the problem. The secret voting idea (with the banner), however, will really encourage people to vote more and more objectively. - Box9
(11) Man, I love this idea... - Gleno
(6) +1 Hiding votes for a period seems like a swell idea - Hemal Pandya
(3) Another +1 for hiding votes for a period. The clock should start when the first answer is posted, not when the question is posed. The problem with random display of equal weight votes is that the first and likely incomplete answer will get an up vote and then subsequent ones will all randomly sort after the first one. Then the first one, with it's one vote keeping it first, can be edited into a better answer. - Josh Heitzman
While I like the idea, its current form would directly (although temporarily) reduce the usefulness of this site, as more highly voted answers wouldn't be placed ahead of others. This is assuming that early votes hold some value as a measure of usefulness. If so, then the question becomes, is a grace period truly worth the tradeoff? - Philip
I proposed basically the same thing, except in the context of eliminating the incentive for tactical downvoting: meta.stackoverflow.com/a/132034/130885 meta.stackoverflow.com/a/132889/130885 - endolith
3
[+104] [2008-09-11 10:36:22] Sam Hasler

I've noticed myself reacting to this too, although I try to mitigate against it as much as possible.

I sometimes try and post a short correct answer as quickly as possible to kind of "mark" the question with my answer to discourage other similar answers. I then I would edit my answer to improve it, using the 5 minutes grace period to edit without it appearing as an edit (although I'll still carry on editing after then if I can improve it, I just use the grace period to edit without worrying about writing edit descriptions).

Conversely, if I find someone has answered the question while I was writing my answer, and I don't feel my answer adds anything, then I've deleted my answer on occasion, although if I provide a link and the previous answer doesn't I'd leave mine. Once I can edit I'd be inclined to remove my answer and add the link to the other answer. Since comments were added I've also been adding to answers by commenting on them, again, once I've got edit rights I might edit directly for simple improvements (adding links, spelling, grammar, formatting, etc.) and just use comments for actual comments.

I've also learnt a few of the keyboard shortcuts for the edit box (hover over the buttons to see them on the tooltips.) The shortcut I use the most is CTRL+L to add a link to selected text. That makes it much quicker to write a quick answer with a few useful links in it.

I've also taken the attitude that there is also a long game to be played. If you can provide a better answer then the initial rush of points that the first answers got is possibly just the tip of the iceberg and there's probably much more points "underwater" to be made once the site is indexed by Google and we start finding answers to questions in Google search results. (although the fact that new users can't vote straight away - you need +15 rep to upvote - might mitigate against this.)

Update: for a really good description of how the long game will play out and why the fastest gun answers aren't a problem see Adam Davis's Answer [1].

[1] http://meta.stackoverflow.com/a/9758

(2) That's exactly what I do, too. However, once you hit submit the 5-minute clock starts ticking, so your short answer only gets as good as you can make it in 5 minutes. - Kyle Cronin
I just carry on editing, take a look at the edit history for this answer. - Sam Hasler
(10) the link is broken in here. - Rosinante
Hmm, it doesn't do a good job moving posts between stackoverflow.com and meta, you lose the edit history and it doesn't preserve inter-question links. - Sam Hasler
I answer a lot from my mobile device. This does not allow for a lot of citing or code examples. I often will have to revisit the answer once I get to a pc so I can elaborate or cite sources. - rlemon
4
[+61] [2008-09-11 23:39:42] quux

The reputation system is valuable, and I find myself emotionally validated by it a bit, but I also have to keep reminding myself that it is an illusion.

There are likely more points to be gained in the first 24 hours of a question than in the whole rest of its lifetime, because a lot of fast guns will be searching the unanswered questions, looking to build rep. And because the person who asked the question is likely to choose an answer and mark it "correct" within this fairly short time, while the question still matters to him. So, for those who see the StackOverflow site as a game where building reputation is how you 'win' ... yeah. Fast guns will 'win'.

But a lot of StackOverflow users will be playing a different game. For them the 'win' happens when they get an answer to the problem they have right now. The people playing this game are probably much less likely to even notice the reputation game. Many of them will probably not even up-vote the answer which works for them. Long term, I wonder what the stats will show: which percentage of asked questions never get an answer chosen by the question-asker? And stats can't really reflect this, but how many people will get an answer without ever posting a question? They may find their answer via Google, or by starting to type a question and the site's own search engine flags a similar question/answer - one that's good enough. Will they bother to vote? heck, a lot of them won't even be able to - because they haven't gained enough rep to vote. They never started playing the rep game.

So, an answer which never even gets rated might be the one that's really solving the most issues for this (potentially huge) group of people who never play the rep game, or never even notice it. That's the illusion: the Illusion Of Perfect Feedback. You're never gonna get perfect feedback. So in this sense, the ordering of the answers will never be perfect. It is possible that it will never even be very good. Because it cannot be.

So, fast guns, slow guns, medium guns: write the best answer you can, no matter how long it takes, and realize that even if you never get a single point on the StackOverflow Rep-O-Meter, you may be helping a lot of people.

I'm not dissing the rep system or the fast guns. They kickstart the process, and that's valuable!


(1) i think the fact the question searching appears to be so unreliable compounds the first point you make - questions seem to get lost in the melee... - flesh
Maybe asker should be able to vote on answers. - Joshua
Actually rep is not an illusion. Rep determines what you can do on the site, it determines not only how much influence you can exert, but also how much you can contribute at all. I cannot even disagree with your answer via downvote here on meta, because of this alleged illusion, simply because I posted something unpopular and on meta "downvotes mean something different", although the fact that I cannot disagree with your answer shows that to be false as well. - Joshua Drake
5
[+36] [2008-09-11 09:20:16] Mendelt

I think that this site has to become more like a wiki where many people add small contributions to existing answers instead of creating their own new answer. Unfortunately the reputation system as it is now doesn't really encourage this yet.

I understand from listening to the podcast that this is the direction the creators of stackoverflow want to go too. Planned features like making questions and answers comunity editable would be a big step in the right direction.


Questions and answers are community editable, but you have to have significantly contributed to the community before you can so so (3,000 reputation) -Adam - Adam Davis
It only requires 2000 rep to edit answers, not 3000. - Chris Upchurch
(3) You're right. Although I ran into interesting interaction today - I deleted comments between I and another user that had no further use in the topic, and they tracked down my email and called me a censor - expecting that our off-topic conversation would last on Sof forever. People won't like edits. - Adam Davis
(1) People don't like other people who judge, decide upon or have the authority to change/delete what they have done. - Bora
6
[+32] [2008-09-11 14:46:22] Aidan

Remove visible scores.

Yes, you hear me right! It feels horrible knocking someone down, especially if you take them into the -1 or less realm! If the scoring system is hidden and you just say "Yup like it" or "Nup not good" and the actual effect that it has is hidden then people will be more likely to rate answers (ecspecialy down) and the chaff may remove itself more effectively.

A second solution would be to not allow ranking until a question had been open for a while (an hour say) then all questions have an equal chance (if you display them in a random order to different users all the better). Compare to this post [1], another problem with being slow :D.

[1] http://beta.stackoverflow.com/questions/56103/fastest-gun-in-the-west-problem#56125

(6) I really like the random ordering among equally rated answers. - Caspin
(1) Hmm... I like the idea. This is going into my Greasemonkey cache. - new123456
(6) But the whole point of visible scores is so that the Google-searchers know which answers to trust... - Kate Gregory
7
[+21] [2008-09-11 09:55:13] Rik

A related problem is the fact that when a question already had 10+ answers, an answer with new information given at this time tends to get lost in the clutter in the bottom and not get the exposure it deserves.

One possibility would be to position an answer based on both its rating and its "age" (for lack of a better word), giving recent but already voted answers a place before older answers with a similar rating.


choose "newest" instead of "votes" ordering to view answers - David Schmitt
(1) But it's not the default, nor - necessarily - the 'best' way of viewing answers. The types of answers referred to will still suffer bias. - Bobby Jack
With "votes" ordering the secondary sort is newest first; so newer questions appear above questions with the same number of votes. - Sam Hasler
@ Sam - is the secondary sort really the newest? Or can you set it up to be? - Richard Corden
8
[+17] [2008-09-11 12:28:02] Gaius

I think macbirdie [1] is on to something. I would recommend a slight enhancement, though:

  • for questions that have no answer marked as "the answer" by the question-asker, sort by "oldest" as the default
  • for question that do have at least one answer marked as "the answer," sort first by "the-answer?" then by "votes", then by "newest" (the current scheme)
[1] http://beta.stackoverflow.com/questions/56103/fastest-gun-in-the-west-problem#56207

9
[+15] [2008-09-11 09:27:12] Bobby Jack

I agree, and fully admit to being guilty of this myself, on occasion. I have, thankfully, seen such 'answers' being modded down, but the level of this occurring varies hugely from question to question - some 'bad' answers are fairly quickly modded down, others are left, possibly, indefinitely.

I think the point about wiki-editing is significant. If the barrier was much lower, we'd get better quality answers, and people would also be less keen to run around, scrabbling for rep. I think Jeff's reservations about this are slightly too great - wikipedia has proven to work without such draconian restrictions.

I also think it would help a lot if up-voting also cost, just like down-voting does. Currently, there is nothing to discourage people from rushing around, blindly up-voting everything they come across.

UPDATE: I wonder if there should be a limit - either time, or number of views / votes / answers - before answers can be voted on at all ...

UPDATE: To clarify the delay concept: this would only apply to answers (but all answers), and could either be a 'reasonable' time to mask 'first posts' (e.g. 10 minutes), or could be dynamic based on number of answers, number of views, etc. "InSciTek Jeff" had a great idea relating to 'blind' votes (see question comments).


"Currently, there is nothing to discourage people from rushing around, blindly up-voting everything they come across." other than the 30 votes per day limit. - Sam Hasler
Yeah, I must admit I rarely run into that. It's still quite a lot of potentially 'random' up-votes for all but the most obsessive ;) - Bobby Jack
(1) I like the "I wonder if there should be a limit - either time, or number of views / votes / answers - before answers can be voted on at all ..." Jeff nuked an idea i posted to uservoice regarding a delay period for vote to be shown. "A pathology of its own", he said. - Stu Thompson
10
[+14] [2008-09-11 11:50:38] Omer van Kloeten

Many comments here refer to the wikification of the answering process: Merging comments, editing each other's comments, etc.

As far as I see it, we're talking about two different types of sites here: Collaborative and Competitive.

  • a Collaborative site is one where the strive is to reach the correct answer for the sake of the correct answer, without regard for selfish advancement. Everyone selflessly adds input, comments on each other's input, etc.
  • a Competitive site is one in which each contributor's work is measured and compared to others'. This is the approach which brings more user involvement (who doesn't want another badge?), but has some other downsides to it too.

SO is a competitive site, rewarding each of its members for their individual effort, so I'm looking for answers that go more along those lines.

I don't pretend to try and change the whole perspective of how the site operates to being a collaborative site.


It's both. So users can get whatever intrinsic or extrinsic reward they want for answering questions but the askers still get their answers. - Sam Hasler
+1 Excellent point. It cannot be collaborative because we do not have joint goals beyond are own; and the points proces is clearly about personal points only,there are no points for collaboration,for getting to the most correct answer. - PerformanceDBA
11
[+12] [2008-09-11 10:18:42] macbirdie

One thing that could possibly help in this case would be to force the "Oldest" sorting mode for the question owner so he goes through all the answers.


(2) i raised a user voice for exactly this, much like a forum, there is value in being forced (as much as you can force anyone) to read the answers people have provided rather than lazily upvoting the top answer because it's there.. - flesh
12
[+12] [2008-09-11 10:53:56] sdfx

You could let old answers "fade away", unless someone votes on them and they are not one of the top n answers. Hacker News uses this technique to gray out downvoted comments so you can ignore the bad answers and focus on the good ones.

But I guess the better way would be to change the culture of SO:

  • If you ask a question, you should own the answering process. Vote, comment and clarify heavy on the answers. People will see that you care and keep answering even on older questions.
  • As a reader, use voting more deliberately. Especially the downvote-button is useful to push new answers above inaccurate old ones.
  • Too many rules are confusing and rarely the solution to more than one specific problem. Provide a framework with few, simple rules and let the community do the rest.

Yes, I know: if we don't have all these safeguards in place, the barbarians will burn down the town, once we open up the gates. But a lot of the rules are there to make things nicer, not safer.


Just a preference, but I think downvotes should only be used for incorrect or misleading answers. Upvotes should be used to push the cream up, and ownvotes to push the rotten items down, but I wouldn't want people to downvote a question when it applies just because it's not as good as another... - Adam Davis
True, that is probably what jeff had in mind, but if downvoting had no negative effect on your reputation you could use it that way. Btw: 800 views and only ~60 votes on the answers? Are they all bad, or does the general user only vote on really good stuff? Why not encourage them to vote a lot? - sdfx
Once we get out of beta yo'll see far less than 10% of the viewers voting. It would be great if everyone voted often for all good answers, but it's either human nature or culture that we really only vote up the best one if it's currently an underdog, or really exceptional - votes are scarce. -Adam - Adam Davis
13
[+10] [2008-09-11 15:09:43] Apocalisp

Get a grip. The idea of the site is to get answers to straightforward questions and then get back to friggin' work. If one asks a question, one is going to be looking for what one thinks is the right answer, so one is going to read all of them.

Before SO: I'm looking at the entire internet full of content that isn't remotely relevant to my problem. After SO: I'm looking at an unordered set of answers to my specific question, written by real people, and one of them is probably a nudge in the right direction.


14
[+8] [2008-09-11 09:39:44] tonylo

I completely agree. The pattern of a lot of questions and answers almost appear to look like reputation trading rather than providing quality information. Obviously this is an annectdotal opinion.

The system does appear to reward incomplete but quick answers.


(1) ...and does not reward those who think an answer out completely. - Stu Thompson
15
[+8] [2008-09-11 11:13:13] Martín Marconcini

I think that it works ok the way it is. What is the problem of 10 short answers BEFORE yours if yours is good? In the end, people will hopefully vote yours up.

Out of my mind a few things thay may help are:

The asker may have the ability to "hide" certain useless answers, or the ability to MERGE two (or more) answers into one single text, making it easier for the reader to follow the answers. Like "The Answer" is conformed with text from answer1,2,3,4 and 5. Preserving the author's name for each "part" of "The Answer". (Sorta like SVN Blame) ;)

Some sort of visual cue indicators that "help" the reader to see that there are more good answers "below". And stuff like that.

Maybe all this makes no sense at all. :)


(1) If there are 10 short answers before a good and correct one, then that good and correct one will be on the second page that no one reads. - Josef
"No one" apparently includes you and me. - harpo
(1) While I like the idea of letting the OP hide or merge (or otherwise define the sorting/visibility) of answers, I think you are neglecting the lemming mentality we seem to see with new questions and their respective answers. The OP should not have to actively manage posts in thread for the first day - Stu Thompson
16
[+8] [2008-09-11 12:37:15] Nick Pierpoint

Following on from Omer's distinction between a Collaborative site and a Competitive site...

It would seem that we need the best of both worlds. There's nothing wrong with a bit of healthy competition so that people strive to produce timely, high quality answers. However, we don't want the answer to be lost between several people's contributions. Certainly marking answers up and down helps the good stuff filter through to the top, but if there's lots of good answers you really need a combined view.

I suggest the question owner take on (at least initially) driving the collaborative element and start the 'wikificiation' process of combining the best bits of several answers into (drum roll) the answer.

The question owner can do this today by adding another answer, but really it's an additional feature that should be put into the tool.

I'd say that most (90%?) of questions wouldn't need this extra step, but where there are several high quality answers having a single point that draws them together would be excellent.


17
[+7] [2008-09-11 09:46:48] John the Seagull

I think that a time buffer is a solution, but the buffer should, instead of buffer the appearance time of the answers, decide when people can vote. Answers will appear as they are being written, but votes can be issued after a half hour or ten minutes, whatever.

Given that you won't be upvoted in that period of time you know you can take 10 minutes or a half hour to prepare the answer.

The saddest part of it all is: What is reputation actually good for? I mean, besides a little ego trip? I'm as guilty as the next guy, but it's still sad.


(1) Yes, rep is definitely a head trip. But it's also tied to rights, and there are obviously a lot of users who are eager to contribute and make the site better. I was so excited when I could retag! Yes, it is sad. - harpo
@harpo +1 people on meta often seem to forget that rep is more than gold stars. - Joshua Drake
18
[+5] [2008-09-11 09:22:40] Galwegian

I agree to a point, although I have to guess that the site has not even come close to settling down. Users are still in a mad panic, scrabbling for reputation points and badges of any shade - what you are describing is bound to happen... initially.

It will calm down.

Having said that, I've noticed a pleasant tendency for people to vote for the actual best answer, most of the time.

Despite all the bilge posts, it seems the cream does often rise to the top!


19
[+5] [2008-09-11 23:57:31] aku

@ Wedge [1],
I would disagree. It would be too idealistic to believe in collaborative intelligence. Ivan Hamilton has some interesting thoughts [2] on this problem. Most people wouldn’t spend their time to check correctness of someone's answer. In many cases they don’t have enough expertise to make correct judgment. Most of the time people vote because they like some answer or feel that it's correct. That’s why trivial answers have huge vote count.

It’s a basic psychology. Herd instinct [3] makes people to believe in quality of up-voted answers. Here is a typical situation: Someone posts a short answer that looks nice (but might be incorrect or just off-topic) In few moments it gains some votes. After a while popular answer becomes covered with votes like a snowball. Even if someone posts an answer that is really correct but doesn’t look so nice, it will be literally buried under a mass of other answers.

Of course there are good examples when SO works as expected – stupid answers get down voted, good answers bubble up. But it’s too naive to suppose that this model works in most cases.

Some people talk about long-term perspective: “when site goes public a lot of people will vote and answer author will gain scores. Good answers go up, bad answers go down”. When site goes public most people will come just for a quick answer for their problem, it’s the same way search engines work. Highest ranked question will be chosen.

Now try to recall how often you scroll to next page of search engine result. How often do you scroll to the end of the first page? When question polluted with tons of answers it’s really hard to choose, unless you’re proficient in given field.

In general case vote count is measurement of popularity, not quality. To mitigate this problem I propose following scenario:

For N minutes since question posted, people can’t vote on any answer.

If question gathered some feedback in N minutes it must be shown on a main page. Answers must be sorted in random order for each viewer.

Now competition begins. There is N equal answers to choose (displayed in random order)

That’s way most answers will have equal chance. Since there will no way to gather reputation in first N minutes, people will spend more time on their answers. Now when question reappears on main page, it will contain N equal answers sorted in random order. People will not be affected by influence of first-the-most-voted answer; they will see N answers and will be forced to read most of them to make a choice. Since questions are sorted in random order, there is equal chance for all answers to be selected.

Key point is to split process into 2 stages:

  • accumulation of answers
  • selection of best answer
[1] #57579
[2] http://ivan.blogs.chimerical.com.au/post/2008/09/07/Date-raped-by-StackOverflowcom.aspx
[3] http://en.wikipedia.org/wiki/Herd_instinct

20
[+4] [2008-09-11 14:09:35] OwenP

I do my best to answer with the detail that I find is appropriate. In my limited experience, I have found that the "fastest draw" does get some up votes and initially my post falls beneath them. However, I find that if your answer truly has more value than any of the other answers, yours will quickly get voted much higher than the previous ones.

Here's an example [1] of this in action. Eric's post was correct, but little more than a code snippet. I decided it's better to teach someone to fish rather than to catch it for them so I explained why the code snippet worked. By the time my explanation was finished, Eric had 3 or 4 up-votes. Notice how over time, my answer got more votes than Eric.

I think SO corrects this itself. I browse answered questions from time to time and up-vote the good responses, particularly when the post teaches me something I didn't know. There's not really a solution to this problem, other than hiding the reputation number so people won't be carefully calculating their every move to maximize their reputation boost. If you decide that the up-vote is more important than providing the best answer, then you are part of the problem. If instead you'd rather be known for always providing a detailed and correct answer, I think you'll find that the "problem" evaporates.

I'm here to help other people and get help myself; reputation is just an arbitrary score that indicates how many times I've had the most interesting/correct/accepted answer to a question.

[1] http://beta.stackoverflow.com/questions/44408/how-do-you-generate-a-random-number-in-c

21
[+4] [2008-09-12 06:46:12] Peter

I think allowing multiple answers to be selected as correct allows the questioner to acknowledge caveats, different opinions and nuances. Often an answer does not cover all sides to the question, but two or three might.


22
[+3] [2008-09-11 09:59:19] Vaibhav

Here's what I have noticed, and tried as well:

Put a one liner real quick, and then edit the answer and add more details. But this is not a solution to the problem. Once again, the only reason i would do this is because if I am spending time to put out a well constructed answer, I would like it to be read.

For a solution the (programmer's) mind suggests that you should take a token to begin an answer, and then the answers are posted in the order the token was taken regardless of who submits first. Of course, there can be many problems around this.


23
[+3] [2008-09-11 13:48:45] Jason Cohen

@ Adam [1]: I appreciate your argument, but I respectfully disagree.

I find that now there's a flood of answers within minutes. Yes, it used to be one fast and several later, but now it's 2-5 immediately, often with very similar information, often with shallow, trivial information. I picked a random question current at the time of this comment, and you can see this effect for yourself [2].

These linger around forever. Usually people don't down-vote, perhaps because of the combination of losing rep and having just 30 votes per day.

I also disagree that "if someone is looking for the information later, they'll read all the answers." I don't think people carefully look through all the answers, and again I don't think they'll be down/up-voting as they go.

[1] #56557
[2] http://beta.stackoverflow.com/questions/56518/c-is-there-any-difference-between-bool-and-nullablebool

We'll just have to disagree. If i'm working on a project and need an answer, I search Sof. If a question is already here, then I do read through all the answers - so the purpose of Sof is preserved - those looking for information will still get it. - Adam Davis
For general Sof users, who aren't terribly interested in the question, yes, they often won't read all the answers, and they won't vote unless there's a compelling reason to (ie, if the answers look well ordered, why mess with it?). But for people who need the answer, every answer is often used. - Adam Davis
Fair enough. I've now been keeping my eye out, and I think I see both phenomena. Perhaps you're right that, even with all the dups and crap (which undoubtedly is there), there's enough of the effect you're describing that votes happen and good ones bubble up. - Jason Cohen
Well, we can certainly hope so! What we're seeing now might get better or worse once out of beta. This is essentially a control system with a feedback loop, but control systems with humans in the loop are notoriously finicky to adjust. One of the reasons I generally take a wait and see attitude. - Adam Davis
I'm don't see the problem from the example you give. There's a simple question and several highly up voted correct answers. If you require that every answer to a question be prefaced by an extensive, informative thesis on the subject at hand, then I think your requirements are excessive. - Wedge
@Wedge: At the time of writing that question didn't have those properties. However, now it does. This does support your argument, so yes I think I'm coming around to your/Adam's point of view. Thanks! - Jason Cohen
I always read through all the answers and select the one that teaches me the most. If there is a tie, I choose the first one. I up-vote all posts that contain a usable resolution, but I cannot down-vote bad answers because my score is too low. This is probably another reason that people do not use their down-votes. - hosch250
24
[+2] [2008-09-11 15:07:57] harpo

I like the idea of hiding the scores [1] on the question page. Knowing an answer's number of votes somehow affects my voting. Clearly we are more liberal with upmodding than with downmodding.

I also like the idea of showing answers in a random order for, say, the first day or two (except the chosen answer). As a result, "bad" answers will sometimes show at the top during this period. This "undue" notice will actually cause the truly bad answers to receive more downvotes, and hence improve the eventual sorting by rank.

[1] http://beta.stackoverflow.com/questions/56103?sort=oldest#56742

25
[+2] [2008-09-11 17:28:28] Brettski

Fastest gun isn't always a bad thing, it only gets annoying when people start counting their position in the post. Like many issues which are solved in development, they can be fixed twice. The initial to fix the immediate problem followed by more thorough fix so it never happens again. As a responder to a question, you can give the quick and dirty response that you are thinking of, then research it and follow up with a more complete, thorough answer.

Since this site has been designed to allow you to re-edit your posts, and others posts (once you have enough reputation) it is perfect for a quick answer followed by a more thorough answer. You never know, somebody's initial thought may help you with your approach.


26
[+2] [2008-09-11 20:44:48] Wedge

I think that to the degree this is a problem it's mostly a problem with the beta. The limited user pool means that sometimes answers may languish a while before they get attention. When the user pool grows by several orders of magnitude I don't think this issue will be as important.


27
[+2] [2008-09-12 05:40:01] helloandre

This may or may not have already been said, and as of right now, there are 39 answers. some very long-winded. I chose to be lazy and not read through them all.

One way to prevent duplicate answers from appearing right after one another is when someone goes to post, let them know if there have been any new posts since they started writing theirs. This would prevent something that's happened to me where someone was quicker to the draw in saying the exact same thing i just did.


(1) It's exactly what I proposed in one of my posts. It's not hard to implement, but will reduce number of dupes. It's already implemented for topic creation - you can see in real time similar questions. I will put this request on uservoice as soon as I get spare votes there. - aku
I absolutely second this. Seeing whether somebody else (possibly even somebody with considerable reputation, increasing the probability of a high quality answer) has started answering the same question would reduce stress, unnecessary duplicate work and redundant research made my two or more people at the same time. - Pëkka
28
[+2] [2008-09-12 06:11:55] aku

@ contagious [1],

I created request on uservoice:

Provide live monitoring of answers being posted [2]

To help us fight against tons of duplicate answers, provide following functionality:

At the time person is writing his answer show excerpts from answers being posted. Color code similar answers.

It can be implemented in the same way it's done for new question posting - i.e. author can see that similar posts already exist.

If it's what you mentioned, please vote for it.

[1] #58370
[2] http://stackoverflow.uservoice.com/pages/general/suggestions/24687

this comment is so I can find this answer in the near future - Kyle Cronin
29
[+2] [2010-10-06 17:04:00] Lance Roberts

I realize now that the FGITW [1] problem is actually the reason that Stack Overflow works well.

If you don't know enough about a topic, then you won't be able to post a quick answer.

Those who post a good quick answer, tend to be those who have more expertise in that area. This gives a higher probability of getting good expert answers.

[1] http://meta.stackoverflow.com/questions/19478/the-many-memes-of-meta/19533#19533

(8) The problem is not good quick answers, but bad quick answers. It generally takes longer to explain why an obvious solution that a non-expert might suggest is not a good one, than it does for a novice to post that flawed solution. - Peter Boughton
@Peter, Yes, but I'm saying that the probability will be that you get more of the good expert answers, than of the bad answers, because the non-experts won't be able to come up with those answers quick enough. So overall you'll get a strong tendency (as has been seen) for good quick expert answers to dominate. - Lance Roberts
(1) "tend to be" is an incorrect deduction here. Those who post first, simply post quickly. They can later edit their answer to include anything they missed, and wrongly recieve credit for providing a "good fast answer", when in fact they merely provided a "fast answer". - TLP
30
[+2] [2013-01-09 03:38:27] Ramy Al Zuhouri

Sometimes I even see users giving replies like: "change this line of code with this one", not caring about giving an explanation of what they say. "Just do this, so I get my reputation".

And you could see low quality answers being edited and becoming better and better again in the time, but they figure first in the list of all answers, even if who gave it was slower than an user who gave a high quality and complete answer without editing it.


31
[+1] [2008-09-11 09:30:34] Mark Ingram

I agree too. I've done the same thing. Been waiting for a page to load so I can verify what I'm writing, but it's taking too long so I just post without the verification.

It does reduce the effectiveness of the site I think. Rather than have a few well written responses, you end up with 10 short half-baked answers.

I'm not sure how you'd fix this - what about a delay between the time you post? So it's not like you're all rushing to answer the question, because if there is a 30 minute delay before the post is viewed, you can safely assume someone else will have already answered the question.


32
[+1] [2008-09-11 09:59:27] Christian Hagelid

I feel that it works fairly well as it is. Using the default sort of newest first, the long thorough answer will show up first for the answers with the same vote. I do think that the 'best' answer will in most cases end up at the top of the list. If a user browsing the site really care about finding a good solution to a problem then they will look at all the answers and vote up the one(s) that prove useful.


33
[+1] [2008-09-11 10:04:21] David Schmitt

Another possibility that might help is to give users a better view into what changed recently around their votes. I haven't yet found a good way to see all Q&As I voted on that have changed. Having that, it'd be easier to review one's votes, which in turn might lead to long-term improvements in vote quality.


34
[+1] [2008-09-11 11:53:49] aku

I have been thinking about this problem for some time but didn't find good solution.

Since reputation system exists people want to get some reward for their answers. As number of users growth, number of duplicate answers will grow as well.

One of the useful features would be a real-time update on question's answers while you're writing your own answer. It can be implemented in the same way as search, but "search result" should be updated every N seconds/minutes. That way I can clearly see that some one else post similar answer.

Another feature that I find to be useful is merging - as @Unsliced suggested. There must be a way to merge similar answers. However it's not clear who should do it and how to divide reputation points between answering people.


35
[+1] [2008-09-11 12:04:53] Martín Marconcini

I think SO is not only competitive. The competition is just an incentive to -in the end- provide a good (if not the best) source of information for programmers, written by programmers. The reputation feeds two things: you own ego and the confidence of the one who's reading an answer when it sees that it comes from a "guy who's not a newbie in the site and knows what he's talking about". Ot at least, you could change all that quote to: "a guy who has been hanging aroung a lot of time in SO". (which doesn't mean he's good). Badges kind of help with that. You could have 10.000 rep points, and little good badges, or 1000 rep and a few silver…

In either case, time will tell. I think that this is both competitive and collaborative with strong and weak points in each.


36
[+1] [2008-09-11 12:26:53] Mel

Could an answer's length be factored into the sorting algorithm somehow? How about letting the reader choose how they want their answers sorted? Much like I can go to Best Buy or Circuit City's sites and sort my search answers by brand, price, or alphabetically, I might be inclined to sort my answers descending by length if given the option... at least as a short-term experiment, although if I find the quality of answers to be better that way, I might just leave it like that forever.


Long does not equal good, you know :) - Omer van Kloeten
(1) And it is easy to cheat. - Ladybug Killer
37
[+1] [2008-09-11 12:28:38] Gulzar

This is a very valid problem, which I have been thinking a lot too. I have some ideas for this.

Redesign ranking functionality a bit. People can upvote (good ones can range from 1-10) or downvote (bad ones will always show as -1) but users will NOT be able to see their actual rep scores.

Use only Badges (or Grades) for display. Ranking can be used internally to determine the badges but the user will never see his rep points.

This has a psychological effect. Instead of posting a quick reply and refreshing the screen for the rep points to shoot up, users will try to write a more useful and conservative reply with code samples and will know they will get more points in the long run.

Allow users who posted the question to only Accept the answer after a day. If you want to get too draconian, allow him to accept only after upvoting or downvoting all the existing answers. :)

To eliminate fastest gun replies, answers will show up only after 5 minutes. That way, even if 10 quick guns are composing replies and posting, all those answers will show up at the same time. This will negate people to post quick replies and allow them to think at least for 5 minutes.


38
[+1] [2008-09-11 12:52:55] aku

@ Mel [1],

You have some interesting point. It will be nice to differentiate answers by various criteria, not only by votes count. Length of the answer can matter for me. Event if it's wrong, it serves as an indication that person spent some time thinking on his answer.

For example, when I browse through answers I'm interesting not only in "common opinion" but in experts decision. I don't care how much "reputation" answered person has. But I'm interested in his skill in given technology. For example there might be a little statistics attached to answer - total votes, how much people with average score per answer (with same\similar tags) > N voted, etc. Of course there might be a case when someone with tiny "reputation" provides best answer. But it's really a matter of statistics. Probability that quality of answer given by "experts" would be good is quite high. I prefer to not see "reputation" of answering person. In my opinion "reputation" doesn't matter, but specific skill does.

[1] #56409

39
[+1] [2008-09-11 13:16:41] bruceatk

Early in my experience on Stackoverflow my gut reaction to this issue was that it was a problem. I also hadn't asked any questions at that time. I soon changed my mind. I believe over time that the most correct answers do bubble up to the top.

If I ask the question I will be reading all the answers no matter what order they are in. As someone looking for an answer I will most likely be doing the same thing. To me what order they are in and how much they have been upvoted doesn't really matter if I need the information.


40
[+1] [2008-09-11 14:25:36] Tyler

A few ideas, mostly independent/combinable:

(1) Have a buffer time for upvotes. You upvote an answer, but it doesn't count on the rankings for a little while.

(2) Show different rankings to different people randomly, weighted by upvotes and newness. This way newer answers still get a chance, but don't drown upvoted ones (or vice versa).

(3) Temporarily place new answers at the top. Give them N/5 views at that position, where N = # views so far.


41
[+1] [2008-09-12 02:38:55] aku

@ Justin Standard [1],

I think some kind of grace period might be the best thing we can come up with. Also secrecy of the ballot is the way to avoid biased opinions. But your proposition is not solving following problem:

Boundary (first/last) answers receive more attention (and votes).

It means that when votes will be revealed, there is a high probability that mostly first\last answers will be up-voted like it happens in current implementation.

Since there are "oldest" & "newest" sort modes, some answers will be less noticeable than other. To solve this issue (first->most popular answer) I proposed to disable voting for some time and pop question to main page after a while to indicate that it's ready for answer selection.

However disabling voting will make it harder for question author to select some answer in the first few minutes after grace period. So your idea might be better.

[1] #58169

As some other posters have pointed out, FAST answers are a GOOD thing -- we don't want to discourage them. However, we still want BEST answers to get enough exposure to win out in the end. If a fast answer gets a lot of votes, maybe it is the intersection of FAST and BEST. - Justin Standard
42
[+1] [2009-12-14 16:14:19] bobobobo

Just allow the OP to select "how long before the first answer gets displayed", say, 10 minutes. If T is the original post time, then we'll call T + 10 minutes "Answer Open Time". Or we could call it Business Time [1], whatever you prefer.

This makes every question like a mini-contest, where people have 10 minutes or so to submit their answers. Answers are then displayed, but "early" answers are timestamped with Business Time, not the time the server originally received them. This effectively removes the fastest gun problem.

[1] http://www.youtube.com/watch?v=WGOohBytKTU

Interesting idea. Would require the OP to remember to do this though. - ChrisF
(6) I don't like this at all - you'd end up with a bunch of identical answers. It would result in a lot of wasted (redundant) effort when people put time into a question that is already well answered. Besides, FGITW was largely replaced with SCITE. Any method, including this one, will be gamed just as heavily as the method that it's 'fixing'. But I believe the unintended consequences of a 10 minute wait time (redundant redundancy) greatly outweigh any benefit it might provide... - Adam Davis
43
[+1] [2010-12-13 17:40:58] Brian M. Hunt

I've put together a different but as it happens related question ( Expiration of answers for questions with novel solutions [1]), and a proposed solution [2], that I think helps resolve (or at least significantly advances discussion about) the "Fastest Gun in the West Problem" by having the "fastest answers" replaced by "better answers" as old and perhaps deprecated answer-sets are revolved by the community (i.e. wiped, to-be replaced by new sets of answers).

[1] http://meta.stackoverflow.com/questions/71522
[2] http://meta.stackoverflow.com/questions/71522/expiration-of-answers-for-questions-with-novel-solutions/71534#71534

Seems to me that what you're proposing is essentially a much more naïve version of Reddit's comment sorting algorithm. - Aarobot
@Aarobot: That's an interesting reference, but I'm not sure my idea has much in common with Wilson Weighting (WW), does it? WW is a form of decay/normalization such as what I've proposed (albeit convolutedly) elsewhere: meta.stackoverflow.com/questions/6662/8101#8101. The 'revolving' idea is different because (1) it's methods are different: qualified user decisions, not an automatic weighting (2) it has a different effect: revolving answer sets, not prioritizing existing answers, and (3) its purpose is different: revitalizing questions for current answers, not weighing existing answers. - Brian M. Hunt
Yeah, I know the actual implementation doesn't have much in common, but you've posted this as a solution to the FGW, and Reddit's algorithm solves it in a simple, elegant, and effective way; no offense but you're proposing something that's far more complicated (for the users) and also flimsier (since it relies on community users digging up really old questions and uses a shotgun approach to promote the best answers). The SO team has demonstrated that they can do sophisticated sorting, vis-a-vis the new "Interesting" tab on SO's front page; I'd rather see the same sort of thing for answers. - Aarobot
@Aarobot: Thanks. I'm not confident I've effectively communicated my proposal to you, which proposal resolves more issues than the FGW problem, and it obviates the need for WW-style sorting. Hence my suggestion about how the site can be changed, without hurting it, to make the FGW problem go away. :) - Brian M. Hunt
44
[0] [2008-09-11 09:18:06] Chris Roberts

I think it's got to be down to the community to a large extent - and to the question owner.

Perhaps the question owner should be given more power to change the order of answers to a greater degree?

The only problem with that is that a lot of people asking questions will be 'drive-by' users, just wanting a quick answer to their problem. They're not the sort of user who will hang around to collate and manage responses.


45
[0] [2008-09-11 10:06:16] Gishu

Well.. Good answers will rise up. So dont worry, post a helpful answer that doesnt exceed half a page. Instead of read this or link to that, be objective and comprehensive. Also the first/last line (bold) AFAPossible should be the direct answer to the OP's question. These kind of answers will be rewarded with votes.
As for not reading all the way down... Well its your loss if you don't read all the answers to find the gem. But someone else definitely will.. So post and help away
Hey I made my first 1000 ... post! post! post! just kidding... 'The work is its own reward' as Jeff likes to quote. :)


46
[0] [2008-09-11 11:41:40] Unsliced

The central problem here is that while there might be several answers to a given question, in an ideal world there would only be one representative of each answer, rather than a whole muddle of similar duplicates.

Obviously we want our answer to be that representative. Our fear is that if we spend too long composing the definitive response someone else will get there first - and if they 'win' they'll get the rep (and, of course, the rep will be going to another answer that can't be as good as ours).

I like the ideal of good answers rising, but what we actually need here is probably some concept of answer merging.

It's probably not for the questioner to make the decision, they can accept an answer but there should be some mechanism for respondees to be able to merge another's answer into theirs and the product would be owned by the community wiki.


47
[0] [2008-09-11 14:14:01] Josef

Maybe there could be a "challenge" to the selected answer. So the person asking a question gets a response and marks that as their answer, even though it may, for whatever reason, be an invalid answer. Part of it could be by adding a comment to the "selected answer" but those are somewhat buried to anyone reading the answer.

If there were a way to show that there is some dispute/challenge from people other than the questioner that the selected answer is the best answer, it should foster some discussion and perhaps even cause the questioner to change their selected answer.

Of course, this way lay flame wars, perhaps, but I'm just using this as a starting point.


48
[-4] [2011-05-21 19:20:03] TLP

The problem

Fast and correct, or comprehensive and insightful. (Scroll down to see suggestion)

I do not think that either type of answer is inherently better. A "good enough" answer will solve the problem and get you back on track. An insightful answer will teach you something new, make you more skilled.

The problem is that if you focus only on rewarding fast answers, many of the comprehensive answers will never get posted at all.

As it is now, the quick answers float to the top, and stay there. People do not always bother to check all the answers, and they might not consider the depth of the question, the way the asker would. If a top voted answer is good enough, chances are it will recieve more votes than it deserves, merely because it is voted highest. That is an error of conformity of the reviewing audience.

Blind votes

Unless votes are made in the blind, you are always getting a systematic error. Simply put, here's the three biggest culprits:

  1. How many upvoted it?
  2. How high up on the answer page is it?
  3. Who made the post?

That is: People will tend to conform to the majority's opinion. People will only read answers to a certain thread depth. People will tend to agree with "authorities", e.g. people with high rep, many badges, etc.

These are all basic concepts in psychological testing [1]. There are many kind of errors that prevent you from getting correct answers. Here [2] is a wikipedia entry on bias, take a look at Bandwagon effect and Primacy Effect.

The correct way to do it, is blind tests [3]

The suggestion

Note: This only affects people who:

  • Are eligible to vote (due to rep or other constraints)
  • Still hold the option to vote for the question.
  • Are not the person posting the question

Anyone who just want to see the voting results can do so by clicking "finished voting".

Blind votes:

  • The author's name & rep is hidden.
  • Other people's votes are hidden until you are finished voting.
  • Answers are presented in semi-random order (1).

(1): A clearly poor answer should be nudged towards the bottom, a clearly good answer should be nudged to the top. Otherwise we are not using the expertise of our users. However, to avoid tainting by position, order is determined by vote intervals, with random order inside the intervals.

Those who do not vote

What about users who are only after answers? If you cannot or choose not to vote on a question, all the information becomes visible to you, and you can use this additional information to evaluate the quality of the answer.

The effect

By having users only vote on the merits of an answer, the validity of the vote tally will be maximized. In other words, we will know that the votes cast reflect the quality of the answer.

As it is now, the votes reflect something more than a good answer. Two identical answers may get a completely different score. Not because one person was faster in solving the problem, but because they shoved a foot in the door fastest.

How will this affect quick and dirty answers? It will still allow users to get rep for quick answers, but it will reduce the exponential effect of "being first", and also disperse some of the rep towards those who feel they have something to add, despite not being first to reply.

In short, it will be less of an "all-or-nothing" effect to being first.

Note: Please give feedback in the comments. The feedback so far has really been helpful.

[1] http://en.wikipedia.org/wiki/Psychological_testing
[2] http://en.wikipedia.org/wiki/List_of_cognitive_biases
[3] http://en.wikipedia.org/wiki/Blind_experiment

(3) But this ignores the population of people who look at votes not to decide how to vote (they may not even have an account) but only to know which answer to trust. That is the larger community. - Kate Gregory
@Kate Gregory Partially true. If you do not know which answer to trust, then I doubt that answer would benefit you much anyway. You must have some comprehension on the subject matter to understand the answers. How about if you a) cannot vote due to rep, or b) forgo voting on this question, then you can see the other votes. - TLP
(1) Consider stackoverflow.com/questions/6076717/where-to-put-dllimport as a single example (I just grabbed a multi answer question from one of my tags that newbies ask in a lot) - the top vote getter is clearly better than the other two. And the whole point of this site is to serve those who cannot tell which answer is better, and to help them. - Kate Gregory
@Kate Gregory Well, with my suggestion that a) those who can't vote, and b) those who don't want to vote, can see the votes, that is not an issue. MY point is that if you wish to vote, then you don't get to see how everyone else voted, then you have to actually read the answers and judge them on their merit. - TLP
(1) I have to say I strongly disagree with this. Yes, you can look at data which shows that higher rep users get more upvotes for more-or-less equivalent answers. However, generally speaking users with higher rep make answers that are simply better -- that's what gets them the rep in the first place. Furthermore, being able to point at an answer and say "I wrote this" is useful for employers as well as for employees showing off what they're capable of. This suggestion is completely contrary to the function of the sites which is why I downvoted. - Billy ONeal
E.g. I think this "cure" is worse than the disease. Furthermore it makes it impossible to identify problem users, because answers are anonymized, and moderators and people with moderator tools can't simply go to a user's profile and check their answer history to see if they're consistently a problem. Moreover, it "solves" a problem which really isn't a problem in the first place -- there are cases where this happens, but it's certainly not widespread enough to require that huge of an engine change. Finally, this answer does not answer the original question about the FGITW problem. - Billy ONeal
@Billy ONeal Then you have not understood my suggestion. You CAN point to an answer and say "I wrote this" because your name is only hidden to those who are still eligible to vote. The point is simply to avoid any bias that takes place before a user votes, nothing else. - TLP
(3) -1 Appeal to authority fallacy. That you have a university degree in psychology is quite irrelevant. This needs to be substantiated not with philosophical argument, but rather with empirical proof that people actually do vote based on a user's rep. I don't think that exists, in fact I've seen quite the opposite. - Cody Gray
@TLP: Not true. Sure, if I'm logged in I can show an employer something, but what if I'm just posting a link? The employer is an "eligible voter" and therefore would not be able to see who wrote the answer. - Billy ONeal
@Billy ONeal The employer can then simply click the "done voting" button and see the results. Assuming he had enough rep to vote to begin with. I do not see the problem. - TLP
(1) @TLP: Well, then we can agree to disagree. I think it's a horrible idea. However, this answer would still get a -1 from me if nothing else that it does not talk at all about the FGITW problem (the topic of this question). - Billy ONeal
@Cody Then you would say that Billy ONeal also made this mistake by saying that higher rep people generally post better answers? And as such, you do agree that showing rep before votes are cast is an "appeal to authority fallacy". - TLP
@TLP: Cody is saying that your answer, which is using "psychological tests", itself is a logical fallacy. Just because psychological tests say something in general does not prove a damn thing about voting behavior. If you could show some data about voting patterns showing your point then you might be going somewhere, but as written this answer ironically falls into the same trap you're campaigning against. - Billy ONeal
@Billy ONeal The FGITW problem has it's source in voting, in that by being fast, one gets some quick votes, and thereby assume a leader position, and thus gets even more votes. By keeping such information hidden, the problem is solved. I did not think I had to spell it out, but perhaps you are right. - TLP
(1) Not necessarily. Reputation is a direct indicator of the quality of answers you've posted on this site. A university degree isn't necessarily the same thing. Although that part of the comment was kind of a joke, contrasting your bristling at downvotes and justification of your theories with the very same underlying logic that you appear to criticize. - Cody Gray
But honestly, it's hard to reply here, because I don't think the "FGITW" phenomenon is actually a problem. I don't think we need to "fix" it, because I don't think it's broken. And I still don't buy that users vote primarily based on reputation. Look at Neil Butterworth. He came back a short while ago with a completely unknown user name, and has quickly skyrocketed back to the top of the C++ tag. All that by starting at 1. In my experience, people vote for good answers, not answers by users with more rep. Hiding the rep is the wrong solution. Just suppress voting for a period of time. - Cody Gray
@TLP: Hmm.... I don't think that argument works. The FGITW problem is about how quickly an answer gets posted. Your argument is about the badges and reputation of the poster. - Billy ONeal
@Cody Granted, it was cheap to appeal to my degree. I was lazy with my research in bias. Here is a wiki page that lists some of the biases en.wikipedia.org/wiki/List_of_cognitive_biases . Have a closer look at Bandwagon effect, and Primacy effect. I could not find the authority effect, but it is most certainly there. - TLP
(1) @Cody I am not saying votes are primarily based on rep, I am saying that they boost the effect of a good answer. But I do not think rep is the biggest factor in this case. More likely a high score and top position causes more bias than rep. - TLP
@Billy Oneal I clearly state three causes of bias, two of which are not related to rep. Those two biases (high score and high placement) are directly connected to how fast an answer is posted. - TLP
Edited to downplay the importance of rep.. perhaps I should remove it completely, as it seems to be a sensitive subject. - TLP
@Billy ONeal I rolled back my initial "intro" to the answer.. I removed it for legibility, but perhaps it changed the focus of my answer. So, in case you are interested, there it is. - TLP
(2) "Fast and correct, or comprehensive and insightful." In my experience, this is very much a false choice. The FGITW problem is generally understood to mean that someone will post a fast and correct answer to start with, and then (usually within the 5 minute window) edit it to be comprehensive and insightful. In the cases where I've used my fast gun, this is what I do. In the cases I've seen Jon Skeet et. al. use his fast gun, this is what he does. It's not necessary to pick one or the other. It generally ends up that the answer that gets the upvotes deserves most, if not all, of those. - Cody Gray
@Cody Gray And yet, if there was no purpose in answering first, you would not do it. So there is, and implicitly you know that by beating someone to the punch, you earn more rep. With blind tests, you still get the early votes, but you know for a fact afterwards that the votes you got was on merit, not on sorting order. Blind tests raise the validity of the vote, and the validity of the rep you earn. And on the whole, they raise the validity of the knowledge base in this community. - TLP
@TLP: Yes, the purpose in answering first is to get upvotes. The idea is, if you have a correct answer posted when others click on the question to answer it, they'll upvote yours instead of taking the time to write their own. It has nothing to do with the current rep of the answerer. It's merely a timing thing, and that's why blinding people to the rep of the answerers is the wrong solution. As I mentioned before, one way to fix this problem is to prevent people from voting for an arbitrary window of time after the question is posted. But this has other consequences we don't want to accept. - Cody Gray
@Cody: I don't think that you quite understand my suggestion, and that you are getting stuck on something about rep. Nothing that you are saying is really argument against this idea, mainly it is argument against change. Take a look at this stackoverflow.com/questions/6099506/… question, and explain to me how ysth's answer is 8 times better than mine, or 4 times better than the answer that got accepted. - TLP
How should I know? I don't know anything about Perl. I wouldn't vote on answers to that question for exactly that reason. I can't assess which is better. And if you want to talk about specific cases, browse the answers in my SO profile. There are tons of them that I personally think are good but don't have very many votes. Not everyone has identical standards. - Cody Gray
@Cody: You didn't take the time to look. I assume you assume it is a special case, and that any conclusion would support my argument. I actually would only like for you to say what you have been saying all along: ysth posted first, and got 8 times as much rep for it as I did. - TLP
@TLP: I think ysth answered his question more directly than the others. Although you and the accepted answerer provided insight into how to fix the bigger problems with his code. This is why I don't answer as many questions, mostly users just want the "quick fix" and use SO in lieu of proper debugging (low signal to noise ratio w.r.t. questions). So if you don't like the FGITW problem, don't play it. Only answer old questions or new questions that have merit. - user7116
@TLP -1 While I agree with your thoughts on this, this is really really just a terrible idea. If this were put into place, if someone whats to rep hunt all they need to do is copy and paste every Jon Skeet answer and they know due to random chance they will get upvotes. This would mean that every question would have a sea of duplicated or near duplicated answers. The FGITW creates a disincentive to posting dupe or near dupe answers (even legitimate ones). In fact I will delete my answer if someone duped it but was even a second faster. This is a good thing. - Some Helpful Commenter
@Conrad That's a good point. However, they can do that today as well. All they have to do is be fast. In fact, I see a lot of users abusing the current system to do it. Post a weak answer, and then steal stuff from my answer in their edits, and get credit for it. How are these problems handled today? - TLP
@sixlettervariables Actually, the asker preferred the long answers. The voters liked the short answer. I think that what happened in that case is, people saw an answer they could read fast, with some upvotes, and a good rep, and they said "yeah that'll work, I'm not reading those walls of texts below." I've seen many times that voters prefer short answers. - TLP
@TLP: to be fair, ysth answered the question at hand. I would upvote for that alone (I dislike asking a question only to receive answers that do not address it adequately). I would also upvote yours and the accepted answer for being "+1 Insightful". - user7116
@sixlettervariables Did he? I am not so sure. An expert might like a direct answer, but the asker in this case was clearly not an expert. But you are right, that may be why others upvoted it. I still think that the difference in voting is very off. - TLP
Very likely that question fell off the face of the earth in about 10 minutes. Once that happens you're only going to get a trickle of votes unless it is really popular. You can avoid playing the game and only answer old questions :) - user7116
@sixlettervariables Actually no, there are rather few perl questions. At least half the votes came hours after all three answers were in. Don't get me wrong, I think the fast answers are a really good feature of SO, but the voting is in some cases so unfair as to not merit answering at all. - TLP
49
[-6] [2012-09-15 10:19:05] Sahil Mahajan Mj

I know, I am quite late in reacting o this question.

So my answer will suffer with the problem what this post is trying to state

This answer would be the last one in the list, a lot below the highly upvoted answers

Now the solution I would like to suggest for the problem is:

  1. Give priority to new suggestions and answers. I am not omitting the usefulness of the old ones, but we should be soft enough to welcome the new suggestions.

  2. If upvoted questions are given priority over other ones, then downvoted questions should be given minority over others.

  3. The mark-it-as-a-useful answer trick should be used to decide the priority of a good answer and the mark-it-as-spam trick should be used to decide the location of the answer down the hill.


(8) If you order the answers by "active" then your answer will appear first. - Lix
50