Wikipedia talk:Requests for adminship/Ironholds 2/Post-mortem

I'll start this one off, then. The intent of the experiment was to allow issues to be dealt with before the "voting", as a common experience is 1) someone posts diffs as an oppose reason 2) Pileon of "Per X" by users who may or may not return for any explanation 3) explanations met with "oppose per badgering" comments. An additional advantage to this type of RfA is that it allowed the readers/commenters to get a much greater insight into the candidate than the standard system. The new system failed to prevent issues cropping up during the voting stage, leading to the normal result, which is fair enough as users are perfectly entitled to post reasoning in the voting stage. The system did allow greater insight into the candidate, but many felt that the question period was too long, and that there were too many questions.

The solution to the second one is simple; I propose that the question period for the next experimental RfA (going up, say, a week from now) be set to 2 or 3 days rather than the original 4. 2 would probably be better, but 3 allows for any latecomers. A suggestion brought up at the London wikimeet was to have a 'crat clerk the page, removing inappropriate questions (repeated ones, for example); I'd be interested in hearing people's responses to this. I feel that in future questions should be more tailored towards the candidate rather than general, far-ranging ones; this would help keep the numbers down and also make the questions more useful.

The first problem is a tricky one. One idea would be to allow simple "oppose" "support" and "neutral" with no "because..." to encourage people to bring up their issues through questions rather than "vote" comments. Issues with this are fairly obvious; firstly, it furthers the idea that RfA is a vote, which it is not, second it makes it impossible for a closing 'crat to rule out silly reasons and third it makes it almost impossible for failed candidates to gain insight into their faults, as it were. Possibly a system were comments are visible to the crat but not to other users? This would probably require MediaWiki messing, however, and again has issues.

I'd like to thank the people who've participated in this RfA and will hopefully take part in the post-mortem. I'm creating a section now for people who want to scrap the whole thing, and another for people who believe it would work with changes (please specify what if commenting); hopefully other headings will turn up as needed. Ironholds (talk) 12:37, 13 October 2008 (UTC)

Would work, but with changes

• Well, kudos to you for trying. As for the future... I think maybe what would help (47 questions is probably a bit much) is not only to shorten the time period, but to limit the number of questions. I figure the questions come in three basic flavours:

1. Standard questions - we've already got this
2. Policy questions - hypotheticals, trying to gauge candidate understanding of policy
3. Candidate-specific questions - e.g., "In this situation, you did this. Why?"

So how do we limit the second two categories? That's tough to say, but maybe there's a time limit on that, too. Maybe the process should look like:

1. Candidate announces they are RfA in the usual way
2. Discussion page opens for three days to hash out the questions that need to be asked, say, 10 in each category. The candidate does not reply to these questions. Maybe--I don't even know if this is possible?--the candidate is unable to view the discussion page at this point. Candidate can respond to the standard questions as per usual.
3. Questions transcluded to main RfA page, candidate may respond for three days. Each commenter gets to ask one followup question if they feel clarification is needed. Maybe to a maximum of x followup questions per each question.
4. Voting opened after three days. But--again, I really don't know if this is possible--votes are hidden to everyone except crats until the end of the voting period. This allows for early closes per SNOW and NOTNOW.
5. Votes displayed after voting closes.

My reasoning for the hiding is this: pileons happen both for and against. It might make more sense for each person voting to vote strictly according to what they see in the responses. I don't know. As for hashing out the questions for a couple of days beforehand, this allows people to distill what they see as any potential issues with the candidate before questioning opens. Plus, at least in terms of policy questions, this could end up showing that there are x questions that everyone always asks, so we should make them part of the standard set. Just some thoughts. [ roux ] [x] was prince of canada 12:59, 13 October 2008 (UTC)

Scrap it

1. You got the same opposes you would have got without the questions. Nobody really seemed to read the answers to the questions, to be honest. It was a great idea, but like many others, simply failed to work in practice. Garden. 13:07, 13 October 2008 (UTC)
2. Insidious bureaucracy. This experiment resulted in 48 questions. That's so far beyond absurd it defies description. This way of conducing an RfA will dramatically reduce passing rates. Bad, bad, bad. Based on this graph I'd go the other direction. Reduce time an RfA is open to three days. If people can't find glaring problems in three days, it's unlikely they will. There isn't anything an admin can do that can't be undone anyways. --Hammersoft (talk) 13:47, 13 October 2008 (UTC)

   I'd attach a health warning to that chart - although it's useful, the data is about a year out of date now and needs to be refreshed. Gazimoff 14:23, 13 October 2008 (UTC)

3. I appreciate the intention behind the experiment but ultimately believe it was a failure. I believe it did very little to alter the outcome of the request or to change the way things happened in the voting phase. I'm afraid that this format, while being excessively taxing on the nominee, can result in a sort of tl;dr response from the casual participant who will skim (or outright skip) the questions in favor of a quick read of the voting section and judge accordingly. Shereth 14:11, 13 October 2008 (UTC)
4. Yup. I didn't really appreciate the tonguelashing people got on that RfA's talk page for questioning the big diff filled oppose, but that is the way of things. We held an experiment to specifically avoid having a long oppose like that and a pile on without an attempt at discussion with the candidate. Maybe it is a good thing that we didn't have to wait until the next experiment to realize we failed. The formatting change didn't solve the primary problem it was meant to solve--peoeple being persuaded by an argument in the oppose/neutral section which might have been better raised in the discussion section. Protonk (talk) 15:48, 13 October 2008 (UTC)
5. I had high hopes for it, and some reservations, but ultimately it appears as though the format did nothing to prevent the typical problems associated with RfA. I doubt very much that nearly all the people who voted read each question and answer thoroughly. Kudos to ironholds though. Wisdom89 _{(T / ^C)} 19:11, 13 October 2008 (UTC)
6. I am sorry to say that I agree with the sentiments here. The problem is not the RfA format, but the willingness of too many people to thoroughly ignore a candidate's full record of contributions in favour of yanking isolated comments out of context and magnifying them a thousandfold in order to create a distorted history. The result would have been the same had it been conducted in the regular format. Quite frankly, Ironholds did not fail in his RfA - the RfA process failed him. Ecoleetage (talk) 00:08, 14 October 2008 (UTC)
7. Far too many unnecessary questions, which ultimately served no purpose except to waste the candidate's time. I would have ignored at least 15. Oh, there weren't 48 questions but rather 51. It was worth a shot, but barring active bureaucrat involvement in culling irrelevant/stupid/spammed questions it's a big step in the wrong direction. MER-C 09:26, 14 October 2008 (UTC)
8. We did learn something though, and the shortcomings raised that may be addressed in other forms! - Mailer Diablo 16:16, 19 October 2008 (UTC)

Oppose/Support per

This ultimately boils down to whether or not RfA is a vote or not. Since the primary factor in anyone getting the mop is getting a certain percentage or greater of support, then it is obviously a vote that simply allows the voters to argue with each other. If it is just a discussion, then people are not weighing in to say if they support or not. However, since the "discussion" is about trust, it becomes a vote because there needs to be clear evidence that a candidate has the trust of the community. One editor might give 14 difs supporting a candidate's brilliance, while there might be only one example of bad behavior. However, if that bad behavior is bad enough to eliminate the community's trust, then members of the community necessarily need a format that allows them to "pile on" to a given reason to indicate their lack of trust.

To that end, the 'Crat as Clerk idea is strong (if we can find a way to make the workload balanced) only if we have established reasons that are legitimate to support or oppose over. That is to say, if a pile on starts up because a candidate has a stupid signature, the 'Crat needs to step in. Hiberniantears (talk) 13:41, 13 October 2008 (UTC)

I think it could work

I think we are confusing the candidate's failure with the format's failure. If the point of the format was to make every candidate succeed, the format was a failure. If Acalamari had asked a question about the diffs before opposing, the whole thing might have turned out very differently. I think we need to actively encourage asking questions about concerns they might have before they oppose. If they get a good answer, much pile-on voting would be averted. If they don't get a good answer, they oppose and people pile on top of them. We should make a rule that no more than 2 people can have an oppose per x !vote, so it would look like Oppose- User x, Oppose per x (if that is the case), Oppose per x (if that is the case), Oppose- User y, and so on. That would automatically decrease pile-on voting and keep it from turning into a single issue (or diff) RfA. Erik the Red 2 ~~~~ 14:41, 13 October 2008 (UTC) Having a crat clerk the page also is a good idea. Erik the Red 2 ~~~~ 14:42, 13 October 2008 (UTC)

The "2 per votes" idea is a bit iffy; you get a sort of first-come-first-served system, however. Besides, if person number 3 turns up and finds themselves unable to oppose "per X" as they were planning, they may simply pick a different reason to get their "oppose" in. Ironholds (talk) 14:44, 13 October 2008 (UTC)

The 2 per vote is simply a dumb idea. Sorry, but it is. Would we allow the correlary? Only 2 supports per nom? No... in fact, we don't have to justify the support at all. Is an oppose who paraphrases another oppose a unique oppose or should be excluded because s/he used the same rationale as somebody else who has already had "2 per"?---Balloonman ^PoppaBalloon 14:47, 13 October 2008 (UTC)

Is it possible to hide votes until everything's all said and done? [ roux ] [x] was prince of canada 14:50, 13 October 2008 (UTC)

That would be a better idea, if it could work. So every user has to come up with their own idea to oppose/support rather than using a diff someone else brought up. By the way, how long is this post-mortem supposed to last? Erik the Red 2 ~~~~ 14:54, 13 October 2008 (UTC)

Yep.. because all of those diffs/questions should be up by the time voting opens. But (like I said way above), crats could still see the results, which lets them still do early closes as required. Is this even programmatically possible, though? I think whether or not we stick with this format, the old format, a hybrid, or just throwing darts at a wall of names, hiding the votes until after the voting period can only be a positive development. [ roux ] [x] was prince of canada 15:02, 13 October 2008 (UTC)

(ec)Secret ballots would require a new way of doing things here at Wikipedia... there currently isn't a method for it. But if we went with secret ballots, then it would be a blanket statement that RfA IS a vote, not a discussion. Also, you wouldn't alleviate the the issue of pile ons because people would still be able to post the reason why they are oppossing, unless you are proposing killing all conversation. Now that I've shot the idea down, I do like Secret Ballots for a reason. First, there are people who (IMHO) wait until a candidates fate is determined before supporting/opposing. They only will follow the crowd which ever way it is going. Second, there are people (and I know this for a fact because I'm one of them) don't get involved with RfA's where the fate appears to be a forgone conclusion. EG if I look at an RfA and it is 70-2, I'm not going to support or oppose. Likewise, if it is 50-50, I won't get involved. I've talked to others who have said, "I would have opposed candidate X, but by the time I showed up, it was obvious they were going to pass, so why make enemies." A secret ballot would mean that we are getting more honest answers. People would be !voting based upon the merits of what people have said, not based upon how many people have echoed those sentaments.---Balloonman ^PoppaBalloon 15:07, 13 October 2008 (UTC)

Bingo. I think we all know that RfA is a hybrid of consensus and voting. So let's let the consensus come in the form of the questions asked of the candidate, and the voting come in the form of...er...voting. I would say that the ballot should be made public after the vote is concluded. Oh.. and actually.. it would be highly trivial to set it up. new user: VoteBot. Email the bot through its userpage with a subject line of: RFA USERNAME !vote. Counting them is as simple as filtering the messages and counting. Not sure how you could then programmatically copy the !votes back onto WP, but I'm sure someone knows how it could be done. Recanting votes is likewise easy: RFA USERNAME CHANGE !vote. Or something. [ roux ] [x] was prince of canada 15:15, 13 October 2008 (UTC)

• I think we need to look back to the WT:RFA page to see the distribution of "support" votes. A lot of that (IMO) is social pressure. If we could vote "oppose" in secret, without any record of our decision, the distribution of voter habits wouldn't have a mean at ~80% support. It would be lower. There are obvious drawbacks to voting in public (and its basically a vote), but a shift away from that would mean a lower promotion percentage (Assuming we kept the 75% threshold. Protonk (talk) 15:44, 13 October 2008 (UTC)

Agreed... secret ballots would equate to more opposes... not more supports.---Balloonman ^PoppaBalloon 15:50, 13 October 2008 (UTC)

Would work (with changes) in Utopia, but not on Wikipedia

First, thanks for volunteering to be a guinea pig. I'm convinced that no significant changes to RFA will ever happen based only on some grand discussion somewhere; there's too much inertia. The only way to change anything is by trying little things and seeing if it can be made to work, little by little, just like you did here. This requires someone to stick their neck out, and I'm glad you were willing. An experiment that disproves a hypothesis is still a successful experiment.

The problems with this RFA were:

• Far too many questions. I share much (not all, but much) of Balloonman's disdaid for questions in general, and this many questions was, as someone said above, tl:dr. I had hoped that there would be more discussion first, rather than that many more questions. Leading me to...
• Far too little discussion. For example, if Acalamari had brought up his concerns during the discussion period, and the community had hashed out whether they were big deals or little deals, that would have been what I had in mind. That didn't happen, and I now suspect that it won't. Without the draw of an ongoing vote, it will be too hard to get people to participate in a preliminary discussion.
• Too much focus in the actual RFA about the fact that it was an experimental RFA. Very disappointing, but not too surprising. It happened in the last two experimental RFA's, and I don't think it's too surprising it happened here. That said, I strongly suspect that the end result would have been the same. The “oppose per format” votes weren’t the turning point.

Finally, time spent talking about secret ballots and other significant changes is time wasted. We are never going to do that, so might as well not spend too much time talking about it.

It's entirely possible that the RFA system we have is the best we're ever going to get. At the very least, I would be surprised if someone else volunteers to be a guinea pig for a while, so talk of what to do in the “next experimental RFA in a week or so” is probably not going to happen. --barneca (talk) 16:11, 13 October 2008 (UTC)

• I think you're dead on is saying that the outcome of this RfA would have been the same were it more traditional. I think we need to focus (in this post mortem) on which parts of the experiment went right or wrong. Did we learn enough new things about the candidate to make this worth the added effort and frustration (probably no). Would this have been different for an obvious "promote" candidate. An obvious "NOTNOW" candidate? A candidate closer to ~75% support? So did any part of this illuminate the way forward to make incremental changes at RfA? Protonk (talk) 16:19, 13 October 2008 (UTC)

The one thing that I did like, but didn't think I would, was the delay between the posting of the RfA and the start of the !voting section. It gave the sense that people had the chance to research the candidate before !voting. Right now, it sometimes feels as if there is a rush to !vote before people have a chance to vet the candidate. I sometimes feel as if there is a rush to get to 30 supports before that person with the oppose comes along and torpedoes the RfA. that being said, 4 days was way too long.---Balloonman ^PoppaBalloon 16:25, 13 October 2008 (UTC)

Since we're talking about making small changes, then: we could try one with a 1-2 day delay between posting and !voting? Ironholds (talk) 17:09, 13 October 2008 (UTC)

I'm thinking 12-24 hours... tops!---Balloonman ^PoppaBalloon 17:18, 13 October 2008 (UTC)

If you can find a good crash-test dummy I support that idea. Ironholds (talk) 17:20, 13 October 2008 (UTC)

Heh, maybe I should volunteer to do it as a reconfirmation RFA. The rapid collision of all the highly-energetic "oppose per format" and "oppose per no-reconfirmation-RFAs" people might be able to produce more new subatomic particles than the Large Hadron Collider (cheaper and less prone to breakdown, too).--barneca (talk) 17:50, 13 October 2008 (UTC)

I don't think that would be a good idea... between that, wallstreet, and the debate on Wednesday, it just might tear the very fabric of our world apart.---Balloonman ^PoppaBalloon 17:53, 13 October 2008 (UTC)

Yeah, so long as the person can commit to being online to answer questions during such a short period. Protonk (talk) 17:38, 13 October 2008 (UTC)

I meant a standard RfA, with the one variation being a 12-24 hour gap between posting and !voting. Ironholds (talk) 17:40, 13 October 2008 (UTC)

Ahh. Well that still means a pile-up of questions to answer. Have you gone back and made a guess at how many hours you put in to answering the questions at your RfA? You had like 40 of them and you seemed to give pretty in-depth, thoughtful answers to each. Must have taken some time. Protonk (talk) 17:42, 13 October 2008 (UTC)

The value of the wait wasn't for the Question and Answers. That failed. The advantage of the delay is that it allows people to vet the candidates without having others influence their investigation. Without people Supporting/Opposing en masse in the first 20 minutes of an RfA.---Balloonman ^PoppaBalloon 17:47, 13 October 2008 (UTC)

Well, I would argue above that we didn't really achieve that. I don't want to blame Acalamari in any fundamental sense (because it was a well reasoned and important opposing statement), but I feel like we had this giant question period and then people opposed on the basis of a big diff filled oppose that came early on. If the purpose of the experiment was to implicitly or explicitly defuse that sort of situation, it wasn't a success. Protonk (talk) 19:30, 13 October 2008 (UTC)

3-4 hours spent on all of them together, probably more but my sense of time is out of wack. The 12-24 hour-delay RfA wouldn't have the "questions, get yer questions here" system this one had, so it wouldn't be an issue, really. Ironholds (talk) 17:44, 13 October 2008 (UTC)

After reading the entire RfA and this discussion, I absolutely oppose this change. The optional questions section has been and always will be a relatively irrelevant part of the RfA process. Its where people with axes to grind go to try to get someone they don't want as an admin to make a mistake and then flog them to death with it. It's where I suspect a great deal of people go to make themselves look important in the RfA process by asking no end of truly inane questions whose answers can be surmised just by properly vetting the candidate. All in all, this looks like yet another idea brought to you by Wikiproject:Bureaucracy. (Yeah, I know it doesn't exist...at least not officially). The RfA process as it stands right now is hopelessly ridiculous...but the least so of every other idea that has been fielded in the past. Trusilver 18:49, 13 October 2008 (UTC)

my comments for what they are worth - I read all the questions and all the answers before voting, which took some time, but there is little point in putting the candidate through 'trial by question' if the voting that takes place is based on voters comments rather than the candidates answers to questions. On the other hand as voting is supposed to create a consensus, votes need to be justified so they can be 'weighed'. This is the nub of the problem! The alternative is an 'actions' based consensus, where the primary reasons for giving admin status is what the candidate has done to gain the trust of the community rather than what he says he is going to do. The current system seems to me to be a mix of the candidates actions, his answers to questions and voters comments creating consensus, but where voters comments can sour the debate. The experiment was too question oriented (far too many at any rate) and did not prevent voter comment from souring the debate. As such it failed to improve on the existing system. In order to improve on the current system the number of questions needs limiting and their validity needs increasing. Voters must be able to justify their votes so they can be weighed, this is vital, but keeping these comments secret until after the vote seems worth an experimentMjchesnel (talk) 19:28, 13 October 2008 (UTC)

Questions

This proposal is to try to get rid of the questions problem we have.

• What does everyone think about having a set of questions that become standard questions. Optional questions cannot be asked unless they concern an action that the candidate has taken, and are being questioned on. If somebody comes up with a question they want to ask - they can't, but if they feel it's a good and helpful question, they may present it to the community with the option of it becoming a new standard question. So the questions for the candidate will become only standard questions that have been voted by the community to be asked to the candidate. Thoughts? iMatthew (talk) 19:04, 13 October 2008 (UTC)

Sounds like a good idea. Couple that with the ability to ask a followup (like I said above, maybe no more than three followups per Q, to avoid insanity), and 3 days of questions followed by 3 days of voting... [ roux ] [x] was prince of canada 19:48, 13 October 2008 (UTC)

• • I would strongly oppose this. Instead of allowing someone with a question about a candidate just ask it, it would just make them add it in the oppose question and would trigger a wave of oppose per X if the candidate couldn't respond in time. (Also, it makes RfA a "have you memorized everything" rather than "do we trust this user?") NuclearWarfare ^{contact me}_{My work} 22:29, 13 October 2008 (UTC)

Somebody proposed a list at wt:rfa and basically it is getting shot down... very bad idea.---Balloonman ^PoppaBalloon 22:34, 13 October 2008 (UTC)

Silly idea to have a general list of questions; people searching for good answers can simply go to "previous successful RfA's" and paraphrase. Questions should be specific to the candidate in (pardon the word) question. Ironholds (talk) 22:43, 13 October 2008 (UTC)

I think it would be better to get rid of all the standard questions. People have so much time to prepare for them, and can read so many other RFAs in preparation, that the answers are almost meaningless. Alternately, change the standard questions every month, but that's a lot more work, and gives an unfair advantage to people who start their RFAs at the end of the month. Mr.Z-man 02:52, 15 October 2008 (UTC)

Agree with the removal: I think a general consensus from the "this could work" bunch is that 'crat clerking to remove general and stupid questions would be excellent. This could also apply on standard RfA's, really; having the same "optional" questions over and over again in every RfA that lasts more than 2 days makes paraphrasing and bullshitting an easy way out; they don't neccessarily reveal anything about the candidate other than his skills with the "copy and paste" function.Ironholds (talk) 03:00, 15 October 2008 (UTC)

• I kind of like the standard questions, but I'm not completely opposed to their removal. We do have to be aware of the possibility that people would respond to the removal of those questions by re-inserting them (or variations on the theme) as optional questions. I also think that we might be approaching the questions from the wrong standpoint. I don't think that time to prepare for questions is the important factor. No one is going to ask a question that a week of prep time will solve where an hour of prep time would be inadequate. We also shouldn't be viewing questions as a means to reveal something that is impossible to reveal from a look at the contributions of the candidate. In this sense the "job interview" questions don't fit the mold--questions like xeno's vandal question (and others, not to pick on xeno) are meant to suss out how the candidate feels though their answer to the question. there are automatic flags (like denying the unblock request or indefing or something) and general clues (how the candidate feels about vandalism or blocks), but in the end the "gotcha" part of the question is neutralized by the fact that the "answer" can be looked up. We have an interesting advantage here at Wikipedia because unlike in a job interview, we can see perfectly all previous actions of a candidate. That informational edge should be considered when asking questions. I was honestly pretty impressed by the quality of questions in your last RfA, Ironholds. Some were exceptional. Neat to see what happens when people have to ask questions. Protonk (talk) 04:26, 15 October 2008 (UTC)

Ideas for the next test, then

Idea 1:

• The same format/layout as last time.
• The "question" period is reduced to two days.
• A 'crat (if one willing to do it can be found) clerks the page, removing silly redundant questions and rewording/moving "Oppose due to diffs......" to the discussion/questions section to get in the spirit of the original RfA and stick them in a forum where they can be replied to before "Per X" pileon. The oppose stays where it is, but with a link to area X in the discussion/question bit.
• Should this one turn out to be an epic fail as well, we'll tank the idea of a third one.

Idea 2:

• A standard RfA, with the minor difference that there is a 12 hour gap between RfA posting and the opening of !voting. This would give people time to assess the candidate instead of people feeling like they need to insta-vote within 30 seconds of seeing the RfA.

Comments? Ironholds (talk) 22:48, 13 October 2008 (UTC)

I'd add to idea 1 question period two days to ask questions before candidate then has 2 days to answer the questions and engage in discussion with the community, then voting with comments hidden to the voters and revealed after the voting is closed.Mjchesnel (talk) 11:07, 14 October 2008 (UTC)

Is there a failure in communication here? Another test akin to this one? Look at the comments in the scrap it section alone, much less the insane amount of controversy this experiment produced. You don't need to throw a lead duck twice to learn that it really isn't going to fly any better the second time. --Hammersoft (talk) 13:02, 14 October 2008 (UTC)

No, but if you refine said duck after the first throw so it has rocket-powered feet and eyes that shoot lasers it might work. I find your comments on this page frankly unhelpful; we get it, you think it is a bad idea. If you have nothing to say to possibly improve the process then there's no need to post similar comments everywhere; the first few "this sucks" messages were read. Ironholds (talk) 13:10, 14 October 2008 (UTC)

• Taking essentially the same format and shortening the amount of time for "discussion" isn't adding rocket propulsion and eyes that shoot lasers. It's more like filling the lead duck with helium. I'm sorry you find my comments unhelpful, but it really isn't my problem. In the note you are responding to, I tried to bring attention to the fact that most people responding on this page don't want this experiment to continue further. That's being ignored by you and certain others. Fine, if you want to go ahead with your experiment, more power to you. Just don't be surprised when it crashes and burns. But my comments here ARE intended to improve the process by preventing another crash and burn experiment from happening again. NOT encouraging something to happen can be just as productive as encouraging it. --Hammersoft (talk) 13:53, 14 October 2008 (UTC)

I happen to agree with Hammersoft. It isn't the RfA process that is messed up, it's the way that Wikipedian's approach the RfA process. You can tinker with the processes as much as you want, but until you find a way to prevent other people's agendas, biases and general petty asshattery from mucking up the system, this is really all beyond the realm of pointless. The only legitimate change I can see to make would be to have the Crats unilaterally toss out EVERY vote (call it what you want) that does not directly address whether or not the editor can be trusted with the tools. Nowhere has this been as apparent to me as when I participated in an RfA six months or so back that ran into thousands of lines of text all perpetuated by a couple of dinks who decided it was relevant to reject an admin candidate because he's a Christian and anyone who isn't an Atheist OBVIOUSLY can't be trusted with the tools. Trusilver 17:20, 14 October 2008 (UTC)

The problem with modelling RfA

I've been thinking long and hard about how to respond to this post mortem, and why I think that there's a lot to learn from it. But I think in order to understand it a little better, it's probably worth me explaining why I thought it was a good idea, and why it probably wasn't as successful as I hoped.

One of the things I often work on is business processes, whether it's (as I've done in the past) auditing them, verifying that they work as required and that each stage of the process has some form of governance and control, or whether it's (as I do now) designing new processes and getting buy-in from various functional groups as part of my design work on new products and services to take to market. One of the things I tend to look for when designing something new is a model to base it on - something that already exists that works to an acceptable level, that I can then mimic or use as the basis for a new design. If there isn't, I'll have to generate something new. But that's rarer than you might think.

So, with this in mind, I've been looking at RfA and the output of WP:RREV in particular. I was trying to understand why the majority of concerns and responses were revolving around the whole debate/election part of the process. I thought that if I could improve this part of the process, the part that allows conributors to analyse a candidate and come to a decision about them in order to build an overall view of the connunity consensus, then the overall path through RfA would be trouble-free. So I looked at other group-decision making processes to see if there was anything else that would lend itself to this model. Unfortunately, there isn't anything that scales well with the number of participants we get at RfA nowadays. So I started to build up a set of requirements about the process. This led me to believe that the forcible breaking of the debate and election phases of the process would be a positive step - it would remove the need for bold support or oppose votes, as everything could be fed in, discussed and weighed by the community during the question phase. I thought it would allow for more candidate-specific questioning, allowing for a very smooth election process afterwards where editors could cite questions that they were either pleased or concerned about the responses with. More than that though, it would allow for a candidate to reply to every single concern raised about them without it appearing to be badgering opposers. This debate-vote style mechanism is used in many instances already, and works quite well at allowing for decisions to be made with a large number of people.

So, why didn't it work so well? Well, Wikipedia is the only ballot that allows for the debate to continue while voting takes place. In fact, it allows for the vote to start along with the debate, before the first concern has been raised. But more than this, it allows for someone to raise a concern about the candidate, which can affect voting, and which the candidate may not immediately be able to respond to. It's this scenario that I was trying to eliminate, as I don't feel that it treats the candidate in a fair and balanced manner. More than that though, I was hoping that a lengthly debate process would involve asking more questions relevant to the candidate. Instead, we had a high number of template or crib-sheet style questions and a handful that were directed at the candidate's past behaviour. I think that I agree with others in that I feel we need to get away from this desire to ask templated questions - does it really help the process?

As for Acalamari's oppose, I was really hoping that this approach would have lent itself against it, and I was hoping that contributors would have seen what we were trying to achieve and gone with it. WilliamH had a similar comment that may have raised itself as an oppose, but was quickly discussed and agreed not to have as much of an impact. I do feel that if Acalamari had raised the oppose as a question or a comment during the debate phase, we may have seen a different result to the RfA. Either way though, it would have allowed the candidate to respond to a situaton without already finding a stack of "oppose per" votes before he even had a chance to respond. I feel that lenthly opposes left unchallenged can attract this, and I know that when I was going through RfA myself I was regularly checking it at all hours for such an eventuality (which is, in itself, unhealthy).

Where do we go from here? I don't know. 48 questions is a lot, people admit they didn't read them or simply skimmed through them then read down the list of opposes before deciding to support or oppose. Yet other candidates still come forward, and candidates still pass RfA. And some of the opposes based on an experiemnt were uncivil and bordering on offensive, questioning that people dare even try to improve things. So while I think it could have worked with minor tweaks, I think human nature being what it is was the key problem. Humans are creatures of habit and like to do things in the same way all the time. People are uncomfortable with change, especially change that they don't have a choice over, whatevere the pros and cons. And while I still feel that RfA can and should be improved, I also feel that the community needs to want that change to happen. Currently, the overwhelming consensis is that no change is required. Until we get to a stage where we have no more candidates, I don't think change will happen.

Sorry for the long-winded post. I've been reflecting on my own involvement within RfA while I've ben writing it, and I think I may have become too passionate and am starting to lose my objectivity around it. With that in mind, I'm going to take a break from the RfA arena for a while and see how things develop. Till then, thanks for reading. Gazimoff 08:04, 14 October 2008 (UTC)

Make discussion compulsory

The answer to people giving reasons in an oppose that they didn't bring up during the discussion is simply to ban such behaviour. Oppose reasons (supports don't need reasons since supporting should be the default) can only be of the form "Per X" and X must be someone that commented during the discussion phase. (You can, of course, say "Oppose per me" if you commented before.) This would require everything to be brought up in advance and would require people to read the discussion before they (!)vote. People could lie about their reasons, of course, but that's always going to be problem. --Tango (talk) 15:41, 14 October 2008 (UTC)

Consensus, sorta

I think a general consensus from the "this could work" bunch is that 'crat clerking to remove general and stupid questions would be excellent. This could also apply on standard RfA's, really; having the same "optional" questions over and over again in every RfA that lasts more than 2 days makes paraphrasing and bullshitting an easy way out; they don't neccessarily reveal anything about the candidate other than his skills with the "copy and paste" function.Ironholds (talk) 03:00, 15 October 2008 (UTC)

The flaw here is evident in your first line: "a general consensus from the "this could work" bunch. The problem here is that for every member of the "this could work" bunch, there are two people in the "uh, no" bunch. It's akin to making the statement that "the consensus among socialists is that socialism is pretty damn good." Here's a great idea that I'm sure you would have arrived at if given the proper amount of time - Instead of trying to maximize the potential of an idea that most people here don't like, how about using the input you have gotten to create a better idea. In other words... instead of using the input of people that agree with you and conveniently ignoring the input of those who don't, how about you come to terms with the fact that in order to come to a community consensus, you are going to need to address each and every opposing argument there is. Until you do that, this whole brainstorming session and test run is little more than an exercise in intellectual masturbation. Trusilver 06:39, 15 October 2008 (UTC)