Okay, I suck

[chipotle]

...for I have taken a stupid poll linked from LiveJournal, specifically the "What D&D Character Are You?" poll. According to it, I am a neutral good half-elf bard ranger.

Which sounds good, one supposes. The ex-analyst in me can't help but notice a flaw in the quiz, though, revealed in the thoughtful inclusion of detailed results at the end of the results page that show you your scores. The flaw is that if you have a tie in a category, it always chooses the first one on its list. For instance, it lists the races in the order human, half-elf, elf, halfling, dwarf. I have a score of 2 in "half-elf" and "halfing"; since half-elf is first, that's what I am. Likewise, I'm tied for "neutral good" and "lawful neutral," so I'm neutral good because in the program's logic, the goods come first, then the neutrals and then the evils.

And indeed, I noticed that most of my friends have very similar results to mine. It's probably very difficult to get a non-good alignment (much less an evil one), and difficult to get something other than an elf or half-elf, if my suspicion that the scoring is biased against boring old humans is correct. The classes may be less subject to bias by virtue of being more obviously varied (it's easy to pose questions that separate the thieves from the clerics, but not the elves from the half-elves).

Okay, really I suck because I'm an ex-analyst. Someone hit me before I take the quiz's Javascript and analyze the possible result distributions in a spreadsheet.

Comments

[roho.livejournal.com]

Heh, it does the same thing even with the primary/secondary class. It picks the first, highest one on the list, (in my case, Ranger). It then starts at the top again, and apparently picks the next one equal to or lower than the first...without excluding the one you already got. So even though I had a score of '2' for Ranger, and '2' for a couple other classes, I was a True Neutral Elf Ranger/Ranger.
(Guess it is possible to get non-good alignments...I got a very strong True Neutral score. But the bias from other tests does seem to incline towards good alignments.)

Results distribution...

[chipuni.livejournal.com]

Methinks that you're taking the test a little too seriously...

I invite you to write a program that generates Javascript for web pages, that also randomly picks among equal choices.

[chipotle.livejournal.com]

As the Rev. Ivan Stang once said about the Subgenius, "Take the church seriously, but don't take the seriously seriously." If someone can derive idle amusement from finding (or generating) online polls, I can certainly derive idle amusement from casually analyzing a couple. I trust nobody's going to replace the Keirsey Temperament Sorter with "What Classic Video Game Character Are You?" But effectively, these online polls are aspiring amateur personality tests, and I've watched several of them inspire a lot of conversation. It's interesting to me to see what's under their hoods.

I invite you to write a program that generates Javascript for web pages that also randomly picks among equal choices.

Well, the idea isn't to be random, after all--it's to be unweighted, so the test isn't mathematically biased. :) An algorithm for ranking a list of n possibilities in an unweighted fashion with a series of questions exists (in a non-computer form) in the job-hunting book What Color is Your Parachute. Using it as the basis for a questionnaire system like this wouldn't be difficult, although you would end up with dozens of "choose A or B" questions rather than one or two dozen multiple choices. A variant on the logic in it could be used for "agree, disagree, no preference" questions, I suspect.

As Prickvixen pointed out, though, the choice of the questions also influences results. The Parachute approach would (I think) help minimize that problem, but I doubt it could be eliminated.

Like I said (or at least implied), I enjoy this stuff. I have no formal statistics training, but both my jobs at Intermedia involved data analysis, and I've become the sort of person who puts my current employers' SEC filings into spreadsheets so I can do my own calculations of burn rate, cash flow and financial life expectancy. (My own projections have always said we stay six months ahead of collapse, while the filings cheerfully say 12-24. We just switched auditing companies, though, and darned if the most recent filings don't agree almost perfectly with my projection--which is, essentially, death in July without a significant capital infusion or massive layoffs.)

[paka.livejournal.com]

I fiddled around trying to "break" the test because I'd noticed that preponderance as well. I found that answering questions at random got me half-elf results, but got dwarf as a result by intentionally choosing "dwarf" answers.