chipotle | Reading

https://tacit.livejournal.com/681575.html

I am more active on Quora than any other social media site. I’ve been there since 2012, in which time I’ve written over 66,000 answers that have received over 1.3 billion views.

It’s no secret that the site has gone steeply downhill recently, with wave after wave of scammers and, now, ch*ld p*rn profiles growing like a cancer on the site. I recently wrote a very long answer about why that is, and how Quora’s policies and procedures basically rolled out the red carpet for people selling ch*ld p*rn (there are now a number of organized CP rings active on Quora). Quora deleted that answer, so I’m re-posting it, with expansions and addendums, here.

If you read this on Quora before it was deleted, feel free to skip to the end, where I’ve added new material.

Why is Quora allowing itself to become a spam and porn site? There are lots of real porn sites without corrupting what used to be an intelligent debate forum. Also, too much scammer spam. Why aren't the moderators doing their job?

The moderators aren’t doing their jobs because, and I say this as someone who has interacted with many moderators and high level admins and had many lengthy conversations with them, because they cannot.

I don’t mean they can’t as in they don’t know how to…well, no, that’s not true. Some of them don’t know how to.

Sorry, this answer got really, really, really long. It’s my analysis of the many failure modes of Quora leadership and moderation based on hundreds of interactions with Quora employees, moderators, and administrators, including cofounder and CEO Adam D’Angelo, about tens of thousands of Quora scammers and spammers. It’s also based on multiple security issues and bug reports I have made to Quora, and what happened after, and on being stalked, doxxed, and harassed on quora (and having my father and my wife doxxed and harassed on Quora), and what happened after.

But you asked, so here we go.

*** CAUTION *** CAUTION *** CAUTION ***

This answer is my opinion, based on my experiences with Quora. I do not work for Quora (well, I might as well do, with all the bug reports and reports of scammers I send them, but I’m not paid for it), I have not seen Quora’s back-end code, and I don’t have any insights into Quora’s management beyond my personal interactions with Quora admins. So take this with a grain of salt.

Problem 1: Absent Leadership

Let me start at the top. I’ve met Adam D’Angelo in person twice at Quora-sponsored events. In person, he comes across as an introverted, painfully shy dude with limited or no theory of mind and no real understanding of how social media works. Stick a pin in that, we’ll come back to it in a bit.

These days, he’s an absentee landlord. He’s on the board of directors of OpenAI, and pays very little attention to Quora these days.

And yet, at the same time, I’ve talked to Quora mid-level employees who have expressed frustration that they would love to implement technical solutions to address some of the worst problems they see with scammers and spammers, but they can’t do so without sign-off from upper management, which is pretty much absent. That’s one problem. Quora is, from a leadership perspective, a rudderless ship, adrift without a captain.

Problem 2: No built-in anti abuse defenses

I run a very small Mac troubleshooting forum, and I also run half a dozen blogs. All of those sites have simple anti-abuse measures like flood control, dupe control, and username control. That means I can, for example, ban creation of certain usernames. That means, with the click of a button, I can stop this from happening:

And I can stop this from happening:

Quora can’t.

These are all user profiles that are active on Quora right now. Quora literally lacks the capability to block usernames with certain words or phrases. It was never part of the codebase from the start.

Quora also cannot do dupe control (flagging or blocking when a user posts the same word for word identical content over and over and over) or flood control (flag or block when one user posts 80 times per second, which obviously means a spambot and not a real human being).

In 1997, I ran a forum for a few years that had automated, built-in username filtering, dupe control, and flood control.

In 1997.

This is what I mean when I say that Adam D’Angelo has no understanding of how social media works. He was the CTO of Facebook, and he does not have the slightest clue how people use social media, how people interact with social media, or how people abuse social media.

Problem 3: Buggy code riddled with security holes

In December 2018, hackers penetrated Quora using significant security holes and stole the entire Quora user database. They got everything, including passwords, because Quora stored the user passwords in plain text, not encrypted, on disk.

This is Security 101. You never, ever, ever, ever, ever, ever store passwords in plain text. The way every site, and operating system, stores passwords, and has since 1976, is you store passwords encrypted. When someone types a password, you encrypt it, then compare it to the encrypted password on disk to see if they are the same.

I had a TRS-80 as a kid in the 70s. It let you lock files on floppy disk with a password. It stored the password encrypted on disk so someone with a disk editor couldn’t find it.

Quora did not. Quora, a site with hundreds of millions of users, stored everyone’s password in plain text.

If that makes you deeply worried about Quora’s approach to security, you should be, because…

Problem 4: Quora’s codebase is an insecure mess

Quora has no Chief Security Officer. Quora’s codebase is riddled with security flaws, in part because they insist on writing their own code to do everything rather than using public libraries, and Quora’s developers from the earliest days onward did not know about and did not think about security. (See Problem 3. Nobody stores 100,000,000 users with plain-text passwords. Nobody.)

I have personally reported several security vulnerabilities that were actively being exploited to Quora. I’ve never heard back except for a bland “thank you for your bug report, we will pass it along to our developers.” In at least one of those cases, I saw the vulnerability being explited months after I reported it.

The vulnerabilities I reported all had to do with flaws in the way Quora handles Unicode.

Brief (I hope) technical digression about what that means: “Unicode” is a way to represent text characters. Computers were largely invented in the US and Britain, so they started out being able to understand only the uppercase and lowercase Latin alphabet, numbers, punctuation, and some special contol characters. That was it.

That means that for the first decades of the computer revolution, you could not type

Naïve

美丽

товарищ

For decades, you typed unaccented Latin characters or you typed nothing. No accented characters like the ï in naïve, no Cyrillic, sure as hell no Chinese.

Unicode was a system developed in the late 80s/early 90s to extend the old way that computers represented text, to allow for everything from accents to foreign-language alphabets to idiographic text to, later, “emoji” like 😮 and ✅.

The problem is that it had to be backward compatible with the old way to represent text or else every single computer program on earth ever written in English text would not work with the new system.

So the answer was a new way to represent text and symbols that still worked with the old system but added onto it to allow support for millions of characters, but that would still show old-fashioned characters right.

As you can imagine, Unicode is massively complex. Massively. Like unbelievably bogglingly complex.

Lots of people have written free open-source libraries for handling, storing, retrieving, and displaying Unicode. Quora refused to use them.

Instead, Quora wrote its own Unicode handling software. The thing about Unicode is that some characters are just represented by one-byte numbers (the uppercase letter A is represented by the number 97, or 61 in computer hexadecimal (base-16) numbers) and some are represented by two bytes (the lowercase a with a grave accent, à, is represented in Unicode as U+00E0), and some characters are represented as a list of instructions (basically “draw this letter and make these marks over it). Each mark is represented by a series of numbers.

That means that some Unicode combinations are illegal, not allowed, they don’t produce anything. These are called “invalid character sequences.” Invalid sequences are supposed to be detected and print as .

Quora doesn’t do this. Because of bugs in how Quora handles Unicode, some invalid character sequences aren’t detected as being invalid. This is how trolls can create usernames that do not show up on Quora and can’t be clicked. If you see a troll answer where the name of the person who wrote the answer is just a blank, there’s nothing there, the troll is exploiting a flaw in Quora’s home-grown Unicode.

Worse, you can smuggle commands to Quora’s software by packaging the commands inside of invalid Unicode. This is similar to SQL injection but instead of wrapping the command in quote marks or SQL comment strings you wrap the commands in broken Unicode.

I’ve reported two different Unicode injection vulnerabilities to Quora. One of them was still actively being abused months later.

Problem 5: Quora does not take security or abuse seriously, and so Quora has become one of the favorite places for scammers and hackers on the Internet

Right now, Quora is struggling with a massive, staggering influx of people selling child abuse images.

I typically report anywhere from 100 to 300 or more romance scam and child abuse accounts to Quora every single day. I log and track every account I report. Yesterday I reported 164 accounts. 33 of those were offering child abuse images for sale, 23 were offering preteen child abuse images for sale, and 3 were offering toddler child abuse images for sale. I spend about an hour a day doing it and it makes me sick to my stomach but I cannot, I cannot stop doing it. I’ve tried. I just…I cannot see it and not do anything.

There is a site called Black Hat World. It is a site where scammers, spammers, computer virus distributors, ransomware distributors, child abuse sellers, and other scum and vermin get together to talk about ways to make the world a shittier place.

I sometimes read Black Hat World. They talk about Quora a lot on Black Hat World. They exchange tips and techniques for running scams and selling child abuse images on Quora. There are at least four organized child abuse rings operating on Quora right now [edit: five, I’ve found another], in addition to all the various random independent child abusers running on Quora.

Black Hat World loves Quora because of its combination of poor security, weak or nonexistent automated controls, and lax, permissive moderation. There are tutorials on Black Hat World for scammers and spammers wanting to do their thing on Quora. Actual step by step tutorials.

This all started because of this woman:

Well, not directly because of her, it wasn’t her fault.

This is Paige Spiranac.

Ms. Spiranac is a pro golfer and a model. Almost exactly two years ago, a romance scammer arrived on Quora and used stolen photos of Ms. Spiranac to run his romance scams.

I saw the account and reported it to Quora.

Nothing happened.

I reported it again.

Nothing happened.

I reported it a total of eleven times.

Nothing happened.

I emailed Ms. Spiranac’s agent and said, “hey, just so you know, your client’s identity has been stolen and her photo is being used as part of a romance scam operation on a social media site called Quora, here’s the profile that is using her photo.”

The next day I got a very polite email from Octagon Agency, the company representing her at the time, thanking me for my email. The day after that, the scam account was taken down, I assume because Ms. Spiranac sent Quora a legal DMCA takedown order.

But it was too little too late.

The scammer running the account ran to Black Hat World and was like “hey, everyone, there’s this site called Quora that permits romance scammers!” and the floodgates opened.

Now here’s the thing:

Any site that allows romance scammers will get flooded with romance scammers, obviously. But as the concentration of romance scammers rises, pretty soon there are tons of scammers competing for the same pool of lonely, gullible victims.

So the scammers start specializing. A new wave of scammers arrives who try to scam people with very specific tastes. They’ll pretend to be trans women to appeal to trans chasers. They’ll pretend to be BDSM dominants to try to scam thirsty, gullible subbies. They’ll pretend to be foot fetishists to appeal to people with foot fetishes.

If that second wave goes unchecked, then the third wave arrives, people who pretend to be underage children in order to appeal to…well, you know.

If that third wave goes unchecked, the child abuse rings are like “oh my God this site permits romance scammers that pretend to be children, we have free reign” and the fourth wave is people selling child abuse images.

This is exactly what played out on Quora.

It took about eighteen months between that one scammer going to Black Hat World and saying “hey everyone, run your scams on Quora” and the child abusers arriving in force.

There’s a lesson here: If you run a social media site, and if you do not crack down immediately and hard at the first sign of romance scammers, you will, you will attract child abusers. It’s inevitable.

At this point, Quora cannot keep up. Of the four child abuse rings I’ve seen here, each makes on average about 20 new profiles a day. You can tell who they are because they all use the same contact information for purchasing their child abuse images. You can tell they’re using bots because they all use word for word identical profiles, the same usernames, and the same images over and over again.

Remember Point 2: No built-in anti-abuse measures. Quora has no automated way to detect identical profiles, nor to block or flag based on certain usernames or certain strings in the profile descriptions. That means Quora moderators are having to do manual searches.

And they’re bad at it. Say a child abuse ring uses the name “Tina.” (This is an example; to my knowledge, they don’t.) They’ll use a bot to create identical profiles over and over. They might, for example, be

Tina-1207
Tina-1208
Tina-1209
Tina-1210
Tina-1211
Tina-1213

and so on.

Quora moderation will ban Tina-1209 and Tina-1211 but leave the others, because you have to do a hand search to find the others and it’s tedious.

That leads to two more problems:

Problem 6: Quora’s back end tools are badly broken

I’ll give you an example:

On my own Quora space, I will often write about the child abuse profiles I report to Quora. These posts often get deleted by Quora moderation.

If Quora would delete child abuse profiles as aggressively as it deletes Spaces posts about child abuse on Quora, we wouldn’t be here, but moving on:

When Quora moderation deletes a post in a Space, when I appeal, there’s a little dance I have to do.

Quora will usually send an answer that says “We cannot undelete this content because a Spaces admin deleted it.”

Then I send back “no, you deleted it, look at this” with a screenshot that clearly says Quora deleted the post.

Then I get an answer that says “we’re so sorry, our back-end administration tool shows that you deleted the post, it’s a bug in our moderation tools, we will undelete it” and they fix it.

I’ve done this over. And over. And over. And over.

They know there’s a bug in their moderation software, one that wrongly displays to Quora moderators that a Spaces post that was deleted by Quora was actually deleted by a Space admin.

You have to keep reminding them about this bug over and over because different employees handle the appeals and each employee doesn’t know about the bug so you have to tell them “look closer, there’s a bug in your software” and they’re like “Oh! Look at that, you’re right!”

They have never fixed the bug.

They have never trained their staff that the bug exists.

Every time, you’re starting from scratch because this poor training means Quora has no institutional memory of the flaws and bugs in their own site administration software.

This same sloppy, shoddy approach to their back-end tooling exists at every level of the Quora stack from top to bottom.

For example, a few days ago I went through another little dance with Quora moderation. I had an answer deleted for spam. Then I appealed, and it was undeleted. Minutes later, it was deleted again.

10:36: I got an email saying they’d looked at the answer and decided it wasn’t spam.
10:38: They undeleted it.
11:03: They deleted it again.

I appealed again and it was undeleted again. This morning, it was deleted again.

Quora’s tools have no provision for a human moderator saying “Quora moderation bot, we’ve looked at this answer, it’s fine.”

That costs Quora money, because every time this happens, a Quora moderator has to stop what he’s doing, check the answer again, and undelete it again.

There are a ton of other, more subtle flaws, too.

After Quora deletes a child abuse profile, they sometimes delete the profile description, which usually contains an address to buy child abuse images, and sometimes they do not; the profile will stay deleted by the profile description advertising child abuse images for sale, and the address to buy them, will remain.

I asked a Quora admin about this. I got a replay telling me it was a problem in their moderation tool and they’re “aware of it and working on it.”

What’s worse is that they never delete the profile Credentials, so the child abuse rings have learned to put the ads for child abuse images inside the credentials, where they remain visible even if the profile is banned.

I wrote a rather angry email to Quora admins about this and here’s what I got back:

Here’s the thing:

This is wrong. This is not correct. You do not have to visit the deleted profile by a direct link to see this. The screenshot above is not a direct link to the profile. A deleted profile’s credentials remain visible in countless places through Quora, including in other users’ Followers and Following lists.

Quora’s own admins and moderators DO NOT KNOW HOW QUORA OPERATES.

I don’t believe this Quora employee was trying to lie to me. I believe this Quora employee honestly, seriously doesn’t understand how Quora’s software works.

Problem 7: Quora’s moderators are incurious and not proactive, probably because they’re overworked and underpaid

Say you report a profile like Keanu-Reeves-359 for impersonation.

Quora admins will delete it. What they will not do is say “oh, if there’s a fake Keanu Reeves #359, I wonder if there is a fake Keanu Reeves #358. And a fake Keanu Reeves #357. And a fake Keanu Reeves #356.”

Nope. They will delete Keanu Reeves #359 and move on.

This is especially bad with the child abuse profiles.

If you report two profiles, one a child abuse profile that is using the name Tina-1208 and another, created a few milliseconds later and identical to it called Tina-1209, they won’t go “huh, a bot is making child abuse profiles one right after the other like a machine gun. I better look at Tina-1207 and Tina-1210, too.”

Nope.

They also don’t stop and ask themselves what profile names mean if they aren’t in English.

I reported this troll profile 7 times. The first time I reported it, it was banned a few hours later. I reported it six more times after it was banned because, well, see for yourself:

Quora policy forbids hate speech in usernames. When a profile whose username contains hate speech is banned, Quora is supposed to delete the username as well.

Which they usually do. If the username is English.

Six more times I reported this profile, explaining what the username means in English. Six more times they did nothing.

Why did I keep reporting it after it was banned?

Finally, finally, after seven reports, finally, after I emailed my Quora contact directly with a screenshot of the user profile AND a screenshot of Google Translate, finally Quora removed the username:

Quora is totally fine with a username “We Must Exterminate the Jews”…as long as it is not in English.

These problems, broken tools and incurious admins, arise from the next problem:

Problem 8: Quora has no money for, or apparently interest in, paying moderators, hiring developers, or fixing the toolchain

Quora started out with no revenue model. When Quora was first founded, it was pitched to investors as a site that would collect and distill human knowledge and make it searchable.

In 2019, it had a valuation of $2 billion.

Then ChatGPT came along and overnight iQuora lost three-quarters of its valuation, from $2 billion to $500 million, because investors were like “why would someone ask Quora if they can ask ChatGPT?”

That’s why Adam D’Angelo pivoted to AI and why he now sits on the board of OpenAI. It’s why Quora is a rudderless ship.

In 2021 or thereabouts, Quora started to run out of money. With the advent of LLMs, the venture capitalists didn’t see the value in Quora anymore. Its valuation collapsed by 75%. The VCs closed the money spigots and Quora was left to sink or swim on its own.

Quora responded by…

…firing the moderation team.

Adam is pitching an AI moderation bot for sale to other social media sites.

This AI moderation bot cannot look at usernames and ban based on users calling themselves Keanu Reeves or Elon Musk.

This AI moderation bot cannot say “this Telegram username is associated with a seller of child abuse images so I will flag or delete posts where this Telegram username appears.”

This AI moderation bot cannot automatically spot and ban profiles called “Fuck All N----rs.”

Quora keeps trying to train their AI moderation bot to spot things like fake Keanu Reeves profiles or child abuse profiles using LLMs or whatever because once you’ve scaled to hundreds of millions of people and billions of posts, it becomes difficult to add basic features like flood control or username filtering after the fact.

They could do it, but it would be expensive, so they’re left trying to fine-tune their recipe for chicken cordon bleu while the entire kitchen burns down around them.

I’ve had so many conversations about the romance scam problem and the child abuse problem with everyone from frontline Quora employees to high-level Quora admins and I 100% believe that nobody, nobody at Quora, nobody understands the scale of the problem, nor how hard it is to get rid of these people once they’ve established a presence.

I actually have more to say, there are at least three more points in my head I could make including a significant worldview issue on the part of Mr. D’Angelo, but I’ve already spent hours on this answer and it’s way, way longer than a Quora answer should be.

If you’ve read this far, congratulations! Welcome to my world. As a user who genuinely loves Quora, it’s disheartening and kind of sickening.

I do love Quora. Quora’s been good to me. I’ve met so many people who have become personal friends in the real world outside Quora. I’ve met a lover and co-author here.

But it’s getting harder and harder to stay. I reported a string of profiles selling child abuse images of toddlers—toddlers!—yesterday and it made me want to throw up. When I was done I had to leave the house and go to a coffee shop to get the stain out of my head. It’s wearing me down and I still can’t stop, because if I’m not reporting these, who is?

tl;dr: Quora was founded by someone who doesn’t understand computer security or social media. Quora has never, ever been proactive about preventing abuse. As a result, Quora never implemented the most basic front-line security or anti-abuse measures, measures that were available in free open-source software in 1997, and now lacks the resources to address the problem.

Quora’s own employees also don’t understand Quora itself, their own software, or the scale of the problem in front of them.

I’ve saved this post. In the event Quora deletes it, which I put at about a 50/50 chance, I will make it available on my blog.

So that’s the Quora answer.

After I posted this, it was deleted by Quora admins, then undeleted, then deleted, then undeleted, then deleted again. As I type this right now, it’s still deleted, but I’ve filed another appeal so it will be interesting to see if it gets undeleted again.

Whilst it was available, several folks asked if I would expand on the part where I said I have more points to make, so here they are:

Problem 9: Quora’s algorithm is broken

Like most social media sites, every Quora user sees a different feed. There’s too much content to show anyone the firehose directly, so the Quora algorithm listens to your interactions to learn what content you want to see. For example, if you downvote content, Quora tries to show you less of that kind of content. If you upvote content, Quora interprets that to mean you would like to see more like that. The more you interact, the more Quora tunes your feed.

Trouble is, Quora sometimes gets its wires crossed.

Quora interprets downvoting and muting as negative signals, and commenting and upvoting as positive signals. But bizarrely, it interprets using the Report feature to report users or content as a positive signal.

If you report lots of romance scammers, you start to see more and more romance scammers. If you report spammers, you see more spammers.

Even worse, Quora sends customized “digests” in your email. I get a digest full of stuff that Quora thinks I might like to see in email every day. Usually it’s full of answers on topics like science or linguistics or computers or math.

Lately it's been full of romance scammers.

I want you to take a step back and let the magnitude of that sink in. Quora sends out romance scam content in emailed digests. Today’s digest included nine pieces of content. Three of them were romance scam posts.

Problem 10: Quora is remarkably tolerant of sexual abuse

Amazon AWS is one of the largest Web hosts and storage engines on the planet. A staggering amount of content, including Quora itself, runs on AWS.

Whatever you may think of Amazon (and there’s plenty to dislike about Amazon), Amazon is fanatical about dealing with ch*ld p*rn. Amazon despises child abuse.

Amazon donates a tremendous amount of money, millions a year, to support the National Center for Missing and Exploited Children (NCMEC).

Amazon maintains an internal team, separate from their normal abuse team, to deal solely with reports of child sexual abuse on their networks.

Amazon, as a matter of policy, logs and tracks every single child abuse report it receives. This information, again as a matter of policy, is forwarded to Amazon contacts within the FBI, and to NCMEC.

Amazon maintains a database of child abusers, and hashes of child abuse images, which it makes available to law enforcement.

Amazon does not fuck around when it comes to child abuse. They have an ultra-strict policy, and they will strike down with great vengeance and furious anger anyone who uses their network for child sexual abuse. Hosting CP on Amazon is like calling down a targeted missile strike on your own location.

Quora, which is hosted on Amazon AWS...does not.

If you create a profile, or five profiles, or a hundred and fifty profiles, on Quora offering child sex abuse materials for sale, Quora will (well, I say will, Quora might) ban your account. It will not do anything beyond that.

The sellers of child abuse materials on Quora know that they need fear no repercussions beyond having their accounts banned...and maybe not even that. They operate brazenly and boldly on Quora, even posting profiles that literally say “CP for sale here, all ages available!”, because they know nothing will happen to them.

Why the pizza emoji? The slice of pizza emoji has become something of a universal signifier of those selling child abuse images. CP: Cheese Pizza. CP: Ch*ld P*rn. Get it?

How did Quora get here? What systemic failures led Quora to be the Internet’s hotspot for romance scammers and ch*ld p*rnographers?

Problem 11: Ayn Rand

Adam D’Angelo, Quora’s cofounder and absentee CEO, is the kind of Big-L Libertarian who mainlines Ayn Rand directly into his veins.

He’s one of those techbro Libertarians who believes, I mean really truly believes, that the solution to bad speech is more speech, as if more speech is a magic wand that somehow magically erases bad actors, scammers, spammers and ch*ld p*rnographers.

His fundamental worldview is one where acting against any speech, even “we have pictures of toddelers being raped and would you like to buy them?”, is anethema.

I believe this is why Quora has no built-in mechanisms to prevent any Tom , Dick, and Harry from creating an account called “Elon Musk” and putting up posts offering free Bitcoin if you just deposit money into an account to, you know, pay for “fees.” It’s why you can create an account called Keanu Reeves or Sandra Bullock and the system will just let you do it, because hey, we wouldn’t want to risk the real Keanu Reeves making an account and running into some kind of barrier, right? It’s why there are thousands of fake Keanu Reeves and thousands of fake Elon Musks and so on, and why Quora’s moderation, what’s left of it, is purely reactive and not proactive.

The problem is, we’ve seen over and over and over again that this approach does not work. It’s empirically not true. But it’s a religious idea among a certain kind of techbro; they want it to be true, so they treat it as Revealed Gospel, never to be questioned.

https://tacit.livejournal.com/681575.html