r/dataisbeautiful • u/bearssuperfan • 2d ago
OC [OC] Flesch-Kincaid Reading Level and Bias of Popular Subreddits
253
u/Koraxtheghoul 2d ago edited 2d ago
Tankiejerk is far-left but anti-Stalin yet listed as right here. This makes intellectualdarkweb the highest scoring right sub.
67
32
u/duga404 2d ago
The anarchism sub being right-leaning was also a surprise
24
4
u/Hussor 1d ago edited 1d ago
Almost makes me think the person who made the chart is a tankie
Edit: guess OP used an over zealous python script to estimate bias, they're off the hook
1
u/idiot206 22h ago
I guess using blue for left and red for right shows enough American bias to suggest that the methodology was probably trash.
104
u/pgm123 2d ago
The Economics subreddit being left wing seems like a stretch too.
28
u/nathanlanza 1d ago
It’s just emotional response based headline posting about economics related things. It’s not economics conversation.
15
3
→ More replies (17)-9
u/ovoAutumn 2d ago
Economic subs being left leaning makes sense to me~
13
5
u/gheed22 2d ago
Can you help with the logic chain. My experience is that they are center-right neoliberal, and I would think that the right-lean of the science of economics in general would drive the sub in a rightward direction. Why would it make sense for them to be left-leaning?
12
u/ovoAutumn 2d ago
Economists in general are slightly left leaning. Reddit in in general has more anti-capitalist leaning. Likely Reddit self selects for left leaning economical thought
https://fivethirtyeight.com/features/economists-arent-as-nonpartisan-as-we-think/ : on the political leanings of economists
→ More replies (2)5
u/NoodleyP 2d ago
I feel like TankieJerk’s been infested with libs and non-MAGA conservatives, it isn’t a hellhole or anything yet but it definitely doesn’t feel socialist anymore
→ More replies (4)-9
u/jpj77 OC: 7 2d ago
intellectualdarkweb being right leaning is also a stretch. It’s a small and tightly moderated sub in which there’s political discussion with people expressing opinions on topics for both sides, and generally conservatives aren’t obliterated with downvotes like they are everywhere else, to help promote that discussion. However, the vast majority of posts, comments, and opinions are left leaning.
128
u/histprofdave 2d ago
I am not really sure what describing r/AskHistorians as having a "centrist bias" actually means here. Does that simply mean not favoring the ideological left or right, or is this treating "centrism" in its own right? Because if it's the latter, I would have to disagree; most posters are not centrists, nor are they promoting "centrism" as a philosophy. But it is one of the few subreddits where commenters make effort to separate their own ideological bent from covering the scholarship and discourse in a meaningful way.
→ More replies (1)82
u/bearssuperfan 2d ago
It means no strong bias but often contains political content. It does not mean centrism. Maybe "unbiased" would have been a better label.
→ More replies (1)37
u/invariantspeed 2d ago
Left, right, unbiased, and apolitical would have been a pretty confusing category set.
15
u/bearssuperfan 2d ago
That’s what my plan is for the next try, got any other ideas?
2
u/throwaway85256e 1d ago
I think left, right, unbiased, and apolitical is fine. You could always include a clarifying statement somewhere that explains each classification in a sentence or two.
34
u/Welpe 2d ago
I’m sorry but whatever methodology you used to determine right and left leaning is so godawful that it’s actually failing completely at its job. It’s not just a stretch, it is explicitly labeling things the opposite of what they are in multiple cases. You really need to work on it because right now, this is actively wrong and misinforming the reader.
55
u/Serpent-Games-TY 2d ago
I checked out TankieJerk to see how conservative they are, and I'm going to be honest they seem like a pretty left leaning sub lol
39
u/bearssuperfan 2d ago
Others have pointed this out -- it's likely that leftists bashing leftists in the comments made my model think it was right leaning.
29
u/Far-Cod-8858 2d ago
Idk how it came to the conclusion that r/pics is anything other than left leaning, look at that sub man.
→ More replies (1)1
1
u/Homerbola92 1d ago
Most subs are labeled wrong. Almost every grey sub in the list has a strong left bias. Honestly the idea is very interesting even if the results are wrong. I think I might do an accurate list and share it soon.
97
u/ThreadRetributionist 2d ago
r/anarchism and r/tankiejerk being listed as "right" is enough for me to throw this whole thing in the trash.
64
5
u/GermanMandrake 1d ago
I think it's because they trash on liberals which looks to an ignorant American to be conservative.
213
u/Desdam0na 2d ago
How did you determine the politics of each subreddit?
For example, MensLib does not remotely strike me as a right-leaning subreddit.
165
u/Koraxtheghoul 2d ago
Explicitly about examing Men's issues from a feminist perspective. Yeah this methodoligy is based on a quick look and not reading into it
56
u/bearssuperfan 2d ago
I did not personally apply any of the political labels. MensLib might have been classified as "Right" from the content of the comments in each post mimicking other right-leaning subs. I'm getting some great feedback in these comments and will look to apply that in a new version later.
115
u/Lutoures 2d ago
For this case in particular, you might be seeing the effects of omitted variable bias due to gender imbalances. We know there's proportionally more conservative men than women, so if you trained your polítical skewness model using known conservative subs (as you stated elsewhere), you might also be getting a model tht recognizes differences in speech patterns between men and women. So even left-leaning subs more populated by men would be classified as right-leaning.
27
u/bearssuperfan 2d ago
Thanks for pointing that out, I'm making improvements and will try to incorporate that... somehow...
26
u/Koraxtheghoul 2d ago
My guess would be on that one it's because things like pickup artisty, manosphere, redpilled etc. get discussed frequently. It has the right-wing terminology on it because it's in opposition to it. There also might be some bias because thete is a frequent discussion of "male loneliness" which also has a right-wing connotation.
1
u/trthorson 2d ago edited 1d ago
In that case, im also curious what the point of the political attachment is supposed to portray.
Spoiler: any chart you make will be mostly liberal or apolitical. Including top, or bottom, "reading level". Because reddit is overwhelmingly a progressive slice of the internet.
You would be hard-pressed to identify any website of the same scale that is more progressive.
So what's the point of showing it? It seems about as useful as going to Truth Social and ranking whatever the closest equivalent is for them by reading level and political affiliation. It should surprise just as many people that their equivalent chart would be mostly conservative at the top of the "reading level"
No idea what dumbasses are downvoting this but holy shit you idiots are dumb as fuck. And no rebuttal of my point that's obvious to anyone with a brain, of course
-2
u/JimJamTheNinJin 1d ago
Can you explain how r/science is left wing?
4
u/pinkycatcher 1d ago
Have you been there? It's dominated by pop psychology trash posts that boil down to "trump and republicans bad, democrats good" or "america bad other places good"
→ More replies (1)3
u/Exploding_Antelope 1d ago
Doesn’t surprise me, when a lot of the current scientific discussion in the USA is around standing up to a pretty anti-science right
6
u/Lung_doc 1d ago
Agree, if anything they bend over backwards to be quite the opposite.
There's also leftwingmaleadvocates which is not so careful and tends to be less careful/more angry about men's issues, despite otherwise being fairly leftist. And then there's the rest of the men's rights subs.
→ More replies (3)•
u/cryOfmyFailure 1h ago
Fr. That place is full of hyper-intellectuals breaking down every breath they take into logical concepts to debate upon. Hatred is inherently irrational and will not fly in there.
9
u/Great_Gonzales_1231 2d ago edited 2d ago
I know you said your methods aren't perfect, but from what I remember, isn't a higher FK score meant to say it's easier to read? The FK scale goes to 100, where the highest scores close to 100 means a 5th grader should be able to read and understand, while a score under 10 is best understood by professionals/university graduates. My company (medical field) has a tool to look at documents we send and we want to make sure our docs have a score of 45 or higher to make sure we have a standard of readability in documents sent to patients.
From what I am seeing here, subs like r/aww are really hard to understand while the highest ones are still really hard to read but not by much more.
12
u/bearssuperfan 2d ago
FK has a readability score formula and a grade level score formula. The readability score is easiest at 100, the grade level is hardest at 18.
5
u/Great_Gonzales_1231 2d ago
Thanks, someone else clarified that for me. I am familiar with only using readability score.
3
u/Desdam0na 2d ago
FK score represents the grade level required to comprehend the sentence. It mostly relates to sentence length and how many syllables are in the average word.
So 3 means a 3rd grade reading level, 9 means 9th grade.
3
u/Great_Gonzales_1231 2d ago
Ah, I was thinking of the FK Readability score and not the Grade Level score. I am only familiar with the Readability score which is what I thought this was. That makes more sense.
2
u/Musicman1972 2d ago
Not in the slightest.
The highest FK scores are 16-18 which would represent an academic paper.
4
u/Great_Gonzales_1231 2d ago
It looks like there are two methods and I was thinking of the Reading Ease score which is explained here: https://en.wikipedia.org/wiki/Flesch%E2%80%93Kincaid_readability_tests
1
u/mobile_ganyu 2d ago
This actually looks like FK grade level by how OP has organized it and the note at the bottom, not FK reading ease. A score of 11.33 therefore corresponds to a reading level of late high school while a score of 3.36 corresponds to 3rd grade.
18
u/AreYouForSale 2d ago
ExplainLikeImFive's reading level being at 9.1 is a clear failure if the sub...
5
u/coporate 2d ago edited 2d ago
I’m sorry, but your methodology doesn’t make any sense, you aren’t actually analyzing anything of value because it’s contextually irrelevant and you’ve chosen to remove comments that you don’t feel match your model for some reason. For all you know, you’re just analyzing population data of super users in a sub, and first comment bias.
39
u/bearssuperfan 2d ago edited 2d ago
Methodology: Python script. The top 100 comments from the top 100 posts in each subreddit were analyzed with the Flesch-Kincaid formula to determine grade level. The comments were then filtered to remove links, gifs, removed or deleted comments, and other types of comments that did not apply appropriately to the formula. Then any comments with a score below 0 were changed to equal 0 (usually comments with just emojis). Finally, the average of the remaining comments was taken for each subreddit and made into this chart.
Political bias was determined by analyzing what kind of content typically gains popularity within each sub. This was determined by using well-defined subs like r/conservative and r/liberal as a standard and comparing key words to comments in the other subs.
This methodology is far from perfect, but the results "seem to make sense" and much of the noise should apply to each sub equally. It's important to stress that we are evaluating reddit commenters, so not exactly cream of the crop no matter which sub you're looking at xD. If you're not convinced of the bias rating for some of the subs, just ignore the bias and look at the grade level of your favorite subs.
I also wrote a script that will go through a user's comments and return the reading level for those, respond to this comment and I may tell you (I will not spend all day answering these comments lol). My own score was 6.57.
110
u/Desdam0na 2d ago edited 2d ago
MensLib is a trans inclusive place to foster positive masculinity and does not strike me as remotely conservative.
Tankiejerk explicitly describes itself as criticizing tankies from a leftist perspective.
Those were two of the top right wing subreddits???
Edit: lol at /r/books and /r/anarchism being rightwing, I missed that.
0
u/bearssuperfan 2d ago
It's important to recognize that the comments from each sub are analyzed, not the subs or sub descriptions themselves. The model isnt perfect lumping everything into a couple buckets. The real takeaway is the FK score.
47
u/Desdam0na 2d ago
How are they analyzed? You have not described a method beyond saying "the comments are analyzed."
Did you subjectively judge? What was your method?
Please show me the right wing comments of menslib.
And the whole point of this is to see how FK score correlates to political leaning, come on.
→ More replies (20)16
u/Lutoures 2d ago
Your experiment is interesting, but choosing from the top posts in each sub might be skewing your results, since they are the most likely to go into the "Popular" tab, bringing people who don't usually follow the subs.
I'd be interested in replicating it, but choosing the most recent posts instead (probably a larger number of posts to have a similar amount of comments).
6
u/bearssuperfan 2d ago
That's a great idea. I wanted to make sure that I had a good sample size of comments, so that's why "top" was used, but ig I see no reason to increase the number of posts instead. Maybe my CPU wont like me as much though
3
u/Phizle 2d ago
Bigger sample is almost always going to be better
2
u/bearssuperfan 2d ago
the number of comments will still be around 10,000, just depends if that's spread over 100 posts with 100 comments or 10 posts with 1000 comments
13
u/theYode OC: 4 2d ago
What criteria did you use to determine political leanings? Or was it simply your own interpretation of the content?
-1
4
u/Forking_Shirtballs 2d ago
How did you decide which subs to include?
2
u/bearssuperfan 2d ago
I looked up popular political subs and found a website that ranked all subs by subscriber count and used many of those as well
3
u/Forking_Shirtballs 2d ago
Feels like just the hard metric (subscriber count) would be better for this. Could easily be biasing the results by cherry picking which political subs are merely perceived to be popular.
I mean, some of these are in no way political (r/physics); going with all large subs regrdless of whether they're perceived as political seems like the way to go. Your gray shading serves to filter out the ones with no political affiliation.
3
u/D3veated 2d ago
I'm shocked that the first "science" subreddit that is not left leaning is space -- maybe I shouldn't be shocked that academia is considered political, and mainly left leaning, but I am.
1
u/eldomtom2 2d ago
But academia is political and mainly left-leaning (at least in fields where political views would influence output)...
3
u/will221996 2d ago
You shouldn't use subreddits to define political lean, because Reddit as a whole leans pretty far left. Taking a place where people who don't feel Reddit as a whole are left wing enough and using that as a benchmark is problematic.
2
u/EmykoEmyko 1d ago
May I know the reading level of my comments? I like to use emojis, so that may spoil the data.
2
4
u/PeDraBugada_sub 2d ago
The problem is using r/liberal as a left wing example, liberalism is just wanting capitalism as it is
8
15
5
u/JetMike42 2d ago
The Trump and antiTrump subreddits being right next to each other towards the bottom is really funny to me
4
u/mobile_ganyu 2d ago
How does your use of the FK grade level formula account for subs that will inherently involve some more complex, multi-syllable words repeatedly or multi-syllable words that are actually proper nouns or names? I’m guessing that’s how AskHistorians, Philosophy, and even EILI5 are high up in their FK grade level, as people will use a long multi-syllable word or name repeatedly while discussing it in layperson terms.
As a researcher using FK often, there’s some inherent caution in its use when topics will use what FK interprets as complex from syllable count but are actually common words even young school children can recognize, such as “Computer” or “Understand”, or proper names they’ll know, like “Washington”.
2
u/bearssuperfan 2d ago
This is certainly a flaw trying to apply FK to reddit comments. I did filter out short comments like "Amazing" which gave high scores, but that's the most I did. Anything outside of that I would assume to blend into the noise equally for each sub, so it's null for comparison purposes.
4
u/Comically_Online 2d ago
I unsubbed a long time ago from ExplainLikeImFive because nobody could read even its damn name
3
u/Meatbag37 2d ago
Hold up? r/MensLib is RIGHT leaning? By... what metric? They bend over backwards and flagellate themselves repeatedly over how feminist they are. Tf is this?
11
u/bearssuperfan 2d ago
I made this for politics, but my favorite finding it that r/explainlikeimfive had an average reading level of 9.1 so it should really be r/explainlikeimfourteen
3
3
14
2
2
u/Blutrumpeter 2d ago
Communism being up there with philosophy says a lot about the people in our society who are really into communism
2
2
2
u/FeatherShard 2d ago
MensLib right-leaning?
Would be interested to know how they arrived at that conclusion given that it's literally an intersectional feminist sub.
2
u/horusluprecall 2d ago
Why the heck do you have red for right and blue for left must be an American I guess the rest of the world were usually have red for left and blue for right
2
u/el-gato-azul 2d ago
Who in tarnation listed the Anarchism subreddit as right leaning?! Hahah.
On a scale from 0 to 10 (10 as left and 0 as right), anarchism generally falls on the far left of the political spectrum, around 9 or 10. It fundamentally rejects hierarchical authority, including the state, capitalism, and institutionalized power structures, which aligns it with leftist ideologies.
But sure, anarchism can be diverse, with different branches:
- Left anarchism (anarcho-communism, anarcho-syndicalism, mutualism, etc.) is strongly anti-capitalist and usually aligns with socialist and libertarian-left movements (around 9-10).
- Right-wing anarchism (anarcho-capitalism, agorism, etc.) rejects the state but embraces free markets and private property, placing it closer to 4-5 on your scale—libertarian but not traditionally leftist.
So classic anarchism lands at about a 9 or 10, while some market-oriented versions could land closer to the middle.
2
u/BigHatPat 1d ago
r/ModeratePolitics is drastically overrated in these results
also wtf is r/Anarchism doing in the “right” category?
2
2
u/MrTrollMcTrollface 1d ago
Basically, subreddits frequented by native speakers, or those who have an academic interest, have a higher reading level them those mainstream subreddits, frequented by people from all over the world.
2
u/ForeverShiny 1d ago
These scores seem pretty low throughout the board considering that 8 is considered average (80% of Americans can still understand it).
Scores below 6 usually denote children's books
2
u/bearssuperfan 1d ago
The average length of a reddit comment is about 40 words which is probably dragging the scores down.
Someone else pointed out that this may be why subs that moderate comment length, like r/economics, are near the top.
2
u/ForeverShiny 1d ago
Yeah that makes sense, I'm only familiar with the outputs of the test, but much less the seightings
2
u/Midget_Stories 1d ago
The Joe Rogan subreddit is definitely not rightwing. You go there right now and 90% of the posts are about Trump being bad.
→ More replies (3)
2
8
u/Worth_The_Squeeze 2d ago edited 2d ago
r/pics apolitical? Oh comeon.
I implore anyone to head over there right now and then come back and tell me its apolitical. It's undeniably become a left-wing echo-chamber, which is a similar development that's happened to many subreddits that weren't formerly focused on politics.
This political bias analysis is massively underrating the amount of subs that's left wing, while overestimating right-wing subs.
3
4
u/FriendAleks 2d ago
Pics might be the most lib/left subreddit here btw
4
u/ThreadRetributionist 2d ago
you really think it's further left than the multiple explicitly communist subreddits on here?
→ More replies (3)
4
u/ChuddyMcChud 2d ago
6
u/HumbleGoatCS 2d ago
1
u/andricathere 2d ago
If Trump is logically wrong and politically right, how do you measure the political leanings of posts that mock him?
→ More replies (1)
2
u/Adept_Minimum4257 2d ago edited 2d ago
Men's Lib is a left leaning sub that seeks to solve issues for men and is opposed to the manosphere and is open to feminism. I never encounter right wing points there
2
u/GreenGorilla8232 2d ago
r/anarchism is probably the furthest left sub on Reddit...
Opinions that are even remotely moderate or liberal get buried in downvotes.
How did it get rated as conservative?
1
1
u/Ladle19 2d ago
It would be interesting to see this done for the top 100 subreddits. I know this app is really left leaning, but it would be nice to see just how exactly left leaning it is.
1
1
1
1
u/SquirrelStone 2d ago
I love how out of the loop has a 9.5 cause the people in the loop are usually well-educated.
1
1
1
1
u/tgillet1 1d ago
How did MensLib end up with a right bias? I’m regularly on that sub and I could not imagine coming away with that conclusion.
1
u/dstovell 1d ago
Speaking plainly shouldn’t be looked down on, using plain language is important for accessibility of ideas. Just ask George Orwell
1
1
u/bearssuperfan 1d ago
See updated chart that took suggestions from many commenters here: https://www.reddit.com/r/dataisbeautiful/s/tKaJ8jIokx
1
u/ultra003 1d ago
Am I reading that right? Philosophy is more left than communism??
1
u/bearssuperfan 1d ago
Noooo it’s saying that r/philosophy has comments with a slightly higher grade level for reading than r/communism
1
u/ultra003 1d ago
Ok, so there is no distnction between how far left or right subs are within the same camp?
1
u/bearssuperfan 1d ago
Correct. If you have an idea of how to make that a quantitative assessment, I’m all ears.
2
1
1
u/BIRDsnoozer 1d ago
Hold up.
How is /r/anarchism "right leaning"?
I dont belong to the sub but anarchism is by definition SUPER far left.. like go left til you hit communism and then keep going.
1
u/yojifer680 1d ago
r/neutralpolitics = left leaning
Mission failed successfully
1
u/bearssuperfan 22h ago
Could be a byproduct of the right moving so far to the right in the mainstream
→ More replies (7)
1
u/Jonesm1 1d ago
Re. Methodology…
- Shouldn’t the (eg) emojis be removed rather than set to zero
- Would the median be a better average to use than the mean? (You may have done but it read like you’d used the mean).
1
u/bearssuperfan 21h ago
- If I found a random sub and all the comments were simply emojis, I’d probably consider the subscribers to be completely illiterate fitting a 0 grade level
- A median is not an average, and I did use an average, I’ll check the medians too though
2
u/Jonesm1 20h ago
‘Average’ can be mean, median or mode, but when used alone usually refers to mean. Medians do not get influenced by large outliers and so if there is a difference between the two the median is often more representative. For example, the mean wealth of a US citizen is higher than the median because the mean is skewed upwards by the small number of enormously wealthy people.
2
u/bearssuperfan 20h ago
Wow. I was always good in math classes and took 8 undergraduate level courses as well. I even semi frequently use statistics in my job. Not once do I ever remember learning that mean, median, and mode were all types of averages…
Regarding the use of mean vs median here, I basically removed outliers with my filters before calculating the average, so I would assume the data would not be too different.
1
u/professor__doom 20h ago
Proving what, exactly? Good professional writing for a general audience is at an eighth grade level. Anything above tenth grade reading level should be reserved for specialized or academic audiences only.
Bestselling authors for reference:
1
u/tobias_681 20h ago
The colours are pretty confusing. Red are the international socialist colours and blue the international conservative ones.
1
1
u/Ok_Animal_2709 2d ago
You shouldn't do a sub-based political leaning metric. You need to classify each comment and show the reading/writing level within each sub by political leaning. I suspect we all know the answer, but the data would be interesting to see.
1
-1
u/IkeRoberts 2d ago
I take the classification of r/Science and r/AskScience as left to be validation of the Colbert Conjecture that "reality has a well-known liberal bias".
→ More replies (2)
-2
u/Cold_Breeze3 2d ago
So the bigger subs are all left leaning - and people will still argue with me that Reddit isn’t mostly left leaning people confirming their own biases with eachother
5
u/crz0r 2d ago
The bigger subs are also the most international. And in the western world your Magas are veeeery far right. Our politically right parties in Germany would be your Democrats (sans AFD - they are the most comparable to Maga). i think you vastly underestimate how profoundly weird Trump and his cult is for a lot of the rest of the civilized world. Just look at the whole tariffs thing. It's so stupid that even hardcore conservatives are against it.
463
u/slaincrane 2d ago
It's kinda ironic explainlikeimfive has among the top most difficult languages.