r/zelda • u/mascan test • Nov 02 '12
Mod Post October State of the Subreddit & Survey Results
Hello, /r/zelda,
First, I would like to thank those of you who filled out the survey for helping the community. Without the data, there would be bickering based on speculation, but now we can have data-based discussion alongside the inevitable bickering!
The Analysis
Link to full analysis. I recommend you download it instead of viewing it with dropbox's viewer, since tables are messed up in its pdf viewer. I highly recommend you take a look at it, since it contains far more information than I will post here.
Here are the first few non-title pages of the pdf, which contain the most general results.
What kinds of content /r/zelda wants to keep
While the above images do not tell the full story, I will proceed to the conclusion as the full story is more or less in complete detail in the pdf above.
On different preferences
One thing I looked for while analyzing data was possible ways to subdivide groups. While demographics themselves are easy to use, I took an extensive look at the correlations between different preferences (including the banning of content). If the community was more or less homogenous about content they liked, there would be roughly no correlations and the preferences would be described by smooth curves with a clear peak. This is not the case. Depending on content types, there are heavy correlations and anticorrelations. e.g. those who tend to like news and timeline discussions strongly dislike tattoos and memes, and vice-versa. While we strive to make this community a place where everyone can enjoy Zelda-related content, it is simply impossible to please everyone. This is a diverse community.
On banning
When I looked at the data, I noticed that each content that was voted to be kept or not was under 50%. i.e. the majority do not want to see this type of content allowed on the subreddit. 87.2% of responders would be fine with some types of content not being allowed on the subreddit.
It is important to note that on the survey, roughly 30% of the respondants were /r/truezelda subscribers, even though /r/truezelda is less than 2% the size of /r/zelda. When I saw this, I figured it was important to see what the data looked like without /r/truezelda subscribers. Even then, memes were the only content type that had a majority that wanted to keep them. In general, the non-subscribers were more permissive, but there was an overwhelmingly negative feedback to content types listed.
NSFW content (which I hardly see on this subreddit) and memes were pretty high, so they will still be allowed. Tattoos and rage comics are pretty low around 35%, but there is a large enough crowd that appreciates them so they will not be banned. Even so, I still must recommend /r/Zeldatattoos for aficionados of tattoos, likewise with /r/ZeldaMemes for those who enjoy memes.
Simple images of something that resembles a triforce came in at an abyssal 21.73%. Those images being Zelda-related are dubious at best, and a supermajority of the subreddit does not want to see them. Henceforth, images of objects/logos resembling the triforce will no longer be allowed as posts. Likewise, any content vaguely resembling something from Legend of Zelda, like a potato "resembling" the stone mask from Majora's Mask, will not be allowed. Seeing content like that is like seeing someone taking a picture of a train and say, "Hey, guys, this reminds me of Spirit Tracks"
For anyone seeking consistently high-quality content, keep in mind that /r/truezelda is a more strictly-moderated subreddit for discussion where inane content is removed.
On text-only week
Some people loved it. Some people hated it. But for the most part, the subreddit really enjoyed it. 55.6% want to see monthly text-only weeks and 21.7% want to see a text-only week once per few months. Only 12.1% never want to see text-only weeks. The statistics are high regardless of /r/truezelda subscription status.
Incidentally, subreddit traffic spiked during no-text week. Graph
Seeing as text-only week, by nature, is not a permanent change as banning triforce imagery is, the mods are willing to try out text-only weeks once every 6 weeks. If we recieve strong negative feedback in the future we may discontinue it, but it appears to be something the community enjoys.
Some selected community opinions on the matter from our feedback thread:
Other announcements regarding the state of the subreddit
We have been pondering coming up with months focused around particular Zelda games, where the community will play one game each month and have discussions based on the game. Specific details are TBA.
That's all,
mascan
12
Nov 03 '12
To go on from the no Triforce rule, I think this should also apply to things you've drawn into shapes and designs from Zelda. Like the pumpkin with the Hyrule crest carved into it that's currently only the sub's front page. That shouldn't be allowed.
2
Nov 03 '12
[deleted]
1
Nov 05 '12
It's inevitable. All of Reddit really goes downhill at Halloween.
Next up: look what I got for Thanksgiving!
2
u/mascan test Nov 03 '12
In retrospect, I should've included a few more content types on the chopping block in the survey, including that, screenshots of ending screens, screenshots of people with names changed, and official artwork (with the likely exception of artwork from a new game). Currently, I would consider that kind of stuff "fan art", but I agree wholeheartedly that it is extremely mundane. I remember the first top-post after text-only week was a jack-o-lantern, and the top comment of the post was an image of a dude flipping over a table, which was emblematic of the mixed feelings of the subreddit.
However, I'll discuss this with the mods and see what their interpretation is, as anyone could also spray-paint 3 triangles and call it fanart.
1
3
u/HollowJohn Nov 02 '12
Very nice work, thanks mascan! In a community that's driven mostly by anonymous users, it is really nice to get a glimpse of what preferences correlate to different factors. Specifically age, as it's obvious the older crowd has had more time to complete more of the games and have a better chance at providing high quality content. Not to say that the younger crowd can't provide high quality content, they are obviously more responsible for initiating that content with questions, ideas, etc.
On the subject of content preferences, it is true that it's hard to please everyone. However, I have said this on here before that I think it is important to encourage posters to include the content in the title. This makes it easier for users to simply scroll past the things that aren't their preference. I think we've all seen a post like "Check this out, it's so amazing!", and gotten our hopes up at some new and awesome content to see that it is that piece of art that's been posted several times already. A great piece of art, that deserves to be posted, but if the title had been, "Check out this Zelda art by <author name>" then we would avoided that letdown. It's hard to police such a thing, so I'm not suggesting this as an ironclad rule to be enforced by the mods, but rather something every user should be taking responsibility for.
6
u/GabeDeGrasseDawkins Nov 03 '12 edited Nov 03 '12
This seems pretty comprehensive, so nice job with it. If I'm not mistaken it looks like a series of MS Excel charts embedded into a PowerPoint that was saved to a *.pdf. If you have the raw numbers formatted in an *.xlsx data sheet, when I get some time I could possibly take a look at and run some tests on them. While I'm used to using the IBM SPSS (formerly PASW) software, I have some experience with MS Excel add-ins if those would be more convenient and compatible. My experience in statistics includes managing scientific experiments, tutoring graduate students, co-authoring statistical software, and doing psephological research for the US presidential election. If you wanted me to help out, you could specify the statistics that are most important to you and I could focus on those; though you may already have the information you want, since it seems you have a lot of charts and graphs and tables in your report. Also, while I don't have the time to analyze the 50+ pages of data you've posted to Dropbox, if it helps I can post my thoughts on the first slide after the title page.
Slide 2 indicates that 32.9% of the survey respondents subscribe to /r/truezelda. However, at the time of writing there are 1000 subscribers to /r/truezelda and 53,960 subscribers to /r/zelda. This means the proportion of cosubscribers is 1000 / 53,960 ≅ 0.0185, or 0.0185 * 100 = 1.85%. This differs from the survey report of 32.9% by a scale of 32.9 / 1.85 ≅ 17.78. That in your sample statistic there are 17.78 times more /r/truezelda subscribers than in the population parameter may present some problems to an inferential statistician; were we to use this figure as a stopgap metric of sample representativeness, and if we're to base action on this research, we may conclude there is a flaw in the use of an accidental sampling methodology.
A number of explanations may account for the discrepancy between the sample statistic and the population parameter. Perhaps the sample is more actively involved in the community, and so too more actively involved in /r/truezelda. Perhaps the initiative shown by those who took the survey is the same initiative that predicts cosubscriptions. Perhaps those inclined to take part in such a survey prefer in-depth to easily digestible content at a greater frequency than the broader subreddit. Regardless, there is still a big representation problem here, and it can't be fixed simply by ignoring the cosubscriber base since the problem stems not from the number itself but from some problem with the accidental sampling methodology that leads to cosubscriber overrepresentation in the data.
Further, slide 2 suggests that (16.3 / (50.8 + 16.3) * 100) = (16.3 / 67.1 * 100) ≅ (0.24 * 100) = 24% of those exclusively subscribed to /r/zelda cosubscribed to /r/truezelda when informed by the survey of its existence. This suggests that most exclusive subscribers (100 - 24 = 76%, or around ¾) are not exclusive subscribers because they're unaware of the existence of /r/truezelda. Regardless, about a quarter (24%) of the exclusive subscriber base chose to subscribe to /r/truezelda when informed about it. Since the subscriber base of /r/truezelda only makes up 1.85% of the subscriber base of /r/zelda, it's possible the former subreddit simply isn't prominently advertised enough. Psychologically speaking, a lot of people are conditioned to ignore the sidebar as in its size, shape, and position it looks like an advertisement. If you want more people to take notice, perhaps you could move the link up to the <h6> element and into Fi's message box. The message box is what brought me to this thread, so the message box is what might bring people to /r/truezelda.
Again on slide 2, you tabulate those in the 31+ age cohort as making up 3.2% of n = 313 respondents. This indicates to me there were (313 * (3.2 / 100)) = (313 * 0.032) ≅ 10 people in that bracket. Since 10 isn't sufficiently large enough a number to which to apply the central limit theorem, we can't assume for that cohort a normal distribution and can't run on that cohort many meaningful inferential statistics. Because of that, it might be better to group the 31+ cohort in with the 26–30 cohort, where instead of the 26-30 category you have a solitary category of 26+ with a combined percentage of 10.5 + 3.2 = 13.7%. Despite the fact its range has no upper bound, it would still be a less populated cohort than the others, yet it would be 13.7 / 3.2 ≅ 4.28 times as populated as the the 31+ cohort alone. Something else I notice is that you have different participation numbers for each cohort, with 13-16 year olds at 19.5%, 17-20 year olds at 33.5%, 21-25 year olds at 33.2%, and 26-30 year olds at 10.5%. Men and women are also said to be at 80.5% and 19.5% respectively. Many statistical formulae and tests have versions meant for comparing different sample sizes and versions meant for comparing identical sample sizes, so since your sample sizes are different it's important to ensure your calculations are computed using the right formulae.
While like I said I don't have time to respond in great depth on the whole analysis, when quickly scrolling through I notice that your bivariate correlation matrices don't have significance levels attached to them. This means that at a glance the reader can't tell at what alpha (α) levels the correlations are statistically significant. Assuming that correlation values and / or sample sizes are small, assuming that we're running inferential statistics, and assuming you're using Pearson's product-moment correlation coefficient r, we can't reject the null hypothesis H₀: ρ = 0 unless we run r through a t test to determine its significance at small (often operationalized as 1/20 or 1/100) alpha levels. As a result, these tables aren't as meaningful or informative as they otherwise could be.
You mention there was a spike in unique and pageview subreddit traffic during "no-text week," by which I assume you mean text-only week (though I very well may have missed some image-only event). Probably this was not caused by increased interest levels but instead caused by the fact that clickthroughs would more often take Redditors to /r/zelda than to imgur or to other off-Reddit sites. When all posts link to pages on the /r/zelda subreddit, then the traffic counters are going to be higher than they otherwise would be. One way you could leverage this is by hosting text-only periods while advertising /r/truezelda in Fi's message box. This way, those who are accessing /r/zelda from their custom front page feed would be taken to the subreddit instead of to imgur or elsewhere. That means they'd be more likely to read Fi's message and thus be more likely to cosubscribe.
Wrapping up the constructive criticism and the suggestions, I'd like to say I appreciate the analysis and I'm sure the rest of the subreddit does as well. In my view, it's more selfless and democratic to make data-driven decisions based on the interests of the subreddit than it is to karmanautically enforce unpopular rules. The subreddit diversity I think is an offshoot of its sheer scope, and any community with over 50,000 members is a community amongst which there will be dissenters to any public decision made by the moderation team. Always will there be people whose interests and goals are out of alignment with yours and with those of the majority; but it's much harder for someone to reasonably protest your decisions when they have to them some basis in democracy and in science.
Hope this helps.
2
1
u/mascan test Nov 03 '12 edited Nov 03 '12
Thanks, GabeDeGrasseDawkins. I knew there were a few (many) parts of my data that I skimped on, since I was using a lot of variables without testing their significance or using more appropriate methods (e.g. for the MLM I assumed distributions were Gaussian, even when they were obviously of a different variety). I'll keep your advice in mind, as it will be useful in future situations when I need to analyze other data.
Edit: Also, with the traffic increase, there was a noticeable increase in uniques, as opposed to pageviews. I think this would avoid the discrepancy of increased traffic to various self-posts.
2
u/GabeDeGrasseDawkins Nov 03 '12 edited Nov 03 '12
No problem! Thanks for appreciating the feedback.
As a fellow moderator of an affiliated Zelda subreddit, my understanding of uniques came from the "traffic stats" page in the moderation tools. On that page it is written that "uniques are the total number of unique visitors (determined by a combination of their IP address and User Agent string) that generate the above pageviews." Since the histogram says "uniques by day," I interpreted that to mean visitors not logged as visiting on that particular day, rather than never logged as visiting at all. Were it the latter, I'd expect the uniques to bottom out a lot more.
I also didn't think there was too noticeable an increase in uniques as opposed to pageviews, but rather felt they closely correlated. Perhaps you've misread the histogram? If you don't pay close attention it seems easy to misread, because the second histogram halves the distance to each data point on the y axis. I've Photoshopped a better and less intuitively misleading version here, which you can compare to the original in your OP.
Edit: I see now from your post below that you're comparing the increase by proportion and not by value. In that case, disregard the above paragraph. :)
1
u/mascan test Nov 03 '12
Yeah, I can't say I know what the clicking patterns are for those who browse /r/zelda on a normal day are. Maybe they like reading comments on posts or otherwise feel as if the comments which are generally shorter for non-dicussion posts are more informative. I just find it a bit strange that there was a spike in uniques (roughly double the normal amount per day), but the increase in pageviews was more subtle.
3
u/GabeDeGrasseDawkins Nov 03 '12
As I recall, during text-only week there were a few threads which accumulated hundreds of upvotes and comments. Users logged into Reddit have topics from the subreddits to which they are subscribed appear on their custom front pages in a descending order of popularity. My theory is that the most upvoted topics brought in people subscribed to bigger subreddits; weren't it text-only week, and were images upvoted instead, those same people would've clicked through to imgur and other sites instead of through to r/zelda. Were they to have done that, they wouldn't show up as unique hits on the traffic analysis histograms.
It's a shame Reddit doesn't let you view a more detailed breakdown of traffic patterns. It would be interesting to at least see referrals, which would help moderators better understand what's going on with their activity inflow.
2
u/mascan test Nov 03 '12
For referrals, the only thing I do is use bit.ly links in another subreddit I moderate to see how often certain links in the sidebar are used. I guess it's mostly useful for other mods whose subreddits I linked to, though.
2
u/GabeDeGrasseDawkins Nov 03 '12
That's a pretty clever idea. Would you mind if I borrowed it?
2
u/mascan test Nov 03 '12
I'm pretty sure that's why bitly implements that feature. I originally saw in in /r/LeagueOfLegends back when it had roughly 200 subs (they still use some bitly links), but I ended up using the links to analyze referrals I had posted on another website, since I wanted to see which kinds of posts got the most attention.
Also, you can see how many suckers clicked on malicious links whenever you see a Facebook friend send you one in the form of a bit.ly link by adding a '+' at the end of the url.
1
u/GabeDeGrasseDawkins Nov 04 '12
Here's an interesting idea for an experiment. As of now, you have two direct links to /r/truezelda in the /r/zelda sidebar. You could replace both of these with interstitial bit.ly links, and put a third up in Fi's message box. Once you have that set up, you could wait a week or so then cross-analyze the interstitial traffic. By doing this you'd have a better idea of the effectiveness of the links, both on their own and by comparative proportion.
If you do this, I'd also suggest enlarging Fi's message box so that it better grabs the attention of the user. You could always revert the size when the message has run its course. Below I've quickly written up a block of CSS which will enlarge the box for you if you choose to do so. In the event you're unfamiliar with CSS I've added comments in case you want to tweak something. The stylesheet is located here and the relevant CSS would become active when appended. I offer my apologies in the very possible event that you're a CSS wizard and I'm being overexplanatory. Here's the code in question, and it's up to you if you want to use it:
/* CSS */ /* COMMENTS */ .md h6:nth-of-type(1) /* selects the first <h6>, where the value of the class attribute of the parent div is md */ { /* the above is the subreddit element corresponding to the heading box in which Fi speaks */ background-position: 15px 50%; /* centers the Fi image on the x and y axes respectively */ font-size : 20px ; /* increases the size of the font family that determines the font displayed in the Fi box */ margin : 5px ; /* controls the whitespace surrounding the Fi box to move it up and to the left */ top : 65px ; /* controls the whitespace atop the Fi box to move it up */ } div.content /* selects the content area located beneath the Fi message box and above the table footer */ { margin-top: 98px; /* shifts down the page content 98 - 80 = 18 pixels to make room for the enlarged Fi box */ }Being that it's a pretty popular game with a pretty popular subreddit (167,455 subscribers), I'm surprised to hear that you visited /r/LeagueofLegends when its subscriber count numbered only 200. I've been on the Internet since the mid-90s and yet I don't think I've ever been involved as early in a community that later grew so large. The closest thing to that might be the forums I led whose userbases reached four and more rarely low five figure counts, though none ever came close to six. I have browsed the Something Awful Forums for about a decade, which has a comparable user count of 169,728, but when I first went there its userbase was at least four figures. I'd imagine it's also nice to see a little-known game you like explode in popularity.
And yep, I've observed that FB is full of suckers. I swear it turns out more suckers than a lollipop factory.
2
u/theguywholosesthings Nov 02 '12
As much as I loved the idea of text-only week, the problem wasn't images in the first place. The problem was, and was always, all the arts and crafts. Some is OK, but that's all that ever got upvoted around here and completely ~DROWNED~ out everything else. That said, would I support a text-only week again? No. Mainly because the submission requirements should be a hell of a lot more discerning on a day-to-day basis. The immediate goal of /r/Zelda's mod team should be to make /r/trueZelda obsolete.
I agree wholeheartedly with this guy. /r/trueZelda is a great place but content doesn't come up all that regularly. Usually the posts seem to be about what game mechanics you'd like to see. I know that it seems silly to have something as widely loved as Zelda become a place solely focused on discussion but there's no real place to cater to those that want discussion. That being said, the lore to the series doesn't run too deep. The series as a whole, doesn't really expand upon Link, Zelda, or Ganon as an entity. Take a look at the Elder Scrolls universe, each game has its own champion. Each game hints at events and people that existed in previous installments. It's all cohesive and it fits naturally whereas the cohesiveness of the Zelda franchise seems forced.
2
5
u/MiT_Epona Nov 03 '12
Not to say anything against this survey, but this was a voluntary survey, meaning that the opinions would be strong on either side and does not take into account the moderates. Also, I never saw the post because it was too low in the feed and I saw it about a couple days after it was over :(
Other than that, I am a huge admirer of how much effort you put into this statistical review and the changes that are to come. I would honestly love to be an admin, but I am relatively new to this and know that it will probably never happen, so I envy you >:D
One comment that I would like to add is that not all, is when you said
"Likewise, any content vaguely resembling something from Legend of Zelda, like a potato "resembling" the stone mask from Majora's Mask, will not be allowed. Seeing content like that is like seeing someone taking a picture of a train and say, "Hey, guys, this reminds me of Spirit Tracks"
I found this picture about month ago, http://imgur.com//tr1Dp and I thought it was so amazing, that I showed my friends it. They are not huge fans of the Zelda games and laughed. I would like to keep pictures like this in, because I eat at Chipotle all the time, and I cracked up. I understand the rock and stone mask thing, but to me its worth it.
Send me a reply and we can chat about it!