Tag Archives: quarter in review

August to October 2016 in review

This is my fourth quarterly review, and it covers the months from August to October of 2016.

Things continue to be busy on the job front. There were a few big job-related projects that happened this quarter. I expect the next quarter to be more relaxed on the job front, partly because of a winter break slowdown and partly because some of the big projects are over.

Updates to Wikipedia Views

I made a number of updates to the code underlying Wikipedia Views, a tool I build to make it easy for people to look up views of multiple Wikipedia pages over long time periods. Many of these changes were motivated by work I’m doing on a Wikipedia decline blog post discussed later.

The chief changes I made include:

  • Added support for looking up mobile web, mobile app, desktop spider, and mobile web spider data from July 2015 onward, using the Wikimedia API. Also, to make the desktop data consistent with the other sources, I switched desktop to use data from the Wikimedia API starting July 2015 (previously, it had used the Wikimedia API only starting January 2016). This has retroactively changed some pageview counts on Wikipedia Views.
  • Added support for cumulative Facebook shares starting October 2016, and drawing on the Facebook API. This data only includes pages that were successfully captured at the end of the month.
  • Added automatic graph generation when viewing data for multiple months or years. The graphs, shown below the table, help give a bird’s eye view of the numbers in the table.

Other changes I hope to make in the next few weeks are listed below.

  • Re-enable CSV export, a feature I had disabled while adding new drildown-based features.
  • Complete documenting and fixing bugs in the existing features.
  • Highlight pages that currently redirect to other pages.
  • Add support for showing data on breakdown by referrers for a few months in 2015 and 2016 when Wikipedia clickstream data is available.
  • Allow for comparison of data across multiple languages in the same query.

Wikipedia editing

For background information, see my site page about Wikipedia

I created two new pages this quarter: Alan C. Nelson and Gladys Block. Both of them were created in the latter half of October, after a long hiatus from new Wikipedia page creation.

I expect to resume Wikipedia page creation after I finish work on Wikipedia Views and on the “Wikipedia decline” post I discuss later in this review.

While reviewing my early Wikipedia page creation history, I realized that a little over 100 pages I had created prior to 2010, that have survived deletion, were not included in the Wikipedia Views tag for pages I created. After adding them, the total count of pages I have created (including a few redirects) stands at 417.

In total, pages I created over my lifetime got 584,534 desktop pageviews, 354,365 mobile web pageviews, and 8,081 mobile app pageviews. In addition, they were viewed 141,778 times by desktop spiders and 14,885 times by mobile web spiders.

You can see the lifetime data for the tag as a whole here. I have also attached a graph from the link; the highest line is the total and the second highest is the desktop pageviews. You can also access a gigantic table (with 223,095 cells of which 71,307 are legitimately filled) of all pages I created across all months and all device types here (cautionary note: this is a large page and may take several seconds to load).

Liftetime pageviews of pages created by Vipul Naik
Liftetime pageviews of pages created by Vipul Naik

Now that I’ve started recording cumulative Facebook shares, I have data that pages I created over my lifetime had received a total of 2904 likes+comments+shares on Facebook as of the end of October 2016. You can see the data by page here. Couple of caveats:

  • This excludes 106 pages I recently realized I had created a while ago, so the total is an underestimate. The excluded pages show the message “cannot retrieve this data” indicating that data on cumulative shares was not captured during the month.
  • The counts are probably underestimates since we have evidence that Facebook has misplaced likes, comments, and shares that occurred in 2013 or earlier.

Sponsored Wikipedia editing

For background information, see my site page about sponsored Wikipedia editing

Sponsored Wikipedia editing operations continued, though at a lower overall level than previously. Many of the people recruited either left permanently, or have significantly reduced output because their academic term (high school or college) has begun.

Issa Rice (pages and payments) and Sebastian Sanchez (pages and payments) have continued to work at a similar rate as before.

Sebastian in particular has continued to work on pages in two broad themes: disease-related timelines (18 pages so far, on which I’ve spent $1466) and timelines of healthcare by country (14 pages so far, on which I’ve spent $1381). He has also created four other pages in themes broadly related to healthcare and global health, and done one translation.

I plan to more comprehensively review sponsored Wikipedia editing at the end of the year. By then, I expect to enter the data into a database so that I can generate a variety of summary reports.

WikiHow

For background information, see my site page about WikiHow

I wrote one new wikiHow article: How to Understand Your Website Traffic Variation with Time. The article got a lot of views in the first week (getting to about 800 pageviews), likely because it was featured prominently on various landing pages and lists of top articles. Its growth has subsequently slowed down to between 5 and 10 new pageviews a day. The draft history (before I pushed it to WikiHow) can be found by looking at the history for the version in my personal Git repository.

This post is the second in a series of posts I intend to write on understanding website traffic. My previous post, published last quarter, was How to Understand Your Website Audience Profile.

I also began drafting a new article in wikiHow style: How to Master Online Surveys. I don’t know if I’ll try publishing it on wikiHow (as there are many very similarly named articles there right now).

Personal server migration

I finished migrating a personal server (that hosts a variety of websites I manage). I had begun this migration last quarter.

Wikipedia decline blog post

I worked together with Issa Rice on a blog post that continues to explore questions originally raised in my post The great decline in Wikipedia pageviews. The new post is far thorougher than the older post. It uses new, more reliable pageview data, Internet surveys, and better graphing and visualization tools.

The first iteration of the post was done by Issa Rice over about ten days in September, and can be seen here.

Since the post was turning out to be fairly involved and we realized that a lot of pieces were missing, I decided to take over the completion of the post, since I could coordinate that better with improvements to Wikipedia Views that would help make research and exploration for the post easier. A copy of the current (very incomplete) draft can be found here.

Writing the post has been a challenging exercise of going back and forth between simplifying the content for presentation, and diving even deeper into the empirics and data in order to obtain clearer conclusions. Realistically, I expect that we will have the post ready for publication in January 2017.

Market wiki updates

Some posts by Buck Shlegeris linked to the price bundling page on the Market Wiki that I had written a while back. Buck raised a few questions (not specific to the page, but related to the topic of price bundling). Inspired by these, I worked to significantly improve coverage of price bundling. I also spent part of some weekends and some BART rides on improvements to the articles Effect of sales tax on market price and quantity traded and effect of price ceiling on economic surplus.

I believe that this wiki is one of the more promising things to edit when I’m in the mood for doing something that is intellectually interesting but doesn’t involve too much of the “arbitrary real world” i.e., doesn’t involve too much research on specific current events or history. There are times when I want to take a break from the huge amount of arbitrary information the real world entails, but I still want to do something in the economics or sociology realm rather than deal with pure mathematical abstractions. The Market Wiki fits this mood niche, and I believe that it’s one of the most high-impact things I could do that fit the mood niche.

Entertainment

I watched a few more Korean dramas:

I also watched two Japanese dramas, Mischievous Kiss: Love in Tokyo and its sequel, Mischievous Kiss 2: Love in Tokyo. They were based on the Japanese anime Itazura na Kiss. In addition, I finished watching The Flower in Prison, a semi-historical Korean drama based on events in the mid-16th century Joseon era.

May to July 2016 in review

This is my third quarterly review, and it covers the months from May to July of 2016.

Things continue to be busy on the job front. I also wrapped up some of my personal bureaucracy by the end of April, so this quarter was focused more on personal projects. However, the main personal project I worked on in the quarter was working with Issa Rice to scale up sponsored Wikipedia editing.

Wikipedia editing

For background information, see my site page about Wikipedia

I created 1 page in May, 2 pages in June, and 2 pages in July. In total, I created 5 Wikipedia pages.

As I mentioned in my previous quarterly review, the pages I am creating right now are not as optimized for pageviews as pages in the past have been. Rather, they are pages in domains where I want to acquire a deeper understanding and believe the research process involved in creating the page will help me. All the pages I created in the quarter were related to migration.

In total, pages I created over my lifetime got 474,772 pageviews over the quarter. You can see the data for all of 2016 (only three of the seven months are in this quarter) here. This is between the 25th percentile estimate (460,000 views) and 50th percentile estimate (520,000 views) that I made in my previous quarterly review. So it was a little lower than my median estimate but well within the usual range,

Forecast for future impact: I expect the pageview counts to be roughly similar to the previous quarter: a baseline of around 155,000 pageviews per month, plus a possibility of minor spikes because of some pages being topical. The estimates are a little lower than those I made last quarter, and the variance is a little higher.

  • 2.5th percentile: 310,000 views.
  • 10th percentile: 375,000 views.
  • 25th percentile: 445,000 views.
  • 50th percentile: 500,000 views.
  • 75th percentile: 635,000 views.
  • 90th percentile: 720,000 views.
  • 97.5th percentile: 800,000 views.

Sponsored Wikipedia editing

For background information, see my site page about sponsored Wikipedia editing

With help from Issa Rice and Ethan Bashkansky, I significantly scaled up sponsored Wikipedia editing operations. You can see a full list of people I have sponsored, along with links to their contributions I have paid for, here.

It has been (and continues to be) an interesting set of challenges in recruitment, training, motivation, figuring out payment, and negotiating the Wikipedia bureaucracy. Issa or I might write more about the experience in the future. The jury is still out and things will be clearer by the end of the next quarter. A few people who have contributed significantly to the project, and deserve special shoutouts, are Issa Rice (pages and payments), Sebastian Sanchez (pages and payments), Jesse Clifton (pages and payments), Ethan Bashkansky (primarily for recruitment efforts) (pages and payments as well as recruitment efforts) and Alex K. Chen (pages and payments).

WikiHow

For background information, see my site page about WikiHow

I wrote one new WikiHow article: How to Understand Your Website Audience Profile. The article got a lot of views in the first week (getting to about 800 pageviews), likely because it was featured prominently on various landing pages and lists of top articles. Its growth has subsequently slowed down to between 5 and 10 new pageviews a day. The draft history (before I pushed it to WikiHow) can be found by looking at the history for the version in my personal Git repository.

The post is the first of many I intend to write related to understanding and interpreting trends and patterns, with an initial focus on the web.

I also edited some of my older WikiHow articles, including How to Gauge the Popularity of a Topic Online. I added and fixed up pictures for some other articles.

In total, pages I have created on WikiHow over my lifetime have about 9000 pageviews, and are getting between 1000 and 2000 additional pageviews every month. I had started editing WikiHow in May 2015.

Other written content

In response to charity evaluator GiveWell’s May 13, 2016 report on their annual traffic and money moved in 2015, I wrote a post reviewing my own past estimates of money moved, and discussing some implications.

I finally got around to publishing the blog post High-skilled hacks: a (very) brief overview of H-1Bs (more to follow). The post built on my reading and research into high-skilled work visas. I hope to do many other posts in the broad area.

Personal server migration

I began but have not fully completed the migration of all my websites to a cleaner Linode installation. The change will make it faster to quickly spin up a new Linode and transfer the key content to that. Basically, I am moving my server away from the snowflake server it had become in the last few years, and standardizing some of the procedures.

Entertainment

Two Korean dramas I have been watching in the last few months are The Flower in Prison (Viki) and Doctors (Viki). I’m watching both on Viki. An older drama that I binge-watched was Neighborhood Lawyer Jo Deul Ho.

I can now understand much of the Korean vocabulary used in the dramas, though these are probably unrepresentative of real-life Korean. Often, I watch an episode without subtitles and get most of the gist of it, but I miss a few nuances that I later fill in by watching with subtitles.

If you are interested in getting some drama recommendations, check out Evelyn Lee’s mid-2016 review.

I am also cancelling my ErosNow subscription since I don’t get a lot of time to watch Bollywood movies, and the smaller chunks of Korean dramas (1-hour episodes) make them fit better with my time.

February to April 2016 in review

Last quarter, I switched my review frequency to quarterly, in the hope of being able to provide more substantive updates in each review. Here I am, with my second quarterly review, this time covering the months of February to April 2016.

Things continue to be busy on the job front, but they are also more stable. In addition, I attended to some personal bureaucracy, including submitting my tax return and getting my passport reissued. With most of these items out of the way, I have started focusing more on various long-term projects, and hope to have a lot to report next quarter.

Wikipedia editing

For background information, see my site page about Wikipedia

I created 8 pages in February (the somewhat increased page creation count is because I took a total of four weekdays off that month, in addition to weekends). I created 2 pages in March and 2 pages in April. In total, I created 12 pages in the quarter.

As I mentioned in my previous quarterly review, the pages I am creating right now are not as optimized for pageviews as pages in the past have been. Rather, they are pages in domains where I want to acquire a deeper understanding and believe the research process involved in creating the page will help me. As was the case last time, my focus remains on issues surrounding migration. In fact, of the 12 pages I created in the quarter, only one, namely the page on Amazon Underground, was unrelated to migration.

In addition to creating pages, I also significantly edited and expanded some existing pages, most of them in the domain of migration. Some of the pages I significantly edited in this quarter were: F visa, I-20 (form), H-2A Visa, and Optional Practical Training.

In total, pages I created over my lifetime got 574,352 pageviews over the quarter. You can see the data for all of 2016 (including January, not included in the quarter) here. This is between the 50th percentile estimate (495,000 views) and 75th percentile estimate (640,000 views) that I made in my previous quarterly review. The main reason they were higher than my median estimate was that one page (Park Yeon-mi) got a lot of traffic in March due to related news coverage.

Forecast for future impact: I expect the pageview counts to be roughly similar to the previous quarter: a baseline of around 165,000 pageviews per month, plus a possibility of minor spikes because of some pages being topical. I have reduced the variation in my distribution since I am now more confident of the stability of pageview counts. The distribution is skewed to the right because of potentially huge traffic as a result of some events being highly topical.

  • 2.5th percentile: 350,000 views.
  • 10th percentile: 400,000 views.
  • 25th percentile: 460,000 views.
  • 50th percentile: 520,000 views.
  • 75th percentile: 645,000 views.
  • 90th percentile: 720,000 views.
  • 97.5th percentile: 800,000 views.

Sponsored Wikipedia editing

For background information, see my site page about sponsored Wikipedia editing

Through the quarter, I sponsored Wikipedia page creation by three people:

  • Alex K. Chen (updated page list here): Alex has worked on a large number of pages, including technology company timelines, as well as angel investors and venture capitalists and their funds. In addition to the pages I pay him to create, he also creates a number of other pages.
  • Issa Rice (updated page list here): After completing some pages on taxes and migration, Issa has shifted focus to global health and poverty.
  • Ethan Bashkansky (updated page list here): Ethan has continued to do sporadic work on tax forms.

In April, I announced on the Effective Altruism Forum that I was looking for people interested in Wikipedia editing. As a result of this announcement, I got two new people who have contributed significantly and expect to continue contributing for the next few months:

  • Jesse Clifton (updated page list here) is primarily working on topics related to animal welfare and rights. His first article was a timeline of animal welfare and rights. He has subsequently been working on articles on animal welfare and rights by country. I hope to share more exciting details of his work in my next quarterly review.
  • Sebastian Sanchez (updated page list here) is currently working on topics related to global health. I hope to share exciting details about his progress next quarter.

There are also other editors who are getting started with editing, and I expect to see some of them (and/or other recruits) make further progress during the quarter.

Subject wikis

For background information, see my site page about the subject wikis

I did active work on the subject wikis for a few segments of a few hours, but not in any systematic way that would affect the overall level of content or change the experience of typical visitors. The subject wikis have been doing well traffic-wise, but this is not due to any additional effort on my part.

Groupprops got a total of 261,975 pageviews this quarter compared to 244,136 for the corresponding quarter last year. The seasonal pattern in traffic was very similar between quarters, so much so that it is visually striking when you look at the graphs in Google Analytics. Market declined (47,781 compared to 65,367) and Calculus stayed stable (60,241 versus 58,499).

WikiHow

For background information, see my site page about WikiHow

I wrote one new WikiHow article: How to Avoid Crowds in Mass Transit. The article has received 491 views so far. I estimate that about 200 of those views were driven by my Facebook share of the article, and another 50 were a result of initial publication, plus my own views. The remaining views are probably organic.

Two of the WikiHow articles I have created over my lifetime now have over 1,000 views: How to Use Git Effectively (at 1,467 views) and How to Ride BART from the East Bay to San Francisco (At 1,362 views).

Other written content

I finally got around to publishing the blog post How did we get here? Chinese Exclusion Act buildup (1848-1872). The post summarized my understanding of the historical backdrop that built the momentum for the Chinese Exclusion Act in the United States.

I also worked on a few other drafts. I expect to publish some of these drafts this quarter, as well as start on (and possibly finish) some new posts.

I also posted to the Effective Altruism Forum to recruit Wikipedia editors.

Entertainment

I finished watching a few Korean dramas on Viki as they wrapped up: Madame Antoine, Descendants of the Sun, and Marriage Contract. In addition, I watched one older Korean drama: City Hunter.

I expect to be spending less time watching Korean dramas going forward, as I start picking up the pace on some more long-term projects during the weekends, now that some of the bureaucratic activities are completed.

November 2015 to January 2016 in review

After October 2015, I decided to switch away from monthly reviews to quarterly reviews. Depending on whether I feel I have clearer, more substantive updates at a quarterly level, and whether my readers agree, I will maintain this review frequency. This post reviews the past quarter (November 2015 to January 2016) while also offering a preview into the current quarter (February to April 2016).

Things have been busy on the job front in a variety of ways, but are now getting more stable (our company hired a bit more). I expect to not get a lot of time for the other activities but I do expect that the time I get for these will come in a more steady stream than it has in the past few months.

Wikipedia editing

For background information, see my site page about Wikipedia

I made one page in November, two pages in December, and four pages in January. You can see the lists of pages I created by month using the tags in Wikipedia Views. In particular, you can get the data at these respective links for the three months: November, December, and January.

I intend to create somewhere between 3 and 10 pages a month for each of the months of February, March, and April. So far in February, I have already created 5 pages, and have some more drafts ready to be pushed out.

The pages I am creating right now are not as optimized for pageviews as pages in the past have been. Rather, they are pages in domains where I want to acquire a deeper understanding and believe the research process involved in creating the page will help me. At present, my focus is on key aspects of migration laws, migration forms, and migration procedures in the United States. I intend to branch out more to other countries and to return to migration history (that I worked on in October and November) after finishing up a few more articles on migration procedures.

I do still hope to facilitate the creation of pages that are generically popular and underprovisioned. In recent months, I have successfully done so through sponsored Wikipedia editing. The people I have been paying include:

  • Alex K. Chen (updated page list here): Alex worked on timeline pages in January and has been working on pages about investors starting February.
  • Issa Rice (updated page list here): Issa created many great pages on important forms related to taxation in the United States, and is now transitioning to work on migration forms. See for instance the pageview counts in January for pages Issa created in January.
  • Ethan Bashkansky (updated page list here): Ethan is starting out with tax forms.

Measurement changes: With some help from Issa, I switched my data source for Wikipedia Views from stats.grok.se (which had stopped updating) to the Wikimedia API, which is probably a better and more stable option going forward. One consequence of this change is that it is now possible to exclude obvious bots and spiders from the view counts, and data from January 2016 onward excludes those numbers. Hence, there is an artificial drop from December 2015 to January 2016 arising directly as a result of that exclusion. The drop mostly affects the “long tail” pages that get very few monthly views.

Impact: Here are the pageview counts for pages I have created over my lifetime in the three months of the quarter: 180,352 in November, 196,227 in December, and 172,159 in January. The total across the three months was 548,738. New content I created has gotten under 1000 views per month. The drop from December to January is explained by the data source change. The low pageview count of newly created pages is explained by the focus on maximizing my learning rather than creating pages with a huge amount of interest.

Forecast for future impact: I expect the pageview counts in the next quarter to come out a little lower than the previous quarter, primarily due to the data source change. My estimate range for pages I created is as follows:

  • 2.5th percentile: 275,000 views.
  • 10th percentile: 340,000 views.
  • 25th percentile: 410,000 views.
  • 50th percentile: 495,000 views.
  • 75th percentile: 640,000 views.
  • 90th percentile: 725,000 views.
  • 97.5th percentile: 880,000 views.

The estimates are a little lower than the past because of the bot-filtering, as well as the shorter length of the forthcoming quarter: 89 days as opposed to 91-92 days for most quarters.

The dispersion of the estimates is similar to that for a month. This is because the greater centrality arising from summing up over more data points is balanced out by the greater uncertainty arising from looking farther out into the future.

Subject wikis

For background information, see my site page about the subject wikis

I did active work on the subject wikis for a few segments of a few hours, but not in any systematic way that would affect the overall level of content or change the experience of typical visitors. The subject wikis have been doing well traffic-wise, but this is not due to any additional effort on my part.

Groupprops got a total of 254,100 pageviews this quarter compared to 232,786 for the corresponding quarter last year. The seasonal pattern in traffic was very similar between quarters, so much so that it is visually striking when you look at the graphs in Google Analytics. Market declined (47,429 compared to 58,823) and Calculus stayed stable (56,258 versus 54,466).

WikiHow

For background information, see my site page about WikiHow

I wrote two new WikiHow articles: How to Use Amazon Route 53 and How to Use Git Effectively. The latter of these articles appears to be the fastest of my WikiHow articles in acquiring pageviews, though it’s still early days to know just how far it will rise.

EA Forum blog post making forecasts about GiveWell money moved

In December 2015, I published a blog post in the Effective Altruism Forum with an analysis of the amounts of money moved to GiveWell top charities and forecasts about money moved for 2015, based on the information available at the time. I also published a follow-up blog post and some comments on my own posts. The actual money moved turned out to be at around the 90th percentile of my estimate, as I noted in a later update at the bottom of the post.

Entertainment

I have continued to watch the Korean TV series Remember: War of the Son on Viki, which is finishing up in February. Other Korean TV series I have been watching include Madame Antoine. Thanks to these TV series, I have picked up many words and phrases in Korean, and can generally understand a decent fraction (excluding some nuances) without subtitles if I am also watching the video alongside (so I can see people’s facial expressions and other contextual cues).

I also watched the Bollywood movie Jazbaa, which I later discovered was a remake of a Korean film Seven Days (something that is not quite surprising, given that some of the themes seemed distinctly reminiscent of the Korean TV series I had watched).

I also found a blog, E Drama Clouds by Evelyn Lee, that offers high-quality commentary on Korean TV series.

On the music front, I bought a few MP3 songs and listened to some more. My recent purchases include Hello, Wake Me Up, and Dark Horse.