Recently, I stopped publishing quarterly reviews. In this blog post, I explain the reasons, and describe what I’m doing instead.

Quarterly reviews previously served to provide periodic updates of three things:

1. My recent activity.
2. Total pageviews and other reach and impact metrics of my recent and lifetime activity.
3. Updates to my beliefs and habits.

Unfortunately, they weren’t an ideal vehicle for any of the three. Among the problems:

• The updates were too infrequent to provide insight into what I’ve worked on and where, and to allow others to track my work.
• There was too much number-crunching involved in collating data from different sources.
• The quarterly period was an unnatural way of subdividing my work and reporting incremental progress, and therefore created extra load on me in terms of efforts to summarize progress.
• The things that people liked most (my belief and habit updates) received too little attention, as I (rightly) believed in the importance of reporting objectively on activity and reach/impact measures.

My solution going forward is as follows:

1. My recent activity: With a few stated exceptions (as described below), all my activity can be tracked by following me on GitHub. I explain this in more depth below. This is near-real-time, and provides a lot more granularity and insight.
2. Reach and impact metrics: I am working to make these available in publicly accessible locations, updated at approximately daily frequency, so that people can check them out any time.
3. Updates to beliefs and habits: I will publish these occasionally, at logically natural points, just as I occasionally publish blog posts or articles summarizing work I’ve been doing or key insights I’ve had. I don’t try to pressure myself to do these; I write them quickly when the ideas are in my head.

Note: As I hint at right above, I try to write these “updates to beliefs and habits” posts quickly, without thinking through them too much. Otherwise they wouldn’t pass a cost-benefit analysis for me. So I basically wrote the current post in one straight sitting of about three hours (which included a bunch of other website updates that were triggered by this draft). That might explain why this doesn’t look like it has undergone editorial review and improvement.

#1: Following my activity

My activity is of roughly three kinds:

• Self-contained projects, that usually involve a mix of code (to render results, allow for programmatic querying) and data. Each such project has its own GitHub repository, as well as a public-facing website. Currently, there are three projects I am working on: contractwork (website: contractwork.vipulnaik.com), donations (website: donations.vipulnaik.com), and Wikipedia Views (website: wikipediaviews.org). I also have a project on BART (intended website: bart.vipulnaik.com) that does not yet have a public website, but some of the backend code is written.
• Miscellaneous written content I create or improve, in the form of wiki pages, blog posts, articles, Facebook posts, and comments. Some of it is related to a larger project of understanding a specific aspect of the world, but it isn’t a self-contained project. This content is scattered across numerous venues. To name a few: my blog (the one you are reading right now), the Timelines wiki, wikiHow, Market subwiki, the Effective Altruism Forum, LessWrong. I generally keep a copy of the content I create (or, in case of significant updates, the content I update) in the working-drafts repository mirrored on GitHub. In case of updates, I usually keep a copy of the entire content, though if my updates are modular enough, I might only keep a copy of my updates. I also use this repository to maintain copies of drafts for content not yet published, or background material I want to link people to without “formally” publishing it as a finished piece of work. In some cases, I also keep copies of comments I leave on blog posts, Facebook posts, Facebook comments, survey responses, and (drafts of) emails I send to people.
• Other forms of activity: The bulk of my activity on Facebook, and almost all of my activity on PredictionBook, is not mirrored in working-drafts. To keep track of these, you’d need to follow me on Facebook and check me out on PredictionBook.

The upshot: following all my public activity on GitHub (for instance, by cloning all the GitHub repositories and pulling updates), as well as on Facebook and PredictionBook, should give a reasonably comprehensive picture of what I’ve been up to and how I work. Note that these are not comprehensive ways of seeing all work I’ve ever done; I’ve only started mirroring my edits/updates in the working-drafts folder in the last twelve months or so.

I have included more detail on the first two bullet points on my website page about GitHub.

EDIT: Starting in October 2018, I started using daily-updates issues to provide more information on my activities at daily granularity (outside my day job and daily sustenance activities). See my post on the why and how of daily updates for more.

#2: Keeping track of total pageviews, reach and impact metrics

For my Wikipedia content, you can keep track of it through Wikipedia Views, as described on my site page about Wikipedia. Specifically, here’s the links to get data for pages I created:

• by month
• by year

For websites that I own or manage, I am trying to make the website data public in three ways:

• Quantcast Measure (QM), with details on pageviews, visits, uniques, and demographic and geographic distribution of visitors: QM has a limit of ten sites, and does not show sites with too little data, so I am restricting its use to relatively high-traffic sites. Currently it applies to the five top subject wikis (you can see the list of subject wikis with links to their Quantcast pages, at subwiki.org). In addition, I’ve also activated QM for openborders.info. QM numbers are usually a little lower than those of Google Analytics, but in the same ballpark, and they help capture the relative trends reasonably well. The main downside is that QM works only from the point in time that it was turned on. I turned QM on for openborders.info, calculus.subwiki.org, and market.subwiki.org on November 27, 2015, and for groupprops.subwiki.org, topospaces.subwiki.org, and mech.subwiki.org around April 2, 2017.
• SimilarWeb: I am trying to connect SimilarWeb to Google Analytics, so that it uses certified metrics from Google Analytics when displaying data. I have submitted information to SimilarWeb; as of the time of writing this (April 8, 2017) it does not seem to have started incorporating Google Analytics data.
• Using the Google Analytics API to make key metrics from Google Analytics publicly visible: I expect to do this with some help from Issa Rice some time in the next few months. This will give everybody access to key metrics from Google Analytics that are currently visible only to people I’ve given access (we probably won’t make all metrics available, to maintain simplicity and preserve privacy). EDIT: This website is live at analytics.vipulnaik.com.

Beyond this, I will also update subwiki.org annually with total Google Analytics pageview counts for subject wikis, as I have been doing.

I have not worked out a suitable solution yet for websites that I do not own or manage, although once we have the setup to make Google Analytics data publicly available, I can apply that to websites I do not own but have (or can get) Google Analytics access for.

#3: Updates to my beliefs and habits

This is the aspect of my quarterly reviews and other writings that has attracted the most positive feedback. Freed of the need to publish quarterly reviews (given the superior alternatives that now exist), I can devote more time to writing up updates in my thoughts. Broadly, there are two kinds of posts I expect to write, and both will be fairly occasional:

• “Wrapper” posts that provide a summary of a project I am working on, and of key findings. These wrapper posts could be to advertise the project, to checkpoint an accomplishment milestone (because the project per se never ends), and to convey key conclusions that might be of interest to people who aren’t interested in the project per se. The “wrapper” jargon is borrowed from distill.pub, which says “Non-traditional contributions often don’t get credit unless authors wrap them with proxy papers. Unfortunately, this multiplies effort and divides attention.”
• General posts about my thought processes, beliefs about the world, ways of working, etc. that are one level “meta” and are not tied to any project I am working on. My three guiding principles and Debugging my apparent 2016 stagnation blog posts are examples.

My goal with both these kinds of posts is to, essentially, write them only once the ideas are all in my head and reasonably clear, so that it’s just a straight exercise of transcribing from my head to the computer. Given that I have no particular pressure to “publish”, I believe it does not make much sense to artificially try to put in custom, “hacky” effort to push out posts of either of the above kinds in a way that interrupts the flow of the larger projects I am working on.

To give an idea of what I used to do, and am now choosing against: there is this strategy where I would decide to work hard on pushing out a wrapper post about a topic I’ve been learning about, even if I didn’t feel like I was fully ready to write it, or knew all the relevant facts. Examples of the kinds of topics I am talking about: understanding trends in Wikipedia pageviews, or understanding the history of immigration enforcement in the United States since 1986. My past strategy was: I would just draft it, pull in a fact from here and a fact from here, revisit, redraft, rewrite, and soon get something that looked okay. And there was a time, early on, when I found that this kind of effort helped me focus and collate information that I would not otherwise have interest in systematically grasping. And I still respect this approach.

However, my current belief is that in this sort of situation, it’s better to just keep collating background information in accessible formats. such as continued work on the Wikipedia Views website, to make it easier and easier to look up Wikipedia view trends, or work on timeline of immigration enforcement in the United States. And then to start working on the wrapper post only when I feel I have enough to say that I can just sit down and say it.

(To be honest, I’ve always followed my current approach in some areas — including group theory, my Ph.D. subject — which also incidentally led to me spending too much time on general learning and too little on cracking Ph.D.-worthy problems. But I’ve tended to give a lot of weight to my approach of “push out at all costs” in some domains, and I am now giving it less weight. I still think that approach can be quite good when I am just starting out in an area, and don’t have the general landscape knowledge and wherewithal to embark on the right sort of larger, more long-term projects.)

There are times when I feel like I can’t do more background work on a project (i.e., I don’t know how to proceed next), but I don’t have a wrapper post in my head either. In that case, I might start an exercise of faux-writing the wrapper post: just starting to draft it, and notice the pain points that are stopping me from writing it. And then use those to decide where next to continue to make progress with the project. The wrapper post thus serves as a diagnostic rather than a goal.

Now moving to general posts about my thought processes, i.e., the ones that do not correspond to any specific project, and are one level meta. I generally feel the need to write these when there is a huge dissonance between things I have said or believed in the past, and the way I am behaving right now. But even then, I don’t actually write them unless I feel I have some sort of explanation, or at minimum decent hypotheses, that I can quickly articulate. For instance, the current post (that you are reading right now) was born of a dissonance between my past intent to publish quarterly reviews and frequent wrapper posts, and the current reality of no quarterly review. I had been experiencing the dissonance for a while but did not write the post until now, because it was only now that I felt I could articulate things well enough. Unlike many people in the rationality community whom I know (and know of), I don’t engage in goal-setting and introspection exercises of my own volition, but rather, I wait for things to boil over before I decide to write about it.

Concluding thoughts on transparency and ease of understanding

I want others, including my future self, to be able to easily figure out what I am doing, where it is headed, and what impact it is having. At the same time, I want this “transparency” to be low-friction and effortless for me. I have moved away from an approach of periodically summarizing things, to one of continuously revealing information as part of my modus operandi (and therefore requiring no special effort).

Is this style of transparency actually more helpful to others? Yes and no. Because there is more information available in real time, in a more “computable” format (commit histories on GitHub, public website data at daily granularity), people can dig in much deeper than having to wait for a quarterly review. On the other hand, a quarterly review provides an important function of summarization, making it easier for a casual follower to get the big picture (as sumarized by me).

In the longer run (and I thank Issa Rice for this insightful observation), I expect that the disadvantages of lack of hand-crafted summarization will be overcome by the advantages of more granular, computable, real time data. That’s because more tools will become available for analysis and summarization of activity. These include some I might create myself, through code improvements in the projects, and general ones such as GitHub’s activity graphs.