This is a series of posts about how software development and programmers have become increasingly reliant on developer communities like Stack Overflow and search engines to help them develop and ship code. We think this is a fundamental shift in the nature of software development that has wide ranging implications.

Part 4 – Causes & Effects of SAP

So the completely fake condition SAP is actually probably real and apparently affects millions of programmers, but what causes it? While researches have been studying social media and the role of web 2.0 in software development for a while, the full impact of this shift in programming practice is really only starting becoming apparent now. In this post we combine speculation and bits of empirical research to try to understand what is causing this shift in programmer practices and the impact it may have on software development.

Increasing Solution Complexity

Programmers are increasingly faced with having to program in multiple languages and across multiple platforms. In the last several years a shift in user experience expectations and the rise of mobile and touch centric hardware has caused a splintering in the set of development technologies required to deploy a product to the widest possible audience. This paradigm shift can be seen most clearly in two parallel trends: the explosion of JavaScript frameworks for the web (jQuery, NodeJS, Bootstrap) and the introduction of rich client mobile applications and their associated marketplaces (iOS, Android, Windows).

The software development landscape is now an acronym soup of competing development platforms with no clear winner in sight. Our CSCW2012 paper on how developers evaluate other developers indicates that developers, companies and recruiters are adapting to this new environment. Whereas once companies looked for employees with deep experience in a particular technology stack, now companies are starting to look for developers that can show a wide range of skills across multiple technology stacks and an ability to learn quickly. For many programmers this confusion of development platforms means being able to develop expertise in a single technology stack is now an unaffordable luxury. Rather programmers are expected (using resources like Stack Overflow) to be able to be instantly conversant in multiple languages and platforms and to be able to integrate with whatever library, component or service that may be required to deliver a solution.

This is speculation, but there is likely an interesting feedback loop here between the amount of indexed programming knowledge available on sites like Stack Overflow and the increase in the complexity and heterogeneity of solutions that programmers are developing. It’s very possible that one trend enables and promotes the other. For example an increasing number of posts on a wide range of programming technologies on Stack Overflow increases the confidence programmers have in their ability to successfully combine more disparate technologies into a functioning solution. This in turn inevitably leads to more questions and answers being posted to Stack Overflow about the individual technologies and how to combine them. Which leads to greater programmer confidence….which leads…, rinse and repeat.

I am not aware of research that looks at how adventurous developers are in taking on complicated multi-language/platform projects the presence or absence of the documentation provided by developer communities. But it would be a good paper.

Poor Official Documentation

Another factor pushing the increasing reliance of developers on search and sites like Stack Overflow is the state of official documentation. The official technical documentation published by platform and language vendors often pales in quality in comparison to the crowd documentation generated by developers contributing to sites like Stack Overflow. There have been several interesting studies done on this topic looking at what kinds of crowd documentation is available and how much depth and coverage it provides vs. the official documentation.

An obvious differentiator between official static documentation and the type of crowd documentation provided by something like the Stack Overflow community is the interactivity and the sheer number of potential contributors. Mamykina et al 2011 show that a question asked on stack overflow has a median answer time of 11 minutes but with the large number of users on Stack Overflow some questions can start receiving answers mere seconds after posting. Very few vendors can ever hope to offer the level of documentation or developer support that can be generated by the Stack Overflow community machine. However, this does not mean that all official documentation is redundant or unnecessary, in fact our study of links posted to Stack Overflow shows that developers do actually frequently refer to official vendor documentation in their posts.

Again I speculate that there is the possibility that we might be witnessing or about to witness another interesting feedback loop where increasing quality and coverage of crowd documentation reduces the impetus for vendors to provide good quality documentation. Alternatively we may see vendors change their documentation and developer support strategies to incorporate features from sites like Stack Overflow or to start officially participating and providing support through these types of sites (this has been happening for quite some time at the individual level but not sure if its official policy for many vendors yet).

The search engine as the World’s Fastest Debugger

Ok so think about this scenario, you are a programmer writing some code on a particular platform and you encounter a problem. What is the probability that you are the first programmer in the world to ever encounter that error/bug? It happens yes, but as long as you are not working in something very exotic, I think you will agree that the likelihood is pretty low. Now add to this scenario a thousands of communities like Stack Overflow where millions of other programmers are continually posting questions and answers about these types of errors. Finally index it and make the whole thing easily accessible through powerful search engines.

Congratulations you have just created the world’s fastest and smartest distributed debugger. (Note I can’t remember if I came up with the search engine as a debugger or if I am sealing it from someone else, if I am let me know and I will cite you)

Now I am not suggesting that the act of debugging is replaced by having access to Stack Overflow and excellent search engines (after all someone has to post the first answer), rather I am talking about the effects of this stored debugging knowledge is having on the process of debugging. Why would a developer spend time self-debugging when they can quickly access the results of another developers debugging efforts? I know I do it all the time. When I encounter an error, where the solution isn’t immediately obvious, I do a quick search. The cost to doing the search is so low and the potential benefit so large that it’s a no-brainer.

I like to think of the search engine as a debugger concept as an important (but overlooked) part of the trend towards reducing the edit-compile-run-debug cycle that is occurring across all parts of the programming world. The search engine as a debugger gives programmers more immediate feedback on how to solve their programming problems, hopefully allowing them to solve the problem and move on more quickly. And if you find that you actually are the first one to encounter and solve the problem you can always post the answer to Stack Overflow and get some nice karma!

More importantly though I think the search engine as the debugger is a symptom not just of a simple one dimensional change in programmer practices but rather a wholesale shift in what it means to be a programmer. Our next post asks the question, is it time to refactor our idea of the programmer?