This is a series of posts about how software development and programmers have become increasingly reliant on developer communities like Stack Overflow and search engines to help them develop and ship code. We think this is a fundamental shift in the nature of software development that has wide ranging implications.

Part 5 – Refactoring the Programmer

How programmers develop software is changing rapidly. Software developer communities and powerful search engines are becoming an increasingly integral (and often unacknowledged) part of the software development process. Driven by increasing solution complexity and the proliferation of frameworks, libraries and platforms required to deliver a single solution, programmers (both novice and expert) frequently find themselves faced with having to integrate and debug multiple heterogeneous pieces of code but without the time to develop even a modest understanding of those components. Search engines and developer communities like Stack Overflow provide a lifeline to the programmer allowing them to mine the stored collective intelligence of vast communities of fellow software developers. However, the reliance of programmers on these communities raises some interesting questions for software companies, software engineering researchers and educators alike.

Since coming to work with Chisel, I and others like Margaret-Anne Storey, Christoph Treude and Leif Singer have attempted to capture the impact that large developer knowledge stores like Stack Overflow (indexed and made accessible through search engines) are having on programmers and software development in general. The Social Programmer is a concept that attempts to sum up the new reality faced by many programmers and asks the question do we need to rethink our concept of what a programmer is and what they do? I this post we ask do we need to refactor the programmer?

(Note the following sections are based on our FCSD 2011 paper where we introduced the concept of the social programmer)

The Evolution of the Social Programmer

The emergence of Stack Overflow as a repository of programming knowledge is the latest evolution of a historical trend with programmers inventing or adopting a “social” technology to meet their need to discuss what it is they do with other programmers. However, with sites such as Stack Overflow indexed and made available through search engines, are we approaching the point where the archive of stored programming knowledge reaches a critical mass and where new programming practices and behaviors will emerge? Have we already reached that point and what kinds of impact might we see on programmer practices and the software development industry as a result?

We think the programmer has already been refactored, it’s happening day by day largely invisibly but evidenced in the ever increasing contributions by programmers to communities like Stack Overflow and GitHub and by the proliferation of programming languages, libraries and platforms. Together these online services for programmers (infrastructure), the programmers that participate and a shared philosophy make up the social programmer ecosystem. We are currently in the process of charting this eco-system.

What Makes a Good Programmer?

For a programmer in the social programmer ecosystem, where a large percentage of programming knowledge is archived and curated by millions of “experts”, do we have to redefine the attributes of a good programmer? Is the metric of a good programmer someone with a deep understanding of programming and software engineering principles, or someone who can leverage and synthesize the programming community to achieve the same results? When you are very unlikely to be the first person in the world to encounter a particular problem, does a smart programmer attempt to diagnose the problem independently or just ask the community? Will an entirely new category of programmers emerge, a Just in Time Programmer, without formal training but with the ability to combine snippets to craft solutions that will just meet their needs? What tools will these new programmers require?

Software Development as a Massively Distributed Activity

If future programmers will require the ability to synthesize a single solution from the contributions of the many, it raises interesting questions (both positive and negative) about the nature of the software that those programmers produce. Give an environment where practically all software is developed with reliance on a shared knowledge base and community, who actually owns the intellectual property? Is there a risk that programmers do not really understand how their software works? Or will it in fact lead to better efficiency by reducing time spent fixing bugs or re-inventing the wheel? Could the distributed development approach to programming actually increase quality by promoting best practice solutions? Is this a realization of software componentization, different from the vision of component based development perhaps but effectively a similar result? What organizational changes will this shift entail in terms of social offshoring and the ad hoc creation of teams?

Conclusion

Ultimately it’s too early to make a judgment on the social programmer and the impact the phenomenon will have on software development practice. We are already starting to see changes in in the skills companies and recruiters are looking for in programmers. At the moment this appears to be confined to a distinct subset of programmers but it will be interesting to see if it spreads and how far. There will also always be programmers who find it difficult to fully participate in these communities because of intellectual property or security restrictions for example. How we bring the benefits of the social programmer ecosystem to these programmers is an interesting challenge.

A final though. An argument frequently raised when discussing these topics is quality. What is this trend going to do to software quality? Are programmers going to become copy and paste programmers, findings snippets of code on the web and then pulling them together to craft a solution without any understanding of how it works. Frankly I’m sure this is already happening but it’s not fully clear if this is; a new trend, can be stopped or is an entirely bad thing. We will probably see the emergence of a group of programmers that lack foundational computer science and software engineering concepts, but this is not necessarily new, many practicing programmers were not trained as programmers. The potential benefit of the social programmer ecosystem is that these programmers can interact with and learn from communities of knowledgeable programmers where expertise is valued and where best practices can be disseminated.

Well, we hope