A possibly unpopular message
Tue, 9th Oct 2012 / 17:07
Social media analysis has outgrown its marketing and advertising roots. Now it must look beyond these industries and learn from other social and academic disciplines. Carl Miller (@carljackmiller), Research Director, Centre for the Analysis of Social Media, Demos, explains.
The academic community has much to thank the marketing and advertising (both B2B and B2C) industries for. Quick and agile, in the face of a keenly felt opportunity, it was this industry who first understood that there was an unprecedented opportunity to listen and learn about people and society through understanding the explosion of digital social spaces. It was here also that ‘big data’ technology and analytics were first used as a new, exciting, even revolutionary way to do this. The industry built up years of background and experience using social media analytics (SMA) to understand people before the glacial drift of academia started to move in the same direction with the same, shared, if belated, sense of opportunity and excitement.
But now it has. And the (possibly unpopular) message of this post is that SMA’s big next step is to grow beyond its roots in the marketing and advertising industries, and draw also upon other disciplines and specialisms. Fundamentally, and as a field in general, social media analytics is not currently making our decisions as smart as it ought. Whilst the specific concerns might differ – driving brand awareness, shaping products, generating leads, cultivating traffic or (as is mine) learning about people to inform policy making – the united and overarching aim is generate information to drive decision-making, and good decisions about any of these topics are based on a certain type of information: information that is valid, explanatorily powerful and ethically sound.
First, the information must be valid. The most important and meaningful criterion of any social research, at its heart it is a measure of the integrity of the conclusions that the research produces. Validity has many different aspects, but most crucial is external validity – the concern with the generalisability of the conclusions beyond the specific context of the analysis, and for this a representative sample is key. Samples drawn with equal and random chance from the whole population you wish your conclusions to refer to – the golden standard – avoids bias, systemic error and allows important external validity measurements such as statistical confidence. In social media analytics, however, unprecedentedly giant samples often obscure poor sample quality. In the Reading the Riots project, analysis of 2.5m tweets implied that “social media was used far more by those seeking to follow – or avoid – events than to incite trouble”. But are these 2.5million tweets representative of the whole of twitter? The tweets were given (by Twitter) to the project team because they were members of the hundred or so most popular riot-related #tags. It’s intuitively striking that people inciting trouble probably had an interest in not attaching their tweets to these high-volume #tags, but at the very least this method doesn’t convincingly deal with the possibility that this, or another systemic bias, is lurking in the data.
Second, the results must be powerful. Especially if you’re seeking to change behaviour, you want to understand not only what your clients are doing, but also why they are doing it. This ‘why’ has itself two important aspects: causation, and explanation, and deriving either of these kinds of meaning from data is far from a straightforward reading of the ‘obvious’ implications of the data. Broadly, SMA has yet to move beyond the construction of descriptive, raw metrics to a serious, statistical treatment of causal significance. Most examples that do, rely on correlations and comparisons that only weakly suggestion causation. For instance, this piece of research, offering to tell you how you can get more likes, comments and shares, correlates share/comment/like performance of a million-odd facebook posts with attributes such as post length, sentiment, self-referential words, time, and day of the week. There is nothing in the research that establishes that people share, comment or like because of any of these attributes however. Any conclusion resting on this assumption (as these are) are shot full of holes. Moving from causation to explanation is equally tough. Humans are complex, and considerations of culture, context, group, language and psychology must be taken into account, as must the full range of theories established by sociology, psychology, and the other humanities that claim insight into why people do the things they do.
Thirdly, SMA must be ethically sound. It is important that SMA users know that the data is being collected and used in a way that doesn’t compromise their organisation’s own values and principles, won’t drive their clients away, and won’t harm their reputation. At its best, social media analytics is an ethically considered application of non-intrusive, open-source research. At its worst, it is the surveillance of people in ways that they would never agree to if they knew it was happening. There is precious little legal or regulatory guidance to fall back on to tell the difference, but too often social media analytics efforts err on the side of abandon, and take the lack of guidance to be an invitation for whatever maximalist interpretation of what is permissible they judge to be most convenient. This is wrong, and one day it is going to catch up with them. Yet, sociology and its allied disciplines are concerned with identifying and limiting the harms entailed through researching – anonymously or not – human beings. These long-considered frameworks and principles must shape and inform social media research ethics as it continues to become more popular and powerful.
In each of these three ways SMA has something to learn from the academic disciplines that are concerned with learning about human through evidencing and interpreting their behaviour. In fact, it needs to make the leap from the metrics-based traditions of its origins into a full-blown applied academic inter-discipline. The think tank Demos has launched The Centre for the Analysis of Social Media to do exactly this for the practical purpose of informing policy making. Bringing together policy specialists, social scientists, computer scientists and tech entrepreneurs, it aims to contribute to the next important and exciting evolution of this discipline. We’re not yet sure of the results, only that more needs to be done.