BetterEvaluation is on the ground at the European Evaluation Society 10th Biannual conference in Helsinki this week. We’ll presenting BetterEvaluation on Thursday but we're also attending lots of interesting sessions, and can share some highlights through this site.
I’ve just attended a session titled “Network effects on evaluation and organisation”, which is very closely related to the overall theme of the conference: Evaluation in the networked society. My hope was to learn more about the evaluation of networks themselves, an area of interest for me. After a quick summary of the session I wanted to add to this discussion by sharing a few resources on this subject, as well as some useful tools (mainly Social Network Analysis software).
The session comprised of three presentations about particular networks, but from very different perspectives: The first (Martina Rillo Otero) was an analysis of a network of evaluators in Brazil, the second (Wolfgang Meyer) talked about the evaluations of two so called ‘governance networks’ in Germany, and the third (Juha Latikka) talked about how network mapping can help decisions about researching funding in Finland – so quite a mixed bag.
The common ground between the presentations was the use of network visualisation to present relational data. The first used the visualisations alongside network analytics such as degree centrality and betweeness centrality to understand the interconnectedness of the evaluation field in Brazil; the second used visualizations to demonstrate how networks are evolving over time (they found that small networks grew while the bigger networks shrank); and the third used visualisations to show how different types of research proposals are related to each other.
For me though, there is a lot more to network analysis and evaluation of networks than visualisation – although this is perhaps the sexy part of it because you end up with pretty pictures which can bring data alive and tell fantastic stories (if used well).
For a start I’d like to see more discussion about what networks are as it’s a very overused term and anyone who uses it should lay out clear boundaries for what they are treating as network and what they are not (e.g. coalitions, partnerships, communities of practice, associations, unions are all forms of networks). This paper provides a very clear functional definition of networks rather than structural – it’s more useful to understand a network by what it does than what it looks like.
In BetterEvaluation we are planning to develop a glossary (or perhaps more appropriately, a Rosetta Stone) of terms useful in evaluation, and we will start by looking at how these different network related terms are used.
The other issues that I think are essential in looking at networks (which I don't want to get into here, but I hope we can get to later) are power dynamics and value creation in networks - particularly using these lenses to analyse network strategies in critical ways.
By way of broadening this discussion, I’d like to share some resources that you might find helpful - please add to this list by suggesting your resources in the comments:
-
Next Generation Network Evaluation, by iScale (2010): a review of approaches to evaluating networks
-
Measuring while you manage: Planning, monitoring and evaluating knowledge networks (2001), by Heather Creech
-
Participation, Relationships and Dynamic Change: New Thinking on Evaluating the Work of International Networks (2002), by Madeline Church et al
-
Evaluating the Effects of International Advocacy Networks (2007), by Ricardo Wilson-Grau
-
Promoting and assessing value creation in communities and networks: a conceptual framework (2011), Etienne Wenger, Beverly Trayner, Maarten de Laat
-
Evaluating Inter-Organizational Relations (2006), Charles Lusthaus, Katrina Rojas and Christine Milton
-
Evaluating Networks, IDRC (2005) – great literature reviews and examples of evaluations
-
Network mapping as a diagnostic tool, Louise Clark
-
Social Network Analysis in Program Evaluation (2005), special edition of the New Directions for Evaluation journal
-
NetMap Toolbox: a great participatory research method based on network mapping.
Rick Davies also keeps a well maintained list of Social Network Analysis resources on his website (mande.co.uk) as well as some useful insights on the use of network models in M&E.
And here are some links to my favorite free Social Network Analysis software packages.
-
UCINET - it was built by academics for academics so it's got all the analytics and fairly easy to use.
-
Gephi - has an easier graphical interface than UCINET and is becoming more popular
-
NodeXL - works with MS Excel which makes it very easy to import data, it's still under development and getting better all the time.
Those are my suggestions but here is a complete list: http://en.wikipedia.org/wiki/Social_network_analysis_software
And if you are really interested in this, then you might like to join me in an online course on SNA that has just started, run by University of Michigan: https://www.coursera.org/course/sna.
Image (top-left): 2010 - February - NodeXL - cscw Twitter Network scaled by followers by Marc_Smith, on Flickr
Image (top-right): A network map of evaluators in Brazil - from the EES session by Cristina Sette
Comments
Re "For a start I’d like to see more discussion about what networks are as it’s a very overused term
In SNA a network is any set of entities or events that are connected by any kind of relationship. In many SNA software packages you should be able to enter data about the entities and the relationships.
Alt: a network is a structure arising from a set of entities/events connected by relationships
This may seem very broad but it is functional
regards, rick davies
Thanks Rick. These definitions are very useful in their simplicity but they are so broad that anything can be described as a network. Do you think there's a need for more specificity or distinctions between different types of network? My sense is that we risk talking at cross purposes unless we have a clearer way of describing these various forms of organization.
You are right in a sense. There is an element of talking at cross-purposes. In SNA my definition applies, then types of networks are described by differences in their structural features, again defined by SNA metrics. But these dont readily match how development workers, for example, would describe networks of various kinds. Though some cross translation could usefully be attempted.
There are many different purposes for being connected (creating a network)...kinship, political, economic, financial, social, interest groups, coalitions, task force, and even the WWW and food systems, etc. I would think there is a need to be clear on nature and objective/goal of the network and, as Rick points out, the structure (density, centrality, reciprocity) helps us understand differences. Based on the nature of the network, the structure may or may not be conducive to its purpose/goal. In a development context, if the structure of the network, low density of connections, is not conducive to its goal (for example an NGO coalition on Child Protection) with then network analysis/metrics provides an understanding to begin weaving the network to achieve its goal/objectives.
Please login or register to comment