Summary Innometrics
Lecture 1. Measuring Innovation
Why measure innovation?
Innovation is key to the addressing societal challenges, growth of output and productivity.
Innovation policy and management are increasingly evidence-based. (feiten gebaseerd)
Innovation data needed to better understand innovation and its relation to economic
growth/societal challenges; to provide indicators for benchmarking
national/regional/sectoral performance.
Overheden en bedrijven moeten begrijpen wat er aan de hand is om te handelen en adequaat in te
grijpen. Daarom is het interessant om de dynamiek van innovatie te bestuderen.
Meta Model of Innovation
The starting point in this course is provided by knowledge dynamics. Additional dynamics are related
to the geography and economy of novelty production
“Economics is a science of thinking in terms of models joined to the art of choosing models which are
relevant…” (Keynes, 1938)
What is a model on innovation?
A model of innovation is a conceptual framework developed for understanding the process of
translating an idea into a good or a service that creates value.
A model of innovation can be seen as a core concept that expresses how change is produced in
society.
Dominant models of innovation results from the efforts of diverse disciplines, including economics,
management science, sociology, geography and political science, that come together to explain the
production of novelty in society.
When used appropriately, models serve society extremely well.
Why models?
Models are key to the “scientific” nature of economics (and other social sciences)
Understand complex social reality by laying bare a very large variety of causal relationships, one (a
few) at a time
Economics/Social sciences advances not by settling on “the model,” but by generating useful
collection of models
An inventory of partial explanations
Non-universality and context-specificity
,Scientific research and big data
Big Data promises to revolutionise the production of knowledge by enabling novel, highly efficient
ways to plan, conduct (uitvoeren), disseminate (verspreiden) and assess research.
The last few decades have witnessed the creation of novel ways to produce, store, and analyse data,
culminating in the emergence of the field of data science, which brings together computational,
algorithmic, statistical and mathematical techniques towards extrapolating knowledge from big data.
Big data, Big issues
Rather than being dependent on surveys and interviews, the traditional data sources of empirical
social science, digital data present us with the opportunity to scrape, generate, analyse and archive
comparative data of unprecedented quantity.
Locating crime spots, or the next outbreak of a contagious disease, Big Data promises benefits for
science, society as well as business.
But more means messier. Do policy-makers and managers (and researchers!) know how to use this
scale of datadriven decision-making in an effective way? -> Hoe deze gegevens doeltreffend
gebruiken?
But big data holds threats for social scientists too.
The technological challenge is ever present. To generate their own big data, researchers and students
must learn to code!
Big data
We are witnessing a progressive “datafication” of social life. Human activities and interactions with the
environment are being monitored and recorded with increasing effectiveness, generating an enormous digital
footprint.
The story is the same in one field after another, in science, politics, crime prevention, public health,
sports and industries as varied as energy, news and advertising.
All are being transformed by data-driven discovery and decision-making.
Traditionally qualitative jobs are becoming data driven: journalism, literature studies, policy makers…
Anderson claims the end of models and theory in WIRED (bekabeld): The Data Deluge Makes the
Scientific Method Obsolete -> de stortvloed aan gegevens maakt de wetenschappelijke methode
overbodig
The emerging emphasis on big data signals the rise of a data-centric approach to research; data are
viewed as central to discovery.
The emergence of data-centrism highlights the challenges involved in gathering, classifying and
interpreting data, and the concepts, technologies and institutions that surround these processes.
In practice, however, access to data is fraught with conceptual, technical, legal and ethical
implications; and even when access can be granted, it does not guarantee that the data can be
fruitfully used to spur further research.
Furthermore, the mathematical and computational tools developed to analyse big data are often
opaque (ondoorzichtig) in their functioning and assumptions, leading to results whose scientific
meaning and credibility may be difficult to assess. This increases the worry that big data science may
be grounded upon, and ultimately supporting, the process of making human ingenuity hostage to an
alien, artificial and ultimately unintelligible intelligence.
,Victor Mayer-Schönberger on Big data: Traditionally, researchers and analysts relied on a sample to do their
analysis. Now, we have the technology to gather and analyze much more data – is some cases even ALL the
data about a phenomenon. (eerst werden alleen steekproeven gedaan, door big data is het mogelijk om alle
data te gebruiken)
As the scale of information increases, so does the number of inaccuracies (onnauwkeurigheden). In a
sample, it is important that the figures are as correct as possible.
With big data, Mayer-Schönberger argues that the amount of data gives us a more valuable output,
even though more errors may occur.
A tendency to move from causality to correlations is the third characteristic described by Mayer-
Schönberger. New data-mining techniques can give us information about what is happening, without
explaining why. (what, not why)
1. Science, technology and innovation indicators: The dramatic growth over the last 20 years in the use of
science, technology and innovation (STI) models and indicators is the result of a combination of the ease of
computerized access to an increasing number of measures of STI and (’big data’), on the other hand, the
interest in a growing number of public policy and private business circles in such models and measurements.
In this introduction, key concepts such as model, indicator, hypothesis, data and statistics will be discussed.
2. Patents:
Patents provide a wealth of information to measure innovative developments. Like scientometric
data, patent data are not without shortcomings.
The conceptual and methodological problems of ‘measuring’ technology are discussed, with a
classification of the types of information which can be drawn from patent databases of both
innovations and the innovative efforts of firms and countries.
3. Scientometrics: Scientometrics is the science of measuring and analysing science. Modern scientometrics is
mostly based on the work of Derek J. de Solla Price and Eugene Garfield. The latter founded the Institute for
Scientific Information (ISI) which is still heavily used for scientometric analysis.
4. Social Network analysis: Innovation is the result of the interaction among an network of institutions in the
public and private sectors whose activities and interactions initiate, import, modify and diffuse new
technologies, and the term ‘innovation system’ is used to emphasize this.
5. Evolutionary models: Evolutionary models in science and innovation have focused mostly on the issue of
changes in technology and routines. If the change occurs constantly in the economy, then some kind of
evolutionary process must be in act, and there has been a proposal that this process is Darwinian in nature.
, Lecture 2. Science & technology indicators
Science and technology (S&T) indicators are widely used in policy documents, firm strategies as well as in
science, technology and innovation studies.
STI indicators: An “indicator” is a set of facts or observations that tells us something meaningful about the
underlying phenomenon of interest, in this case science, technology and innovation (henceforth STI).
Use of STI indicators
In addition to understanding the data underlying an indicator, how those data were processed, and the
relationship of the data and its processing to a framework for analysis of the STI system, evaluating indicators
also requires an understanding of the purposes for which they are used.
For example, past research has suggested that all else equal, a greater intensity of investment in new
knowledge will lead to higher rates of productivity growth and income growth. For this reason, one might focus
on the R&D/GDP ratio over time or across countries as a benchmark of innovative activity.
Performance assessment and benchmarking: Some indicators serve as performance measures that
give an assessment of whether the STI system or some component thereof is doing better or worse
over time, and better or worse than some comparison group (e.g. other countries).
Informing public policy decisions: An important function of STI indicators is to provide an informed
basis for public policy decisions. But of course, the policy is not intended to affect the indicators; it is
intended to affect the underlying concepts of interest.
Informing private sector decision-making: Firms and individuals in the for-profit sector also use STI
indicators to make business decisions, and not-for-profit organizations (e.g., universities) use them to
make decisions in pursuit of their missions.
Facilitating social science research: As noted above, social scientists use data to test the implications
of models and thereby refine the models. Hence their interest in the indicators is more in the
collection and availability of data than in indicators per se.
Dramatic Growth
The dramatic growth in the use of STI indicators is the result of the interaction between supply and
demand.
o Supply: ICT
o Demand: Competition at the level of nations, industries, firms and individuals
Like any other statistics, indicators of science, technology and innovation (STI) can be both used and
abused. (gebruikt en misbruikt)
o From straightforward ignorance of the sources, definitions and methods (door onwetenheid)
o from a sort of STI version of Goodhart’s law: once STI indicators are made targets for STI
policy, such indicators lose most of the information content.
Up to a level, the evolution of STI indicators can be read as a story of searching in progressively
‘darker’ places as our ‘key-detection’ antennae become more finely attuned over time. (dronkaard die
zoekt naar zijn sleutel bij de lantaarnpaal omdat daar het licht is)
