Summary Innometrics→ everything together
LECTURE 1
Introduction to Innometrics
What is it? Quantitative analysis of Science, Technology and Innovation (STI) →
how can we analyze data that tells us something about STI, we only look at the
creation of innovation instead of the use of it
- measure dynamics of novelty and knowledge creation
- using appropriate data, metrics and indicators
- at different levels of aggregation (organisations but also countries and regions)
- considering relevant theories and “models” of innovation → the data itself
is not enough
Science - systematic study to gain more/new knowledge
Innovation - novel idea used for tackling a problem (products, new practices…)
Diffusion - you have to have people that use the innovation
Cooking recipe - codified knowledge
Tacit - learning through practice, hard to communicate, hard to codify
Why measure knowledge dynamics of innovation? Innovation is key to the
addressing of societal challenges, technological process and productivity growth
→ data needed to better understand developments in science, technology and
innovation and their relation the economy and society at large
Policy decisions are increasingly evidence-based - metrics inform budget and
funding allocation, research priorities, or the design own new ethical and
regulatory frameworks → company managers use data-driven input for better
decision-making to monitoring markets and competitors, and predict future trend
Evaluating and benchmarking national/regional/industry/firm performances based on “good”
indicators
So because of the fact that governments and companies need to understand what is going
on to act and intervene appropriately
Why is it interesting? Knowledge production is changing.
Growing body of knowledge → patents, scientific publications
Bigger teams in science and invention → higher degree of collaboration, higher
distances
Higher degrees of specialization of people → scientists and inventors are less
likely to “jump fields”: more specialists than generalists
Increasing diversity and changing direction in research due to societal changes →
STI and gender: increase in female-focused research, driven by female-focused
teams, STI and local problems: link between local issues and local research
Ongoing global trends: open source
Some evidence for the burden of knowledge (the lengthening of knowledge slows down
innovation)
- the age at which people achieve certain innovation milestones is rising
- more and more innovation is conducted by larger and larger teams
, - people are specializing more and more
Theoretical perspectives for Innometrics → (evolutionary) economics
(evolutionary models of technological change and innovation), system of
innovation approaches (networks and inter-organisational learning) and
territorial innovation approaches (the role of location and clusters)
How do we study innovation?
Levels of analysis → micro - meso - macro
Data-driven - structured data (publication, patents, innovation statistics) & unstructured data
(text)
Translate innovation theory into indicators → explorative and interpretative,
bridge theory and methods
LECTURE 2
Measuring innovation: models and indicators
Innovation involves the process of and outcome from the exploitation of knowledge creation.
It involves the translation of new knowledge and ideas into new products, production and
business processes.
Implementation - newness → new to… world, country,
industry/film/sector/household → value creation
Measuring innovation is difficult because.. Innovation is change (change is happening - how
to quantify?), innovation is value creation (which value? of value for whom and when?),
innovation is newness/uniqueness (understanding which kind?)
“Models” of innovation
→ a conceptual framework developed for understanding the process of
translating an idea into a good or a service that creates value
A model can be seen as a “conceptualization”, a “perspective”, a “narrative”, “figure” or “tool”
that expresses how change is produced in society
Why? To reduce and focus on a perspective (but be aware of the limitations)
- provide an inventory of partial explanations → the goal is not universality
but context-specificity
- understand complex social reality by laying bare a very large variety of (causal)
relationships, one or a few at a time
- models are key to the scientific nature of discipline, also in the social
sciences → travels easily between scholars and between the latter and
policy makers. Calling a narrative or tool “model” facilitates its
propagation.
Simple/linear models
- from basic research to innovation
- a simple knowledge production function
Invention - innovation - diffusion
Basic research gives rise to applied research and technological development, which in turn
generates innovation
,Public good: Governments must therefore invest where industry does not →
market failure as argument
Input → black box → output
An input-output relationship in which inputs are directed at research activities, which produce
results (output) and, ultimately, impacts (economic productivity, growth)
You don’t know what is in the black box
Criticism:
- it is a loew information model: no clear link between sciences/R&D inputs and
innovation output
- Emerging policy implications not very helpful!
- it is an idealized model
Systemic approaches
- territorial innovation approaches: NIS, RIS, …
- sectoral approaches: TIS, SIS, …
- Triple Helix and quadruple helix
Other relevant conceptualisations of knowledge production
- modes of knowledge production: mode1 vs. mode2
- modes of innovation: scientific and technologically-based innovation (STI) vs.
innovating based on learning-by-doing, by-using, and by-interacting (DUI)
In the past, the focus has been on linear models. Nowadays, innovation is linked to
contemporary social and environmental challenges, calling for societal transformation.
System perspective developed in the 90s
Knowledge and information are regarded the primary sources of value and economic growth
Turning from industrialized economies based on labor, tangible capital and material
resources into economies based on the creation and exploitation of knowledge
The emergence of innovation system approach was also a way to open the black box
Innovation systems and indicators
Innovation system thinking has formed the basis for the development of a variety of
innovation indicators, and the application of benchmarking tools and dashboards
to measure, evaluate and compare system performance
can detect problems in the innovation system
real world (phenomenon under study) → conceptual world (theoretical
definitions, concepts) → methodological world (indicators, data, analysis)
Construct validity: is the concept measured the right way?
What is an indicator?
General: an indicator is a set of factors or observations that tells us something meaningful
about the underlying phenomenon of interest. An indicator provides information in a
simplified and often aggregated form, which facilitates a focused evaluation.
Practical: a statistical summary measure of an…
, Indicators are a ‘technology” and technologies change behavior. I.e. indicators change the
behavior of those who work with it (producers and users).
Innovation indicator producers are usually inter-/supra-national organizations with the
aim to develop a common language and standard as codified in various innovation manuals
for data collection, interpretation and use
Users are various organizations in society who may change their behavior and/or
may provide a motivation for revision of the indicators
Indicators are a means of governance: use of indicators to “steer” individuals or
organizations. Example innovation rankings, benchmarking classifications
→ danger of endogenous distortion of indicators and measurements (publishing
also “bad” articles for example if the indicator is publications)
Data quality issues: extent, reliability, validity
Data extent: scale and scope of the data: is the sample big enough, do we have access to
the data that we need
Data reliability: reproducibility of data collected and results: what happens if the study is
repeated? Same or different results?
Data validity: relevance and representativeness: how closely does data correspond with the
underlying concept? Is the data relevant for measuring what we wanted?
Why is innovation so difficult to measure? Subjective by itself, difficult to know
the casualties → we assume a lot
Indicators developers tend to concentrate (first) on developing indicators of those things that
easiest to measure, which may not be the variables most pertinent to STI policy or
management
However, the evaluation of STI indicators can be read as a story of searching in
progressively “darker” places as our “key-detection” antennae become more finely attuned
over time
We have certain biases: see slides!
L2 READINGS
Hall BH, Jaffe AB (2018) What is an Indicator? In: Hall and Jaffe (eds). Measuring
Science, Technology, and Innovation: A Review. Chapter 1, pp. 2-12
