From Positivism to Critical Realism: Philosophical Paradigms and Knowledge Creation in the Twenty-First Century; A conceptual Review

Full text

Text extracted from the article PDF for reading and search. Tables, figures and equations are in the PDF, which remains the authoritative version.

1. Introduction

Questions about what can be known, how knowledge claims are justified and whose accounts are accepted as authoritative remain central to research. These questions are not merely technical. They depend on assumptions about reality (ontology), the relationship between the knower and what is known (epistemology), the procedures used to investigate a question (methodology), and the influence of values on inquiry (axiology). Research methods therefore do not operate in a philosophical vacuum. Decisions to measure, interpret, compare or explain phenomena reflect assumptions about what counts as evidence and what constitutes a satisfactory answer (Crotty, 2020; Guba & Lincoln, 1994) [9, 14].

Positivism has been influential in establishing systematic observation, measurement and the search for regularities as central features of scientific inquiry. Yet research on human action and complex social systems often requires attention to meaning, context, institutions and processes that cannot be adequately represented by observable associations alone. Interpretivist, constructionist and pragmatic approaches developed distinctive responses to these concerns, while critical realism sought to defend the existence of a reality independent of human descriptions without assuming that knowledge of it is direct, complete or infallible (Bhaskar, 2013; Danermark et al., 2019) [4, 10].

These debates have renewed relevance in the twenty-first century. Artificial intelligence (AI), large-scale datasets, digital scholarship, climate change and persistent inequalities create opportunities to identify patterns while also raising questions about explanation, bias, context and authority. Predictive accuracy, for example, does not by itself explain the causal processes behind an outcome or establish that a decision is fair. Flanagan et al., (2023) similarly notes that interpreting machine-learning systems and understanding their reasoning are important challenges in scientific practice. This paper provides a conceptual comparison of major research paradigms and evaluates the contribution and limitations of critical realism for contemporary knowledge creation. It does not report an empirical study or claim to offer a systematic review. Its argument is that critical realism can be particularly useful where researchers need to move from describing patterns to developing context-sensitive causal explanations, but that it should remain in dialogue with other philosophical traditions and locally grounded forms of knowledge.

2. Approach and Scope of the Paper

This paper is a conceptual study based on a purposive, interpretive synthesis of selected works in the philosophy of science, social research methodology, critical realism, and the sociology of knowledge. It brings into dialogue Kuhn’s account of paradigmatic change in science, Bhaskar’s realist philosophy of science, Berger and Luckmann’s analysis of the social construction of knowledge, and methodological debates concerning research paradigms, explanation, and the relationship between theory and evidence. These traditions are examined not as an exhaustive historical survey, but as complementary resources for developing an argument about how knowledge is produced, justified, and contested in the twenty-first century.

The paper then considers the contemporary significance of this argument through a focused discussion of selected literature on artificial intelligence, data-intensive research, and the politics of knowledge. These contemporary examples are used illustratively: they demonstrate how questions of explanation, interpretation, power, context, and epistemic authority arise in current knowledge practices, but they are not treated as a systematically sampled empirical literature. The analysis is therefore guided by conceptual relevance rather than by a comprehensive search strategy or formal inclusion and exclusion criteria.

No original dataset was collected, and the paper does not report primary empirical findings. It also does not claim to be a systematic review, scoping review, bibliometric study, or meta-analysis. Its contribution is instead theoretical and synthetic: it reconstructs connections among major philosophical and methodological positions and uses selected contemporary debates to clarify their implications. The paper should consequently be read as a theoretically informed argument grounded in an interpretive engagement with the literature, rather than as a comprehensive account of all scholarship in the field. Claims about particular technologies, institutions, or social outcomes remain illustrative and should not be interpreted as general empirical findings beyond the sources discussed.

3. Historical Foundations: Knowledge, Science and

3.1. Ontology, Epistemology and Methodology

Ontology concerns what exists and the nature of the reality under investigation. Epistemology concerns what can be known and how knowledge claims may be warranted. Methodology concerns the logic and organisation of inquiry, including the relationship between questions, evidence and analytical procedures. Methods such as surveys, interviews, experiments and document analysis are practical tools; they are not, by themselves, complete philosophical positions. The same method can be used within different paradigms if its purpose and interpretation differ (Crotty, 2020) [9]. The distinction matters because an observable event may be interpreted in different ways. A statistical association can establish a pattern, but its meaning and causal significance depend on the research design, the quality of measurement, theoretical assumptions and alternative explanations. Likewise, participants’ accounts may illuminate meanings and experiences without necessarily providing a complete account of every structural condition shaping those experiences. Rigorous inquiry therefore requires alignment between the question being asked, the assumptions adopted and the evidence used.

3.2. Paradigms and Scientific Change

Kuhn, (1970) challenged the idea that scientific development consists only of a steady accumulation of facts. He argued that scientific communities work within paradigms that shape legitimate problems, methods and standards of solution. Anomalies may eventually contribute to a crisis and a reorganisation of scientific practice, although not every change in research philosophy should be described as a Kuhnian revolution. The development of alternative paradigms is better understood as a continuing debate about the strengths, limits and purposes of scientific explanation than as a simple linear replacement of one worldview by another.

The history of epistemology also includes rationalist and empiricist traditions, as well as later attempts to clarify the role of concepts, experience and social context in knowledge. These traditions should not be collapsed into a single progression from ancient philosophy to contemporary research: philosophical positions have coexisted, influenced one another and been reinterpreted across disciplines. For present purposes, their importance lies in demonstrating that evidence is essential but that its meaning and justification are also matters of philosophical argument.

Implications

4.1. Positivism

Positivism is associated particularly with Auguste Comte’s argument as cited in Brown, (1994) that systematic scientific inquiry should focus on observable phenomena and regular relationships rather than speculative metaphysical explanations. In research practice, positivist approaches generally privilege clearly specified concepts, systematic measurement, hypothesis testing and the identification of regularities. Quantitative surveys, experiments and statistical modelling can be appropriate where variables can be measured reliably and the research question concerns distributions, associations or effects.

Positivist approaches have contributed to the development of standardised measurement, reproducible procedures and evidence-based decision-making. They can be especially useful for estimating prevalence, comparing groups, testing specified hypotheses and evaluating interventions under defined conditions. Nevertheless, the claim that positivism simply assumes a wholly value-free researcher or that all positivist work seeks immutable universal laws is too broad. Actual research traditions vary, and careful quantitative inquiry must address measurement error, uncertainty, model assumptions and the social conditions under which data are produced.

A further limitation arises when observable regularities are treated as sufficient explanations. A correlation between household income and programme participation, for instance, does not by itself show why participation differs or how institutions, trust, social norms and access constraints shape it. This is not a reason to dismiss quantitative evidence; it is a reason to distinguish description and prediction from causal explanation.

4.2. Post-Positivism

Post-positivism retains the idea that a world exists independently of researchers but rejects the assumption that it can be known with complete certainty. Popper, (2005) argued that scientific theories should be exposed to serious attempts at refutation rather than regarded as conclusively verified by accumulated confirming observations. Knowledge is consequently provisional: claims remain open to revision in light of new evidence, methodological criticism and competing explanations.

Post-positivist inquiry commonly emphasises explicit hypotheses, critical testing, attention to bias, triangulation and the transparent treatment of uncertainty. Quantitative methods remain important, but findings are interpreted as fallible rather than final. This orientation is valuable in policy and applied research because it encourages researchers to ask what evidence would challenge a conclusion. However, post-positivism does not automatically require mixed methods; method selection should be justified by the question and the design.

4.3. Interpretivism

Interpretivism emphasises that human action is meaningful and that social phenomena cannot always be understood by treating people as if they were objects governed by the same kinds of regularities sought in some natural sciences. Associated with Weber, (1978) concept of Verstehen, interpretive inquiry seeks to understand how people make sense of their actions and circumstances. Interviews, ethnography, phenomenology, case studies and narrative approaches can help researchers examine meanings in context.

Interpretivism is particularly relevant when the research question concerns identity, lived experience, institutional trust, cultural practice or the meanings attached to a policy or technology. Its strength is contextual depth, not a guarantee that every interpretation is equally convincing. Researchers must show how interpretations are grounded in evidence, consider divergent accounts and explain their own role in producing the analysis.

4.4. Social Constructionism

Berger & Luckmann, (1979) argued that much of everyday social reality is sustained through interaction, institutionalisation and shared systems of meaning. Constructionist approaches investigate how categories, identities, norms and institutional facts are produced and maintained through language and social practice. Examples include the changing meanings of professional status, legal identity, gender roles or what counts as legitimate expertise. Social constructionism does not necessarily imply that all reality is invented or that material conditions do not matter. Rather, it draws attention to how people collectively produce and stabilise meanings through which the social world is understood and organised. In digital environments, collaborative encyclopedias, online communities and social media illustrate how knowledge can be produced and contested beyond formal academic institutions. Such settings also make credibility, moderation, misinformation and unequal visibility important research concerns.

4.5. Pragmatism

Pragmatism focuses on the consequences of ideas and the role of inquiry in resolving problems. In the tradition of Dewey, (1929) knowledge is connected to action, experience and the revision of beliefs through investigation. Pragmatism does not mean that any claim is true simply because it is convenient. Rather, claims and methods are assessed in relation to the problem, the consequences of acting on them and the quality of inquiry through which they are developed. Pragmatism can support methodological pluralism, including mixed-methods designs that combine measurement with analysis of experience and context. For example, an evaluation may use quantitative indicators to estimate programme outcomes and interviews to understand implementation barriers. The methods are combined because they contribute different kinds of evidence to the same practical question, not because mixing methods is automatically superior in every study.

4.6. Critical Theory and Postmodern Critiques

Critical theory examines how power, ideology and institutional arrangements shape social life and knowledge. It asks whose interests are served by dominant explanations and whether inquiry can expose or challenge relations that reproduce inequality. Postmodern and poststructuralist approaches have further questioned universalising claims and examined how discourse, classification and institutional authority shape what societies recognise as truth. Foucault, ( 2013) for example, analysed relationships between knowledge and power rather than treating accepted knowledge as wholly separate from institutional practices. These approaches are important correctives to accounts of knowledge that ignore exclusion, history or power. However, it would be inaccurate to treat all postmodern thinkers as claiming that an external reality does not exist or that no claim can be better supported than another. Different authors make different arguments, and the strongest analysis engages those arguments precisely rather than reducing them to a single doctrine.

4.7. Critical Realism

Critical realism, developed most prominently by Roy Bhaskar, seeks to preserve the independence of reality from human knowledge while recognising that knowledge is produced through historically situated concepts, practices and institutions (Bhaskar, 2014) [5]. Its central distinction is between ontology and epistemology: the world does not depend on what researchers believe about it, but explanations of that world remain fallible and revisable. A key feature is a stratified understanding of reality, commonly explained through the empirical, actual and real domains. The empirical concerns experiences and observations; the actual concerns events that occur whether or not they are observed; and the real concerns structures, powers and mechanisms capable of generating events. These domains are related but not identical. An event may occur without being witnessed, and a mechanism may exist without producing the same outcome in every context. Causal explanation therefore requires attention to the conditions under which mechanisms operate (Bhaskar, 2014; Danermark et al., 2019) [5, 10].

Critical realism is also associated with retroduction: reasoning from an observed phenomenon towards hypotheses about the conditions or mechanisms that would make it possible. Researchers develop and compare explanatory accounts, then assess them against empirical evidence, theoretical coherence and rival explanations. Retroduction is not a licence to invent invisible causes; proposed mechanisms require justification and must remain open to criticism. Because social systems are often open systems in which several mechanisms interact, critical realism does not assume that causal processes always produce constant, observable regularities. It can therefore be useful when a programme has different effects in different communities or when similar outcomes arise through different processes. Danermark shows how realist social research connects abstract concepts and empirical inquiry to develop explanatory accounts.

5. Comparing the Paradigms

Table 1: Comparative overview of major research paradigms Paradigm View of reality and knowledge Research emphasis Potential contribution Positivism Reality is objective and observable. Measurement, hypothesis testing, and statistical analysis.

Identifies patterns, relationships, and predictions.

Post-positivism Reality exists independently, but knowledge is fallible.

Critical testing, uncertainty, and triangulation.

Produces cautious, evidence-based explanations.

Interpretivism Reality is understood through subjective meanings and experiences.

Interviews, ethnography, and case studies.

Explains experiences and context-specific meanings.

Constructionism Knowledge and meanings are socially constructed.

Discourse, narrative, and interaction analysis.

Explains how meanings and social categories emerge.

Pragmatism Knowledge is evaluated through practical consequences.

Problem-oriented and mixed-methods research.

Supports practical solutions and decision-making.

Critical theory and post-structural approaches Knowledge and social categories are shaped by power and discourse.

Ideology, discourse, and institutional analysis.

Exposes inequalities and challenges dominant assumptions.

Critical realism Reality exists independently, but knowledge is partial; events arise from underlying mechanisms.

Explanatory inquiry and causal mechanism analysis.

Explains outcomes through underlying mechanisms and contexts.

Source: Author synthesis based on (Bhaskar, 2014; Crotty, 2020; Danermark et al., 2019; Guba & Lincoln, 1994). Note: Each paradigm contains internal diversity, and researchers sometimes combine insights from more than one tradition. The table should therefore be used as a guide to philosophical differences, not as a classification system that mechanically determines which method must be used.

6. Critical Realism and Knowledge Creation in the

6.1. Artificial Intelligence and Data-Intensive Research

AI and data-intensive research are methods for obtaining, processing and analyzing large volumes of data and identifying patterns and making predictions. While these methods enable scientific discovery, it is important to note that the presence of a pattern in data is not an automatic causal explanation. Outputs of models are dependent on the data available, the objectives and labels chosen, and the assumptions built in to the systems. The interpretability of machine learning systems remains challenging, particularly when scientists and decision makers need to understand how a machine came to a conclusion as opposed to simply knowing that the machine made the right prediction (Cartwright, 2023) [8].

According to a critical realist perspective, researchers are invited to distinguish between the level of a model’s predictive performance and the mechanisms that generate the outcome to be predicted. In this framework, empirical regularities are important, however they do not by themselves demonstrate the causal powers, structures or context dependent processes, which make a given outcome possible (Benton, 2013; Brown et al., 2021) [1, 6]. For example, an algorithm may identify households at high risk of poverty, but this classification alone does not explain the processes that produce poverty or identify which intervention is likely to be effective. Researchers may therefore need to examine labour markets, land access, household composition, public services, institutional decisions, and local histories, using evidence suited to each question.

This means that critical realism helps develop explanatory questions along with the assessment of technical options. Instead of viewing data-intensive systems as surrogate techniques of causality, it promotes the examination of how practices of data generation along with the researcher’s modelling, institutional and social choices, shape the A.I. system’s capabilities and what it can detect and classify. This requires researchers to move beyond formal or functional assessment of A.I. systems towards contextual and explanatory assessment(Cartwright, 2023; Benton, 2013; Brown et al., 2021) [1, 6, 8]. Consequently, AI-generated knowledge is strongest when there is a combination of predictive evidence and theoretical explanation, contextual analysis, and a critical assessment of the assumptions that link data to conclusions.

6.2. Big Data, Climate Change and Complex Systems

Large data sets help to illuminate patterns and trends over space and time. But addressing social and environmental issues requires acknowledging that there are interacting processes. Take for example, the changing climate, which encompasses biophysical changes and human-made infrastructure and systems, different ways of living and making a living, governance and different levels of exposure to risk. Critical realism suggests that outcome X may be attributable to multiple mechanisms (interacting processes) that need to be addressed to fully understand a given issue. (Bhaskar, 2010) [3].

Integrating this perspective can enrich remote sensing, environmental monitoring, statistics and community-based research. Satellite observations can show changes in vegetation cover, but understanding why these changes happened may need other forms of evidence, including rainfall, grazing, land management, markets, tenure and local decisions. It’s not to substitute philosophical argument for measurements; it is to relate the measurements to questions for which explanations are sought, and to make use of relevant forms of evidence to test the explanations. (Khazem, 2018) [15].

6.3. Mixed Methods and Explanatory Research

Critical realism permits the use of both qualitative and quantitative methods, since they answer different types of questions. Quantitative methods are useful to indicate distributions, associations and variations in outcomes, while qualitative methods are used to analyze meanings and to explain decisions and to illuminate institutional practices. Documentary, historical or observational evidence may be necessary to assess explanations. The value of combined methods is judged by whether they result in a better explanation. Contradictory findings, if any, should be identified and explained (Danermark et al., 2019) [10]. Look at a rural development intervention that is adopted in some communities but not others. Survey data may show adoption to be related to household resources. Interviews may show perceptions of the program’s legitimacy. Differences in implementation may be found in the institutional records. A critical realist analysis may compare the reasons and mechanisms that may explain adoption or non-adoption. The analysis should result in a defensible, revisable explanation. It should not be the case that one paradigm is assumed to automatically reveal the single hidden cause.

6.4. From ‘Does It Work?’ to Context-Sensitive

Explanation An intervention is likely to create different results in different places because there are likely to be differences in the resources, institutional relationships and social conditions required for the intervention to be successful. Evaluation may, therefore, focus not only on determining the relationship between an intervention and outcome but also on for whom and in what circumstances the intervention worked, by what mechanisms and what unintended consequences it had. The interest is, therefore, in contexts and the mechanisms (pathways) that link contexts to outcomes. This is in keeping with the realist approach to evaluation (Pawson & Tilley, 1997) [17].

7. Critical Realism in Development Studies

Development problems such as persistent poverty, unequal access to education, gender inequality and uneven participation in public programmes rarely have a single cause. Household income, for example, may be influenced by access to land and markets, education, infrastructure, finance, labour relations, public policy and historical inequalities. These factors can interact, and their importance may differ between places and groups.

A quantitative study can estimate the distribution of poverty or test associations between income and selected variables. An interpretive study can explore how households experience insecurity and understand their choices. A critical realist study can draw on such evidence to investigate the structures and mechanisms that reproduce particular outcomes, while remaining attentive to actors’ interpretations and the context in which they act. Such an approach is especially useful when the aim is to explain why an intervention succeeds in one setting but produces limited effects in another. Nevertheless, critical realism should not be represented as a complete policy solution. Explanatory depth does not guarantee that a recommendation is politically feasible, ethically acceptable or effective in practice. Researchers must also consider distributional consequences, implementation capacity, uncertainty and the voices of people affected by decisions. Better explanation can inform policy, but transformation depends on institutional and political processes beyond research alone.

8. Knowledge, Power and Decolonisation in Africa

Debates about decolonising knowledge encompass a number of things, from the content of educational curricula to choosing local illustrative examples. However, most importantly, it concerns the power to decide research problems and frame the questions that shape the universe of discourse. It refers to the selection of conceptual apparatus and the types of evidence that will warrant an answer to the questions, as well as the beneficiaries of knowledge. The postcolonial critique of knowledge has affirmed that knowledge institutions can serve to reinforce the colonial hierarchy and privilege certain languages, methodologies and intellectual traditions. The conditions under which research is initiated, sponsored, implemented, and used merit more critical analysis. Indigenous and traditional knowledge systems incorporate different levels of observations about the environment and interactions of human beings with their environment and surroundings. They develop memory about events and interactions in human history in a way that documentation and science may not cover. While such knowledge systems should not be generalised, essentialised and absolutised, they should be understood in the right context. Critical realism allows researchers to recognize the reality of processes from the historically specific accounts that provide understanding. It facilitates conversations between scientific measurement and locale-based knowledge without assuming that either one of them is complete. Critical realism, however, is not a decolonial methodology.

Decolonising research implies a change in, among others, the setting of research agendas, authorship, language, ownership, control and the governance of data and the distribution of research benefits. Ubuntu, with its emphasis on engagement with and concern for others, provides a basis for a social theory of knowledge. It should be read in its own history and interpreted faithfully rather than reduced to popular sayings. Attention should be paid to the historical and social specificity in which different knowledge systems emerged rather than to their superficial similarities. From a decolonising perspective, critical philosophy should be employed so as to acknowledge scholarly work in various intellectual traditions while being attentive to the asymmetry in scholarly powers, as well as the uneven distribution of institutional power and finances in the world. This should also include a recognition of the value of local knowledge.

9. Limitations and Critical Evaluation of Critical

Realism Critical realism has several strengths, but it also presents challenges. First, underlying mechanisms are not always directly observable, and researchers may disagree about whether a proposed mechanism is adequately supported. Retroduction can become speculative if researchers move too quickly from an outcome to a preferred explanation without considering alternatives or gathering further evidence. Second, critical realist concepts can be difficult to operationalise. The distinction between the empirical, actual and real domains is philosophically useful, but it does not automatically tell researchers which data to collect, how to measure a construct or how to compare rival mechanisms. Those decisions require substantive theory, transparent methods and careful justification.

Third, an emphasis on multiple interacting mechanisms can produce overly elaborate explanations. If every outcome is attributed to a long list of possible causes, an explanation may become difficult to evaluate. Researchers should therefore identify the mechanisms most relevant to the question, explain why they are plausible, specify the contexts in which they are expected to operate and consider evidence that would count against the proposed account.

Finally, the distinction between ontological realism and fallible knowledge does not eliminate disagreement over what constitutes the best explanation. Critical realism is strongest when treated as a disciplined orientation to inquiry rather than a guarantee of explanatory truth. It should be assessed alongside alternative approaches according to the clarity of its concepts, the quality of its evidence, its treatment of rival explanations and its usefulness for the research problem.

10. Implications for Researchers

Make philosophical assumptions explicit. Researchers should state why a particular view of reality and knowledge is appropriate to the research question and explain how it informs the design.

Distinguish description, prediction and explanation. A reliable pattern or accurate prediction is valuable, but neither necessarily identifies the processes that generated an outcome.

Align methods with the question. Methods should be selected for their evidential contribution, not merely because they are conventionally associated with a paradigm. Test rival explanations. Researchers should document how alternative accounts were considered and what evidence supports or weakens each account.

Treat context as analytically important. Institutional, historical, cultural, economic and environmental conditions can shape how processes operate and whether outcomes vary. Practise reflexivity. Researchers should consider how their assumptions, positionality, institutional location and choices affect the questions asked and interpretations produced. Engage knowledge holders ethically. Where community and indigenous knowledge is involved, consent, attribution, ownership, reciprocal benefit and the right to contest interpretations should be addressed.

Communicate uncertainty. Claims should be proportionate to the evidence, limitations should be transparent and conclusions should remain open to revision.

11. Conclusion

The development of philosophical paradigms demonstrates that knowledge creation involves more than the collection of facts. Positivism has contributed systematic measurement and the investigation of regularities; post-positivism emphasises fallibility and critical testing; interpretivism foregrounds meaning; constructionism examines the social production of knowledge; pragmatism links inquiry to practical problems; and critical approaches examine power and exclusion. These traditions differ in their assumptions and purposes, and none should be reduced to a caricature. Critical realism offers a valuable framework for research that seeks to explain how observable outcomes are generated by underlying structures and mechanisms in particular contexts. Its distinction between reality and fallible knowledge, together with its support for explanatory inquiry and methodological pluralism, is relevant to complex development problems, environmental change and the growing use of AI and data-intensive systems. However, it does not replace the need for empirical rigour, interpretive understanding, ethical engagement or critical analysis of power.

The future of knowledge creation should therefore not be framed as the inevitable victory of critical realism over other paradigms. A more defensible conclusion is that contemporary research requires philosophical reflexivity, careful alignment between questions and methods, transparent evaluation of explanations and meaningful engagement with diverse knowledge traditions. Where critical realism helps researchers move from patterns to plausible, testable explanations, it can make an important contribution to scientific and socially relevant inquiry. Its contribution will be strongest when its own assumptions and limitations remain open to scrutiny.