THE RELATIONAL EVALUATION AND THE EVALUATION OF THE SOCIAL IMPACT.
EVALUATION OF THE LESMES FOUNDATION OF BURGOS (SPAIN).
This research is carried out with a double meaning. First of all, the work consists in the design and application of a relational evaluation model, and of a social and economic impact model of organisations and social intervention projects in the local realm. Secondly, with the aim of verifying the feasibility and the utility of the model, we proceed to its application, making a relational evaluation of the Lesmes de Burgos Foundation, and of the social and economic impact of its programmes and its social and labour insertion companies. Thus, four hypotheses are formulated:
– H1: The social and labour insertion programmes of the Lesmes Foundation (CEFE) and its insertion companies (CEISLABUR, GRM and REUSAD) are efficacious and obtain more positive results than other programmes.
– H2: The social and labour insertion programmes of the Lesmes Foundation and its insertion companies are more efficient (ratio of results / resources invested) than another type of similar actions.
– H3: The society receives an economic return from the social and labour insertion programmes of the Lesmes Foundation and its insertion companies greater than the investment.
– H4: It is possible and feasible the structuring and application of a relational evaluation model, and of an impact model of the implementation of organisations and projects of social intervention that facilitate the decision-making in order to optimise its results.
The Lesmes Foundation is a local organisation of the third sector of social action with over 20 years of experience, the purpose of which is the social and labour insertion of people who are in situation or at risk of social exclusion. For it, it develops projects of integration of outcasts into society and projects of social and labour insertion, and it promotes insertion companies.
After a notional and practical review of the status of the issue, the evaluation of the social intervention, especially going in depth into the impact, and analysing the attendant conceptual field (the phenomenon of the social exclusion, the third sector, and the theory itself and the practice of the social evaluation), it has adopted a notional, methodological, integrating and adapted approach.
A model based on two components is designed: the relational evaluation and the evaluation of the social and economic impact. On the one hand, the model includes the relational evaluation with a foremost qualitative complexion, and with an innovative and notional approach, starting from a generalisation of the ecological theory by the psychologist Urie Bronfenbrenner. The starting principle is founded on the fact that the development of a social organisation and its programmes are the consequence of its relationships organised in systems and subsystems. On the other hand, it contemplates the quantitative evaluation of the social and economic impact, which consists of measuring the net effects of the intervention, establishing a causal relationship that leads us to isolate the deadweight, the decay and the attribution, and to try to mitigate, as far as possible, the displacement. 39 similar works developed in 14 different countries are revised.
For the so-called relational or nodal evaluation, 6 systems of development of the social organisation of reference (onto, micro, meso, macro, exo and chronosystem) are defined. Nodes and relationships are identified, and practical and prospective conclusions are obtained. Action points (improvement, risk, attention and opportunity) are detected. The information obtained from this component of the evaluation complements and is used for the interpretation of the results of the second component, which is the impact one. Qualitative techniques are used, such as the Delphi technique, and also quantitative one so as to classify the relationships.
The evaluation of the social and economic impact is carried out with a quantitative methodology through an ex post facto design. A sample of 608 people, 457 of which are an experimental group from the Lesmes Foundation and 151 of which are a control group from other programmes, is selected. The independent variable is defined, which is the sort of intervention (education, guidance, insertion company, …), and the dependent one, which is the difference between the number of contribution days the two years after the intervention and the previous two years. Other variables, such as the sex, the age, the studies and the profile, are valued and controlled, as well as the current employment situation, the percentage of working days and the employment affinity with the education, when applicable. Interventions carried out over 10 years, between 2006 and 2015 (observations between 2004 and 2017) are studied.
It is intended to establish a causal relationship between the IV and the DV, empirically confirming the existence of a covariation between both variables, showing the direction of the causal link and isolating the strange variables. After verifying that the control and experimental groups start from a homogeneity in the baseline of the DV, we proceed to evaluate the effectiveness with the generated contribution days, and the efficiency with the calculation of the benefit and the economic return of the interventions, depending on the savings in passive policies and the profit according to the generated taxes and contributions. This component ends with the hypothesis contrast, using the Student’s t-test. The results are the following:
- The active employment policies are efficacious. They achieve a significant improvement of the employment situation (between 127 and 184 days).
- The active employment policies are profitable, generate an economic impact and a greater return than the necessary investment (between 1’60 € and 2’08 € for each euro invested).
- The education programmes and those which include work experience or on-the-job training obtain better insertion results than those which are limited to guidance and labour intermediation actions (mean difference: 48 days for the education programmes and 58 days for those which include work experience).
- The insertion companies are more efficacious than the rest of the interventions (mean difference: 66 days).
- The insertion companies, in addition to being the most effective one, are the most efficient interventions, since the differential economic impact is greater than the rest of the programmes (difference of 0’96 € for each euro invested).
- The programmes and especially the social and labour insertion companies of the Lesmes Foundation offer better and more effective results than other active employment policies (mean difference: 57 days).
- The social and labour interventions of the Lesmes Foundation are more efficient than other similar interventions. The average profitability is higher (difference of 0’48 € for each euro invested).
Integrating both components, the differential results found for the rated entity are explained through some arguments:
- Framework of complementary and coordinated programmes, with a wide range of sources of funding. Optimised administrative and technical structure. Appropriate infrastructure and material resources.
- Count on insertion companies.
- Rooted and local territorial implementation.
- Belong to the third sector.
- Interdisciplinary, involved and experienced human team. Democratic internal structure.
- Stability of the programmes developed. Lasting learning. Long-term planning.
In summary, a model of evaluation of the social intervention has been designed and applied, and its feasibility and utility have been proven. It integrates different theoretical-practical approaches, and a mixed qualitative and quantitative methodology. It is set within a theory of the social change. It aims to find a causal relationship between the intervention and the improvement of the employment situation. It evaluates the effectiveness and the efficiency. Finally, it has a marked prescriptive nature, offering information and relevant elements for the decision-making that improve the intervention, and guiding its future planning.
KEY WORDS.
Social impact assessment, relational assessment, nodal evaluation, relationship, node, system, social intervention, third sector, socio-labour insertion, social exclusion, Lesmes Foundation, effectiveness, efficiency, causal relationship, economic impact, return.